Context Switching
To solve the hardware disparity between the number of active processes and the limited number of physical CPU cores, the Operating System acts as a master scheduler. The mechanism it uses to share the CPU among multiple processes is called Context Switching.
The Context Switching Mechanism
At a fundamental level, the CPU is completely unaware of different "applications." It simply executes instructions sequentially. It is the Operating System that orchestrates the switching by forcefully swapping out the memory and registers.
When the OS decides that Process A has had enough time on the CPU and it is time to give Process B a turn, it executes a strict sequence of events:
- Pause Process A: The OS interrupts the CPU.
- Save Process A State: It takes all the CPU registers (most importantly, the Program Counter) and saves them to memory so it knows exactly where Process A left off.
- Load Process B State: It retrieves the previously saved registers and Program Counter for Process B and loads them into the CPU.
- Run Process B: Process B executes on the CPU for a few milliseconds (its "time slice").
- Pause Process B: The OS interrupts the CPU again.
- Resume Process A: The OS saves B, loads A's old Program Counter, and Process A resumes exactly as if it had never been interrupted.
The Result: Concurrency
This switching process is heavily tested in backend engineering because it defines the concept of Concurrency.
Concurrency means that multiple tasks are making progress at the same time, but they are not necessarily executing at the exact same physical millisecond.
Because the OS performs this Save A -> Run B -> Resume A cycle thousands of times per second, human users perceive it as both processes running simultaneously. However, under the hood, the single CPU core is still strictly executing one task at a time.
[!WARNING] The Cost of Switching (Thread Thrashing): Context Switching is fast, but it is not free. Saving and loading registers takes a microscopic amount of CPU time. If your system spawns 50,000 threads, the OS will spend so much time just saving and loading states (Context Switching) that the CPU will have 0% capacity left to actually execute your code. This fatal bottleneck is called Thread Thrashing.