Introduction to Multithreading

Program, Process & Threads

A Program (or an application) is an executable file on the disk. When we execute any program it starts a Process. A Process (or Context) is a running instance of the program located in the memory. A Thread is a unit of execution in the process. It is also known as a lightweight process.

A Process consists of several things few of which are PID (Process ID), application code, heap memory and at least one thread named the main thread. It may contain other threads as well.

A Thread consists of several things few of which are an Instruction Pointer and Stack memory. Each thread inside a process has its own Instruction Pointer and Stack Memory, the rest of them are shared between them.

Multitasking Process vs Multitasking Threads

Multitasking at the process level allows multiple processes to run concurrently. In this type of multitasking a process is the smallest unit of code that can be dispatched by the scheduler. Since each process requires its addresses, they become heavy-weight tasks carrying the expensive overhead of context switches. Inter-process communication between processes is also expensive. This approach should be preferred If the tasks are unrelated, or If we don't want the tasks to share memory or If stability and security are our main concerns. While Java programs make use of a process-based multitasking environment, they are not directly under Java's control.

Multitasking threads allows multiple threads to run concurrently. In this type of multitasking a Thread is the smallest unit of code that can be dispatched by the scheduler. Unlike process, threads share memory among other threads that belong to the same process. Interthread communication and context switches between threads are inexpensive.

💡

Concurrency: By multi-tasking between different threads our system can create an illusion that all those tasks are happening in parallel. This kind of multitasking is called concurrency. Even with a single-core machine, we can achieve concurrency.

💡

Context Switch: When the OS switches from one thread to another that is called as context switch. Context switching is an expensive task. Generally, context switching between threads belonging to the same process is less expensive because threads share some resources within the process. If our application has too many threads then it can cause thrashing.

💡

Thrashing is when the application takes more time in context switches than to execute the threads.

Why Multithreading?

The 2 main reasons to use multithreading are Responsiveness & Performance:

Responsiveness is when the users are not made to wait after they perform any action. If one user submits a request to a web server then the other user should not have to wait until the first user’s request is served. We can achieve this by using multiple threads. Responsiveness is extremely important in applications with a user interface. This can be achieved using concurrency.

Using multiple cores we can run multiple tasks in parallel. This will help complete complex tasks in a fraction of the time than the single-threaded program.

The single-threaded program makes use of infinite event loop polling. In this model, a single thread of control runs in an infinite loop, polling (checking) a single event queue to decide what to do next. For example, if we have a single-threaded program which has to download a file and also do some computation unrelated to the file. In this program, the execution will not proceed ahead until the file has been downloaded because the main thread is polling to check if the download has been completed to proceed further.

Thread Scheduling

In case of multiple threads, the OS has to schedule threads so that they can be executed. There exist many different types of algorithms for thread scheduling, few of them are:

First Come First Serve:
A thread that comes first gets executed first. A common disadvantage of this kind of scheduling is that if a thread having a long execution time comes first then it can lead to starvation of other threads. Starvation is when a process waits for an indefinite time to acquire the resource.
Shortest Job First:
The thread with the shortest execution time gets executed first. However, this also has its disadvantages, If the CPU keeps executing the shortest job first then it is possible that it might not be able to execute the threads that have a long execution time.

As we have observed, each one of them has its advantages and disadvantages and is not ideal for practical scenarios.

Practical Implementation of Thread Scheduling:
In Practice, The OS divides the time frame into moderately sized pieces called Epochs. In each epoch, the OS allocates a different time slice of a thread. Not all the threads are completed in an epoch. This decision is taken based on the dynamic priority. In the below image, 4 threads are executed in an epoch. The threads may or may not get completed in an epoch.

Dynamic Priority = Priority + Bonus (Bonus can be negative). Static priority is set by the developer and Bonus is adjusted by the OS in each epoch for each thread. This helps the OS to keep giving preference to the time-sensitive threads like UI threads but also keep executing the actual tasks. We should treat priority as a polite suggestion for the OS.

What's Next?

Now we are aware of what threads are. In the next post, we will be exploring how to use the Thread class in Java to interact with threads.

References:

Rizwan Shaikh's Blog