- Vikyath Shirva
This little article is an introduction to something that confuses almost everyone at some point:
Concurrency, Parallelism, SIMD, MIMD...
You’ve probably heard these terms in system design discussions, backend interviews, or while reading about CPUs and GPUs. And most of the time, the explanations feel either too complicated or too abstract to really stick.
So let me break it down in the simplest way possible.
By the time you finish reading this, you should have a clear mental model of what these actually mean.
We are going to use something very basic.
Wearing your shoes.
The confusion mainly comes from the fact that these words sound similar, and people often use them interchangeably. But they are not the same thing.
At a very high level, you can think of it like this: concurrency is about managing multiple tasks, while parallelism is about doing multiple tasks at the same time. That sounds simple, but it becomes much clearer once you see it in action.
Imagine you are getting ready to go out. You have two socks, two shoes, and only two hands. You cannot physically tie both shoes at the same time, so you naturally end up switching between tasks.
You might wear your left sock first, then the right one, then move on to tying your left shoe, and finally the right shoe. At any given moment, you are only doing one thing, but over time, you are making progress on multiple things.
That is concurrency.
You are not doing things simultaneously; instead, you are handling multiple tasks by switching between them. This is very similar to how a single CPU core works. It rapidly switches between different tasks, giving the illusion that everything is happening at the same time, even though it is not.
Now imagine that something changes. Instead of just two hands, you either have four hands or someone else is helping you.
In this case, one hand (or one person) can take care of the left shoe while another handles the right shoe. Both tasks are now happening at the same time.
This is parallelism.
Here, multiple tasks are being executed simultaneously because you have more resources available. This is how multi-core CPUs work in reality. Different cores can execute different tasks at the same time, rather than switching between them.
Now let’s go one level deeper.
Imagine you give a single instruction: “tie the laces.” Somehow, both your hands execute that exact same instruction at the same time, but on different shoes.
The instruction is the same, but the data (the shoes) is different.
This is SIMD: Single Instruction, Multiple Data.
This is exactly how GPUs operate. They take a single operation and apply it across a large number of data points simultaneously, such as pixels in an image or elements in a matrix. Everything moves in sync, following the same instruction.
Now consider a more realistic scenario.
You are not alone anymore. You are tying your shoe, your friend is polishing another shoe, someone else is cooking in the kitchen, and another person is checking their phone.
Everyone is doing something different, on different things, completely independently.
This is MIMD: Multiple Instruction, Multiple Data.
This is how modern systems actually work. Multiple cores run different programs or threads, each executing its own instructions independently. There is no requirement for everything to stay in sync like in SIMD.
If you step back and look at it:
Concurrency is about one person managing multiple tasks by switching between them. Parallelism is about multiple workers handling tasks at the same time. SIMD is when the same action is applied everywhere, and MIMD is when different actions happen independently.
These are not just theoretical concepts. They directly map to how real systems are designed and optimized.
If you simplify it down:
Concurrency is about managing tasks.
Parallelism is about executing tasks simultaneously.
SIMD is uniform execution.
MIMD is independent execution.
Once you start thinking in terms of hands as resources, the brain as control, and tasks as instructions, these ideas stop being abstract. They become something physical that you can visualize.
And once you can visualize them, you won’t forget them.