Summary
This chapter explains how to use multiprocessing to achieve true parallelism by spawning separate processes with isolated memory spaces, demonstrating process creation, starting, joining, and data sharing via queues for interprocess communication.

General domain of usage
data crunching

Picture your Python process as a restaurant franchise, not just a single kitchen. In multiprocessing, you don't just hire more chefs for one kitchen—you actually open several kitchens in different buildings. Each kitchen (process) has its own chef, fridge, stove, and even its own 'Master Knife'—there's no sharing. This means Chef A in Kitchen A and Chef B in Kitchen B can both chop, cook, and serve at the same time, truly in parallel, because they're not waiting for a shared resource like the GIL in threading.

This is the core advantage of multiprocessing: each process runs in its own memory space, so they never bump into each other. It's perfect for heavy tasks such as mathematical calculations, image processing, or data crunching, where real parallelism can speed things up dramatically.

However, if Chef A in Kitchen A needs to pass an ingredient to Chef B in Kitchen B, it's not as easy as just handing it over the counter. They need a delivery truck—this is Inter-Process Communication (IPC). IPC is slow and expensive compared to sharing ingredients in the same kitchen, so you should only use it when necessary.

Let's see how this looks in Python using the multiprocessing module. First, you'll see how to launch two processes that each perform a task in parallel:

```python
from multiprocessing import Process
import time

def print_numbers(name):
    for i in range(1, 4):
        print(f"Process {name}: {i}")
        time.sleep(0.5)

# Create process objects
p1 = Process(target=print_numbers, args=("A",))
p2 = Process(target=print_numbers, args=("B",))
```

Each process is like a chef in their own kitchen. Next, start and join the processes:

```python
p1.start()
p2.start()

p1.join()
p2.join()

print("Both processes have finished.")
```

The output will show both processes running in parallel, with their print statements interleaved. This is true parallelism—each process is independent and can use the CPU at the same time.

But what if you need to share data? You must use special tools, like queues or pipes, which act like delivery trucks between kitchens. Here's how to use a queue for IPC:

```python
from multiprocessing import Process, Queue

def worker(q):
    q.put('Ingredient from worker')

queue = Queue()
p = Process(target=worker, args=(queue,))
p.start()
print(queue.get())
p.join()
```

This approach is slower than simply sharing data between threads, so use multiprocessing when you need real parallelism for CPU-heavy tasks, and keep communication between processes to a minimum. 



Python's **`multiprocessing`** module enables you to achieve **true parallelism** by spawning separate processes, each with its own Python interpreter and memory space. Unlike threads, which share memory and run in the same process, **processes are completely isolated** from each other. This isolation avoids the **Global Interpreter Lock (GIL)** limitation, allowing multiple CPU cores to work in parallel on CPU-bound tasks. After watching the video above, you should understand that **process-based parallelism** is especially useful when you need to perform heavy computations or take advantage of multiple cores for tasks that would otherwise be bottlenecked by the GIL. The **`multiprocessing`** module provides a familiar API similar to the **`threading`** module, making it easy to switch between threading and processing depending on your needs.

Represents an independent process. Use this class to create and manage separate processes.

A process-safe FIFO queue for sharing data between processes. 

Provides a connection between two processes for two-way communication.

Manages a pool of worker processes for parallel execution of a function across data. 

A synchronization primitive for controlling access to shared resources between processes.

Allows sharing a single value of a specific type between processes.

Allows sharing an array of values between processes. 

Creates a server process that holds Python objects and allows other processes to manipulate them using proxies.

Returns a reference to the currently running process object.

Returns the number of CPU cores available on the system.

Use these classes and functions to build robust parallel applications, manage data sharing, and coordinate process execution in Python.

import multiprocessing
import time

def print_message(message, delay):
    time.sleep(delay)
    print(message)

process1 = multiprocessing.Process(target=print_message, args=("Process 1 finished", 2))
process2 = multiprocessing.Process(target=print_message, args=("Process 2 finished", 1))

process1.start()
process2.start()

process1.join()
process2.join()


This code sample demonstrates how to use Python's **`multiprocessing`** module to run tasks in parallel by creating separate processes. The example uses simple time delays and prints messages to show how processes can execute independently and simultaneously.

#### Key Concepts Illustrated

- **Process Creation:**
  - You create new processes using the `multiprocessing.Process` class;
  - Each process is given a target function (`print_message`) and arguments to pass to that function.

- **Process Starting:**
  - The `start()` method launches each process, allowing them to run concurrently;
  - Once started, each process runs its target function independently of the others.

- **Parallel Execution:**
  - Both processes sleep for different durations (`2` and `1` seconds) using `time.sleep()`;
  - Because they run in parallel, the process with the shorter delay finishes first, even though it was started after the other.

- **Process Joining:**
  - The `join()` method ensures that the main program waits for both processes to finish before moving on;
  - Without `join()`, the main program could exit before the processes complete.

This example shows that with multiprocessing, tasks can run at the same time on multiple CPU cores, making it ideal for CPU-bound operations that benefit from parallel execution.

Which statement best describes the difference between threads and processes in Python?

A deep dive into advanced structural programming concepts in Python, focusing on robust file handling, error management, concurrency, asynchronous programming and testing.

Master Python's exception system for robust error handling and debugging.

Explore advanced techniques for file operations and resource management using context managers.

Compare and apply Python's multithreading and multiprocessing modules for concurrent programming.

Master asynchronous programming in Python using the asyncio library for scalable I/O-bound applications.

Explore how to write, organize, and run tests using Python's unittest and pytest frameworks.

Working with Processes

Key Concepts Illustrated