Python的multiprocessing模塊通過繞過GIL限制實現(xiàn)並行處理，適合CPU密集型任務(wù)。其核心是Process類，可啟動獨立進程，如示例中的show_pid函數(shù)被兩個進程分別執(zhí)行。數(shù)據(jù)共享主要通過Queue或Pipe及Value/Array實現(xiàn)，如使用隊列傳遞數(shù)據(jù)確保進程安全。對於批量任務(wù)，Pool提供了高效的解決方案，支持map()和apply_async()方法分配任務(wù)。但需注意性能權(quán)衡：進程啟動開銷、大數(shù)據(jù)傳輸延遲、共享狀態(tài)復(fù)雜性及平臺差異可能影響效率。

Python's multiprocessing module is a solid way to handle CPU-bound tasks that can benefit from running in parallel. Unlike threading, which is limited by the GIL (Global Interpreter Lock), multiprocessing spawns separate processes, each with its own Python interpreter and memory space. That makes it ideal for tasks like heavy computations, image processing, or any job that can be split into independent chunks.

Understanding the Basics of Multiprocessing

At the core of the multiprocessing module is the Process class. It works similarly to threading's Thread , but instead of running threads within the same process, it launches new processes. Each process runs independently, so you're not blocked by Python's GIL.

Here's a basic example:

 from multiprocessing import Process
import os

def show_pid():
    print(f'Hello from process {os.getpid()}')

if __name__ == '__main__':
    p1 = Process(target=show_pid)
    p2 = Process(target=show_pid)
    p1.start()
    p2.start()
    p1.join()
    p2.join()

Each call to start() launches a new process. The join() method ensures the main script waits until both processes finish. This is how you typically start using multiprocessing — by defining target functions and launching them in separate processes.

Sharing Data Between Processes

Because each process has its own memory space, sharing data between them isn't as straightforward as with threads. If you try to use global variables or regular Python objects, changes won't be reflected across processes.

There are two main ways to share data:

• Using Queue or Pipe – These are thread-safe and process-safe communication channels.
• Using shared memory constructs like Value or Array – These allow multiple processes to read/write the same memory block.

For example, using a queue:

 from multiprocessing import Process, Queue

def worker(q):
    q.put("Data from process")

if __name__ == '__main__':
    q = Queue()
    p = Process(target=worker, args=(q,))
    p.start()
    print(q.get()) # Will print "Data from process"
    p.join()

Queues are often the preferred choice when you need to pass complex or dynamic data between processes.

Using Pools for Parallel Processing

If you have a large number of similar tasks to run, using a Pool is more efficient than manually managing processes. A pool creates a fixed number of worker processes and distributes tasks among them.

The most commonly used methods are:

• map() – Blocks until all items are processed
• apply_async() – Non-blocking, allows you to retrieve results later via a callback

Example using map() :

 from multiprocessing import Pool

def square(x):
    return x * x

if __name__ == '__main__':
    with Pool(4) as pool:
        result = pool.map(square, [1, 2, 3, 4, 5])
        print(result) # Outputs [1, 4, 9, 16, 25]

Pools are especially useful when dealing with batch operations like file processing, scraping, or mathematical transformations.

Performance Considerations and Gotchas

While multiprocessing can speed things up, it's not always faster. Spawning new processes comes with overhead — especially on Windows, where the entire interpreter needs to be re-spawned. So if your tasks are very small or fast, the time spent starting processes may outweigh the benefits.

Also, be careful with:

• Large data transfers between processes — sending big chunks over queues can slow things down.
• Shared state — even though possible, it adds complexity and potential bugs.
• Platform differences — code that works fine on Linux might behave differently on macOS or Windows due to how processes are forked or spawned.

So think carefully about task size, data flow, and platform before diving deep into multiprocessing.

基本上就這些。

以上是探索與Python多處理的並發(fā)的詳細內(nèi)容。更多資訊請關(guān)注PHP中文網(wǎng)其他相關(guān)文章！