亚洲国产日韩欧美一区二区三区,精品亚洲国产成人av在线,国产99视频精品免视看7,99国产精品久久久久久久成人热,欧美日韩亚洲国产综合乱

There is a question: If it is the data in the database, why not execute sql? This is much more efficient than multi-process?
If you have to choose one of the two given, then consider:

1). 如果你要用多進程去數(shù)據(jù)庫獲取數(shù)據(jù)(就算你用了mysql連接池,可以不怎么考慮數(shù)據(jù)庫連接的io消耗),
你每取一次數(shù)據(jù),總要有一次查詢吧, 完了以后,你還要把更新后的數(shù)據(jù)寫入到數(shù)據(jù)庫了, 又是一次數(shù)據(jù)庫操作,
想想這個消耗有多大?
2). 數(shù)據(jù)庫獲取100萬數(shù)據(jù)，然后想多進程去執(zhí)行x++; 這種情況啊,只要計算機內(nèi)存夠(只有100萬數(shù)據(jù),基本是沒問題的), 用python的進程池map一下,確實也是沒什么問題

First store the original data in the queue (queue), as a producer
Then fetch the data from the queue, perform operations, and act as a consumer
At this time, you can open multiple threads on the consumer (of course, if you handle the lock well, Producers can also do multi-threading)

while tmp_queue.empty() is not True:
    x = tmp_queue.get()
    x += 1

In the queue, if there are always elements, the thread will continue to operate.

In fact, the best way to implement multi-processing in Python is to use multiprocessing中的map

Example (Python 3):

# f.py
# 要對某個列表中每個元素都執(zhí)行一次的function
def f(x):
    return x + 1

# main.py
from multiprocessing import pool

from f import f
# 創(chuàng)建進程池
p = pool.Pool(4)
lst = range(100)
# 使用多進程對整個列表進行計算
print(p.map(f, lst))

You can directly write the two functions of data reading and data +1 into one operation, and then use multiple processes to operate. Just use the process pool to operate as mentioned above. Set the size of the process pool according to the number of your cpu cores. Since there is no memory sharing and direct communication between multiple processes, you can first use multiple processes to read all the data from the database, and then use multiple processes to perform val+1