python 多进程并行编程 ProcessPoolExecutor的实现
时间:2021-03-15 09:53:57|栏目:Python代码|点击: 次
使用 ProcessPoolExecutor
from concurrent.futures import ProcessPoolExecutor, as_completed import random
斐波那契数列
当 n 大于 30 时抛出异常
def fib(n):
if n > 30:
raise Exception('can not > 30, now %s' % n)
if n <= 2:
return 1
return fib(n-1) + fib(n-2)
准备数组
nums = [random.randint(0, 33) for _ in range(0, 10)] ''' [13, 17, 0, 22, 19, 33, 7, 12, 8, 16] '''
方案一:submit
submit 输出结果按照子进程执行结束的先后顺序,不可控
with ProcessPoolExecutor(max_workers=3) as executor:
futures = {executor.submit(fib, n):n for n in nums}
for f in as_completed(futures):
try:
print('fib(%s) result is %s.' % (futures[f], f.result()))
except Exception as e:
print(e)
'''
fib(13) result is 233.
fib(17) result is 1597.
fib(0) result is 1.
fib(22) result is 17711.
fib(19) result is 4181.
can not > 30, now 33
fib(7) result is 13.
fib(12) result is 144.
fib(8) result is 21.
fib(16) result is 987.
'''
等价写法:
with ProcessPoolExecutor(max_workers=3) as executor:
futures = {}
for n in nums:
job = executor.submit(fib, n)
futures[job] = n
for job in as_completed(futures):
try:
re = job.result()
n = futures[job]
print('fib(%s) result is %s.' % (n, re))
except Exception as e:
print(e)
'''
fib(13) result is 233.
fib(17) result is 1597.
fib(0) result is 1.
fib(22) result is 17711.
can not > 30, now 33
fib(7) result is 13.
fib(19) result is 4181.
fib(8) result is 21.
fib(12) result is 144.
fib(16) result is 987.
'''
方案二:map
map 输出结果按照输入数组的顺序
缺点:某一子进程异常会导致整体中断
with ProcessPoolExecutor(max_workers=3) as executor:
try:
results = executor.map(fib, nums)
for num, result in zip(nums, results):
print('fib(%s) result is %s.' % (num, result))
except Exception as e:
print(e)
'''
fib(13) result is 233.
fib(17) result is 1597.
fib(0) result is 1.
fib(22) result is 17711.
fib(19) result is 4181.
can not > 30, now 33
'''
上一篇:在Ubuntu系统下安装使用Python的GUI工具wxPython
栏 目:Python代码
下一篇:Python使用matplotlib实现基础绘图功能示例
本文标题:python 多进程并行编程 ProcessPoolExecutor的实现
本文地址:http://www.codeinn.net/misctech/81463.html






