Autograd and Fork - subramen/pytorch GitHub Wiki
TLDR: Use spawn instead of fork.
Autograd engine relies on threads pool, which makes it vulnerable to fork. We detect such situations and warn users to use spawn method of multiprocessing.
So this code will work
import multiprocessing as mp
ctx = mp.get_context('spawn')
simple_autograd_function()
with ctx.Pool(3) as pool:
pool.map(simple_autograd_function, [1, 2, 3])When this code will fail
import multiprocessing as mp
ctx = mp.get_context('fork')
simple_autograd_function()
with ctx.Pool(3) as pool:
pool.map(simple_autograd_function, [1, 2, 3])See https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods for more details.