Is there a way to run a batch in parallel? I wrote a function that I have been using with guild, but it is taking a long time to run so I would like to run a massive batch in parallel. Something like this:
pool = mp.Pool(processes=4)
datas = pool.map(func,cases)
Ideally the cases here would come from my batch, and then Guild would save the results of each case to its own folder. Is there any way to do this currently?