Multitasking - mikahama/mikatools GitHub Wiki

Mikatools makes it extremely easy to run Python code in parallel.

from mikatools import *

def counter(max):
	random.seed()
	w = open_write(str(random.randint(1,101)) + "_tmp.txt")
	for i in range(max):
		w.write(str(i) + "\n")
	w.close()

jobs = [{"max":10000},{"max":10000},{"max":10000},{"max":10000},{"max":10000},{"max":10000}]

t = WorkerRunner(counter, jobs, 2)
t.start()

In the example above, the method counter(max) will be run in parallel with the parameters from jobs.

WokrerRunner(function, kwarg_list, number_of_workers=4, run_as_threads=False)

• function is the method that will be called with each item from the kwarg_list
• kwarg_list is a list of keyword arguments. Once a new process is started, a keyword argument dictionary is passed from this list to the function
• number_of_workers sets the number of parallel processes.
• if run_as_threads is False, the tasks are run as processes, if it is set to True, they are run as threads.

Threads or processes? By default, the jobs are run as processes. The limitation of this approach is that the parallel jobs will not share memory. Therefore, they cannot modify the same global variable, for example. With threads they would run using the same memory... However, threading is broken in Python and does not run truly in parallel, so I would not recommend using it.

WokrerRunner.start(join=True)

This method will start the parallel tasks. If join is True, the script will wait for the tasks to complete before continuing. If join is false, the execution will continue and a Queue object is returned.