Minetest Asynchronous Lua - sapier/minetest GitHub Wiki

Table of Contents Asynchronous Lua Calls Preface Basic concept Details AsyncEngine WorkerThread async_environment job issueing/result handling

ATM minetest contains two separate LUA stacks encapsulated within engines (GUIEngine,ScriptAPI). One used for mainmenu another one used for scriptapi. These stacks share basic components e.g. common functions. Lua functions are organized in submodules which register their functionality to an engine if their initializer is called by the engine.

Asynchornous Lua calls are added by creating (one or more) additional lua engines. Those engines are identical and are used to run the asynchronous functions. The current Engines have full access to either map or menu gui, this isn't feasable for asynchronous calls. If it was possible a asynchronous thread could destroy things done by other threads. While obvisously lot's of functions can't be used from a asynchronous thread, others are perfectly safe. In mainmenu most of them deal with downloading things from internet.

AsyncEngine is responsible for creation of worker threads as well as Job dispatching.
Capabilities of worker threads (supported functions) can be configured prior initialization of AsyncEngine. Configuration is done by registering the functions to the AsyncEngine.

Once initialization is done the worker threads wait at a semaphore for jobs to become available.
Jobs are transfered by GUIEngine/(later Scriptapi) and first stored within a internal list. Once this is done semaphore is counted up to wake up a worker thread. After worker thread has finished the jobs result is passed back to AsyncEngine. The worker will continue waiting at the semaphore until next job is to be processed.

A worker thread is a wrapper around a lua stack. Most of it's time it's sleeping waiting for a job to become ready. Once a job is ready the serialized job function and parameters are pushed onto stack. This stackis passed to lua jobhandler function.
The serialized result is sent back to AsyncEngine and the thread will sleep again.

The async environment is a stripped down lua environment only containing basic helper functions.
In normal operation worker thread does only call job_processor lua function. As there are multiple async environments to handle jobs global variables are NOT available to be used. All data to be processed has to be passed by parameter to job processor. Some engine functions are available within these environments. For GUIEngine these are basicaly all calls that don't require access to non static parts of GUIEngine. This limitation could be fixed for ScriptAPI by using propper environment/map locking!

Job processor receives two parameters, serialized function and serialized data. First both are deserialized, if this suceeds the function is called with data beeing passed as only parameter.

Return value of this function is serialized and sent back to worker thread.

A job consists of three things:

• Job function to be called in async enviroment
  • The job function must comply to the limitations of async environment:
    • No global variables
    • No direct GUIEngine access
    • No userdata variables
  • A job function may or may not have a parameter
  • a return value is mandatory
• data to be passed to this function
  • may not contain userdata values
• function to be called once result is available
  • no limitations as it's run within GUIEngine (not asynchronous)

A job is issued by calling do_async(jobfunction,parameters,callback)