🔙 Home

For a quite detailed reference about Harbour multi thread programming, please visit Alexander Kresin's pages; see also Przemek's introduction on multitrhreading and MULTI THREAD SUPPORT section of <xhb-diff.txt> (by same author); furthermore, you might want to try out a dozen of multi-thread samples, listed into Harbour /tests sub-directory.

• hb_threadStart([<nThreadAttrs>,] <@sStart()>|<bStart>|<cStart> [, <params,...>] ) ➜ pThreadID|NIL
  attempts to create a new thread and start executing the code specified by <@sStart()>|<bStart>|<cStart> which can be a function Symbol or a code-Block or a function Name. <params,...> is an optional list of parameters passed to the function or code-block. [<nThreadAttrs>], if used, specifies the attributes of thread, as they're defined into hbthread.ch
  Returns either a pointer to the newly created thread or NIL on failure.

• hb_threadSelf() ➜ pThreadID> | NIL
  returns either the pointer of thread, from inside which, this function is invoked or NIL if this pointer happens to be NULL.

• hb_threadID([<pThreadID>]) ➜ nThreadNo
  returns the numeric ID of the the current or <pThreadID> thread into the threads' stack.

• hb_threadIsMain([<pThID>]) ➜ lMainHvmThread
  it returns true if the given [<pThID>] thread ID or the current thread (if no ID is given) is main HVM thread.
  (note: this is a new function added on 2017-07-27 with this commit, therefore not available in earlier builds).

• hb_threadJoin(<pThreadID> [, @<xRetCode>]) ➜ lOK
  suspends execution of the current thread, until the thread pointed by <pThreadID> ends executing (exit). If @<xRetCode> passed (by reference), obtains the exit (?) code of the joined thread.

• hb_threadDetach(<pThreadID>) ➜ lOK
  detaches (disconnects) the <pThreadID> thread from current process, leaving it running separately. Detached thread will release all of its resources automatically upon ending running.

• hb_threadQuitRequest(<pThreadID>) ➜ lOK
  sends a QUIT request to given <pThreadID> (child) thread, while suspends the execution of current (sender) thread, waiting for the implementation of request (that is, the termination of <pThreadID> thread).

• hb_threadTerminateAll() ➜ NIL
  sends to all threads a request to complete (terminate) their running and waits until it happens. It can be called only from the main thread.

• hb_threadWaitForAll() ➜ NIL
  waits until all the threads have finished running.

• hb_threadWait(<pThreadID>|<apThreadID>, [<nTimeOut>] [, <lAll>]) ➜ nThreadNo | nThreadCount | 0
  suspends execution of the current thread for <nTimeOut> seconds (or forever, if no time-out specified), waiting for the completion of either the <pThreadID> thread, or one of the threads in the array of pointers <apThreadID>, or all the threads of that array when <lAll> passed as .T..
  Returns the ID number of the thread completed, or the total number of the threads completed in the given period (if <lAll>==.T.), or 0 (zero) if none of the requested thread(s) has been completed and the time-out has elapsed.

• hb_threadOnce(@<onceControl> [, <bAction>|<@sAction()>] ) ➜ lFirstCall

• hb_threadOnceInit(@<item>, <value>) ➜ lInitialized

• __vmCountThreads([@<nStacks>], [@<nThreads>]) ➜ nThreads
  returns the number of active threads.

• hb_mutexCreate() ➜ pMutex
  creates a mutex and returns its handle (pointer).
  In general, a mutex is a kind of a flag visible and respected by all running threads. It is used by a thread (which can hold the mutex by locking it) to "inform" the other threads, that it is entering into a critical stage during which it will use and probably alter(!) some shared resources (usually variables with application-wide scope) and because of this, they do not permitted accessing/utilizing of those resources; therefore, they have to wait until the mutex be unlocked. This mechanism prevents the so called "race conditions" that may happen in a multi-threading application.

• hb_mutexLock(<pMutex> [, <nTimeOut>]) ➜ lLocked
  locks the <pMutex> for the thread that invokes the lock. Practically this means that the thread will have exclusive access of any shared resource contained in the code following the mutex-locking, until either explicitly unlocks the mutex (by 'hb_mutexUnlock()') or the <nTimeOut> (in milliseconds) elapsed.
  Returns .T. on successful locking, .F. otherwise.

• hb_mutexUnlock(<pMutex>) ➜ lOK
  unlocks a previously locked mutex. Returns .T. on successful unlocking, .F. otherwise.

• hb_mutexNotify(<pMutex> [, <xVal>]) ➜ NIL

• hb_mutexNotifyAll(<pMutex> [, <xVal>]) ➜ NIL

• hb_mutexSubscribe(<pMutex>, [<nTimeOut>] [, @<xSubscribed>] ) ➜ lSubscribed

• hb_mutexSubscribeNow(<pMutex>, [<nTimeOut>] [, @<xSubscribed>]) ➜ lSubscribed

• hb_mutexEval(<pMutex>, <bCode>|<@sFunc()> [, <params,...>] ) ➜ xCodeResult

• hb_mutexExists( <pMutex> ) ➜ lExists
  it returns true if the given <pMutex> parameter is pointer item to a fully functional mutex.
  (note: this is a new function added on 2017-07-27 with this commit, therefore not available in earlier builds).

• hb_mutexQueueInfo(<pMutex>, [ @<nWaitersCount>] [, @<nQueueLength>]) ➜ .T.

• hb_mtvm() ➜ lMultiThreadVM
  checks if the application is multi-thread ready, which, by the way, can be enabled by compiling the application with -mt Harbour switch.

^{(as it's stated in this 2008's commit).}

«Harbour threads needs OS threads support. Each Harbour thread is directly mapped to OS thread. It's not very efficient on some older system where cost of thread creation and/or task switching is very expensive but it should not be bigger problem for modern OS-es which can support threads in practice nearly in user space only.
I haven't touched Harbour function calling convention which comes from Clipper. It means that we do not pass pointer to VM to each functions like CLIP or Xbase++. To resolve the problem I have to use thread local storage (TLS) where such pointer is kept. If platform does not support TLS then it can be emulated by us. Anyhow the speed of accessing TLS data and extracting HB_STACK poitner is critical for performance. Some compilers depending on used hardware and OS give native support for TLS (f.e. __thread keyword in GCC/BCC or __declspec( thread ) in MSVC). This should give optimal performance. On other Harbour uses TLS functions like TlsGetValue() (MS-WIN) or pthread_getspecific() (PTHREAD) are used. OS2 gives quite interesting TLS functionality which seems to be quite fast though it will be interesting to know how it is iplemented internally for real multi CPU machines (if it depends on CPU exception then the performance will be bad). We need TLS only for one pointer to HB_STACK structure.
I haven't added any tricks like HB_THREAD_STUB in xHarbour to reduce the cost of TLS access. If it will be necessary for some platform the we can add it.
Except TLS Harbour threads needs OS support for non recursive mutexes or critical sections and conditional variables. If platforms does not support conditional variables (f.e. MS-Win or OS2) then they can be emulated using multistate semaphores. I intentionally didn't create code which may need recursive mutexes. The non recursive ones are often faster and some platforms may not support recursive mutexes so they will have to be emulated by us.
Harbour uses reference counters for complex variables. It means that even readonly access to complex item causes internal write operations necessary to increment/decrement its reference counter. To make such readonly access MT safe we have to make incrementation and decrementation with result checking atomic. By default it's done by mutex inside vm/fm.c but some platforms have native support for atomic inc/dec operations, f.e. Interlocked*() functions in MS-Win. If they are available then such functions should be used to not reduce the performance by mutex call very often used functions. For many CPUs it should be quite easy to implement such atomic inc/dec functionality in assembler. F.e. for GCC and x86@32 it may looks like:

  static __inline__ void hb_atomic_inc32( volatile int * p )
  {
     __asm__ __volatile__(
        "lock incl %0"
        :"=m" (*p) :"m" (*p)
     );
  }

  static __inline__ int hb_atomic_dec32( volatile int * p )
  {
     unsigned char c;
     __asm__ __volatile__(
        "lock decl %0"
        "sete %1"
        :"=m" (*p), "=qm" (c) :"m" (*p) : "memory"
     );
     return c == 0;
  }

and then it's enough to define in hbthreads.h:

      #define HB_ATOM_INC( p )    hb_atomic_inc32( ( volatile int * ) p )
      #define HB_ATOM_DEC( p )    hb_atomic_dec32( ( volatile int * ) p )

Probably I'll make it for some most popular CPUs in the future.
In Harbour each thread which wants to call HVM functions have to allocate it's own HVM stack. It's done hb_vmThreadInit(). The HVM stack is freed by calling hb_vmThreadQuit(). This functions can be called also by 3-rd party threads if they want to call HVM functions or execute .prg code. Calling HVM functions without allocated stack will cause GPF. I moved most of static variables to HVM stack to make them thread local. But some of them like FS errors have their own alternative copy which is used when thread does not allocate HVM stack. It allows to use hb_fs*() functions without HVM stack but programmer have to know that error codes return by hb_fs*Error() functions can be overwritten by other threads which also didn't allocated HVM stack. To execute garbage collector scan and mark pass it's necessary to stop other HVM threads. Otherwise the scan may give false results. It's also possible to not stop threads but protect with mutex all operations on GC items but it will probably cause performance reduction and will force some other modifications. Maybe I'll implement it in the future.
I didn't use any OS level thread KILL or CANCEL calls. All HVM threads have to be cleanly removed without any resource leaks. QUIT command terminate only calling thread. If main (startup) HVM thread call QUIT then it sends QUIT request to all existing threads. In QUIT state ALWAYS statements and destructors are executed. New thread is created by:

      hb_threadStart( <@sStart()> | <bStart> [, <params,...> ] ) -> <pThID>

The returned value is a pointer to internal thread structure which can be used in JOIN or DETACH operations. Each thread should be Joined or DETACHED to avoid resource leaks. If programmer does not store or all instances of are destroyed then thread is automatically detached. I do not know clear method of thread detaching in OS2. If some OS2 users knows it then plase update vm/hbthread.c. When thread terminates then all locked by this thread mutexes are released.
Each thread uses its own memvars (PRIVATEs and PUBLICs) and work areas. When new thread is created then it inherits from parent thread:

  - code page
  - language
  - SETs
  - default RDD

error block is initialized to default value by calling ERRORSYS()
and PUBLIC variable GetList := {} is created.
The following objects are initialized to default value:

     - error block
     - math error handler and math error block
     - macro compiler features setting (hb_setMacro())
       or move them to SETs.   

We can think about inheriting them. It's also possible to add inheriting of all visible memvars but I do not know it's good idea.

Compilation and linking:

For MT mode HVM library should be compiled with HB_MT_VM macro. GNU make automatically creates hbvmmt library which should be linked with Harbour MT programs instead of hbvm. Non GNU make files should be updated. If given compiler support TLS then you can try to set HB_USE_TLS to force using native compiler TLS support. Now it's enabled by default only for BCC. For Linux and GCC builds it may depend also on used GLIBC version. In older system there is no TLS support at all or TLS works only for shared binaries so I haven't enabled it. If you will test some other compiler then please add default native TLS support for them in hbthread.h Users using hb* scripts can simply use -mt switch when they want to create MT program, f.e.:

      hbmk -n -w3 -es2 -mt mttest01.prg   

There are still some minor things which should be done but I'll do them later. Current state seems to be fully functional. The most important and still missing is our own file lock server for RDD synchronization in POSIX systems. Kernel internally recognize POSIX locks by PID and file i-node - not PID and file handle. It means that the same file open more then once by one process shares locks. Because POSIX locks can be overwritten then we do not have any synchronization between aliased workareas or threads using the same table in *nixes. We have to make synchronization ourselves. I'll create such lock server ASAP.

Please test and enjoy using Harbour threads.»

🔙 Home