Servers - Theano/Theano GitHub Wiki

This page try to put in one place information about making a server that will use Theano.

Mailing list thread Advice on Theano as a webservice

Peoples used ZeroMQ

Machine translation demp (used celery, code)

Manual conversion to Javascript (could be useful as an example)

Theano implementation note

• Theano function aren't thread safe. If you use threads, only 1 threads at a time can use a given Theano function. You can use a lock around a Theano function to serialize access or use one copy of the function per threads via import copy; copy.copy(f)
• Theano compilation time can be significant, so you probably do not want to recompile the Theano function at each request. This can be done with a fixed thread pool instead of creating new function every time.
• If you do not want to import your full model on the server, you can pickle the Theano function and just unpickle it on the server. To make this even faster, you can reuse the old optimized graph from the pickle with the Theano flags reoptimize_unpickled_function

import cPickle
# To dump
f = theano.function(...)
with fd as open("fct.pkl", 'w'):
    cPickle.dump(f, fn, -1)
# to load
f2 = cPickle.load(open("fct.pkl", "r"))

• The server will need a compiler that Theano can use (even if you unpickle) most of the time. This is because Theano generate C code during Theano function compilation. We cache that compilation, so it would be done only the first time. If you do not want that, you could copy the compiledir to the servers or compile the Theano function in the master version of the VM/container. Theano currently do not help on that. Do not forget that by default Theano won't reuse already compiled module if they are too old. To tell Theano to reuse older c module, use the Theano flag cmodule.age_thresh_use (new since December 21th 2016).
• A possible long term solution is to have Theano generate a shared library. This would request no [partial] recompilation, linker, c code generation/compilation.