Multi tenanting with a Reverse Proxy - tooltwist/documentation GitHub Wiki

For a summary of options for Multi-tenanting, see Multi Tenanting and Reverse Proxies.

The approach described here allows a single domain name to route requests to multiple servers, based up a tenantId in the first position of the URL.

multi-tenanting-reverse-proxy

There are several advantages to this approach:

A regular ToolTwist application can be used as a multi-tenant application, without the application code's logic understanding the concept of tenants.
The back end servers do not necessarily have to be the same application.
The logical separation of the tenants is guaranteed (assuming they don't share databases, user sessions, etc).

The Reverse Proxy

We use Apache in this wiki page, but similar functionality is available in Nginx (pronounced "Engine-X") and other reverse proxies. The operations that need to be performed are:

Routing the request to the appropriate back end.
Converting the URL to something the back end understands.
Inserting the tenant back into all URLs in the responses from the back end.

This final point is quite important. Given the innumerable types of web content the back end might return (CSS, Javascript, Images, XML, etc) the reverse proxy server has limited understand of the content in the response, so cannot reasonably determine what is a URL and what is not. The simplest option is to convert every instance of a string that is known to only be used in URLs.

For ToolTwist we use /ttsvr/, and this will be replaced whether it occurs in a URL, in the text of HTML, or anywhere else in any file with one the mime-types we will specify. For example, if we call a site with the URL http://mysite.com/chicken/hello.html, and that page contains /ttsvr/ in a URL or anywhere else, then /chicken/ will be replaced in those locations. If you wish to place /ttsvr/ into the page you will need to break up the string in some way.

<body>
    This page cannot output /ttsvr<div/>/  without breaking up the string in some way.
</body>

Apache 2.4 Config

The following is an example Apache configuration used to route requests to two different backend servers, based upon whether the request URL matches /au/... or /nz/....

<VirtualHost *:80>
	ServerAdmin [email protected]

	# Serve up some files directly from Apache    
	DocumentRoot /var/www/site
	<Directory /var/www/site/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride All
		Order deny,allow
		Allow from all
	</Directory>

	ProxyPreserveHost On
	#ProxyHTMLEnable On

	<Location "/au/">
		# https://httpd.apache.org/docs/current/mod/mod_proxy.html
		ProxyPass		http://192.168.200.17:9001/ttsvr/
		ProxyPassReverse	http://192.168.200.17:9001/ttsvr/
		#ProxyPassReverseCookieDomain  "backend.example.com"  "public.example.com"
		#ProxyPassReverseCookiePath  "/ttsvr/"

		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/html
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/css
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/plain
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/x-component
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/javascript
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/json
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/xhtml+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/rss+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/atom+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/vnd.ms-fontobject
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE image/svg+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE image/x-icon
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/x-font-ttf
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE font/opentype

		Substitute "s|/ttsvr/|/au/|i"
	</Location>

	<Location "/nz/">
		# https://httpd.apache.org/docs/current/mod/mod_proxy.html
		ProxyPass		http://192.168.200.17:9002/ttsvr/
		ProxyPassReverse	http://192.168.200.17:9002/ttsvr/
		#ProxyPassReverseCookieDomain  "backend.example.com"  "public.example.com"
		#ProxyPassReverseCookiePath  "/ttsvr/"

		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/html
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/css
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/plain
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/x-component
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/javascript
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/json
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/xhtml+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/rss+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/atom+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/vnd.ms-fontobject
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE image/svg+xml
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE image/x-icon
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/x-font-ttf
		AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE font/opentype

		Substitute "s|/ttsvr/|/nz/|i"
	</Location>    

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

Explanation

The optional top part of the file allows Apache to serve up files as well, if they do not match one of the locations specified further down.

<Location "/au/">...</Location>

The block tells Apache how to handle requests with a specific URL pattern. In our case we're using ProxyPass and ProxyPassReverse to specify that incoming requests should be sent to a back end server with the URL remapped, and that we should perform a reverse mapping of URLs in any headers in the response.

ProxyPass		http://192.168.200.17:9001/ttsvr/
ProxyPassReverse	http://192.168.200.17:9001/ttsvr/

For example, http://mysite.com/au/frog-guts.html will be routed to http://192.168.200.17:9001/ttsvr/frog-guts.html, and http://mysite.com/nz/frog-guts.html will be routed to http://192.168.200.17:9002/ttsvr/frog-guts.html. In this example only the port number is changing for the different location, but you can use different IP addresses or domain just names as easily.

Note that ProxyPassReverse only looks after the response headers, not the content of responses.

The ProxyPassReverseCookieDomain and ProxyPassReverseCookiePath allow URLs to be similarly mapped in cookies, (but I have not tested this functionality).

AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE text/html

The AddOutputFilterByType lines define the Filters to be applied for each mime type. We include HTML, CSS, Javascript, JSON, XML, and other common types. The filters we apply are run in order on the responses from the relevant back end server:

Inflate (un-compress) any gzipped responses from the backend server. We need to do this so that the substitution step that comes next can work. For performance reasons it is best if the back end does not compress it's responses, but we can leave the INFLATE instruction there in any case.
Substitute strings in the response content, to remap our URLs.

Substitute "s|/ttsvr/|/au/|i"
Deflate (compress) the response.

Notes

Once again, the URL prefix used on the backend server needs to be unique, because it will be substituted anywhere it occurs in responses, not just in URLs. For ToolTwist we use /ttsvr/, but with other server environments (e.g. NodeJS) any unlikely string could be used. Using something like ZIUUWHHA would be ideal, because it's never going to be included in a normal web page or static files.

On a Linux machine you will need to enable the Apache modules before they can be used in the config file. (There's probably a few more than needed here)

  a2enmod php7.0
  a2enmod rewrite
  a2enmod filter
  a2enmod proxy_http
  a2enmod proxy_html
  a2enmod xml2enc
  a2enmod deflate
  a2enmod substitute

Test Harness

We have a test harness that can be used to test various configurations.

It runs Apache in a Docker container, and three back-end servers written in NodeJS to simulate the ToolTwist /ttsvr/ URLs. From a browser you can check that URLs are mapped correctly in HTML, CSS and Javascript files.

See https://github.com/tooltwist/reverse-proxy-test-harness.

Useful links

Apache module reference - http://httpd.apache.org/docs/current/mod
mod_proxy - http://httpd.apache.org/docs/current/mod/mod_proxy.html
mod_substitute - http://httpd.apache.org/docs/current/mod/mod_substitute.html