Home - Orange-OpenSource/mod_dup GitHub Wiki

Welcome to the mod_dup mod_migrate and mod_compare wiki!

Table of Contents

mod_dup

mod_dup allows to duplicate HTTP requests to one or more destinations, requests are matched on URL,Query String, Headers and Body using regular expressions.
mod_dup may also transform the duplicated version of these matched requests.

Use mod_dup for the following use cases:

Duplicate Requests

with DupDuplicationType HEADER_ONLY (for GET) or COMPLETE_REQUEST (all methods)
Use this mode to transparently perform the request on two or more different servers for different purposes:

  • Making sure preproduction and staging platforms receive real production traffic.
  • Performance/Load/Capacity testing with real production requests and a duplication multiplier
  • Mirroring
  • Database migration (with mod_migrate) or upgrade with no downtime
    • Setup a new database with a new WebService that calls it
    • Install mod_dup on the old WebService to Duplicate requests from the old WebService to the new, so that inserts are performed on both DBs
    • Perform a backup up the old DB and a restore on the new DB, restoring only what is not newer on the new DB than the backup
    • Voila, you have a fully consistent new DB

Duplicate and Compare

To be used with DupDuplicationType REQUEST_WITH_ANSWER and mod_compare on the receiving end
Use this mode to pre-validate an N+1 code version of your Rest or SOAP API.

  • Setup your new WebService or Rest API code on a new server
  • Install mod_compare on the new Server
  • Install mod_dup (and mod_migrate) on the old server
  • mod_compare will tell you which requests yielded a different result on N and N+1 servers

Configuration

To configure the request duplication module see: mod_dup configuration

mod_compare

mod_compare allows to compare in realtime the response header and body of HTTP Requests played identically on two Rest or SOAP APIs.
it is made to be installed on server(s) B with mod_dup installed on server(s) A. mod_compare receives an http request which contains the response header and body of the a Web Service that will be compared to the response of the web service installed on the same apache alongside mod_compare. For more details on the request format: mod_compare serialized request format

Two operating modes are possible:

Realtime Response Comparison

Use this mode to perform the response comparison of two different servers.
If the responses do not match, mod_compare writes the differences in a file.
This is the default mode.

No Comparison: write to file

Use this mode to not perform the comparison but to write the two responses to a file, for later offline comparison.

Comparison output formats

mod_compare supports two output diff formats

  • Human Readable Text: similar to diff
  • json for easy usage with Elastic Search + LogStash + Kibana.
For details, see here: mod_compare logs

Configuration

To configure the request comparison module see: mod_compare configuration

mod_migrate

mod_migrate is basically mod_rewrite with the capability of matching on the request body
mod_migrate allows to match HTTP Request URL,Query String, Header and Body using regular expressions, and performs the "migration" of the request if the matching condition is satisfied.
If the content you want to match is in the body, you can use migrate to dynamically set an environment variable, in conjunction with mod_rewrite which will rewrite or proxy based on the presence of that variable.

Use it in the following scenarii:

  • For clients to transparently call a new WebService that conforms to the old one without having to change the DNS or the destination IPs, hence no downtime and limited maintenance on your client's side.
    • The response of the new Webservice will be returned to the client, even though it passed through the old server.
    • To do that, install mod_migrate on the old WebService to migrate requests from the old WebService to the new one.
  • For any case where you wanted to use mod_rewrite but realized it can't match anything in the body of the request.

Configuration

To configure the request migrate module see: mod_migrate configuration

how does this work?

To understand how the different apache modules and filters interact with each other, please visit the page: mod_dup_migrate_compare internals

How is it tested?

  • Unit testing is done with cppunit
  • End-to-end testing is a bit all over the place and needs refactoring
mod_dup testing explained
⚠️ **GitHub.com Fallback** ⚠️