Running the Tests - AmrAdelKhalil/mongo-connector GitHub Wiki

This page describes how to run the tests under supported versions of Python.

Requirements

  1. Copy of the Mongo Connector Github repository

    The tests are not included in the package from PyPI and can only be acquired by cloning this repository on Github:

     git clone https://github.com/10gen-labs/mongo-connector.git

  2. Mongo Orchestration

    Mongo Connector runs MongoDB on its own using another tool called Mongo Orchestration. This package should install automatically if you run python setup.py test, but the Mongo Orchestration server still needs to be started manually before running the tests:

     mongo-orchestration start

    will start the server. To stop it:

     mongo-orchestration stop

  3. Environment variables

    There are a few influential environment variables that affect the tests. These are:

    • DB_USER is the username to use if running the tests with authentication enabled.
    • DB_PASSWORD is the password for the above.
    • MONGO_PORT is the starting port for running MongoDB. Future nodes will be started on sequentially increasing ports.
    • ES_HOST is the hostname on which Elasticsearch is running (for mongo-connector < 2.3).
    • ES_PORT is the port on which Elasticsearch is running (for mongo-connector < 2.3).
    • SOLR_HOST is the hostname on which Solr is running.
    • SOLR_PORT is the port on which Solr is running.
    • MO_ADDRESS is the address to use for Mongo Orchestration (i.e., hostname:port)

Test suite structure

All the tests live in the tests directory. Here is an overview of what's in each test module:

  • setup_cluster.py does not contain any tests, but contains tools for setting up and tearing down the MongoDB clusters used in the tests.
  • test_command_replication.py tests how commands are passed and processed before being sent to a DocManager.
  • test_config.py tests the configuration file / command-line option parser.
  • test_elastic.py tests mongo-connector running against ElasticSearch (for mongo-connector < 2.3)
  • test_elastic_doc_manager.py contains unit tests for the Elastic DocManager (for mongo-connector < 2.3)
  • test_formatters.py tests the Formatter classes, which can restructure MongoDB documents.
  • test_mongo.py tests mongo-connector running against MongoDB (as a destination system)
  • test_mongo_connector.py tests the front-end to mongo-connector (e.g., oplog progress file, command-line parsing, etc.)
  • test_mongo_doc_manager.py contains unit tests for the Mongo DocManager
  • test_oplog_manager.py contains unit tests for the OplogThread class, which is responsible for tailing the MongoDB oplog
  • test_oplog_manager_sharded.py contains the same tests as above, but running against a sharded environment
  • test_rollbacks.py tests OplogThread under various rollback conditions
  • test_solr.py tests mongo-connector running against Solr
  • test_solr_doc_manager.py contains unit tests for the Solr DocManager
  • test_synchronizer.py also tests OplogThread running with a dummy DocManager
  • test_util.py contains unit tests for mongo-connector utilities

Running tests on the command-line

While the tests take care of setting up and tearing down MongoDB clusters on their own, the Solr and ElasticSearch tests require Solr and ElasticSearch to be running, respectively. Make sure to start these before doing a full test run!

You can run all the tests with one command (this works in all supported Python versions):

python setup.py test

In addition, you can be more selective with which tests you run (in Python > 2.6 only)! For example, if you only wanted to run the rollback tests:

python -m unittest tests.test_rollbacks

Or if you only wanted to test the remove method of the Solr DocManager:

python -m unittest tests.test_solr_doc_manager.TestSolrDocManager.test_remove

Error messages

Some of the tests are meant to generate lots of ERROR-level log messages, especially the rollback tests. mongo-connector logs exceptions it encounters while iterating the cursor in the oplog, so we see these in the console output while MongoDB clusters are being torn apart in the tests. As long as all the tests pass with an OK message, all is well.