Architecture: other projects - metaspace2020/metaspace GitHub Wiki

Python Client

To publish a new version, increment the version number in setup.py and use the GitHub Action to publish it to PyPI. It's possible to publish a new version from a branch.

Documentation at ReadTheDocs should update automatically when changes are merged into master, however it's sometimes necessary to manually update the documentation source/template files to reorder important methods or hide unimportant/deprecated methods.

Remember that users usually don't update until a problem forces them to, and it's often very hard for them to find the cause if the API changes between versions. Both backwards and forwards compatibility should be considered with every change!

There are test files for manual tests, however these make some mutations and some tests require data to manually be set up, so they should only be run against your local environment. Before running them, create a test_config file in the python-client directory and populate it with:

host=http://localhost:8888  # Update this if you use a different port
api_key=AAAAAAAAAAAA  # Generate a real API key from http://localhost:8888/user/me

Ansible

See the docs

We use a specific version of Ansible (2.8.6) because a lot of things break if it's updated. It's probably not worth trying to update it.

Lachlan Stuart's msi_recal recalibration/alignment tool

https://github.com/metaspace2020/metaspace/tree/master/metaspace/recal

It was developed as with the intent of improving datasets while requiring virtually no extra tuning (only a bit of metadata). It's reasonably good at automatically fixing miscalibrated datasets, but it wasn't able to reliably improve alignment. Ultimately it wasn't chosen to be included in METASPACE as we couldn't demonstrate an improvement in METASPACE annotations after recalibration.

It uses built-in databases such as CoreMetabolome-v3 and a list of matrix peaks as calibration points. The databases can either be selected manually, or auto-detected based on the provided metadata (polarity, matrix). More info in the --help.

Includes multiple different recalibration and alignment algorithms, both of which has an option based on MSIWarp.

New PyPI versions are published via GitHub Actions. Note that it compiles MSIWarp into the package so that all platforms & python versions can be supported, but sometimes this requires hacky fixes when GitHub Actions' C++ compiler version changes.

Andrew Palmer's recalibration project

https://github.com/alexandrovteam/recalibration

pyimzML

https://github.com/alexandrovteam/pyimzML/

There are pytest tests.

Remember to update the CHANGELOG.md! Many external users depend on this library.

New versions should be published by bumping the version number in pyimzml/__init__.py and manually running the GitHub Actions workflow.

Documentation is hosted at ReadTheDocs. It should update automatically when master is updated.

(c)pyMSpec, (c)pyImagingMSpec

https://github.com/alexandrovteam/

As far as I'm aware these are only used in METASPACE. We haven't needed to update them in a long time. Building & publishing is all manual.

METASPACE uses two different versions of cpyMSpec: 0.3.5 (default - when a dataset config has analysis_version == 1) and 0.4.2 (when analysis_version >= 2). We retained the old version to ensure equivalence of annotated results, as the two versions have small numeric differences in centroid calculation due to different algorithms. However, 0.4.2 can calculate centroids ~2x faster than 0.3.5.