7. Known issues, contribute - GDelevoye/cigarfilter GitHub Wiki

Known issues, contribute

Known issues

The installation is too slow, what can I do ?

Installation can sometimes be slow due to conda-forge. See the detailed explanations for more info. Workardounds :

  1. Don't install the software in an already existing environment !
  2. Use mamba if conda is really slow. See https://github.com/conda/conda/issues/7239 for more info .
  3. Delete conda-forge and conda-metachannel from your channels
  4. Use a strict channel priority using conda config --set channel_priority false
  5. Update conda through conda update -n base conda (Tested with conda 4.10.1)
  6. In your conda configuration, set the "defaults" conda channel at highest priority, and remove all the others

Contribute

PRs are explicitely welcome.

Unexhaustive wishlist of features

Here are a few ideas of how the software can be improved in the future :

  • Faster CI/CD
  • Add CI/CD for other compilers/platforms (in particular, Mac and Windows)

There is no reason to think that cigarfilter does not work there, but it has not been tested

  • More test cases for the CLI
  • cigarfilter_config can be re-implemented with a more maintainable argparse command subparser
  • More tests for cigarfilter_config
  • More examples
  • Doxygen documentation integrated as a Github page
  • Benchmarks

My own benchmarks show about 20x speed increase compared to python, but I guess it greatly depends on the hard drive.

  • Add the possibility to redirect the reads that pass to one file (or pipe), and the reads that don't pass to another file or pipe
  • Add a cxx_compiler package in the conda-build process, so that the user does not require a C++17 compiler

As of Nov. 2022, this breaks the pip install, which is why I preferred to rely on people having a C++17 compiler already installed - as should be the case for most people doing C++ in 2022 anyway

  • Optimization of any sort would always be welcome
  • Safer cigarfilter_config

As of Nov. 2022, many instances of cigarfilter can be launched safely at the same time. But only one instance of cigarfilter_config can be launched safely at the same time, and using cigarfilter while cigarfilter_config is at work a potentially problematic.

  • Binary distribution

Right now, the program is only available from source. Building the package from source ok for mainstream platforms. Sometimes however, the binaries are not available online (e.g boost libraries searched by conan) and therefore the compilation time can be prohibitive.

If you have another proposal, please feel free to open a PR.

Design principles

The design principles are rather simple :

  1. Dependency injection

By compiling plugins rather than one executable for each filter, we allow the user to install the software only once. This is a feature. At the moment, boost is used for that purpose, as it allows to load dynamic libraries in a rather cross-platform manner. If this dependency can be replaced by any standard solution, please let me know.

  1. Simplicity (black box), cross-platform

The cigarfilter_config utility works as a black box that handles the compilation toolchain and the plugins so that the users can focus on their job : science.

The tool must avoid any unnecessary hassle for the user (if it does not, then I failed). It should be deployable easily in +/- any new server/computer just by copy-pasting the code of a filter, and recompile it with cigarfilter_config. Bioinformaticians may indeed run the code on various computation platforms.

Build manually from source

cigarfilter is built with a CMake / conan toolchain. Here is an example of how it can be done in Linux :

git clone https://github.com/GDelevoye/bamfilter/ &&
pip install conan &&
conan remove "*" -s -b -f &&
conan config set general.revisions_enabled=1 &&
conan profile new cigarfilter --detect --force && 
conan profile update settings.compiler.libcxx=libstdc++11 cigarfilter && 
cd bamfilter &&
mkdir -p build && cd build &&
conan install .. --build=missing --profile cigarfilter&&
cmake .. -G "Unix Makefiles" &&
make &&
echo "DONE"

In practice, these steps are performed using conda-build (see "Installation") instead of using platform-dependent scripts.

Possible compilation troubles

In recent Linux systems, the building should be done without problems.

On older distributions however, some issues can occur regarding the C++17 standard.

Also, I have been aware during my own tests that there are sometimes issues (which I do not well understand yet) with some combinations of conan and CLang in MacOS.

In case of troubles :

  • Check your ABI. It must be C++11 compatible
  • In ~/.conan/conan.conf you can also :
    • Disable the lock in the cache (by editing ~/.conan/conan.conf) (uncomment cache_no_locks = True)
    • Overright the path to cmake or compiler, to ones that are compliant with C++17
  • Set your compiler version in your conan profile : conan profile update 'settings.compiler.version=yourversion' default
  • Clear the cache : conan remove '*' -f 
  • If your system has several compiler installed, perhaps the default is not C++17 compliant. Export the right environment vaiable to point to the right one could fix the problem too. Exemple
> export CC="gcc-10"
> export CXX="g++-10"