Introduction

Hey so it's a meta-analysis and a big one at that! which means I have a whole load of random ass data across a variety of primers and insturments. Thats a problem when we want to compare taxonomy. Before this you successfully converted everything to an ASV, great! oh yknow I'll finish this explination later lets just into it (he never came back to finish the explination)

Getting MaLiAmPi up and running

MaLiAmPi has like negative 1 documentation

thanks

We want to enter with an ASV table; however, maliampi only wants an entry point with fastq files or phylotypes. Okay how do we get phylotypes? we use the phylotypes code written by the same people as maliampi. Cool what do we need for phylotypes? we need a jplace file. what is a jplace file? Its a json file that stores genetic sequences fit onto a reference evollutionary tree. OKAY how do we get one of these? We run phylogenetic placement by our lonesome with epa-ng (default package for maliampi) scratch that we'll use DEPP it's faster than EPA-ng and explicitly used by the greengenes database. God forbid any of this easy. Scratch that again. the backbone of greengenes2 is only in a sepp format. Lets see how that plays out.

Greengenes2

This is a minor comment. Greengene2 has two releases so far. a 2024.09 and a 2022.10 release. If you read the changelog of the greengenes2 2024.10 relase it says

This release does not include new backbone or modified topology.
Approximately 2.3M additional V4 ASV fragments, previously placed with
DEPP, where included which were not part of the 2022.10 release.

This means THAT from our perspective on aligning taxonomy the 2022.10 release is just as valid as the 2024.09 release since the backbone does not change (uhh you align your sequences to the backbone). Okay cool. Regardless of which algorithmic placer we use, we'll need a file with all the sequences in it. One with all the taxonomy, and one with the backbone.
You can find the the greengenes (2) releases here:\

https://ftp.microbio.me/greengenes_release/

The backbone is this file and is described thusly:

2022.10.backbone.sepp-reference.qza
A SEPP compatible reference for 16S rRNA fragment insertion.
Unzip the stupid qiime2 artifact, it also contains the aligned sequences.\

RAxML-NG

Okay, to place our sequences with BSCAMPP, we need to estimate the model parameters of the phylogenetic tree. By model parameters i mean like substitution rates and transversions ect. We'll use RAxML-NG bc that is the recommendation from BSCAMPP. \ Just go download the binaries and read the installer to get it running.
Now we still have a shit tone of sequences, so we're gonna randomly subsample 30k sequences otherwise it'll never run within a reasonable amount of time. (PS. at 30k sequences we're already looking at a runtime around 24 CPU days haha)

I will be using seqkit

conda install -c bioconda seqkit

seqkit sample -n 30000 greengenes2_MSA.fasta > greengenes2_MSA_subsam30k.fasta

Then run RAxML-ng to determine the best model

./raxml-ng --msa greengenes2_MSA_subsam30k.fasta --model DNA --threads auto{30}

UPP2

Okay! now we're onto something. You've successfully downloaded the aligned sequences and the tree. Now you need to use UPP2 to align your sequences onto the larger alignment of sequences from the tree. We're using UPP2 to do this bc cDEPP used UPP and UPP2 is more computationally efficient or something.

https://github.com/gillichu/sepp

NOTE: This is a fork of the main sepp package and it is out of date excluding for UPP2 (presumably) so uhhh just dont use this repo to do other things too.\

UPP2

BSCAMPP

Okay, with a model from RAxML-NG and an alignment from UPP2, we can finally run phylogenetic placement.

BSCAMPP

MaLiAmPi

Downloading nextflow.io

Okay MaLiAmPi (this is the last time i will properly capitalize it) needs a couple things. Java 17, docker and nextflow

Nextflow can be installed simply

cd Nextflow
curl -s https://get.nextflow.io | bash
./nextflow run hello

You'll also want to make Nextflow an executable path

export PATH="$PATH:~/home/coder/project/R/nonR_libraries/Nextflow"

Did it work?

./nextflow info

you'll also need docker

DEPP DNU

I'm leaving this section for posterity, but I couldnt get DEPP to work, it required too much >200 gb of ram at model training, the documentation was poor and no one was answering the git issues.

Okay, For this to work we need DEPP to do a couple things. Enough things that it gets it's own wiki page. DEPP

In the cDEPP paper, there is this quote "Multiple models are trained for different regions as evidence shows that the performance of DEPP would degrade if used with highly gappy aligned queries". GREAT. We have ASV's from all over the 16S region. In effect we will need a couple different models. In the Greengenes2 database, they only provide the alignment for the V4 region. At the very least we need to be able to get the V1-3 and V6-8 regions.\

Greengenes2 describes this as "In silico extracted V4 sequences from the backbone based on the EMP 16S primers". But doesn't exactly say how they did it. I did email asking for exactly how this did this to be the most comparable. But at the very least qiime2 has some built in methods. Lets a gooo download qiime2.\

Lets get this going! In silico extraction of V regions from greengenes2