Input files - hillerlab/TOGA2 GitHub Wiki

Contents

Tip

  • For minimal required input, consult the Mandatory input section.
  • For programmatic production of reference input BED file and "semi-mandatory” files, see the prepare-input mode page.
  1. Mandatory input
    1. 2bit genome files
    2. Alignment chain files
      1. Aligning genomes with recommended pipeline
      2. Aligning genomes with custom pipelines
      3. Extracting alignment chains from HAL multiple genome alignment format
      4. Chain file sanity checklist
    3. Reference annotation BED file
      1. BED file sanity checklist
      2. Converting GTF/GFF files into BED12 format
  2. Semi-mandatory input
    1. Isoform table
      1. Isoform file sanity checklist
    2. U12 & non-canonical U2 introns file
    3. SpliceAI output directory
  3. Additional input
    1. CESAR2 profiles
    2. Transcript URL table

Mandatory input

The following file are required for TOGA2 to run.

2bit genome files

TOGA2 accepts reference and query genome sequences in .2bit format. FASTA format sequences can be converted into .2bit with UCSC faToTwoBit utility:

faToTwoBit in.fasta out.2bit

faToTwoBit, among other UCSC utilities, is downloaded to bin/ folder during TOGA2 installation. Compiled binary can be also downloaded from UCSC Download portal or installed via Conda.

Alignment chain files

Aligning genomes with recommended pipeline

TOGA was developed and tested with pairwise genome alignment data produced by the Hiller Lab LASTZ pipeline. The pipeline produces high-scoring, highly contiguous alignment chains ideal for TOGA annotation, and the list of software requirements (Python, Nextflow, UCSC utilities) overlaps significantly with that of TOGA, facilitating their joint installation and use.

Aligning genomes with custom pipelines

Note

To be expanded

Extracting alignment chains from HAL multiple genome alignment format

Note

To be expanded

Chain file sanity checklist

Below is a non-exhaustive list of potential quirks to check your chain files for. For general chain file format description, consult with the UCSC documentation.

  1. Chain entries must be separated by a blank line (see example in the UCSC documentation). This heuristic is used by chaintools for fast chain file parsing, with its violation not being anyhow mitigated in TOGA2. If your chain file does not contain blank line separators, insert them with the following or similar command:

    sed -i ’1p; /chain/s/^/\n/g’ in.chains
    

    While this requirement might be alleviated in future chaintools releases, we encourage the user to stick to the UCSC format to avoid potential problems.

  2. In the same vein, chaintools struggles with parsing metadata lines. This restriction might be alleviated in future chaintools releases as well, otherwise consider removing those lines from your chain file:

    sed -i ’/^#/d’ in.chains
    
  3. The minimal chain score considered by TOGA2 is regulated by parameters -mcs/--min_chain_score and -minscore/--min_orthologous_chain_score; chains with with score x < min_chain_score are filtered out at the initial TOGA step, and those with score x < min_orthologous_chain_score are not used for annotation of loci other than processed pseudogenes/retrogenes. By default, both parameters are set to 15.000, which filters out most of the spurious low-quality chain for vertebrate genome alignments. If your chains are inherently fragmented and low-scored, however, default filters might lead to important orthology data being lost. To check how chain alignment scores are distributed, extract them from the chain files:

    awk ‘($1 == “chain”){print $2}’ in.chain > scores.txt
    

    There is currently no best recipe for optimal cutoff estimation, but generally lowering the threshold below 1000 is not recommended.

Reference annotation BED file

Note

To be expanded

BED file sanity checklist

  1. TOGA2 uses certain special characters as delimiters in the output data, limiting the number of symbols valid for use in input transcripts’ names. The accepted symbols include Latin alphabet letters (upper- and lowercase), digits, dots (‘.’), hyphens (‘-‘), and sharps (‘#’). Transcripts containing any other symbols are discarded at the initial step of TOGA pipeline.
    • Despite being the primary field delimiter in query transcripts’ names, the sharp (‘#’) symbol is accepted in reference transcripts. Annotation files provided with TOGA2 release version contain reference gene names separated from transcript identifiers by sharp (${transcript_id}#${gene_id}) to facilitate results analysis and interpretation, and we highly recommend the users to format transcript names in the similar manner. Note that additional data contained in transcript names are not anyhow used by TOGA2, and gene names added to transcripts do not serve as a substitute for input isoform file.
  2. All transcripts are expected to have a coding sequence of at least one base and a complete reading frame (i.e., coding sequence length divisible by three).
  3. The prepare-input mode automatically tests the input annotation for the points listed above. If you use TOGA2 with a custom BED file input, running preparation pipeline on your input data beforehand is highly recommended.
  4. Alternatively, transcripts that do not comply with the requirements above are discarded at the setup step of the TOGA2 pipeline. For the purpose of gene loss analysis, these transcripts are assigned an N loss status. Reference genes which have all their isoforms discarded due to technical issues are also classified as N.

Converting GTF/GFF files into BED12 format

Note

To be expanded

Semi-mandatory input

The following files are not required for TOGA2 execution but provide the best annotation experience. Starting from version 2.0.7, the respective command line arguments are mandatory in TOGA2; if you do not have or do not want to provide any of these files, you have to set the respective placeholder flag.

Isoform table

A two-column, tab-separated file mapping reference transcripts to the respective reference genes. The file is expected to have no header and contain the following information:

  • the first column contains reference gene names;
  • the second column contains the reference transcripts’ names corresponding to the genes in the first column. One line contains information on one transcript’s mapping, making this file a narrow data format.

Below is an excerpt from the human hg38 assembly isoform file provided with TOGA2 (hg38.toga.isoforms.tsv, v2.0.0 onwards). Gene ENSG00000157388 has seven distinct isoforms, therefore appearing seven times in the isoform file, once for each isoform:

ENSG00000157388	ENST00000288139.11#CACNA1D
ENSG00000157388	ENST00000350061.11#CACNA1D
ENSG00000157388	ENST00000481478.2#CACNA1D
ENSG00000157388	ENST00000636938.1#CACNA1D
ENSG00000157388	ENST00000637424.1#CACNA1D
ENSG00000157388	NM_001128839.3#CACNA1D
ENSG00000157388	XM_005265448.4#CACNA1D

The isoform file is provided to TOGA2 run mode with the --isoform_file option. This option is considered semi-mandatory and is expected by default unless the --no_isoform_file flag is set.

Warning

If the isoform file is not provided, TOGA2 does not report the gene level loss statistics in the loss_summary.tsv file. For other applications where the reference gene composition is relevant, TOGA2 treats each reference transcript as a separate gene. In case of alternative splicing events, the independent isoform treatment might distort the orthology relationship statistics. For example, an otherwise 1:1 gene correspondence is classified as many:one if multiple isoforms of the reference gene map to the same locus in the query.
That being said, the only safe scenario for omitting the isoform file input is if your reference is known to have little to know alternative splicing. For all statistics involving reference genes that are not explicitly reported, use the transcript level statistics instead.

Isoform file sanity checklist

  • Do not include any header in this file. Since TOGA2 does not expect any header item when parsing this file, your column titles will appear in the orthology and loss summary files as Lost/Missing/Undefined items, distorting the overall gene loss and orthology statistics.
  • Genes must have unique names; obviously, two or more genes sharing the same name in your isoform file will be treated as a single gene for the purpose of orthology classification and gene loss assessment.
  • If you reuse the TOGA1 input files, do not use the legacy reg_{number} notation for query gene naming. The reg_ notation is used by TOGA2 for provisional gene naming in the intermediate output files; using the same notation for reference names may result in name collision and lead to compromised orthology classification results.

U12 & non-canonical U2 introns file

Note

To be expanded

SpliceAI output directory

A directory with SpliceAI splice site predictions for the query genome. These predictions are used at the alignment step for improved exon boundary annotation; if --intron_gain_check flag is set, they are also used for query-specific intron annotation. See Using SpliceAI predictions with TOGA2 for more details.

SpliceAI predictions are provided with the --spliceai_dir option. This is argument is semi-mandatory; if you do not want to use the SpliceAI-related features in your TOGA2 run, the --no_spliceai flag is expected. Starting from version v2.0.7, if --input_directory argument is used for “bundled” input specification, the SpliceAI directory is sought for in the query directory.

The directory is expected to store four BigWig files, one for each DNA strand and splice site combination:

  • spliceAiAcceptorMinus.bw
  • spliceAiAcceptorPlus.bw
  • spliceAiDonorMinus.bw
  • spliceAiDonorPlus.bw

All BigWig files are expected to have headers to count the start positions from. Each Wiggle block has both fixed step and span equal to 1. Each line between the headers corresponds to the respective genomic position and contains the corresponding splice site dinucleotide start probability.

HERE BE AN EXAMPLE

The SpliceAI directory with the necessary files can be generated with the TOGA2 spliceai mode. The resulting f See SpliceAI prediction generation with “spliceai" mode for annotation generation and file specification.

Additional input

These files are not necessary for TOGA2 execution or have stable defaults provided with TOGA2.

CESAR2 profiles

TOGA2 uses CESAR2 for exon alignment. Apart from the reference and query sequences, CESAR2 needs nucleotide frequency profiles for intronic sequences bordering the exon, including the splice sites. These profiles are presented as four-column tab-separated tables, with each column standing for each of the four major nucleotides, and each row corresponding to nucleotide position relative to the exon-intron boundary:

  • acceptor splice site profiles have 22 rows, containing frequencies for positions -22 to -1, where -1 is the last base of the upstream intron;
  • for donor splice sites, profile files contain six rows, bearing frequencies at positions 1 to 6, where 1 is the first base of the downstream intron.

In addition to separate acceptor and donor profiles, TOGA2 discriminates splice sites with regard to their intron class (U2 or U12) and canonicity (canonical for GT-AG splice site dinucleotide composition, non-canonical for the rest). Together, that gives 8 separate splice site profile files. In addition, special codon profiles are provided for the first exon’s 22 upstream intergenic bases (“acceptor”, 10 positions) and the last exon’s six downstream intergenic bases (“donor”, 6 positions).

Starting from version v2.0.8, all CESAR profiles except for first exon acceptor and last exon donor can be provided as a single directory with the --cesar_profile_dir option. The files are expected to be named according to their splice site, intron class, and “canonicity”, as follows:

  • canon_U2_acceptor.tsv
  • canon_U2_donor.tsv
  • canon_U12_acceptor.tsv
  • canon_U12_donor.tsv
  • nonCanon_U2_acceptor.tsv
  • nonCanon_U2_donor.tsv
  • nonCanon_U12_acceptor.tsv
  • nonCanon_U12_donor.tsv

All these profiles are generated in a single directory with prepare-input command (see here for more details). It also generates profiles with uniform nucleotide probabilities in all positions (equiprobable_acceptor.tsv and equiprobable_donor.tsv, respectively). These profiles can be used if certain intron classes are underrepresented in your reference data, resulting in suboptimal profile inference; for example, CESAR2 profiles supplied with TOGA2 use the equiprobable acceptor profile instead of nonCanon_U12_acceptor.tsv due to scarcity of respective splice sites in the human annotation used.

Note

As of now (version v2.0.8), special profiles for the first and last exons are not generated by prepare-input. These are supplied with TOGA2 under supply/CESAR2.0/profiles/firstCodon_profile.tsv and supply/CESAR2.0/profiles/lastCodon_profile.tsv.

Transcript URL table

Note

To be expanded

⚠️ **GitHub.com Fallback** ⚠️