Step 1: Create QIIME2 manifest file - shenjean/diversity GitHub Wiki

QIIME2 requires absolute file paths for its manifest file and will throw an error if the file paths are not absolute.

Check if QIIME2 has been loaded correctly

qiime --help

Manifest file format

The manifest file looks like this:

sample-id	forward-absolute-filepath	reverse-absolute-filepath
BCB11H	$PWD/BCB11H_S173_L001_R1_001.fastq.gz	$PWD/BCB11H_S173_L001_R2_001.fastq.gz
BCB11T	$PWD/BCB11T_S175_L001_R1_001.fastq.gz	$PWD/BCB11T_S175_L001_R2_001.fastq.gz
BCB16HL	$PWD/BCB16HL_S182_L001_R1_001.fastq.gz	$PWD/BCB16HL_S182_L001_R2_001.fastq.gz
BCB16SL	$PWD/BCB16SL_S180_L001_R1_001.fastq.gz	$PWD/BCB16SL_S180_L001_R2_001.fastq.gz
BCB16TL	$PWD/BCB16TL_S181_L001_R1_001.fastq.gz	$PWD/BCB16TL_S181_L001_R2_001.fastq.gz
BCB17HL	$PWD/BCB17HL_S221_L001_R1_001.fastq.gz	$PWD/BCB17HL_S221_L001_R2_001.fastq.gz
BCB17SL	$PWD/BCB17SL_S184_L001_R1_001.fastq.gz	$PWD/BCB17SL_S184_L001_R2_001.fastq.gz
BCB17SR	$PWD/BCB17SR_S183_L001_R1_001.fastq.gz	$PWD/BCB17SR_S183_L001_R2_001.fastq.gz
BCB17TL	$PWD/BCB17TL_S187_L001_R1_001.fastq.gz	$PWD/BCB17TL_S187_L001_R2_001.fastq.gz
BCB17TR	$PWD/BCB17TR_S186_L001_R1_001.fastq.gz	$PWD/BCB17TR_S186_L001_R2_001.fastq.gz
BCB9HR	$PWD/BCB9HR_S167_L001_R1_001.fastq.gz	$PWD/BCB9HR_S167_L001_R2_001.fastq.gz
BCB9TL	$PWD/BCB9TL_S169_L001_R1_001.fastq.gz	$PWD/BCB9TL_S169_L001_R2_001.fastq.gz

Generating the manifest file

First, go to the BocaCiegaBay folder by typing cd BocaCiegaBay.

Then, extract the sampleIDs from file names ending with "R1_001.fastq.gz" using the cut command. Here, the delimiter is set to "_" and we extract the first column (e.g. BCB16HL from file: BCB16HL_S182_L001_R1_001.fastq.gz)

ls *R1_001.fastq.gz | cut -d "_" -f1 >SampleID

Extract list of files with file names ending with "R1_001.fastq.gz", then replace the beginning of each file name (^) with "$PWD/" to specify the file path. Note: "$" and "/" are special characters that need to be escaped using the reverse slash "".

ls *R1_001.fastq.gz | sed "s/^/\$PWD\//" >R1

Repeat with file names ending with "R2_001.fastq.gz"

ls *R2_001.fastq.gz | sed "s/^/\$PWD\//" >R2

Remove any intermediate files, if necessary, using the rm command. For example: rm R1 R2

Check whether the output files, sampleID, R1, and R2 have the same number of lines:

wc -l sampleID R1 R2

Output should look like this:

12 sampleID
  12 R1
  12 R2
  36 total

Check contents of files, sampleID, R1, and R2 using the more command:

more sampleID

Output should look like this:

BCB11H
BCB11T
BCB16HL
BCB16SL
BCB16TL
BCB17HL
BCB17SL
BCB17SR
BCB17TL
BCB17TR
BCB9HR
BCB9TL

View the contents of R1

more R1

Output:

$PWD/BCB11H_S173_L001_R1_001.fastq.gz
$PWD/BCB11T_S175_L001_R1_001.fastq.gz
$PWD/BCB16HL_S182_L001_R1_001.fastq.gz
$PWD/BCB16SL_S180_L001_R1_001.fastq.gz
$PWD/BCB16TL_S181_L001_R1_001.fastq.gz
$PWD/BCB17HL_S221_L001_R1_001.fastq.gz
$PWD/BCB17SL_S184_L001_R1_001.fastq.gz
$PWD/BCB17SR_S183_L001_R1_001.fastq.gz
$PWD/BCB17TL_S187_L001_R1_001.fastq.gz
$PWD/BCB17TR_S186_L001_R1_001.fastq.gz
$PWD/BCB9HR_S167_L001_R1_001.fastq.gz
$PWD/BCB9TL_S169_L001_R1_001.fastq.gz

more R2

Output:

$PWD/BCB11H_S173_L001_R2_001.fastq.gz
$PWD/BCB11T_S175_L001_R2_001.fastq.gz
$PWD/BCB16HL_S182_L001_R2_001.fastq.gz
$PWD/BCB16SL_S180_L001_R2_001.fastq.gz
$PWD/BCB16TL_S181_L001_R2_001.fastq.gz
$PWD/BCB17HL_S221_L001_R2_001.fastq.gz
$PWD/BCB17SL_S184_L001_R2_001.fastq.gz
$PWD/BCB17SR_S183_L001_R2_001.fastq.gz
$PWD/BCB17TL_S187_L001_R2_001.fastq.gz
$PWD/BCB17TR_S186_L001_R2_001.fastq.gz
$PWD/BCB9HR_S167_L001_R2_001.fastq.gz
$PWD/BCB9TL_S169_L001_R2_001.fastq.gz

Combine SampleID, R1_fixed, R2_fixed using the paste command:

paste -d "\t" SampleID R1 R2 >manifest

Edit the manifest file using nano manifest and add the following headers separated by the Tab key. Check for spelling/typo errors:

sample-id	forward-absolute-filepath	reverse-absolute-filepath

To save your file in nano, type Ctrl+O. To exit nano, type Ctrl+X.

Check whether each column in the manifest file is formatted correctly. View the contents of column 1:

cut -f1 manifest

Output should be:

sample-id
BCB11H
BCB11T
BCB16HL
BCB16SL
BCB16TL
BCB17HL
BCB17SL
BCB17SR
BCB17TL
BCB17TR
BCB9HR
BCB9TL

View the contents of column 2 in the manifest file:

cut -f2 manifest

Expected output:

forward-absolute-filepath
$PWD/BCB11H_S173_L001_R1_001.fastq.gz
$PWD/BCB11T_S175_L001_R1_001.fastq.gz
$PWD/BCB16HL_S182_L001_R1_001.fastq.gz
$PWD/BCB16SL_S180_L001_R1_001.fastq.gz
$PWD/BCB16TL_S181_L001_R1_001.fastq.gz
$PWD/BCB17HL_S221_L001_R1_001.fastq.gz
$PWD/BCB17SL_S184_L001_R1_001.fastq.gz
$PWD/BCB17SR_S183_L001_R1_001.fastq.gz
$PWD/BCB17TL_S187_L001_R1_001.fastq.gz
$PWD/BCB17TR_S186_L001_R1_001.fastq.gz
$PWD/BCB9HR_S167_L001_R1_001.fastq.gz
$PWD/BCB9TL_S169_L001_R1_001.fastq.gz

View the contents of column 3 in the manifest file:

cut -f3 manifest

Expected output:

reverse-absolute-filepath
$PWD/BCB11H_S173_L001_R2_001.fastq.gz
$PWD/BCB11T_S175_L001_R2_001.fastq.gz
$PWD/BCB16HL_S182_L001_R2_001.fastq.gz
$PWD/BCB16SL_S180_L001_R2_001.fastq.gz
$PWD/BCB16TL_S181_L001_R2_001.fastq.gz
$PWD/BCB17HL_S221_L001_R2_001.fastq.gz
$PWD/BCB17SL_S184_L001_R2_001.fastq.gz
$PWD/BCB17SR_S183_L001_R2_001.fastq.gz
$PWD/BCB17TL_S187_L001_R2_001.fastq.gz
$PWD/BCB17TR_S186_L001_R2_001.fastq.gz
$PWD/BCB9HR_S167_L001_R2_001.fastq.gz
$PWD/BCB9TL_S169_L001_R2_001.fastq.gz