Humans vs IA - AreliDGS/EPP622-2026-SNPCalling GitHub Wiki

Humans vs IA

Model used: Claude Sonnet 5, developed by Anthropic.

Date generated: September 10, 2026

Exact prompt:

Act as an expert bioinformatician. Write a complete, step-by-step short-read SNP calling pipeline for a haploid fungal genome. The pipeline must start from paired-end FASTQ files and end at a filtered VCF file.

Please write the entire response, including all explanations and code comments.
Incorporate industry-standard tools and provide the exact command-line syntax for each of the following steps:
1. Initial read quality control.
2. Read trimming and quality filtering.
3. Reference genome indexing.
4. Read alignment/mapping against the reference genome.
5. Format conversion, sorting, and indexing of alignment files.
6. Marking duplicates.
7. Variant calling specifically optimized for a HAPLOID organism.
8. Strict VCF filtering to retain high-confidence SNPs.

Provide the complete pipeline structured as a clean, end-to-end Bash script with brief comments explaining each step.

Additionally, at the very beginning of your response, please include a brief technical note identifying the exact architecture or version of the model you are (e.g., your specific parameter size, training cutoff context, or release version if applicable) to help me document the model details for my records.

My findings:

1. Three things it got right.

The AI-generated code a workflow structure for high-confidence variant discovery, executing three critical practices correctly. First, it precisely implements the --sample-ploidy 1 flag in both GATK4 HaplotypeCaller and GenotypeGVCFs. This is biologically mandatory for true haploid fungal isolates, ensuring the model does not attempt to calculate spurious heterozygosity from sequencing noise. Second, it successfully choose a tool for each step my prompt request to search the newest and common used at industry-standard benchmark, I can see that integrates GATK4 MarkDuplicates into the post-alignment cleanup stage. Flagging PCR and optical duplicates and avoid artificially inflated by clonal amplification artifacts. Third, it establishes an explicit, rigorous string-filtering using variantfiltration as I did on this pipeline.

2. Three things you don't understand and would want to learn more about before using. Or do you understand all of it? If so be ready to answer questions about it.

As I haven't use the GATK4 tools, I would say that this is the part that I not clearly understand. While the full operational execution of the GATK4 pipeline is clear, the parameters for deeper statistical exploration represent an area of my interest. Specifically, the exact mathematical trade-off behind the StrandOddsRatio (SOR > 3.0) and FisherStrand (FS > 60.0). Both metrics evaluate strand bias—detecting if a variant appears preferentially on forward or reverse reads—but they utilize distinct statistical distributions. Understanding how SOR acts as an odds ratio to minimize penalties in high-coverage or highly asymmetrical regions compared to the traditional Fisher's Test would provide valuable insight into how this pipeline outperforms standard positional alignment approaches in highly polymorphic structural regions. Additionally, learning about the preferable values for the parameters given the data or the objectives is one of the more useful things to learn when I'm using a new tool.

3. How specifically would you assess the results of this pipeline for accuracy?

To evaluate the accuracy of this AI-generated pipeline, the final output file must undergo a strict mathematical validation benchmark. The most robust strategy is to implement an automated positional cross-comparison script, mirroring our class infrastructure with evaluate_snps_v4.py, against an empirically validated golden standard Ground Truth VCF. This process counts the physical overlaps to compute true validation matrices: TP for successfully identified polymorphisms, FP for filtered sequencing noise, and FN for real variants missed by the tool. As evaluate_snps_v4.py did, by converting these counts into values of precision, recall, and F1 values, we can objectively score the pipeline's performance. Additionally, this quantitative audit should be paired with local visual validation using JBrowse 2. Streaming the final alignment tracks allows a direct review of the read evidence at discordant coordinates, visually confirming whether lingering false flags are caused by mapping errors or coverage starvation.

4. What did the labs teach you that turned out to matter for evaluating the AI-generated code?

Our hands-on lab pipeline provided the foundational baseline of the necessary steps to process, clean, and filter a dataset, allowing to evaluate if the AI-generated code go through the same process. Most notably, our experience with genomic ploidy and cohort design. The AI pipeline forces a haploid assumption (ploidy 1), whereas our class labs utilized a flexible diploid framework to successfully capture complex variations. This lab allowed me to immediately recognize that if the AI script were blindly executed on a mixed or diploid dataset like ours, it would cause severe data corruption, erasing true heterozygous variants if someone run this unaware of the implication given the dataset. Additionally, our labs highlighted the efficiency of modern, all-in-one processing tools like fastp for rapid adapter and poly-G tail removal, enabling me to identify that the AI’s choice of Trimmomatic introduces more computational work, and in my point of view this increases the chances of making mistakes.

5. What is the value of learning about a computational genomics task before using AI for it — or is there none?

In my point of view, without a baseline understanding of alignment mechanics, quality metrics, and variant calling models, a researcher becomes entirely dependent on the AI's output, rendering them unable to identify errors or missingness of assumptions. AI models operate on pattern matching and frequently produce code that is biologically or contextually invalid for a specific organism—such as proposing haploid configurations for a diploid dataset. Additionally, I would say that a prior knowledge transforms the researcher from a passive user into a critical supervisor of what is instructed to the AI to build. More than ever I have noticed that someone who uses AI is not able to create a reproducible work, something that is very important in the professional practice. Probably I would use this pipeline in my future projects.