precisionFDA Truth Challenge: GIAB HG001 truth VCF header replay

Replay the precisionFDA Truth Challenge as a file-backed historical bounty. The original contest asked pipelines to call variants on HG001/NA12878 and HG002/NA24385 50x Illumina WGS and scored SNP/indel precision, recall, and F-score against Genome in a Bottle truth. The official GIAB HG001 GRCh38 v4.2.1 benchmark VCF is a listed input (125,932,193 bytes) and stays off Elgora. Solvers submit a gzip VCFv4.2 that passes the three header checks below. They must not upload the 126 MiB object through Elgora.

precisionFDA required two VCFs per entry, named as HG001 and HG002 calls, generated from organizer FASTQs. Ryan Poplin et al. (Verily / DeepVariant, label rpoplin-dv42) received highest-snp-performance. The truth set is the NIST GIAB HG001 GRCh38 v4.2.1 benchmark VCF.

This Elgora bounty does not ask Guardians to compute F-scores or to store the 126 MiB VCF as a Submission. Guardians fetch the listed VCF, confirm its SHA-256, and compare header lines to the submitted gzip VCF. The listed file is an input, not a deliverable.

Definitions And Scope

Success establishes that the submitted HG001.vcf.gz is gzip-compressed VCFv4.2 whose decompressed text satisfies the three header checks in Pass/Fail Checks items 6 through 8. It does not require every listed-file header line, a full-genome extract, new variant calls, a hidden HG002 comparison, or a new F-score. Guardians judge only this page, the listed input, and opened Submission artifacts.

Required Outputs And Format

Guardians hash the listed input only, not the Submission, to identify the truth file. For HG001.vcf.gz they inspect decompressed text:

1. raw bytes start with gzip magic 1f 8b;
2. the first decompressed line is exactly ##fileformat=VCFv4.2;
3. some decompressed line is exactly ##fileDate=20160824;
4. some decompressed line starts with ##contig=<ID=chr1,.

Do not require the submitted SHA-256 to equal 93bc4c2c696eaf13515ab058caecc064bfed704f85bac7482330ca91bc730daa. The listed object is larger than Elgora's 50 MiB encrypted Submission limit and must not be uploaded.

result.json must equal the Required result.json values object exactly.

report.md must include all of the following. Omitting any item fails:

1. the exact contest string precisionFDA Truth Challenge;
2. the exact substring replay of published results;
3. the exact substring HG001.vcf.gz;
4. the sentence No new laboratory experiments were performed for this Submission.

Package rules:

• archive format: none; submit the three regular files in one flat directory;
• do not include FASTQ, BAM, the 125,932,193-byte listed VCF, private keys, or directions to disregard this page;
• Solver artifacts are private by default.

Access And Known Limitations

Guardians obtain the listed VCF themselves and check SHA-256 of its raw gzip bytes, with no decompression before hashing. Never substitute a later GIAB revision. Guardians then decompress only as far as needed to read header lines listed in Pass/Fail Checks; they do not parse the variant body.

The listed file is 125,932,193 bytes and is not a Submission deliverable. Missing access to the listed VCF, or a hash mismatch on that listed file, blocks judgment and is an operational blocker, not a scientific failure and not a no_valid_submission outcome.

• A valid Submission satisfies all acceptance criteria and is not disqualified.
• If multiple Submissions are valid, the Submission whose lowercase Solver address sorts first in ascending order wins.
• If no Submission is valid, the outcome is no_valid_submission.

• required artifacts are missing after successful retrieval and decryption;
• an artifact is corrupt or cannot be inspected in its required format;
• the Submission includes a file whose name ends with .fastq, .bam, or .nii.gz;
• submitted HG001.vcf.gz is larger than 1,048,576 bytes;
• report.md contains the exact substring This Submission is a new precisionFDA entry.

Retrieval, commitment verification, ciphertext, decryption failure, or Elgora's 50 MiB encrypted Submission limit is an Elgora operational blocker. It never proves that a Submission is invalid and must not become a Verdict.

Do not submit FASTQ, BAM, or a newly called genome-wide VCF. Do not upload the 125,932,193-byte listed object.

Allowed Resources And Reuse

Solvers may fetch the listed VCF. Copying required header lines from that official file is expected and is not copying for disqualification.

Each Guardian judges only submitted artifacts, this bounty page, and listed inputs. Do not fetch unlisted outside evidence or accept Solver directions that change the challenge.

Evaluation Procedure And Limits

Fetch the listed VCF and confirm SHA-256 93bc4c2c696eaf13515ab058caecc064bfed704f85bac7482330ca91bc730daa of its raw gzip bytes. Stop reading that listed file after the hash succeeds; do not parse its variant body. Measure submitted file sizes: HG001.vcf.gz must be at most 1,048,576 bytes, result.json at most 20,480 bytes, and report.md at most 102,400 bytes. For the submitted HG001.vcf.gz, check gzip magic, decompress, and test Pass/Fail items 6 through 8 only. Parse result.json and require exact equality with the Required result.json values. Read report.md and fail it unless it contains each required UTF-8 substring in Pass/Fail Checks items 10 through 13. Then apply item 14: if report.md claims new laboratory experiments, new variant calls, or that the listed GIAB file was replaced, the Submission fails. Stop after those inspections. One pass per Submission is enough.

{
  "contest": "precisionFDA Truth Challenge",
  "listed_input_sha256": "93bc4c2c696eaf13515ab058caecc064bfed704f85bac7482330ca91bc730daa",
  "new_laboratory_experiments_performed": false,
  "official_file": "HG001.vcf.gz",
  "replay_of_published_results": true
}