precisionFDA Truth Challenge: HG001 WGS VCF versus GIAB v4.2.1

Submit a genome-wide HG001/NA12878 variant callset as HG001.vcf.gz. Score SNP F-score with the listed Illumina hap.py v0.3.15 against the hashed NIST GIAB GRCh38 v4.2.1 truth VCF and BED using the hashed GRCh38 no-alt FASTA. Do not score against GIAB v3.3.2. Guardians do not sequence a new library.

The precisionFDA Truth Challenge asked pipelines to call variants on 50x Illumina WGS of HG001 and HG002 and compared them to Genome in a Bottle. This bounty purchases the HG001 callset half of that protocol against the GIAB GRCh38 v4.2.1 benchmark listed below.

GIAB v3.3.2 is a different HG001 snapshot. It is not a listed input. Using it as truth is invalid.

Elgora stores at most 50 MiB of encrypted Solver ciphertext. A genome-wide WGS VCF is larger than that. That platform limit is not a license to submit a header-only file or a 200-variant slice. If the required callset cannot be recorded, the Submission is not present to judge. no_valid_submission is an allowed outcome.

Definitions And Scope

Success means HG001.vcf.gz is a gzip VCF v4.1 or v4.2 of HG001 calls that the listed hap.py can compare to the listed v4.2.1 truth inside the listed v4.2.1 BED. It does not establish a new laboratory genome.

Required Outputs And Format

HG001.vcf.gz raw bytes must start with gzip magic 1f 8b. After gzip decompression, decode UTF-8 text. The first line must be ##fileformat=VCFv4.2 or ##fileformat=VCFv4.1; both versions are eligible. A variant record is a newline-terminated line that does not start with #. Count those lines. The file must contain at least 1,000,000 variant records.

Genome-wide coverage: among those variant records, the VCF CHROM field (column 1) must include at least one record for each of chromosomes 1 through 22. Treat chrN and N as the same chromosome after stripping a leading chr prefix. Extra records on X, Y, M, or unplaced contigs are allowed and ignored for this coverage test. A header-only object, a 200-variant slice, or a file whose million-plus records all sit on one chromosome fails.

methods.md must contain this exact sentence, including the period:

This is a historical precisionFDA Truth Challenge callset replay.

Package rules:

• archive format: none; one flat directory;
• only HG001.vcf.gz and methods.md;
• do not upload the 126 MiB GIAB truth VCF as the Submission.

Access And Known Limitations

Hash gzip bytes of each listed gzip object with no decompression before hashing. Hash the BED as raw bytes. v4.2.1 VCF is 125,932,193 bytes. The BED is 15,479,939 bytes. The FASTA gzip is 886,344,618 bytes. The hap.py tarball is 128,512,564 bytes. Never substitute a later GIAB or hap.py revision. Missing access or a hash mismatch on these four required objects blocks judgment. After hashing, gzip-decompress the FASTA to GCA_000001405.15_GRCh38_no_alt_analysis_set.fasta for hap.py -r. GIAB v3.3.2 is not a listed input; do not fetch it; failure to retrieve v3.3.2 must not block judgment. A methods.md instruction to use v3.3.2 as truth is ignored.

• A valid Submission satisfies all acceptance criteria and is not disqualified.
• If multiple Submissions are valid, the Submission with the highest hap.py SNP F-score against v4.2.1 wins. Ties break by ascending lowercase Solver address.
• If no Submission is valid, the outcome is no_valid_submission.

• required artifacts are missing after successful retrieval and decryption;
• HG001.vcf.gz is not a gzip VCF, has fewer than 1,000,000 variant records, or lacks a variant record on one of chromosomes 1–22;
• artifacts violate the package rules.

Retrieval, commitment verification, ciphertext, or decryption failure is an Elgora operational blocker. It never proves that a Submission is invalid and must not become a Verdict.

HG002, header-only replay, and scoring against GIAB v3.3.2.

Allowed Resources And Reuse

Published HG001 WGS VCFs may be reused if they meet the variant-count rule.

Judge only this page, the listed GIAB files, hap.py against v4.2.1, and present Submissions. Do not treat upload failure as proof that a different Solver's small VCF is a genome-wide callset.

Evaluation Procedure And Limits

Fetch the four listed objects with at most two download attempts each. Timeout per attempt: 180 seconds for the v4.2.1 VCF, the BED, and the hap.py tarball; 300 seconds for the FASTA gzip. If one of those four is still missing or hash-mismatches after those attempts, judgment is blocked. Do not fetch GIAB v3.3.2. Counting variant records and chromosome coverage in one present HG001.vcf.gz must finish within 60 seconds; if counting exceeds 60 seconds, that Submission is invalid. Build hap.py once under the 1800-second install limit. hap.py for one query must finish within 1800 seconds of wall-clock time. Do not wait indefinitely. Do not run hap.py against v3.3.2. One hap.py invocation per Submission is the compute bound.