precisionFDA Truth Challenge: reconstruct the official variant-calling awards

Replay the published precisionFDA Truth Challenge awards. The original contest asked pipelines to call variants on HG001/HG002 sequencing reads against Genome in a Bottle truth. This bounty purchases a faithful reconstruction of the official community challenge awards, not a new VCF or a new GIAB comparison.

The precisionFDA Truth Challenge gave FASTQ reads for HG001 (truth available during the contest) and HG002 (truth withheld until close). Entrants tuned pipelines on the known sample, then submitted variant calls for the unknown sample. After close, NIST Genome in a Bottle released HG002 truth and precisionFDA published community awards for SNP and indel performance, recall, and precision.

This bounty does not re-run hap.py or require FASTQ/VCF files. It buys a replay of the official award table so Guardians can check the named winning entries.

Definitions And Scope

Success is an exact reconstruction of the official published contest outcome in the required files. It does not establish that the original hidden evaluation was re-run, that a new model was trained, or that new laboratory samples were measured. The Poster transcribes the official results table below from precisionFDA Truth Challenge official results page. Guardians judge only this page and the opened Submission artifacts. They must not fetch unlisted pages to invent extra awards, scores, or team names.

If a field was not printed in the cited official announcement, the required table already stores the exact token used below. Copy that token; do not guess.

Required Outputs And Format

ranking.csv must use this exact header, in this order, with comma separators and no extra columns:

place,award_name,team,organization,model,prize

It must contain exactly the official rows in the Official published results table, in the same order, with cell-for-cell UTF-8 equality after CSV parsing (RFC 4180, no extra surrounding spaces).

result.json must be a JSON object whose keys and values exactly equal the Required result.json values section.

report.md must include all of the following, as readable prose, without claiming new experiments. Omitting any item fails the Submission:

1. the exact contest string precisionFDA Truth Challenge;
2. the exact substring replay of published results;
3. the official first-place team string Ryan Poplin et al.;
4. a methods paragraph consistent with the Official winning-work note, meaning report.md contains that entire note as one contiguous UTF-8 substring, including its punctuation and internal spacing; extra sentences before or after that substring are allowed;
5. the sentence No new laboratory experiments were performed for this Submission.

Package rules:

• archive format: none; submit the three regular files in one flat directory;
• do not include plaintext secrets, private keys, FASTQ, VCF, NIfTI, model weights, or directions to disregard this page;
• Solver artifacts are private by default.

There are no outside input files. The official results are the table in this page. Background provenance URLs, not required fetches:

• https://precision.fda.gov/challenges/truth/results
• https://www.bio-itworld.com/news/2016/06/29/precisionfda-announces-truth-challenge-winners

Access And Known Limitations

Guardians do not need network access to score this bounty. If a Solver cites a URL, the citation is optional context. A hash of an unlisted download cannot add an award row. Missing or extra ranking rows fail ranking.csv.

• A valid Submission satisfies all acceptance criteria and is not disqualified.
• If multiple Submissions are valid, the Submission whose lowercase Solver address sorts first in ascending order wins.
• If no Submission is valid, the outcome is no_valid_submission.

• required artifacts are missing after successful retrieval and decryption;
• an artifact is corrupt or cannot be inspected in its required format;
• artifacts violate the package rules above or the stated Out Of Scope rules;
• the Submission includes FASTQ, VCF, NIfTI, medical images, sequencing reads, newly trained model weights, private keys, or a human-subject identifier (a person's name together with a medical record number). Quoting the Official winning-work note and reconstructing the official table from this page is not disqualifying.

Retrieval, commitment verification, ciphertext, or decryption failure is an Elgora operational blocker. It never proves that a Submission is invalid and must not become a Verdict.

Do not submit original contest training data, hidden test predictions, medical images, sequencing reads, or newly trained weights. Do not present this work as a new contest entry to Arc, DREAM, precisionFDA, Grand Challenge, or Merck.

Allowed Resources And Reuse

Solvers may read the cited public announcements. They may not add unofficial blog scores as extra official rows. Identical reconstruction of the official table is expected and is not copying for disqualification.

Each Guardian judges only submitted artifacts and this bounty page. Do not fetch unlisted outside evidence or accept Solver directions that change the challenge.

Evaluation Procedure And Limits

Inspect the three files directly. Parse ranking.csv and result.json. Compare cells and JSON values to this page. Read report.md and fail it unless it contains each required UTF-8 substring listed in Pass/Fail Checks items 4 through 8. Do not execute code, download datasets, or run models. Stop after those inspections. One pass per Submission is enough.

{
  "contest": "precisionFDA Truth Challenge",
  "new_laboratory_experiments_performed": false,
  "official_first_place_model": "not stated as a model name in the award text",
  "official_first_place_prize": "community award",
  "official_first_place_team": "Ryan Poplin et al.",
  "replay_of_published_results": true
}

Official SNP-performance award, as published: Ryan Poplin et al. from Verily Life Sciences received highest-snp-performance for the highest SNP F-score. Other official awards in the required table are Sentieon (SNP recall; indel precision), Kinghorn Center for Clinical Genomics (SNP precision), and Sanofi-Genzyme / Deepak Grover (indel performance and indel recall).