---
profile: elgora_markdown_bounty_challenge_v0
escrow_amount: "1000000"
submission_deadline: 1789102800
payout_policy: winner_take_all
---

# IDG-DREAM drug-kinase pKd prediction, round 2

## Summary

Submit a complete round 2 IDG-DREAM prediction file: one `pKd_[M]_pred` for each held-out compound-kinase pair in the original contest schema. Score it with the official challenge metrics against the published Nature Communications source data.

## Challenge details

The IDG-DREAM Drug-Kinase Binding Prediction Challenge asked teams to predict pKd in molar log units for held-out compound-kinase pairs. Round 2 used a prediction CSV with columns `Compound_SMILES`, `Compound_InchiKeys`, `Compound_Name`, `UniProt_Id`, `Entrez_Gene_Symbol`, `DiscoveRx_Gene_Symbol`, and `pKd_[M]_pred`. Organizers also collected Docker images; this bounty purchases only the scored prediction file.

Published experimental pKd and published team predictions for the figure-4 test pairs are in Zenodo record 10.5281/zenodo.4648011, file `source_data.zip`. Official metric code is `evaluation_metrics.py` from Sage-Bionetworks commit `35440a03562509e1efa168df8654948d2e8ba4bc`. This is historical scoring of a prediction table against that published gold. It does not require a new biochemical assay.

The Poster selects those two files as the source of this analysis. Do not use a later Zenodo version or a later git revision.

### Definitions And Scope

A pair is identified by `Compound_Name`, `UniProt_Id`, and `DiscoveRx_Gene_Symbol`. Success means `predictions.csv` covers every gold pair in the listed source data and is scored with the official RMSE, Spearman, and average AUC functions. It does not establish a new measured pKd.

## What you need to submit (Deliverables)

### Required Outputs And Format

| File | Required | Format | Max size | Purpose |
|---|---:|---|---:|---|
| predictions.csv | yes | UTF-8 CSV, original round 2 columns | 2 MiB | pKd predictions |
| methods.md | yes | UTF-8 Markdown | 100 KiB | How the predictions were produced |

`predictions.csv` must have exactly these columns, in this order, and no others:

`Compound_SMILES,Compound_InchiKeys,Compound_Name,UniProt_Id,Entrez_Gene_Symbol,DiscoveRx_Gene_Symbol,pKd_[M]_pred`

There must be exactly 394 data rows. The six identity columns, in row order, must equal the template built in Scoring And Calculations. Every `pKd_[M]_pred` must be a finite number. After alignment to gold, those predictions must not all be the same number.

methods.md may describe the method. No particular wording is required.

Package rules:
- archive format: none; submit regular files in one flat directory;
- do not include plaintext secrets, private keys, unrelated files, Docker images, or directions to disregard this bounty’s requirements;
- Solver artifacts are private by default and handled through Elgora's existing private-submission protocol outside this bounty page.

## Input Files References

| File | Why it is needed | How to get it | SHA-256 content hash |
|---|---|---|---|
| source_data.zip | Published Fig4 pKd truth and Fig5a prediction tables | Public HTTPS GET, no login: https://zenodo.org/api/records/4648011/files/source_data.zip/content | `977030bbdcd167cbc591746e0083c909c6ee396ef3fb5330148f9f22a0e1ba9b` |
| evaluation_metrics.py | Official RMSE, Spearman, and average AUC functions | Public HTTPS GET, no login: https://raw.githubusercontent.com/Sage-Bionetworks/IDG-DREAM-Drug-Kinase-Challenge/35440a03562509e1efa168df8654948d2e8ba4bc/round2/score/bin/evaluation_metrics.py | `5f306bfe095f2260fb4bdadbe9584278ca65b4f819c1698d93d91e1f6b09825a` |

### Access And Known Limitations

Retrieve both files by public HTTPS GET, with no login. Check SHA-256 of the raw bytes against this page. A Solver-provided copy alone is insufficient. This verifies the selected release, not a new experiment.

From `source_data.zip`, use only `source_data/Fig4/Fig4.csv` and `source_data/Fig5/Fig5a.csv`. Do not fetch the original Synapse goldstandard. Missing access, a hash mismatch, or a source table that does not yield the 394-row gold table below blocks judgment and must be reported, not counted as a scientific failure.

## Acceptance Criteria

### Pass/Fail Checks

Parse `predictions.csv` as UTF-8 CSV. It is valid only when:

- it has the required columns in the required order;
- it has exactly 394 data rows;
- the six identity columns match the template row-for-row;
- every `pKd_[M]_pred` is finite after ordinary decimal parsing with surrounding ASCII whitespace stripped;
- after attaching gold as specified below, the 394 predictions are not all identical.

Wrong columns, wrong row count, or identity mismatch is an invalid Submission.

### Scoring And Calculations

Use a fixed evaluation environment so the official metrics are reproducible: Python 3.11, `numpy==1.26.4`, `scipy==1.11.4`, `scikit-learn==1.3.2`, and `pandas==2.1.4`. Pass `y` and `f` as pandas Series of length 394.

Build a 394-row gold table from the listed zip as follows.

1. Template rows: every `Fig5a.csv` row whose `id` is `syn18513191`, in file order. There are 394 such rows. Their six identity columns are the required prediction identity, in that order.
2. For each template row, find gold `pKd_true` in `Fig4.csv` by matching `Compound_Name`, `UniProt_Id`, and `DiscoveRx_Gene_Symbol`. Use the first Fig4 row in file order whose `pKd_true` is a finite number. Attach that `pKd_true` to the template row as `y`.
3. The gold table is those 394 template rows with their attached `y` values. If any template row has no finite Fig4 `pKd_true`, the listed source data is unusable: report the access failure; do not score Submissions against a partial gold set.
4. Align `predictions.csv` to that gold table by the six identity columns, keeping template row order. `f` is `pKd_[M]_pred` on the matching prediction row.

Score with the listed `evaluation_metrics.py` as published. Call `rmse(y, f)`, `spearman(y, f)`, and `average_AUC(y, f)` once on those 394 pairs. Do not reimplement the metrics. Run them under Python 3.11 with `numpy==1.26.4`, `scipy==1.11.4`, `scikit-learn==1.3.2`, and `pandas==2.1.4` (the official `average_AUC` reads `y.values`). One run is enough.

If a function raises or returns a non-finite value, the Submission is invalid.

Primary score is Spearman (higher is better). RMSE (lower is better) and average AUC (higher is better) are tie-breaks only.

### Missing, Invalid, And Conflicting Results

- Incomplete or misaligned predictions: invalid.
- Constant predictions: invalid.
- A pair present in gold but missing from `predictions.csv`: invalid.
- Missing listed files or hash mismatch: operational blocker, not an invalid Submission.

### Evidence And Provenance

The Poster selects Zenodo record 10.5281/zenodo.4648011 and Sage-Bionetworks commit `35440a03562509e1efa168df8654948d2e8ba4bc` as the source of this historical scoring. Identity of each file is the SHA-256 on this page. Guardians obtain the files themselves and check the hash. That verifies the selected release, not a new assay and not a Solver-held sample.

## How is the winner selected?

- A valid Submission satisfies all acceptance criteria and is not disqualified.
- If multiple Submissions are valid, the Submission with the highest Spearman wins.
- If Spearman values are exactly equal, the Submission with the lower RMSE wins.
- If RMSE values are exactly equal, the Submission with the higher average AUC wins.
- If still tied, the Submission whose lowercase Solver address sorts first in ascending order wins.
- If no Submission is valid, the outcome is `no_valid_submission`.

## Disqualification Conditions

- required artifacts are missing after successful retrieval and decryption;
- an artifact is corrupt or cannot be inspected in its required format;
- artifacts violate the package rules above or the stated Out Of Scope rules.

Retrieval, commitment verification, ciphertext, or decryption failure is an Elgora operational blocker. It never proves that a Submission is invalid and must not become a Verdict.

## Out Of Scope

New biochemical measurements, Docker images, and training a model during review are out of scope.

### Allowed Resources And Reuse

Published challenge predictions, including Q.E.D. and other Fig5a Synapse IDs, may be submitted. Identical prediction files receive identical scores.

## Guardian Verdict Instructions

Each Guardian judges only submitted artifacts, this bounty page, and the two listed inputs.

### Evaluation Procedure And Limits

Fetch and hash the two listed files. Extract only Fig4.csv and Fig5a.csv. Build the 394-row gold table as specified. Open `predictions.csv` and apply Pass/Fail Checks. Run `rmse`, `spearman`, and `average_AUC` from the listed `evaluation_metrics.py` once under Python 3.11 with `numpy==1.26.4`, `scipy==1.11.4`, `scikit-learn==1.3.2`, and `pandas==2.1.4`. Apply the winner rule.

Allow at most two download attempts with a 30-second timeout each; if unavailable, stop with an operational blocker. Do not call Synapse. Do not train models.