How reliable are the conclusions drawn from Adaptyv’s complete raw-curve release?

Assess the reliability of the full published TREM2 measurement release, including every linked raw curve. Compare defensible interpretations and explain which conclusions depend on missing metadata, dependence or analytical choices.

Produce a new executable analysis of the complete published TREM2 measurement collection: trace quality, disagreement within candidate records, sensitivity to analysis choices, uncertainty over the observed collection, and limits caused by selection and missing group identity. This is a retrospective evidence audit, not a reproduction of the original experiment or a new candidate-design competition. Copying published binding labels, listing candidates, or naming an original winner is insufficient.

The evidence contains 100 tested candidate records and 215 linked curve JSON files with 940 trace records. The 41 untested designs in the competition coverage account are absent from these inputs. There is no verified AI-versus-human team mapping in the provided evidence. Do not infer it from names, author fields, designMethod, or computational annotations. The analysis must explicitly establish which comparisons remain non-identifiable; it must still perform all possible trace and sensitivity analyses. Do not extract, analyze, or submit sequences, structures, participant identities, biological optimization suggestions, or laboratory protocols.

All roles download the following files with public HTTPS GET, without credentials, before reproduction. trem2.csv, SHA-256 f9ec9368c7719e2353ded3559c6dd4981c17b799a657aabcfa04496a46a80056, is at https://proteinbase.com/api/proteins/download?collectionId=019e0495-7ce5-a11b-95e8-e114da740e31&slug=adaptyv-x-muni-hackathon-ai-agents-vs-humans . It supplies published anonymous record IDs, evaluation entries and curve links. The raw-file appendix below fixes every required curve URL, local input filename and SHA-256. It is part of this page; no additional manifest or external scientific source is required. Unavailable or changed bytes block judging; never silently omit a file or substitute a newer collection.

Parse the CSV as UTF-8 with optional BOM and RFC4180 quoting. Use id as candidate identity and retain original evaluation-array index for provenance. Read only id and evaluations. Do not read the sequence, name, author, or designMethod columns into analytical outputs. Evaluation objects use type, metric, target, value and sometimes unit. Curve references are experimental spr_kinetic_curves or bli_kinetic_curves entries with value.url; target must be trem2. Separate SPR and BLI. Each raw JSON object maps a source trace key to an object with raw, fit, concentration, control, aggregated, and virtual. Preserve those keys and source paths. Raw arrays are raw.t and raw.y; available fits have fit.association and/or fit.dissociation, each with t/y arrays. Missing or empty fits are not successful fits.

Link a raw file to a candidate and assay method only through the candidate's explicit URL-valued evaluation. Do not join binding, expressed, kd, kon, koff or other metric entries to a particular curve by array position, order, proximity, matching values, or an invented measurement identifier. Where no explicit shared identifier exists, use candidate-level multisets and state the limitation. Units absent from a trace stay unspecified; do not label raw concentration, time or response with invented physical units. Retain the reported unit beside any evaluation value and never combine unlike or unknown units.

1. Full evidence graph and structural quality. Inventory all 100 candidate records, 215 unique URL files and every trace. Verify file hashes. Create one trace row keyed by input filename and original trace key, preserving all explicit candidate/method links and raw flags. Report raw point count; mismatched arrays; empty, nonnumeric, nonfinite or nonincreasing time; raw/fit coverage represented by the following counts, not an unspecified coverage score: per trace, raw_point_count is the length of the raw t array and raw_y_count the length of the raw y array (null for a missing/non-array field); per association/dissociation segment, fit_point_count is the length of its structurally valid fit t array, otherwise null with the validation reasons, and compared_points is the number of structurally valid raw points with t inclusively between that valid fit segment's first and last t, otherwise zero with the invalidity reason. A segment is evaluable when both structures are valid and compared_points is positive; per trace report evaluable_segments as the count of evaluable association/dissociation segments (0, 1 or 2). Across the dataset report total trace count, structurally usable trace count, and the count of traces with at least one evaluable segment; available control, virtual and aggregated status; and every undefined diagnostic reason. Reconcile all totals, including files with more than one link if any. Do not delete inconvenient traces, relabel controls, assume concentration zero means a control, or count a virtual/aggregated trace as an independent physical replicate.

2. New trace diagnostics from raw points. A structurally usable raw trace has equal nonempty numeric finite arrays with strictly increasing t. For both raw and fit arrays, numeric means a JSON number excluding booleans; never coerce strings, null, true, or false to numeric values. For every usable trace with at least two points, calculate duration max(t)-min(t), response range, median absolute adjacent response difference, and absolute endpoint change divided by response range. The last diagnostic is null when range is zero. A usable fit segment has t and y arrays of equal length, at least two points, finite numeric values excluding booleans, and strictly increasing t. For simultaneous structural defects, report reasons in this fixed order: missing object or non-array t/y; unequal lengths; fewer than required points; nonnumeric or nonfinite t/y; nonincreasing t. Combine nonnumeric and nonfinite into one reason. Evaluate time monotonicity only if the preceding structural checks passed. An empty, unequal-length, nonnumeric, nonfinite, or nonincreasing segment is invalid: report its exact structural reason and null error statistics, and do not repair or interpolate it. For each usable association or dissociation segment and structurally usable raw trace, interpolate its y linearly onto every original raw t point satisfying segment_min_t <= raw_t <= segment_max_t; use those raw y values for residuals. A structurally invalid raw trace receives no fit comparison and retains its raw structural reason. Signed residual is raw response minus the interpolated fit response; mean signed residual is its arithmetic mean. Report compared-point count, RMSE, mean signed residual and RMSE divided by raw response range, separately for each segment; normalized error is null for zero raw range and all error statistics are null for no overlapping raw points. Do not extrapolate, merge overlapping segments, refit kinetic constants, or treat a residual threshold as proof of biological validity. For the later aggregation, a trace's normalized RMSE is the maximum of its finite segment normalized errors, or null if neither segment supplies one. These descriptive diagnostics do not require physical units and must be labeled accordingly.

3. Candidate-level disagreement and unequal evidence. Within each method separately, summarize the diagnostics first per source file by median across usable nonvirtual, nonaggregated traces, then per candidate by median across its linked source files. Assign the control stratum per trace: exactly boolean true, exactly boolean false, or unknown for every other/missing value. Split traces within a file by these strata before calculating file medians; a mixed-control file may contribute a separate summary to several strata, and a candidate retains separate method/stratum summaries. Never pool control strata at file or candidate level. In the aggregated diagnostic scenarios, include a trace only when both virtual and aggregated are exactly boolean false; true, null, missing or nonboolean values for either flag exclude that trace from those scenarios while retaining it in the full inventory. Unknown control remains its own stratum; it does not imply unknown virtual/aggregated flags are false. For each candidate and diagnostic report count, minimum, maximum, median and interquartile range across files, with contributing IDs and explicit undefined reasons. For the candidate-weighted distribution, include each finite candidate median exactly once, giving every contributing candidate equal weight. For the trace-weighted distribution, pool the finite per-trace diagnostic values from the same method, control stratum and scenario, including each distinct file/trace-key exactly once; do not first average or median within files. Compare these two empirical distributions; quantify the median and interquartile-range differences to show the effect of unequal trace counts. Do not claim the files are independent experimental repeats. Separately form each candidate's binding multiset from evaluations whose type == "experimental", metric == "binding", and target == "trem2" using exact case-sensitive string equality. Retain only values whose JSON type is boolean; count missing and nonboolean values separately without coercion. Classify the resulting multiset as all-true (one or more true and no false), all-false (one or more false and no true), mixed (both), or unavailable (no booleans); define the collection summary as side-by-side counts p (supported), n (nonsupported), and u (unavailable) across the same 100 candidates for each convention, using the exact definitions in analysis 5. Report the resulting candidate decisions and both count triples so their change is explicit; do not substitute an unstated pooled score or rate. Do not assign these labels to individual curves.

4. Sensitivity and conditional uncertainty. For each method and control stratum, repeat the candidate-level summary for two analysis scenarios: all structurally usable nonvirtual/nonaggregated traces, and only those additionally having at least one evaluable fit segment. Then apply a third scenario to the latter: remove whole traces whose trace-level normalized RMSE exceeds that method/stratum's empirical 75th percentile among finite trace-level values. Keep null-error traces accounted for separately; they do not enter that quantile or normalized-error summaries. If no finite values exist, this scenario is undefined and its output is null with reason. Report the number removed, every resulting denominator, and changes in candidate-level median normalized RMSE and endpoint-change ratio. This percentile cutoff is an explicit sensitivity choice, not a scientific rejection standard; retain all original rows and state that unavailable fits are not known bad fits. For each scenario summarize the median of the finite candidate-level values. Calculate 1,000 candidate-cluster bootstrap medians by sampling candidate IDs with replacement, retaining each selected candidate's entire evidence, using a fresh NumPy Generator(PCG64(20260909)) for each method/stratum/scenario. Process endpoint_change_ratio first and normalized_rmse second. For each diagnostic independently, the sampling population is only candidate IDs with a finite candidate-level median for that diagnostic, sorted ascending; its size is n. For each replicate sample n indices with replacement using rng.integers(0,n,size=n). Every selected occurrence retains its complete candidate evidence and contributes its candidate median once, including repeated draws of the same candidate; never collapse duplicate draws to a unique-ID set. The replicate statistic is the median of those n values. Excluded null-median candidates remain in coverage and are counted but cannot enter this statistic. Report n and the eligible IDs; report 2.5th and 97.5th percentile bounds, finite-resample count and replicate medians. For every reported quantile, use numpy.quantile(finite_values, q, method="linear"); the median is q=0.5, quartiles q=0.25 and0.75, and IQR is q75-q25. Bootstrap interval percentiles use q=0.025 and0.975. Empty finite sets yield null summaries with a reason. With fewer than three eligible candidates, report descriptive values but null interval and the exact count. These intervals describe sensitivity over observed candidates, not assay error, causal effects, or generalization to all submitted designs. Do not pool SPR and BLI or substitute trace resampling for candidate resampling.

5. Selection and missing identity. For each of the any-true and all-true published-binding conventions, report observed support p, observed nonsupport n, and unavailable status u among the 100 records. The all-true convention requires at least one explicit boolean and no false; the any-true convention requires at least one true; missing or nonboolean entries are counted and disclosed but are not fabricated observations. Candidates with no explicit booleans are unavailable. For the stated scope of 100 observed and 41 absent untested designs, calculate the no-assumption descriptive range [p/141, (p+u+41)/141]. It bounds a convention applied to reported labels, not a proven biological success rate. Explain why a measured-only rate cannot represent the absent 41. Demonstrate AI-versus-human non-identifiability with two hypothetical assignments of anonymous observed record IDs to two equally sized groups that preserve every observed measurement but give different group support contrasts; clearly label them counterexamples, not recovered identities. If the outcome is constant and different contrasts are impossible, demonstrate the constant case numerically and explain why causal group performance is still unidentifiable. No fabricated team labels may enter the primary analysis.

6. Evidence-backed interpretation. Report at least one quantitative conclusion from structural coverage, trace diagnostics, within-candidate disagreement, weighting/sensitivity, and selection. Link each to generated tables and source trace or evaluation IDs. Explain whether it survives the defined sensitivity scenarios; identify missing assay metadata, missing controls where applicable, unequal/selected evidence, and unverified experiment independence. State what additional existing records would be needed to identify group performance or link reported kinetics to traces, without proposing new lab work. Honest non-identifiability passes where evidence is absent; replacing available raw analysis with caveats fails. No conclusion may claim physical sample authentication, clinical effectiveness, or a recovered AI-versus-human winner.

Include RUN.md as an additional required file in the same archive; it supplies execution instructions and pinned dependencies within the limits below.

Submit one ZIP, at most 50 MB compressed and 150 MB uncompressed, containing report.md (at most 25,000 words), analysis.py, methods.json, machine-readable JSON/CSV tables in results/, and at most 16 PNG figures. Do not include source data, sequences, structures, participant names, nested archives, symlinks, downloaded dependencies, or opaque executables. A table must exist for every required analysis. methods.json documents table columns/types, formulas, undefined reasons, quantiles, bootstrap order and seed, aggregation weights and plot sources. Sort source files, trace keys and candidate IDs in ascending Unicode codepoint order before processing and sampling. Numeric calculations use IEEE-754 binary64 without intermediate rounding. All array ties remain explicit; no candidate ranking is required.

Guardians download and verify all fixed inputs before one complete reproduction. Use Python 3.12, NumPy 2.2.6, pandas 2.2.3, SciPy 1.15.3 and Matplotlib 3.10.3 as the numerical reference versions. Include RUN.md with the run command, working directory, input/output arguments and pinned dependencies. The input collection contains trem2.csv and every appendix raw/ path. Regenerate all outputs into an empty directory without using submitted result files as input. The evaluation budget is at most 4 CPU cores, 8 GB RAM, 2 GB temporary disk and 45 minutes wall time; one retry is allowed only for a documented infrastructure interruption. Each Guardian owns its security and execution setup under Elgora rules.

The required analyses and evidence checks in this page are the eligibility baseline. Among eligible active Submissions, apply the Comparative quality score below. Highest total score wins. Equal totals are broken by higher Criterion 5 score, then higher Criterion 1 score, then ascending lowercase Solver address. If no Submission is eligible, use no_valid_submission. Guardians' written Verdicts name failed mandatory criteria and the selected winner without exposing private Submission content. Retrieval, commitment verification, or decryption failure blocks judgment rather than proving scientific invalidity.

All work required above remains mandatory. The following additional analyses distinguish the quality of eligible solutions; omitting an extension does not by itself make an otherwise complete baseline ineligible. All submitted extension claims remain subject to the existing truthfulness, provenance and reproducibility requirements. Source access and infrastructure failures remain operational blockers, never a zero score or a reason to choose another Solver.

There are five criteria, each with four cumulative evidence levels. Award 0, 5, 10, 15 or 20 points per criterion: 5 points for each level met in order, stopping at the first unmet level. Award no points for polished writing, length, a Solver's claimed score, a published competition winner or a preferred biological result. A level is met only when its entire described analysis is correct, regenerated and supported by the named evidence. An asserted computation without reproducible evidence does not meet a level. Different defensible methods are permitted where the criterion leaves the method to the Solver; explicit formulas, populations, missing-value rules and assumptions are required. Unsupported assumptions cannot be silently treated as source facts.

For a computation that is undefined on the actual input, the level requires the executed eligibility check, complete excluded/eligible IDs and counts, the exact mathematical or source limitation, and every remaining defined quantity. A blanket caveat is insufficient. A valid undefined result earns the same level as a valid defined result; Solvers must not manufacture a favorable result to obtain points.

Criterion 1: Influence analysis — 20 points

1. For every method/control stratum and each of the three required scenarios, analyse endpoint-change ratio and normalized RMSE separately. For each diagnostic, eligible candidates have a finite candidate-level median. Delete each candidate's aggregate contribution in turn, retaining other candidates' values, and compute the median of the remaining finite values. An empty remainder gives null, count zero and a reason. An empty initial population gives no deletions and null influence extrema with a reason.
2. Record all deletion values, denominators, maximum absolute change and every tied most-influential candidate ID; preserve reason-coded undefined results.
3. Repeat by deleting linked source files and compare file-level with candidate-level influence without claiming files are independent experiments.
4. Tie the observed influence to a bounded conclusion about the released collection, with links to all contributing evidence.

Criterion 2: Weighting comparison — 20 points

1. Compare trace-weighted, equal-file-weighted and equal-candidate-weighted distributions within each existing method/control/scenario stratum.
2. Declare formulas, weights and quantile rules; reuse the original trace eligibility and do not pool unlike strata or silently treat unknown flags as false.
3. Report medians, IQRs, effective contributing counts and changes for both normalized residual and endpoint-change ratio under every weighting.
4. Explain numerically which apparent quality conclusions reflect unequal evidence counts; do not imply any weighting authenticates physical samples.

Criterion 3: Fit-availability sensitivity — 20 points

1. Compare the original all-usable and fit-available scenarios using endpoint-change ratio so missing fit residuals are not invented.
2. Retain candidate and trace membership for both scenarios, full exclusion reasons and every finite denominator.
3. For candidates represented in both scenarios, compute paired changes separately from changes caused by different candidate membership.
4. Quantify the two effects and explain why a missing fit is not evidence of a failed experiment or a usable residual of zero.

Criterion 4: Cutoff robustness — 20 points

1. Extend the existing 75th-percentile residual sensitivity with the 50th and 90th percentiles, all computed from the same original finite method/stratum residual population.
2. For each threshold, drop whole traces only from this labelled sensitivity scenario, retain equality at the threshold, and report all candidate/trace counts.
3. Report aggregate diagnostic values and the envelope across these thresholds plus the unfiltered fit-available scenario, preserving undefined cases.
4. State which conclusions survive all computable thresholds and identify that percentile cutoffs are analytical choices, not laboratory validity criteria.

Criterion 5: Independent numerical verification — 20 points

1. Supply a second implementation of structural validation, interpolation residuals and file/candidate aggregation, independent of the primary code and results.
2. Apply it across every raw file and trace, retaining coverage and all undefined diagnostics.
3. Reconcile primary and secondary results with numerical tolerances and exact identity/count checks; implementation agreement does not authenticate a laboratory experiment.
4. Provide a machine-readable claim ledger mapping every headline conclusion to source file/trace or evaluation identities and generated result cells.

Put extension methods in quality_methods.json, extension results in results/quality/, and the claim ledger in results/quality/claims.csv. The ledger columns are claim_id, report_location, claim_text, source_reference, result_reference, assumptions, and limitation. Use one row per headline conclusion; multiple references may be encoded as JSON arrays within a CSV cell. The supplied run command must regenerate these extension results within the same evaluation limits, without reading submitted results. Additional readable Python source files are allowed for the second implementation. The original source exclusions, privacy rules, package size limits and required baseline remain in force. Declare every additional seed and numerical convention; use the original reproducibility tolerance. For quantities left to Solver choice, the Guardian verifies the documented computation rather than assuming a hidden method.

Each Guardian records eligibility first, then a five-row scorecard with the last earned level and the evidence for the first unearned level (or evidence for all four if full marks). Total points equal the sum, from 0 to 100. Apply the tie-break only when totals are exactly equal. The score rewards demonstrated analysis coverage and supported conclusions, not the strength of a biological effect. Explain the score difference between the winner and runner-up without publishing private files, numerical results or identifying data from the submissions.

Each row identifies a required published curve input. These are complete evidence inputs, not files to select by apparent quality. Download each URL to its stated path and verify its hash before review.

AI assistance and reuse of disclosed code are permitted. Explain prior work used in the report; regenerate all required results from the fixed inputs. Identical results alone do not prove copying. No new laboratory work, candidate design or sequence optimization is purchased.