The question
Can a mushroom-body-inspired sparse code and learner find high-fitness protein variants with only a few lab-style measurements, compared with standard search strategies?
Computational biology · protein language models · search under a measurement budget
Better protein representations were not enough. Search strategy was the bottleneck.
A reproducible study of query-budgeted protein fitness-landscape search using ESM-2, ProteinGym, fly-inspired sparse coding, Bayesian optimization, evolutionary search and zero-shot protein language models.
What the experiments found
The representation is useful, but the fly-inspired learner fails to convert it into sample-efficient search. With the same measurement budget, GP-BO, evolutionary search and a graph-adapted AdaLead beat it, and a zero-shot ESM-2 ranking that uses no observed fitness is a strong training-free baseline.
What I built
A budgeted-oracle benchmark harness; six search baselines and the fly learner; a zero-shot ESM-2 scorer; four ablation studies; 2,800 deterministic logged runs on 10 ProteinGym assays; and the replayable demo on this page. Python and TypeScript, with tests.
Search efficiency at matched measurement budgets
How often does each method find a variant in the assay's top 1% after measuring a given number of variants? Mean over 10 unrelated ProteinGym assays × 20 seeds; bands are 95% bootstrap intervals over assays. Random search is itself a strong baseline here (it reaches the top 1% in roughly nine of ten runs by 200 measurements), so 50–200 measurements is the informative regime.
View the numbers
Graph-adapted AdaLead is not the published implementation unchanged (mutation = move to a graph neighbour, GP surrogate, no recombination). The zero-shot ESM-2 ranking is built without any observed fitness measurement; its budget counts the top-ranked candidates then measured against the assay.
Live experiment: the same 200 measurements, two strategies
Both methods start from the same TEM-1 β-lactamase variant and may measure 200 variants of it. Press Start and watch where each one looks. Everything shown is replayed from the recorded run (assay and seed fixed in advance; the run is bit-for-bit the benchmark's); nothing is simulated in your browser. The fly animation is only a picture of the logged search state.
The experiments behind the finding
Five studies, in the order they were run. All negative results are kept.
1Representation: is the signal there?
2Representation ablation: was it compressed away?
3Learner ablation: memory, replay, temperature, learning rate
4Cross-assay pretraining (negative result)
5Real hemibrain wiring vs random wiring
★Final benchmark: matched query budgets
Random search and walk, greedy local search, regularized evolution, GP-BO, graph-adapted AdaLead, zero-shot ESM-2 and the fly learner (with memory and the best configuration tested earlier), on 10 assays × 20 seeds. Unqueried experimental fitness is never visible to a method: it can only ask a budgeted oracle. Per-assay percentile normalisation; raw DMS scores are never averaged across assays. Click a metric or a budget column.
What this project found
What this project does and does not claim
This is
- an in-silico benchmark;
- an evaluation on measured ProteinGym landscapes;
- a study of query-budgeted sequence search;
- an application and evaluation of fly-inspired sparse representations.
This is not
- drug discovery or therapeutic design;
- wet-lab validation;
- a realistic simulation of a fly brain;
- proof that biological neural circuits optimise proteins;
- a novel replacement for established protein-engineering algorithms.
Limitations. 10 assays, mostly growth or stability phenotypes; confidence intervals over assays are wide for the deterministic zero-shot rankings. Embedding-based search methods use ESM-2 8M embeddings (CPU-cheap, cached for every assay) while zero-shot scoring goes up to 650M; masked-marginal scoring was run for 8M and 150M only. Graph-adapted AdaLead, GP-BO and regularized evolution used fixed, untuned settings. The fly learner's best configuration was chosen earlier on one landscape and was not re-tuned for the benchmark. The 2-D map in the demo is a UMAP picture and is not what any method sees.
Engineering
Scientific pipeline (Python)
- Budgeted oracle: methods receive an
Oraclethat reveals a variant's fitness only when queried and raises when the budget is spent; the data object they get has no fitness field. The evaluator alone reads the full landscape. - Methods: random search/walk, steepest-ascent local search with restarts, regularized evolution, exact GP Bayesian optimisation (Matérn-5/2, expected improvement; NumPy/SciPy), graph-adapted AdaLead, the fly learner on a FlyHash sparse code, and zero-shot ESM-2 scoring (wild-type and masked marginals).
- Models and data: PyTorch and Hugging Face Transformers for ESM-2 (8M, 150M, 650M); ProteinGym DMS substitutions; neuprint-python for the hemibrain connectome; one 650M embedding run on a Hugging Face Jobs A10G (67 s).
- Determinism: seeded RNG stream per (seed, method), logged query sequences, SHA-256 of every key artifact, a replay script that re-runs jobs and compares logs bit for bit.
Quality and delivery
- Tests: pytest for budget accounting, no-fitness access, determinism, prefix consistency, trace replay and the learning rule; vitest for the in-browser replay verifier and log hashes.
- Frontend: TypeScript and Vite; Canvas 2D for the landscape and Kenyon-cell views, hand-written SVG for the charts, d3-scale-chromatic for colour. No UI framework.
- This page is a static site deployed on Vercel. Heavy data (the recorded run, the full result tables) is fetched lazily when you scroll to it.
- Honest demo: the page re-applies the logged weight updates and checks them against the logged statistics and hashes in your browser (see the check under the live experiment).