An in-silico version of this loop leans on two kinds of models doing two different jobs. A generator proposes new sequences, and a predictor scores them. These are usually separate models. The Evo2 paper is a good example, using Evo2 to generate DNA and a predictor like Borzoi to guide it.

I got curious whether in context learning could collapse that split. If I show a foundation model a handful of high activity sequences and ask it for one more, will it return something that also looks high activity? No fine tuning, no numeric labels, just a list of “preferred” sequences in the prompt. If it works, the generator is doing the predictor’s job on its own, inferring “what success looks like” from the examples alone.

To test this I needed two kinds of models.

1. As the generator I tried two foundation models. Evo2, the long context DNA language model from the Arc Institute, is trained on a broad slice of natural genomes and transcriptomes; I used the seven billion parameter checkpoint. EVA is a recent state of the art RNA foundation model. One is broad and genome scale, the other is RNA specific, which is what makes the comparison interesting.

2. As the final arbiter of success I used T-Grammar, our proprietary regression model for RNA regulatory activity. Given a 5’ or 3’ UTR, it predicts two things per cell type: activity, how active the sequence is (think expression or stability), and cell specificity, how much that activity differs across cell types. Here T-Grammar is only the judge and is not part of the loop; it scores the generated sequences after the fact and never steers the generation.

The setup

For a single cellular context, I took an in-house dataset measuring the impact of synthetic short variable 5’ UTR sequences on expression and split it into a highly activating set and a highly repressive set based on the experimental measurements. The sequences are fully synthetic and the foundation models have never seen them in training. From each pool, I sampled some number of sequences and concatenated them into a single prompt via a <sep> token.

Then I generated a fixed number of nucleotides at a given temperature, producing a synthetic sequence. I built a bunch of these prompts, half from the activating set and half from the repressive set, generated a batch of sequences per prompt, and passed the sequences through T-Grammar to read out predicted activity and predicted cell-type specificity. Each box and point cloud below summarizes 320 generated sequences per number of examples in the prompt, which matters for reading the significance numbers later.

I also swept the number of in context examples rather than fixing it. For Evo2 I went from two examples in the prompt up to 256. For EVA I went up to 128, which is as far as that model would accommodate. Sweeping the example count is what lets us answer the more interesting question, which is not just whether in context learning happens but how much priming each model needs before the signal becomes clear, and whether activity and specificity behave the same way.

If the in context idea works, the continuations from high prompts should score higher on T-Grammar than the continuations from low prompts. If it does not, the two distributions should look identical. Neither model has any other way to know which prompts came from which pool, because no labels were ever provided.

Evo2:

Let’s start with activity. Across every example count from two to 256, the activating primed generations sit above the repressive primed generations. The activating cluster around 2.4 to 2.45 in predicted center of mass and the repressive sit closer to 2.0 to 2.1. The gap is present in two examples and widens as you add more, mostly because the repressive group drifts down while the activating stays put. This is the more legible of the two signals.

Specificity is the harder readout, and it is a harder biological problem to begin with. The activating primed continuations carry a higher predicted differential signal than the repressive primed ones at every example count, but the two groups sit much closer together than they did for activity. The difference for specificity is also lower in the actual samples used in the prompt. The activating stays near zero, dipping toward the middle of the sweep, while the repressive drifts steadily more negative as you add examples. So the separation is there from the first prompt but it is a smaller gap similar to the input data. Where a sequence pushes activity in one cell type versus another is a finer question than how active it is overall, so it makes sense that it is the harder pattern to pick up.

This panel makes the activity versus specificity comparison easier. Two things to notice. First, both signals are significant across the entire sweep by a wide margin, so neither is borderline anywhere, even at the smallest prompt. Activity is the stronger of the two at every count and pulls further ahead at the high end. Specificity runs consistently above activity, less significant but still far below the line the whole way. Both dip a little toward the middle of the sweep before dropping steeply at the top. Second, the message here is the gap between the two curves rather than the absolute values. With a large sample, a very small p value mostly tells you a separation is real, not how large it is. The reading is that specificity is reliably less separated than activity, even though both clear significance easily.

So the last Evo2 panel asks how large the generated gap is in absolute terms, not just whether it is significant. The dashed line marks the gap between the real activating and repressive MPRA pools, the ceiling we are chasing. For activity, the generated gap climbs as you add examples and by the end of the sweep is reaching that ceiling. For specificity, the generated gap sits above its ceiling across almost the whole range. Put another way, Evo2 produces sequences whose predicted activating minus repressive separation is as large as the real pools for activity and larger than the real pools for specificity, and the separation generally grows with more examples.

So the Evo2 takeaway is two parts. Activity is the cleaner, larger signal and shows up from the first prompt. Specificity is real and significant throughout, but it is the smaller and less separated of the two.

EVA:

EVA also does the task, once it is given more than a couple of examples. It takes the same <sep> delimited prompts, it scales up to the same range, and it produces continuations that T-Grammar can score. On the basic question of whether in context learning happens at all, EVA clears it for activity.

For activity, the activating primed boxes sit above the repressive primed ones across most of the sweep, so the direction is right, with one exception: at the two smallest prompts the groups are essentially tied. After that the activating pulls ahead, and the gap widens as the repressive group drifts down while the activating holds steady. So on the boxes EVA does strengthen with more examples for activity. The separation is still smaller than for Evo2, and as the significance panel shows, it is far less consistent from one count to the next.

The specificity signal is far fainter and, more tellingly, not consistent in direction. The activating and repressive groups sit close together at most counts, and at times the ordering flips, with the repressive group sitting at or above the activating group. At the counts where it does point the right way, the gap is small. This is the same activity versus specificity pattern as before, except for EVA specificity it is weak enough that even the sign is not stable.

The separation curves tell the same story. Activity separation trends upward as you add examples, while specificity separation stays small and flat. So whatever EVA gains from a longer prompt shows up in activity, not in specificity. That is the opposite of Evo2, where both gaps grew toward and past the real pool ceiling.

The significance panel is the clearest place to compare the two, and with a large sample it is unforgiving. For EVA, activity is not significant at all at the two smallest prompts. After that it does separate, but along a jagged, sawtooth path: strong at one count, weak at the next, reaching its best only at the high end, where it finally rivals Evo2. Specificity is worse. It fails the significance line at the smallest prompts and again partway through the sweep, lands right on the threshold at one count, and clears it cleanly at only a few. There is no smooth trend, and at several counts the specificity split is not statistically real. Evo2, by contrast, was significant at every count by a wide margin.

Putting the two side by side, the contrast is sharp. Evo2 separates activating from repressive at every example count, for both readouts, by a wide margin. EVA separates cleanly only at some counts, shows no separation at the smallest prompts, and on specificity is inconsistent enough that the direction sometimes flips. The larger sample is part of why this is now visible, since an earlier, smaller run made EVA look more uniformly significant than it is. I would still be careful about reading too much into why. The two models are different in kind. Evo2 is a broad genome scale model and EVA is RNA specific, the two were run over different ranges of example counts, and this is all in silico. One reasonable hypothesis is that the wider training grammar behind Evo2 carries into the fine structure function relationship that specificity depends on, but that is a guess, and the cleaner test is the wet lab rather than more in silico comparison.

Are the generations copies of the prompt?

One worry hangs over all of this. If the model just parrots the sequences in the prompt, the steering is not steering at all, it is the prompt played back, and the effect would only look stronger as you add more examples. So for every generated sequence I measured its highest identity to any of the in context examples it was shown.

Identity does rise as you add examples, which makes sense, since a longer prompt gives the model more to lean on and more chances at a partial match. But it rises slowly and flattens out well short of copying. Even at the largest prompts a typical generation overlaps its nearest example by only around two thirds, and the bulk stay clear of the copy line at the top of the plot. This holds for both models and both prompt groups. So the sequences are genuinely new.

Why I find this interesting

One, this is in context learning at the sequence level, not the language level. The token here is a nucleotide. The pattern the model latches onto is not English or code but presumably some combination of k-mer composition, motif content, and structural propensity. Whatever that combination is, it transfers to a separate model trained on a separate readout. The two models have never spoken to each other and still agree, and the degree of agreement differed between them, most visibly on the harder of the two outputs.

Two, no training. No fine tuning. No reward model. No labeled dataset on the foundation model side. The cost of getting a usable generator of “high activity 5’ UTRs for this cell line” was the cost of sampling a few examples and gluing them with pipes. If the result holds up under bench validation, that is an efficient way to get there.

Three, this kind of prompt-based steering opens up a different mode of how we use foundation models internally. The path so far has been: collect data, train a specialist (T-Grammar), use the specialist to score or generate. The alternative path this experiment hints at is: collect data, use the specialist as an oracle to filter generations from a foundation model that is being steered by a small number of in context examples. We keep the specialist where it is most useful, predicting activity from sequence with proper calibration, and let the foundation model write plausible nucleotide sequences, without teaching it anything new. The comparison gives an early read on which model recovers the signal more reliably for this task.

One caveat here, we are validating in silico, with one model (T-Grammar) judging the other two (Evo2 and EVA). The interesting next step is the wet lab. If the Lab in the Loop generates a small set of prompt steered sequences, splits them into activating primed and repressive primed batches, and runs them through the same MPRA assay that produced the training data in the first place, the in context signal either survives in cells or it does not. That is the result that ultimately matters, and it is the one that would tell us whether the difference we see here is real or an artifact of one model judging another.

The promise of in-context learning has been around for a few years on the language side. Show a model a few examples, get a model that does the task. Seeing it on RNA sequence design, with two different foundation models agreeing on what counts as a good generation, was a striking thing to watch.

Following the announcement, several reporters expressed interest in learning more about Therna’s n=1 strategy, with Ryan Cross at Endpoints News asking how the FDA’s recent draft guidance, which introduces a framework for accelerating tailored, individualized treatments for ultra-rare diseases, could impact the company. This guidance marks an important moment for the field and an opportunity to define how n=1 approaches can be developed with both speed and scientific rigor. In this blog, I outline our vision and approach to advancing n=1 treatments for patients who have no alternative treatment options. Central to this effort is our AI-enabled RNA platform and its alignment with the scientific and regulatory requirements of individualized medicine.

Therna’s approach is built on the premise that RNA can be designed with extreme precision. We have developed an AI-driven platform trained on massive amounts of proprietary experimental data generated in our labs, enabling us to interrogate how RNA behaves in biological systems.

When creating the model, we asked questions about how RNA interacts with its biological environment and how those interactions influence behavior, translation efficiency, tissue and cell type expression and longevity. We also examined RNA structure, stability, and interactions with RNA-binding proteins, as well as how DNA, RNA, and proteins work together within the cell. As a result, our AI model with RNA intelligence can generatively design RNA sequences with characteristics that we want. For example, we can program the mRNA to be expressed exclusively in a tissue or cell type of interest, to be more durable, and achieve high translation efficiency to protein output. Similarly, we can find unique target sites within the transcript and design small oligonucleotides, such as ASOs and siRNAs, to increase, decrease, or fine tune the expression of the mRNA. Rather than producing transient biological signals, these engineered RNA sequences are designed to function as therapeutics.

By integrating computational design with experimental validation, we can generate better RNA medicines faster and translate those designs into candidates suitable for development. This approach has the potential to improve the cost-efficiency of developing highly individualized therapies, enabling programs that would not have been feasible using conventional drug discovery approaches. In doing so, it opens the door to programmable RNA medicines with previously unattainable properties.

While single-patient programs are an important application of our platform, Therna is advancing a broader pipeline of RNA medicines across multiple disease areas with significant unmet need. These efforts leverage the same design and validation framework to develop therapeutics at scale.

Our rationale for advancing individualized medicine is grounded in four core principles:

First, these patients are often overlooked and have no viable treatment options. There is limited commercial incentive for industry to pursue such therapies, and academic institutions typically lack the resources to develop them at scale. We believe there is a responsibility to address this gap and bring forward solutions where none exist.

Second, our AI-enabled platform allows us to rapidly design and validate effective RNA medicines. For these patients, time is of the essence. We have demonstrated the ability to design and experimentally validate candidates in under three months and advance them into pre-clinical development – an unprecedented acceleration of traditional drug development timelines.

Third, each program generates highly valuable data that both validates and strengthens our platform. From initial RNA design through experimental testing and clinical validation, these data improve our models and expand their predictive capabilities. Over time, this learning builds, benefiting not only future n=1 patients but also our broader therapeutic programs.

Fourth, advancing these programs helps establish a viable framework for individualized medicine. While these efforts are less resource-intensive than traditional drug development due to the absence of large clinical trials, they still require meaningful investment. Demonstrating what is possible is an important step toward enabling broader adoption of individualized approaches for patients who need them.

Therna’s approach to individualized medicine reflects a convergence of scientific innovation, regulatory evolution, and urgent patient need. By combining AI-enabled RNA design with rigorous experimental validation, we are establishing a new model for developing therapies for patients who have historically been left behind by traditional drug development. These early programs not only provide a path forward for individuals with ultra-rare diseases, but also generate the data and experience needed to scale this approach more broadly. As the field continues to evolve, we believe Therna will be at the forefront of developing individualized RNA medicines, expanding what is possible for patients and redefining how therapies are developed.

RNA medicines have already transformed medicine, but the next phase requires deeper control. Control over tissue specificity. Control over durability. Control over safety and expression. Biology has always held these answers, but only recently have we had the computational power and data scale to uncover them. This is the moment Therna Biosciences was built for.

From day one, we set out to integrate deep RNA biology with generative AI in a way that respects biological complexity rather than flattening it. At Therna, we have built an AI platform to design novel RNA medicines that are longer-lasting, safe, and more effective. Our approach is grounded in a lab in the loop system, where models are continuously informed by carefully designed experiments and high quality biological data. Asking the right biological questions has been just as important as building the right models.

What makes Therna Biosciences different is that we are not only an AI company applying models to biology. We are deeply rooted in the data we generate. Our experimental systems are designed to produce biologically meaningful, high resolution datasets that directly shape our models. Better data leads to better models, and better models allow us to ask more precise biological questions. This feedback loop is core to how we operate and why our platform continues to improve with every cycle.

After two years of stealth operation, we launched the company officially this year by securing seed funding from investors (Pear VC, AIX, and Fusion Fund) who believed in our mission and vision. We assembled an experienced team that brings together deep expertise in RNA biology, AI, and translational and preclinical science to streamline the AI-powered discoveries into reality by designing next generation RNA medicines.

We were proud to be accepted into both the NVIDIA Inception program and the Google for Startups Cloud Program. These partnerships reflect the seriousness of our technical ambition and have strengthened the computational foundation behind our work. As part of our collaboration with NVIDIA, we are working with CodonFM, the new RNA foundation model that learns the rules of RNA by reading sequences in biological units called codons to reveal patterns that matter for therapeutic design. Integrating Therna’s proprietary RNA biology data with this class of models and world class compute infrastructure helps accelerate the development of programmable RNA medicines in ways that were not possible before.

Perhaps most importantly, we validated our core premise. RNA is a language, and it can be learned and engineered. With the right data, the right experiments, and the right models, RNA can be designed with intent. Fit for purpose mRNA. Smarter targeting strategies. Faster iteration cycles. Work that once took years can now happen in weeks.

As we look ahead to 2026, the opportunity feels even larger. We are excited to scale our platform, expand the scope of what programmable RNA can do, and translate our advances into medicines that matter. This is still the beginning, but the foundation is strong, and the momentum is real. Therna Biosciences is here to become The RNA company.

Thank you to our team, our partners, and everyone who believes that the future of RNA medicine can be written with purpose.

We are just getting started.

Subscribe now

Reality has been more complicated. Across benchmarks, far smaller supervised models often outperform large pretrained ones. For some, this has led to the conclusion that scaling has failed in biology (or at least to a questioning of the underlying premise).

I think that conclusion is premature. Scaling is alive, but it plays out differently in biology than in text or images. Success depends on scaling the right axes — not just parameters, but context length, data diversity, objectives, tokenization, and architecture. And it depends on evaluating models fairly, with methods that reveal their strengths rather than obscure them.

In this post, I am laying out why scaling still matters for biology, why in fact foundation models are crucial for bringing biology into the next decade, and more importantly what dimensions we should be scaling, and how to think about evaluation.

Supervised Models Excel with Abundant Data, but Biology Is Far More Complex

In genomics, supervised models excel where labels are plentiful. Human and mouse genomes have benefited from massive data-generation consortia. Disease genetics, ENCODE-like catalogs of regulatory activity, popular immortalized cell lines; these are rich ecosystems where supervised models can thrive. Feed them thousands of transcriptomic and epigenomic tracks and they deliver excellent performance.

But biology extends far beyond this tight circle. Step into zebrafish, plants, or microbial consortia, and labeled data quickly thins out. Assays become noisy, annotations sparse, and sample sizes small. Training high-capacity supervised models is simply not feasible in these settings.

This is where foundation models matter. Pretraining across species and contexts lowers the barrier in data-poor domains. A model like Evo 2, trained on thousands of genomes across the tree of life, carries useful priors even for organisms studied by only a handful of labs. Its broad evolutionary grounding allows transfer: patterns learned from one species help interpret another. And remarkably, Evo 2 achieves state-of-the-art performance on both non-coding and coding variant prediction in humans despite being trained only on reference genomes, not on expensive omics tracks.

The lesson: supervised models may be dominating in rich-data niches for the time being. However, foundation models are indispensable in sparse ones. And it should be clear to everyone that much of biology is data-sparse.

Benchmarking Pitfalls: Why Evaluation Choices Matter

I feel like there is a tendency in our field to downplay the value of foundation models. But in reality, benchmarking of models requires care and attention. When foundation models are benchmarked, the results often hinge on methodological details that are easy to overlook.

1. Layer Choice Is Not a Nitpick

Transformers do not distribute information evenly across layers. Early layers capture local features; middle layers often encode rich structural and functional signals; late layers tilt toward autoregressive objectives. Picking the wrong layer to probe can make a model look weak.

Yet many benchmarks report results from a single layer or from a naive averaging of token embeddings. That choice can flip conclusions. For Evo 2, for example, the authors show that probing layer 20 yields excellent classification accuracy on BRCA1 variants; whereas, probing layers 1-3 produces far worse results. Without systematic layer sweeps and thoughtful pooling strategies, comparisons between models become unreliable.

2. Cropping Results Misses the Full Story

Benchmarks like TraitGym illustrate another problem. Evo 2 lags behind specialized models on some complex traits but outperforms them on Mendelian ones. This makes biological sense: Mendelian traits are often driven by strong-effect variants in coding or conserved regulatory regions, where evolutionary pretraining is powerful. Complex traits, by contrast, are polygenic and data-rich, favoring supervised models tuned to phenotype-specific signals.

But over and over again, critics simply share a cropped version of this evaluation. Sharing only the right-hand panel of a benchmark, the slice where supervised models win, gives a skewed impression. Reporting full grids reveals the trade-offs: foundation models recognize evolutionary disruption; supervised models capture subtle statistical associations. Even in humans where we have plenty of data, both stories are true, and both matter.

Noise Today Can Be Structure Tomorrow

One common critique is that most of the genome is “noise.” Intergenic regions make up ~75% of mammalian genomes, and many positions seem unconserved. If only ~10% of nucleotides are conserved across species, the argument goes, why waste model capacity predicting the rest?

This view misrepresents regulatory biology. Intergenic regions are not uniform deserts. They are patchworks of functional elements embedded in less constrained sequence. Enhancers, silencers, and insulators sit scattered across these expanses, looping over vast distances to control genes. Chromatin domains partition the genome into 3D compartments that matter for transcriptional regulation.

The ENCODE 2025 encyclopedia catalogs millions of candidate regulatory elements in human and mouse alone. Many of these lie in “non-conserved” territory. Sequence-level conservation is not the right proxy: regulatory function often persists through network-level conservation, where different motifs evolve to achieve the same control logic.

When models struggle to learn from intergenic regions, it does not mean those regions are meaningless. It means our objectives are insufficient. Absent large swaths of data, standard next-token prediction may miss distal and sparse dependencies. Here, addition of supervised tasks, predicting chromatin contacts, enhancer-promoter loops, or expression changes under perturbation, could teach models to extract real structure from apparent noise.

Scaling Beyond Parameters

In public discourse, scaling is often shorthand for “adding parameters.” Bigger is better. But in biology, parameters are only one axis, and sometimes not even the most important one.

• Context length. Many regulatory interactions span hundreds of kilobases. Standard autoregressive training with limited windows will miss them. Long-context models like Evo 2 and AlphaGenome, with million-token windows, show why context scaling can be more impactful than sheer size.

• Tokenization. Single nucleotide tokens are simple but not necessarily optimal. Codons, kmers, or adaptive schemes can encode biological structure better, letting smaller models outperform larger but poorly tokenized ones.

• Readouts. Linear probes on the right intermediate layer can beat more complex architectures on the wrong one. Benchmarking must separate engineering choices from fundamental model limits.

• Data breadth. A model trained only on human data will struggle with plants. One trained only on coding sequences will miss regulatory grammar. Scaling species diversity, assay types, and experimental contexts often delivers larger gains than adding layers.

Scaling in biology is multi-dimensional. The critical question is not “how big is the model?”, but “what axes are we scaling, and are they aligned with biological signals”?

Architectures for Biology, Not Just Borrowed from Language

Language models advanced because architecture and data evolved together. Self-attention unlocked long-range dependencies; tokenization improved; datasets scaled; instruction tuning taught models to use their knowledge.

Biology will follow a similar trajectory, but with its own twists. The signals we care about are inherently bidirectional, multimodal, and structured in 3D.

• Bidirectionality. Enhancers regulate promoters upstream and downstream. RNA folding depends on base-pairing across both directions. Autoregressive models that only look backward miss half the story.

• Long-range, sparse interactions. Unlike text, where nearby words matter most, biological regulation often skips over large spans. Sparse attention or hierarchical models may capture this better.

• Multi-scale patterns. Chromatin domains, local motifs, 3D genome structure; biology is layered in ways language is not. Standard transformers may not capture these efficiently.

We likely have not yet found the optimal architecture for biological sequences. Borrowing directly from NLP will take us part of the way, but breakthroughs will come from designs tailored to biological invariances.

The Data Frontier: Diversity, Perturbations, and Synthetic Biology

In our field, there is also a sense that we are approaching the limits of meaningful biological data. This is simply wrong. OpenGenome2 and similar collections are milestones, not endpoints. Sequence diversity is vast; microbial communities, metagenomes, plants, and understudied clades remain under-sampled. Functional diversity is even larger. Perturbation assays, multi-modal single-cell data, and time-series measurements are only beginning to scale.

But most importantly, we are no longer limited to natural data. Genome-scale generative processes and genome foundries mean that we can design synthetic sequences and test them at scale, effectively creating new training data beyond what evolution provides. Evo itself demonstrates this: models that generate and evaluate synthetic variation expand the dataset in directions natural diversity never explored.

Data growth in biology is not about chasing the trillion-token thresholds of NLP. It is about expanding along the biologically relevant axes: species, conditions, modalities, perturbations, and designed diversity.

Foundation Modeling Is Needed More Than Ever

Taken together, these points argue for a reframing. Scaling in biology is not dead. It is conditional.

• Foundation modeling helps most where data is sparse.

• Scale fails if we probe the wrong layers or delude ourselves by cropping benchmarks.

• Scale needs objectives that respect biological regulation.

• Scale is multi-dimensional: context length, tokenization, data breadth, architecture, not just parameters.

• Scale depends on data diversity, including synthetic generation.

When these conditions are met, scaling delivers. Evo 2’s performance on variant interpretation, foundation models’ ability to transfer across species, and the early success of long-context architectures are all evidence. The story is not failure; it is refinement.

Conclusion

Scaling has always been about more than size. In biology, it is about matching objectives, architectures, and data to the underlying signals. Dismissing scaling because supervised models win in data-rich niches misses the broader picture.

The right framing is this: scaling is alive, but it must be given the right instructions, such as longer contexts, better tokenization, richer objectives, broader data. Under those conditions, scaling unlocks biological insight in places where no supervised model could even start.

Scaling works. We just need to scale the things that matter.