What If One Scientist Had an AI Army?

AI tools are easier to get than they have ever been. You open a tab, paste in a question, and get back a confident paragraph. Knowing whether that paragraph is useful still takes expertise. Nobody has automated that part.

That gap was the thread through HITL #31. Oscar and I sat down with AI researcher Enrique Noriega to talk about how language models became agents, how his lab pulls knowledge out of biomedical papers, and why evidence matters most when a model sounds most certain. It's a guest episode: one expert and one long conversation, with no news rundown.

Access is not expertise

The episode's title asks what happens when one scientist gets an army of AI agents. The honest answer depends on the scientist. An agent can read faster than any person, but it can't tell you which finding matters. If you don't already know what a good answer looks like, a thousand agents just get you to a wrong answer faster.

The builder version of this is something I keep running into with clients. Building an MVP and scaling a reliable system take different skills. A demo only has to work once. A system has to work on the ten-thousandth input, on the weird edge case, and on the day the upstream data changes shape. AI makes the demo cheap. It doesn't make reliability cheap.

Turning papers into knowledge graphs

The part I found most useful was how Enrique's lab works: turning genes, proteins, and the relationships between them into knowledge graphs. A model reads a paper, finds statements like "protein A regulates gene B," and records them as structured links you can query.

The catch is context. A relationship that holds in one disease, one tissue, or one experimental setup may not hold anywhere else. If extraction strips that context away, you get a graph that looks authoritative and quietly says things the papers never claimed. Most of the hard work is in carrying the context along with each finding.

RAG, agents, and what counts as ground truth

We spent some time on the stack. RAG grounds a model's answers in actual source documents instead of whatever it picked up in training. Agents add planning and tools on top: they decide what to look up, call something, check the result, and keep going. Each layer is useful. None of them makes the model right by default.

That leads to the uncomfortable question: right compared to what? In biomedicine, ground truth comes from curated databases and wet-lab evidence, and both are incomplete. Some relationships are real but nobody has curated them yet. Some curated entries rest on thin evidence. If your evaluation set has gaps, your accuracy numbers have the same gaps.

Accurate but missing the point

This is the idea I'll be thinking about all week. An AI summary of a paper can be accurate, with every sentence true, and still miss the paper's main point. It can give a side result the same weight as the headline finding, or drop the limitation the authors spent a page on.

That failure doesn't show up when you spot-check facts. It only shows up when you already know what the paper was about. That brings us back to the episode's opening point: access isn't expertise.

Research, products, and who gets access

We also talked about the difference between research work and product work, where training data comes from, and what AI-assisted healthcare research could change. Personalized treatment is a real possibility, but costs are real too. Who gets access matters as much as what becomes possible. A breakthrough that only reaches a few people is a smaller breakthrough than the headlines suggest.

We closed with a brief introduction to Enrique's telescope project. Technical issues cut that conversation short, so the episode ends with a short notice and our outro. I'd like to get him back to finish it.

Try this after listening

Pick a paper you know well. Write down its main claim and its biggest limitation. Then ask an AI to summarize it and compare. Does a finding from one cell line keep that context, or does it turn into a claim about people?

Then tell me: what has an AI summary missed in a subject you know well?

Human in the Loop — Watch on YouTube · Listen on Spotify · Listen on Apple Podcasts

Every episode — full show notes, sources, and the archive — lives at podcast.vallyseed.com. The show is produced by VallySeed, where Oscar and I help organizations design, build, and deploy AI systems that create measurable competitive advantage.