Most people who try to build a quant trading agent start with the same picture: one smart model, one score, one list that says buy this, skip that. Feed it data. Let it think. Watch it print answers. It looks clean. It looks like intelligence.

We built that. For a while it looked alive. Then it went quiet. The scary part wasn’t that it stopped working. The scary part was that the screen still said BUY.

This is not a sales pitch. We don’t sell stock picks. We don’t promise returns. This is the honest story of how we tried to find an edge — what we built, what broke, and why we now have three ideas in test and none of them is allowed to make the call.

If that sounds like a strange thing for an AI firm to say out loud, good. That’s the point.

The First Mistake: One Brain for the Whole Market

The first instinct is simple. You have a lot of stocks. You have a model. So you ask the model to rank everything — best idea at the top, worst at the bottom.

That’s the easiest agent to build. It’s also the first one that falls apart.

The model has to learn from the past: did this stock go up or not? Those answers come late, they’re messy, and they change when the market changes. For a few weeks the model and the market agree. The dashboard looks busy. You start to trust it.

Then the market changes. The model doesn’t. The number is still there. The ranking is still there. If you let that ranking pick size or timing, you are now taking advice from something that looks alive and isn’t.

We caught this by asking a boring question: can the model still tell good ideas from bad ones? When the top of the list stopped beating the bottom, we knew it had nothing left to say. The BUY stamp was still on the screen. The conviction was gone.

A lot of teams paper over that moment. We didn’t.

What We Actually Did

Building a quant trading agent, if you’re honest, doesn’t look like a product launch. It looks like a series of demotions.

1. We built the big ranker. One score for the whole market. For a while the scores lined up with what happened next. We wrote everything down.

2. We watched it die. The top names stopped beating the bottom names. The software was fine. No crash. No missing number. It just wasn’t saying anything anymore — and it was still talking.

3. We sat it down. We didn’t delete it. We moved it to the bench. In our shop that has a hard meaning: it can show up in a log. It can show up in a research note. It cannot pick the list, set the size, or approve a ticket. Bench is a job, not a polite word. The model lost the right to decide.

4. We tried a specialist. A second idea: maybe a narrower model, in calmer markets, would do better. Early paper results looked encouraging. Encouraging is not permission. We kept it as a research diary — write down the outcomes, review them on a Friday, don’t plug it into the desk. Nobody gets to say “it was right three weeks, turn it on.”

5. We tried an even narrower specialist. A third idea: only look at one kind of market at a time. Still in test. Still waiting on enough real examples and a person saying yes before it becomes anything more than research.

Three ideas. Three bets. Zero of them allowed to drive.

If you want the bigger picture of how agents check each other, these posts sit next to this one:

>

- Anatomy of a Multi-Agent Trading System
- Inside Project Cortex
- From Research Platform to AI Factory

Three Signals in Test — And Why We Won’t Blend Them

Call them by the job they have, not by a brand name:

  • (A) The old all-market ranker. On the bench. It can leave a note. It cannot sign the ticket.
  • (B) The calmer-market specialist. We’re keeping score on paper. Early signs look better in its lane. Still not promoted. Still not sizing anything.
  • (C) The one-sector helper. Newest and smallest. Not enough examples yet to keep a real scorecard. Still gathering evidence.

The temptation — we’ve felt it — is to mash all three into one number. Average them. Weight them. Make a cocktail.

We won’t.

If you mix them and the mix “works” for a month, you don’t know who did the work. When the mix breaks — and it will — you don’t know who died. You just built a new black box out of parts you made so you could see what was going on.

One idea with a real edge beats a smoothie. Finding edge means isolating it, not hiding it.

What Went Wrong Along the Way

These are things we hit, or caught before they did damage.

Not enough answers from the market. The model only learns after the fact. Train on a thin set and it memorizes last month. That’s what happened to Signal A: sharp in one kind of tape, silent when the tape changed.

A dead number that still shows up on time. This is the dangerous one, because nothing looks broken. No error. The score still arrives. It just doesn’t mean anything. If the rest of the system only asks “did a number show up?” you will keep acting on noise. We watch whether the top of the list still beats the bottom. When that gap dies, the signal is flagged. A flagged signal cannot approve a ticket.

A plumbing problem dressed up as “the market is bad.” Sometimes the model goes quiet because the market really is hostile. Sometimes it goes quiet because a data feed went stale or a pipe broke. Both look like a flat screen. They are not the same. We check the pipes before we trust the model.

The Wednesday tweak. Nothing is setting up. Someone says: loosen the rule, add a filter, just for this week. That isn’t research. That’s making a trade so you don’t have to sit on your hands. Quiet is allowed. If the model has nothing to say, we stand down.

What “Edge” Is Allowed to Mean Right Now

While all three ideas are in test, the only “edge” we’ll claim in public is basic weather:

  • Is this part of the market even open for business?
  • Is the data clean?
  • Did the safety checks pass?
  • Did a person say yes?

That is not a model calling the shots. That is hygiene. It answers whether we’re allowed to operate today — not what the trade should be.

For a model to drive, it has to earn it:

  1. Proof on paper first. Not one good week. Not a pretty backtest. Enough real examples, checked after the fact, on a regular schedule.
  2. A person says yes. A name and a date. Not a timer. Not a script.
  3. No automatic promotion. Being right for a while is necessary. It is not enough.
  4. If it wobbles on the way up, start over. Stale data, a dead gap between good and bad ideas, missing numbers — the clock resets. You don’t average away a bad week.

This feels slow. It is slow. Promoting a bad idea costs more than waiting another month.

Questions to Ask Anyone Selling You a Trading Agent

Skip “what’s the hit rate?” and “what did it pick last week?” Ask these.

What is allowed to decide? Which part can actually approve a ticket? Which part can only leave a note? If they say “the model suggests and then we review,” ask whether that suggestion comes with a rank, a size, or a clock. If it does, the model is deciding. Say so.

What is only allowed to leave a note? A benched model should be blocked from ranking and sizing — not “we usually ignore it.” If it can still write the list, it isn’t on the bench.

What is the bar to go live? If there isn’t a written bar, there isn’t a bar. Look for: enough examples, proof after the fact, more than one kind of market, a human sign-off, and a pause between “looks good” and “turn it on.”

What happens when the model goes quiet? Not when it crashes — when it just goes flat. Do they switch to a backup that was never promoted? Use last week’s number? Keep trading because the market is open? Walk away from that last one.

Can they show you a time they said no? A real day when a ticket could have been written and the system refused — bad data, failed check, dead model. If they can’t show one, it has never been tested. Or it has never said no.

Where We Are

We have three signals in test. None of them is the live brain. We are not embarrassed by that. We are not in a rush to change it.

The hard part of building a quant trading agent is not the math. It is not the data pipes. It is not even finding an edge — though that is genuinely hard. The hard part is sitting still when you don’t have one, and refusing to pretend otherwise because the screen still printed a number.

Our default is do nothing. Every action needs a real reason, written evidence, and a person who said yes.

That’s where we are. When something earns the wheel, we’ll say so. Until then, we’d rather be honest about the process than impressive about the product.

We don’t sell trading signals. We don’t promise returns. If you’re building decision systems in careful environments and want to talk about how this actually works: hello@llamaresearch.ai.