How an AI Agent Evolved a Polymarket Trading Bot On Its Own

Five winning trades in twenty minutes moved a balance from 150 dollars to 152 dollars. That is the whole financial result, and it is almost beside the point. The real story is that no human made those five decisions. An AI agent designed the strategy, ran it, scored it, kept what worked, and threw away what did not, all on its own. I am Madhuranjan Kumar, and I want to follow that self-improving loop from the trading experiment where it was demonstrated into a normal business, because the loop is the transferable part, not the trades.

The loop that started on a trading bot

The pattern comes from an auto-research project Andrej Karpathy shared, originally pointed at training a small nano GPT model. The idea is simple to state and powerful in practice: an agent that improves its own experiments without a person babysitting each step. In this build it was aimed at something completely different, a Polymarket bot hunting arbitrage on short-term Bitcoin markets.

Three pieces make it run. First, a GitHub repository acts as both the code and the research memory, because every experiment becomes a commit and the full history stays available to learn from. Second, a markdown file, a training program document, is the playbook. It defines how a candidate experiment gets chosen, how it runs, how it is scored, and whether it is kept or discarded. Third, the live environment is where each candidate strategy meets reality. The agent reads the playbook, updates the strategy code, spins up a new candidate, runs it, and a score evaluator compares the result against the best strategy kept so far. Beat the metric and it becomes the new baseline. Fall short and it gets discarded and logged so the agent will not repeat it.

The arbitrage detail matters because it shows why a clean score is everything. On the five-minute up-or-down Bitcoin market you could buy the up side for around 49 cents and the down side for around 50, paying 99 cents combined to win one dollar. Because both sides are bought, the trade cannot lose once it fills, so the win rate reads 100 percent by design. That guaranteed number is meaningless on its own. The real constraint is the fill rate, which sat near 50 percent in testing. Optimizing the vanity number would be a waste. Optimizing the fill rate is the actual work. One more guardrail carried the whole thing: because prediction-market data is very noisy, any unusually strong result got re-run once before the agent trusted it. That single confirmation step is the difference between compounding real gains and chasing a lucky fluke.

Now watch what happens when you lift that exact structure out of trading and drop it into an online store.

Chapter one: choosing the one number that actually matters

An online store selling home goods wanted to lift conversions on its best-selling product page. Before any agent got involved, the owner had to answer the same question the trading bot answered: what is the single score that reflects success? Not clicks, not time on page, not add-to-cart rate in isolation. The store settled on the conversion rate on that one product page, purchases divided by visitors, measured over a defined window.

This choice is where most businesses go wrong, and it is the same trap as the 100 percent win rate. It is tempting to optimize the number that always looks good. The store resisted that and picked the metric that maps to money. Everything downstream depended on getting this right, because the agent would relentlessly maximize whatever score it was handed.

Chapter two: writing the playbook the agent follows

Next the store wrote its version of the markdown playbook. It spelled out how a variant is chosen, how long each test runs, how many visitors it needs before a verdict is allowed, and the exact rule for keeping or dropping a variant. It also encoded the confirmation rule borrowed straight from the trading bot: any variant that looks like a large jump must be re-run before it is trusted, because weekend traffic and a surprise promotion can fake a win.

The store's product folder became the research memory. Every experiment would land as a commit, so the team could always open the history and see precisely why the current winning page won. That auditability turned out to matter more than anyone expected, because it meant nobody had to remember or argue about past tests. The record was the argument.

Chapter three: the first candidates go live

With the playbook in place, the agent started proposing variants and shipping each one to a slice of traffic. The first candidate was a rewritten headline. The second was a different hero image. The third was fresh button copy. Each ran against the current best version, and the score evaluator compared conversion rates once the visitor threshold was met.

Early results were modest and mixed, which is exactly what honest experimentation looks like. Some variants lost and were dropped and logged. One headline change edged ahead of the baseline and was committed as the new best. The important thing was that the store was no longer arguing about opinions in a meeting. It was watching evidence accumulate, one commit at a time, the same way the trading bot accumulated confirmed strategies. This is the muscle every business wants for its Facebook and Instagram ad campaigns too, where creative tests either earn their keep or get cut, and the winners become the new control.

Chapter four: the lucky spike that almost got trusted

Around the third week, a variant with a bolder product photo posted a huge jump in conversion. On the surface it looked like the breakout the store had been hoping for. Under the old, opinion-driven way of working, the team would have declared victory and rolled it out everywhere.

The playbook stopped them. The confirmation rule flagged the result as unusually strong and forced a re-run. On the second pass, with the promotional email that had inflated the first window now finished, the variant performed only slightly better than baseline. It was a real but minor improvement, not the breakout it first appeared to be. That is the trading bot's noise guardrail earning its keep in a completely different setting. Without it, the store would have anchored its entire page on a holiday spike and quietly lost ground for months. With it, the store kept the small genuine gain and moved on, unfooled.

Chapter five: twelve weeks of compounding

The value of the loop is not any single test. It is the compounding across many of them. Think of the page's effectiveness on an illustrative scale where the starting version scores around 40. After four weeks of proposing, scoring, keeping, and confirming, the accumulated small wins on headline, image, and layout lifted that to roughly 62. By week twelve, with dozens of experiments logged and only the confirmed winners kept, the page sat near 81 on the same scale. Those numbers are illustrative rather than measured, but they capture the real shape of this method: not a single dramatic jump, but a steady climb built from many small, verified improvements that never regress, because losers are logged and never revisited.

What changed for the store was not just the conversion number. It was the way decisions got made. The page now evolved on evidence instead of on whoever spoke loudest in the room. The full commit history meant a new team member could read exactly how the current design came to be. And the agent kept running quietly in the background, proposing the next candidate while the humans worked on things only humans can do.

Why the confirmation step is the soul of the whole thing

If I had to keep only one idea from this build and throw away the rest, it would be the confirmation re-run. Everything else, the repo as memory, the markdown playbook, the score evaluator, is machinery you could reinvent in a dozen ways. The confirmation step is the discipline that keeps the machinery honest, and it is the exact thing most people skip when they try to build a self-improving system.

Here is the trap it protects against. Any process that keeps the best result it has seen will, over enough tries, get fooled by luck. Run enough experiments in a noisy environment and one of them will look brilliant purely by chance, a variant that caught a holiday, a promotion, or a random good week. A naive system commits that fluke as its new baseline and then spends the following weeks defending a result that was never real. The re-run breaks the spell. By forcing any surprising winner to prove itself a second time, on fresh data, the system separates the genuine improvements from the accidents. The store's bolder-photo variant is the perfect illustration: it looked like a breakout, failed the second pass, and was correctly filed as a small real gain rather than a big fake one. Without that single guardrail, a self-improving loop does not compound. It drifts, chasing ghosts, and slowly gets worse while congratulating itself. With it, the loop only ever locks in things that survived scrutiny, which is why it climbs instead of wandering.

What any business can copy from this

The trading bot and the store are the same machine wearing different clothes. Both start with a repository as memory, both follow a plain markdown playbook, both use a score evaluator to keep the winner, and both re-run surprising results before trusting them. Strip away the domain and you have a template that fits an enormous range of work. Ad-creative tests, pricing experiments, email subject lines, product-page layouts, onboarding flows, and landing-page variants all share the same shape: a clear score, a way to run a trial, and a history to learn from.

It is worth being clear-eyed about where this loop fits and where it does not. It shines wherever you can define a clean score, run many small trials cheaply, and keep an honest history. It struggles wherever success is fuzzy, trials are slow or expensive, or the environment shifts so fast that yesterday's winner is irrelevant today. That is not a flaw in the method, it is the boundary of it, and knowing the boundary keeps you from pointing it at problems it cannot solve. A store optimizing a product page is well inside the boundary. A business making a single irreversible bet, like a rebrand or a one-time launch, is outside it, because you cannot run that a hundred times and keep the best. The skill is recognizing which of your decisions are the repeatable, scorable kind, because those are the ones a self-improving loop quietly compounds while you sleep.

The method reaches beyond conversion, too. That same content-testing foundation feeds SEO and organic search when the winning headlines and product descriptions become the versions search engines index. And the moment a variant lifts conversions, the extra orders flow into the CRM and website stack where follow-up automation handles the next several touches, so an improvement on one page compounds into more repeat business downstream.

If you want to try this yourself, start small and start safe. Turn the folder where you keep experiments into the research memory so the agent can learn from every run. Write the experiment rules into a plain markdown playbook the agent follows on its own. Add the confirmation re-run before you trust any result in a noisy environment, because your traffic is at least as noisy as a prediction market. And prove the whole thing in a dry mode that risks no real money or meaningful traffic before you scale it up. The live trading demo was deliberately tiny, five clean trades that added two dollars, precisely because the dollar figure was never the point. The point was that the structure held, and that position size, or in a store's case traffic share, scales once the structure is proven.

You can absolutely wire these pieces together yourself if you enjoy the tuning and have the patience to pick the metric that truly reflects success rather than the one that always looks good. If you would rather have the loop designed, the playbook written, the score chosen correctly, and the whole thing proven safely for your specific business so it compounds without babysitting, that is exactly the kind of system I set up.