State of AI with Nathan Benaich - Turning compute into intelligence

0:00Turning Compute into Intelligence with Ted Moskovitz, who leads the science of scaling team at Anthropik at Rise 2026. Anthropik bet on scale from the start. You can read it in the company's genealogy. Dario Amode was first author on Baidu's Deep Speech 2, back when stacking more data and more compute on a problem was still a contrarian bet. Ted Moskovitz now runs the team that turns that conviction into a discipline. It is called the science of scaling and its job, in his words, is to work out how to turn compute into smarter models. At Rise, we spent half an hour on what that actually involves, why he insists the word science is doing real work, and how a frontier lab turns scaling into an empirical discipline for cutting uncertainty before it spends the compute. A science, not an art. Ted is firm on the framing. These systems are complex, he says, but they are understandable. The catches that understanding them demands epistemic humility.

1:01because there are always too many variables in flight, and running a controlled experiment is hard. The skill he prizes most is knowing the limits of your own evidence. Knowing what an experiment tells you and what it doesn't tell you is important. That sounds modest. In a field that markets every result as a breakthrough, it is closer to radical. His own path ran through linguistics and neuroscience, but the thing that carried over, he's clear, wasn't brain-inspired architecture. It was the scientific method itself.

1:30rigor, skepticism, doubting your own results. The job also changes relationship to curiosity. In a PhD, you pull on a thread because it's interesting. In a frontier lab, the question is colder. What is the cost benefit? And is the answer going to be interesting but ultimately less useful to making Claude smarter? Experiments that survive that test get fed up to the people deciding the big training runs, where the point is to cut uncertainty before committing the compute.

1:58The metric that matters is the counterfactual. I put a striking set of numbers to him drawn from Anthropics' recent essay, when AI builds itself. More than 80% of the code merged into Anthropics' code base is now authored by Claude. A typical engineer merges eight times as many lines of code per day as in 2024. And on one kernel optimization task, Claude's speedup climbed from roughly three times to 52 times in under a year. The essay's own line was that, We have not yet seen that curve bend." So what, I asked. Is the honest measure of AI actually accelerating? Not benchmarks, Ted said. The real test is what changes the counterfactual. If you took Claude away and coded alone, what would you do differently? Concretely, it reduces to the number of human interventions that are required. How often a person has to step in, how often the model's first pass is the one you accept.

2:58By that measure, the line has moved fast. He pointed to OPUS 4.5 in November as a jump, and the Mythos class models in February as a bigger one. The shift is trust. The models still make mistakes, and you still have to check them. But you can trust them a lot more than you could before. In the labs, people have already stopped supervising every step. They run in bypass mode and let the agent work. When bigger models get cheaper. Does the future belong to one large model?

3:28doing everything, or to companies decomposing tasks across smaller, cheaper ones? Ted leans hard toward the former, and the argument is economic, not sentimental. He cited Noam Brown at OpenAI who plots test time compute curves with cost on the x-axis. A bigger model that reaches an equally good answer in far fewer tokens can come out cheaper than a small one grinding away. It could be more cost effective to just ask the bigger model. His blunter version, People underrate the value of having a really smart model to ask questions to. The counter pressure is real. At Ramp, staff are nudged away from using Opus 4.8 to write emails when Sonnet will do the job. But his bet is that raw intelligence keeps winning on cost as well as quality. The next axis is taste, the frontier. The conversation kept returning to his judgment. The same essay claim models now pick the better next research step about 64% of the time, up from 51% in November.

4:31How do you build a system with taste? Here, Ted turned characteristically vague. He wouldn't say what goes into it, but he would say that when his team started using Mythos this year, it felt like it had better research taste than anything before it. With one caveat, still not as good as your average researcher. Average researcher where, I asked, maybe at Anthropic, a high bar to be measured against and one the models are now climbing. Safety is a capability, not a tax on it.

5:04Ask what safety research has ever done for product quality and TED reaches for cars, seatbelts, crumple zones, airbags, an oven that won't catch fire. Guardrails are what make the product usable at all. And a product nobody can safely use generates no feedback to improve it. The clearest case built the whole category. Reinforcement learning from human feedback began as a safety project, stopping a chatbot from spewing garbage.

5:31and turned out to be the thing that made chatbots good. Alignment and safety really go hand in hand with capabilities, he said. One encouraging trend I put to him is that bigger models are turning out easier to align, not harder. Ted's own framing was more guarded. It has gone different from how many safety researchers expected, he said. But his read is that people feel pretty good about the alignment situation right now, tempered by the obvious caveat that we don't know if we will cross some capability threshold.

6:01and it'll suddenly flip, hence the case for healthy apprehension, where the leverage is. I closed by asking what the highest leverage work in 2026 looks like. Ted was self-aware about his own bias and unhedged anyway. If you believe AGI is close, go to a frontier lab because your lever there is bigger than it is outside one. He counts himself among the converted. I'm definitely more AGI-pilled than when I joined. OpenAI and Anthropik are, he pointed out, still small companies where a newcomer can move things and the window matters. If you want to influence the direction these systems take, sooner is better than later. That includes London. Anthropics office here has gone from 15 or 20 people a few years ago to a couple hundred, with whole strands of frontier work, teds among them, run from the UK rather than mirrored from California. It doesn't feel like we're a satellite, he said.

6:59What the conversation kept circling back to is that the hard part of scaling was never the spending. Anyone can buy more compute. The discipline Ted's team is building is the other half, knowing what each experiment will and won't tell you, and turning that compute into capability you can measure and trust.