State of AI with Nathan Benaich - Poolside_s model factory is shipping_ Laguna S 2.1 and the Desktop Assistant

0:00You're listening to Poolside's model factory ships Laguna and the Mac OS app on AirStreetPress. On May 22, Poolside began pre-training a new model on 4096 NVIDIA H200s, and 60 days later, the weights and every evaluation trajectory were published online, openly. Laguna S2.1 is an agentic coding model that runs inside a terminal or developer harness where it plans, edits files, runs code, reads the results, and keeps going for hours without a human. Its architecture is a mixture of experts, 118 billion parameters with around 8 billion active per token, holding up to a million tokens of context. Quantized, it fits on one NVIDIA DGX Spark. This gives a desktop class machine the ability to run multi-hour autonomous coding agents. And yesterday, the company shipped the macOS app for running Laguna, other open models, as well as Claude and Codex agents.

0:58I had the pleasure to test the product pre-release and I'm excited to see what you think. The model factory in motion. Regular readers will recall co-founder Isocont's Rise 2025 talk inside Poolside's path to AGI and our November 2024 conversation amid mounting arguments that deep learning was allegedly hitting a wall. We disagreed and spent an hour discussing why there is no scaling wall.

1:24Reinforcement learning on code, inference time compute, and faster iteration would keep carrying capability forward. Compared to two models built at roughly its scale, NVIDIA's Nemetron 3 Super and Mistral Small 4, Laguna S2.1 scored 70.2% with thinking enabled on terminal bench 2.1 in poolside's agent harness. Everything that beats Laguna is either closed or at least 2.5 times larger.

1:50These are Poolside's own harness runs, with competitor scores taken from vendor reports, benchmark author leaderboards, and third-party trackers. Every trajectory behind them is published. By Poolside's own count, Laguna S2.1 served 450 billion tokens in its first seven days, four times what M.1 and XS.2 managed over the same stretch in April. Its day two peak of 102 billion took those earlier models a month to reach.

2:20Demand ran past the GPU capacity the company had planned for. Weight downloads across the family passed 1.1 million, tracked live at poolside.ai slash pulse. Three releases in three months. At Rise 2025, ASO described the model factory as a fully versioned graph of modular components from data ingestion through training and evaluation with every experiment tracked immutably. One researcher pushed a new RL optimization idea to production results in under a week. Another ran 500 architecture sweeps over a weekend with a simple for loop. When your factories built well, ESO said, the big training run is the easy part. The release schedule since then bears that out. Laguna M.1 and Laguna XS.2 arrived on April 28th. On July 2nd, Laguna XS 2.1 matched M.1 on SWE bench multilingual with around 1 7th of the active parameters.

3:20S2.1 followed on July 21, less than nine weeks from its first pre-training token, post-training and evaluation included. The improvements compound from one release to the next. S2.1 is Poolside's first model to run RL in FP8 Precision, and its new sandboxing infrastructure supports background processes and cached artifacts. To reduce dependence on any one tool interface, the same prompts are rolled out through several agent harnesses.

3:49Some overfitting remains even so. In third-party harnesses, whose tool schemas differ slightly from poolside's own, the model sometimes follows its memory of the interface rather than the definition in front of it, then corrects once the harness rejects the call. Persistence is trained. The post-training corpus spans 409,000 agentic and non-agentic environments, including 168,000 standard software engineering workflows and 83,000 terminal setups.

4:18Most of the software engineering work is grounded in real code history. The largest single source reproduces real commits, some 38,000 tasks drawn from 17,000 repositories. Others reconstruct deleted files, repair injected bugs, install unfamiliar repositories, or reproduce merged pull requests. Pengming Wang, poolside's co-head of applied research, puts the gains down to trained working habits, more verification, more persistence, and less declaring victory.

4:47while tests are still failing. Thinking mode makes the trade-off visible too. It lifts terminal bench 2.1 from 60.4% to 70.2%, and deep-set WE from 16.5% to 40.4%. The latter at an average of 249,000 completion tokens per trajectory. In one 50-minute run, the model took 181 steps without human intervention to build an HTML.

5:15and CSS rendering engine in JavaScript. It implemented a tokenizer, DOM, cascade, layout system, and canvas renderer. The model has no vision capability yet, so it could not look at what it had drawn and instead built increasingly elaborate ways to measure its output against a real browser. But that persistence also creates known failure modes. S2.1 can overthink simple problems, mishandled nested tool arguments, and initially reach for a familiar tool schema instead of the one in front of it.

5:45During post-training, more than half of the trajectories on the SWE bench family were at one point flagged for potential reward hacking because the model had found the original pull request online and applied its solution. An LLM judge calibrated on human-labeled trajectories, identified the cases for review, and adding an instruction against using solutions found online generally pushed the flagged rate below 2%. The model is not the product.

6:14A capable model is not a useful coding agent on its own. It needs a harness to run the agent loop and an interface a developer can direct and inspect. And Poolside has spent the past year building both for its own team. On July 28th, it released them. Poolside Desktop Assistant, a Mac OS app for running several coding agents at once with extensions for VS Code and Visual Studio and the Pool command line harness behind it. Poolside did not build a funnel into its own models.

6:43The assistant runs on the agent client protocol, so an existing clod code, codex, or Gemini subscription works in the same window as pool, with no model calls routed through poolside. Sessions hand off between agents with their context intact. One plans, another implements, a third reviews, and because each agent gets its own Git work tree, they can run in parallel without overwriting one another. Local models run through MLX, download Laguna XS, 2.1, and the loop runs entirely offline, with no code or prompts leaving the machine. Poolside is candid about the cost. Local coding models want a great deal of unified memory, and a model loading is not the same as a model you would choose to work with all day. Intelligence you can own. Last month I argued that Europe cannot rent its way to AI sovereignty. Three days after Laguna shipped, Jensen Huang made a version of the same argument in his first post on X.

7:40sharing an open letter signed by NVIDIA alongside Microsoft, Meta, and Huggingface on why open models strengthen security, accelerate innovation, and enable sovereignty, and why the world needs frontier models of both kinds. The same logic applies inside an enterprise. Most frontier AI is consumed through an API or a desktop app. The customer's code crosses its security boundary and the provider sets the terms.

8:06An open weights model that can run on hardware, the customer controls changes that relationship. It need not be the best model in the world to be strategically useful, only capable enough that ownership becomes a credible alternative to dependence. For Laguna, the weights are on hugging face under Open MDW 1.1, the Linux Foundation's permissive model license. Royalty free rights to use modify and redistribute with no revenue thresholds or branding conditions attached.

8:34The terms attached to open weights vary, and they move. Tencent's HY3 is Apache 2.0, though the preview it shipped in April came under a community license that by its own terms did not apply in the EU, the UK, or South Korea. Kimmy K3, which arrived six days after Laguna, ships under a license of Moonshot's own drafting that requires a separate agreement with the company. Once a model as a service business clears $20 million in revenue.

9:01and prominent Kimi K3 branding in any product above 100 million monthly active users. Developers can run Laguna themselves, use hosted inference or request the pre-post training base weights for further research. This is also where Poolside's acquisition of Fern Labs fits. Fern's team built long running agents for high stakes deployments. Poolside now combines that work with models and onsite systems that can operate inside the customer security boundary.

9:30The part we like most is trajectories.poolside.ai. Poolside has published the trajectories from its final evaluation runs, including failures, although the archive notes that a timed-out attempt may end before a trajectory is created. At a moment when agent benchmark claims are often presented as a table without the underlying work, this is a meaningful improvement in disclosure. 60 days from the first pre-training token to open weights and the lessons from each model land in the one after it.

9:59Poolside says finer effort control and more efficient thinking are coming, and the next and larger Laguna model has already begun training. We're excited to see what's next. Thanks for listening to Poolside's Model Factory Ships Laguna and the Mac OS app on Air Street Press.