http://www.jjshanks.net/tags/benchmarks/ Recent content in Benchmarks on jjshanks.net Hugo en Tue, 28 Jul 2026 09:00:00 -0800 http://www.jjshanks.net/posts/personal-ai-benchmark/ Tue, 28 Jul 2026 09:00:00 -0800 http://www.jjshanks.net/posts/personal-ai-benchmark/ <p><strong>Quick Take</strong> I stood up a personal agentic AI benchmark using Inspect AI, with tasks running in Docker and OpenRouter as the default inference provider. A first run of six starter tasks against four free models already surprised me: ling-3.0-flash, a model I had never heard of, matched every score while using the fewest tokens.</p> <p>New models ship constantly, and I have no good way of telling whether any of them matter for what I actually do. The published numbers don&rsquo;t help; they&rsquo;re benchmarks I don&rsquo;t run, on tasks I don&rsquo;t have. Meanwhile my own setup runs on auto-pilot: Claude Code with Opus 5 for coding, ChatGPT 5.6 Sol for adversarial reviews, Gemini 3.6 Flash for personal things. It works, but I couldn&rsquo;t defend any of it with evidence.</p>