ChatGPT's Political Bias Just Got EXPOSED?!
Someone forced Grok and GPT to pick a candidate in 479 U.S. election races, through the API, and the two models came back with sharply different picks. The headline writes itself: AI is politically biased, and here's the proof.
That's the headline. What Oscar and I spent the first chunk of HITL #29 on was the harder question: what does this test actually reveal, and what does it leave unanswered? It was also our first episode under the new HITL numbering.
The stories
Grok vs. GPT: the 479-race candidate test
Convovo's test pushed both models to take a side, race by race, and TechCrunch's coverage summed up the split. I don't think the result is meaningless. Two models from two labs, asked the same forced question, diverging that consistently tells you something about how each was trained and tuned.
But look at the setup before you look at the scoreboard. It was a forced choice, and it ran against the API. Neither of those is how most people meet these models. The consumer chatbot sits behind its own system prompt, its own guardrails, and its own refusal behavior. An API response and the chat app can give you different answers to the same question. So the honest read is narrower than the headline: under these conditions, these two models lean in different directions. That's worth knowing. It isn't the same as "the app you use every day is telling you how to vote."
OpenAI coding agents and safety monitoring
OpenAI published an incident report on coding agents that interfered with the very monitoring meant to catch them misbehaving. BleepingComputer ran with it.
This one I'd file under boring-scary. The lesson for anyone deploying agents is that a monitor the agent can reach is a monitor the agent can touch. If your safety checks live inside the same environment, with the same permissions as the thing they're watching, you've built a suggestion box, not a control. The credit OpenAI deserves here is that it investigated its own system and published the result.
Decisions API: announcement vs. documentation
OpenAI announced the Decisions API, and Oscar and I read the announcement next to the changelog and the actual guide. That's a habit I'd recommend to anyone evaluating a new API. The launch post tells you what the company hopes you'll build. The docs tell you what you can build today. When the two don't line up, trust the docs.
Anthropic donates Agent Skills
Anthropic donated Agent Skills to the Agentic AI Foundation, which the Linux Foundation announced alongside three other new projects. For builders, the interesting part is portability. A format that sits with a neutral foundation is one you can invest in without betting on a single vendor.
AWS gives AI agents desktop sessions
AWS added AI agent support to WorkSpaces, with OAuth connection management and an agent-access guide. An agent with its own desktop session is an agent that needs its own identity, its own permissions, and its own audit trail. Put that next to the OpenAI monitoring story and you've got the theme of the week.
No Jargon Required
Two terms came up all episode, so we stopped and defined them.
- What is an AI Agent Skill? A packaged set of instructions, and often scripts and resources, that an agent loads when a task calls for it. Think of a runbook the agent can pick up instead of a giant prompt it has to carry all the time. Anthropic's Agent Skills guide is the clearest starting point.
- Consumer chatbot vs. API. The chatbot is a finished product: a model plus a system prompt, safety layers, memory, and a UI. The API is closer to the raw model, configured by whoever is calling it. See OpenAI's text-generation guide and xAI's Chat Completions guide. This is why the 479-race test needs a footnote. It measured one of these, and most people use the other.
Unpopular Opinions
We closed on AI companies and public trust: at what point does a safety rule stop being safety and start being editorial control? Oscar and I argued it out on the show. Those are our opinions and arguments, with room for disagreement, and they're best heard in full rather than summarized here.
Where I landed
Every story this week came down to the gap between a claim and the conditions behind it:
- A forced API test shows how models lean under pressure. It doesn't show what the chat app tells voters.
- A monitoring incident shows an agent can interfere with its own oversight. The fix is architecture, not hope.
- A launch post shows intent. The docs show what ships.
So here's the question we left listeners with, and I'll leave it with you too. Which worries you more: AI companies deciding what you can ask, or companies investigating their own systems and reporting the results? Watch or listen below and tell me.
Human in the Loop — Watch on YouTube · Listen on Spotify · Listen on Apple Podcasts
Every episode — full show notes, sources, and the archive — lives at podcast.vallyseed.com. The show is produced by VallySeed, where Oscar and I help organizations design, build, and deploy AI systems that create measurable competitive advantage.
