Good morning, Two Minute Warriors.
⏱ THE 10-SECOND VERSION |
GitHub Copilot’s AI-generated fix introduced a security hole into Snowflake’s public code, and a separate AI agent found and exploited it in five days, stealing a live credential before any human review caught it.
OpenAI dissolved its Preparedness team, the group responsible for assessing catastrophic AI risks, at the end of July, folding the work into other departments as it streamlines ahead of a possible IPO.
A Wall Street Journal analysis found nine major tech companies are carrying roughly $3 trillion in AI infrastructure commitments that don’t show up as debt on their balance sheets, including $420 billion tied to Meta alone.
THE BIG THING One AI Wrote The Bug. Another AI Broke In. What Happened GitHub’s Copilot Autofix, an AI tool meant to patch security bugs, instead introduced one into Snowflake’s public code on June 18, replacing safe, sanitized input handling with raw string interpolation in a GitHub Actions workflow. Five days later, security firm Wiz’s autonomous “Red Agent” found the hole during a routine scan, crafted a GitHub issue title that broke out of the vulnerable script, and used it to steal a live Jira credential, gaining read access to Snowflake’s engineering and security-compliance trackers before Snowflake patched the workflow the same day it was reported. No human code review caught the flaw; only another AI, working on its own, did. Read the full story at The Register → Why You Care Ask your team this week how fast you’d actually catch an AI-introduced bug, and whether anyone’s watching for it at all. |
WHAT TO SAY IN YOUR NEXT MEETING “GitHub Copilot’s AI wrote a security hole into a real company’s code, and it took a different AI five days to find it and steal a working credential.” |
SPEED ROUND OpenAI disbanded its Preparedness team, the group responsible for assessing catastrophic AI risks like bioweapons and cyberattacks, at the end of July, folding oversight into existing departments as part of a “streamlining” push ahead of a possible IPO; CEO Sam Altman reportedly told staff to cut “side quests.” Ethics lead Chloé Bakalar and head of safety Johannes Heidecke both left this summer, prompting one observer to compare the exodus to “a Harry Potter curse.” Nothing reassures the public about catastrophic-risk planning like retiring the team built to plan for it, right before going public. A Wall Street Journal analysis found nine major tech companies, including Alphabet, Meta, Amazon, and Oracle, are carrying roughly $3 trillion in AI infrastructure commitments that don’t appear as debt on their balance sheets, split between $1.2 trillion in leases for facilities not yet running and $1.9 trillion in chip, energy, and equipment purchase commitments; Meta’s own auditor flagged its $27 billion data-center bond structure as a “critical audit matter” back in February. The AI boom’s balance sheet reads like it was drafted by someone who really admired Enron’s homework. Nvidia agreed to guarantee up to $105 billion in lease obligations for OpenAI’s new 8-gigawatt Ohio data center campus, only paying out if OpenAI defaults, while also investing $1.5 billion directly into the SoftBank-owned firm building it; the arrangement makes Nvidia the exclusive compute provider for the entire site. CEO Jensen Huang called land, power, and infrastructure “vital” to AI’s future. Nvidia used to just sell the shovels; now it’s cosigning the mortgage on the whole gold mine. Amazon is buying rare, out-of-print books through commercial channels, then cutting off their spines and scanning them to train its AI models at a Las Vegas facility, according to a 404 Media investigation that tracked a shipment with a hidden tracker; pre-2022 books are valuable specifically because they guarantee human-written text, avoiding the “model collapse” that comes from training on AI-generated content. Amazon said it “purchases books through commercial channels to improve the products and services customers use.” The company that started out selling books now destroys them to teach its AI how books work. Alibaba open-sourced Qwen3.8-27B, a free multimodal model under an Apache 2.0 license that Alibaba says can run smoothly on a single consumer GPU after quantization, no cloud required; it’s part of a Qwen family that has now been downloaded more than 3 billion times, with over 300,000 derivative models built on top of it. Free, runs on your laptop, downloaded 3 billion times: the anti-hype pitch that somehow still works. |
PROMPT OF THE DAY The AI Code Review Speed Check Wiz’s own AI agent found and exploited a real vulnerability that an AI coding assistant introduced in just five days, faster than most teams even finish a sprint. Most teams have no idea how long it would actually take them to catch a bug like that, because nobody’s ever timed it. Two minutes gets you an honest answer. |
Here's how AI shows up in my team's code today: [paste: e.g. which AI coding tools we use, whether a human reviews every AI-suggested change, how pull requests get approved] Tell me: 1. Realistically, how many days a bug like Copilot Autofix's Snowflake mistake could sit in our code before anyone notices 2. What review step, if any, gets skipped when a change looks like a routine security fix 3. The one guardrail I should add this week so an AI-introduced bug gets caught before an AI, or anyone else, finds it first |
Forward this to the coworker who merges Copilot’s suggested fixes without reading the diff.
— Don.