Cory ZanerWhat Broke in My Lab This Week: A Blind Spot, a Too-Trusting Health Check, and a Panic Over Nothing
Every week my AI bots write status notes about what they built, what failed and what they're waiting on me for. This is the first of a weekly series where I read those notes out loud: three things that went wrong or got fixed, with nothing dressed up.
Series: Building My Life With AI Bots · Weekly Lab Notes #1
I run my home lab with a crew of AI bots. They build, test, review each other’s work and write down what happened. My rule is simple: nothing gets sent, posted or paid for without my OK. A side effect of that rule is a pile of honest status notes. Most of them say “done.” The interesting ones say “that didn’t work.”
So here’s the new weekly habit: three things that broke (or almost did), why, and what we changed.
1. My SIEM caught every Windows attack, and missed most of the Linux ones#
This week my pen-test bot ran a small set of simulated attack techniques against two throwaway lab VMs, one Windows and one Linux, to see what my SIEM would catch. These were safe, well-known test techniques on machines built to be broken.
What broke: Windows looked great: every technique raised an alert. Linux was a different story. Most of the techniques never showed up. They weren’t really “missed” by the rules. The SIEM never saw them happen, because the Linux box wasn’t recording which programs were run.
Why: Detection rules can only fire on data you actually collect. I had the agent installed and assumed that meant I had visibility. It didn’t. Also, the Windows alerts came from a generic “a process started” rule, not anything that named the technique, so “caught” was a little generous.
What we changed: The finding went to my security bot and my systems bot with a recommendation to turn on program-execution logging on the Linux lab machines, so the SIEM has something to work with.
2. A health check that trusted too much#
One of my bots wrote a small health-check script for a lab service. It restarts things when they fall over and reports back that it’s running.
What broke: Nothing in production, because my security-review bot read the code first. It flagged that the script could be pointed at internal addresses and would follow wherever it was told to go. That’s the kind of helpful feature an attacker could borrow to poke around inside my network.
Why: The first version allowed internal addresses “for convenience.” Convenience is how a lot of security bugs get written.
What we changed: The builder bot fixed it in a second round and the reviewer re-checked. One bot wrote it, another bot broke it on paper, and the first bot fixed it before it ever shipped. That’s the whole point of the setup.
3. The outage that wasn’t#
What broke: One morning my systems bot reported that it couldn’t reach the lab from its workspace. That’s the kind of message that makes you put your coffee down.
Why: The lab was fine. The bot’s remote connection check was faulty. It was a false alarm, not an outage.
What we changed: The bot fixed its check, confirmed the connection, and logged the cause as a false alarm instead of quietly moving on. I like that last part. “It works now” isn’t an explanation, and I want the explanation written down.
What I’d do differently#
I’d test my visibility before I test my detections. Making sure each machine actually records what I need would have surfaced that Linux gap in five minutes instead of during an attack demo.
That’s week one. The lab broke a little, the bots caught most of it, and I learned more from the failures than from the “done” notes.