OpenAI proved again last week that the most dangerous kind of creation is the one not understood by their own creators.
Here's a sentence that should worry you more than it probably does: LLMs don’t act predictably, ever. Ask an LLM the same question twice, and you WILL get two different answers.
It’s not a bug, it will never be “fixed”, it’s the nature of the thing. And it's exactly why I don't let these tools make decisions for me, not in my life and not in my code — only draft, suggest, and speed up the boring parts.
Which is why it’s mind-boggling that people running much more dangerous code don’t seem to care as much as I do. How quickly we’ve forgotten the foundation of computer science!
For the majority of our time writing code, it’s been deterministic. Same input, same output, every time, forever. That's not a small feature, it's the entire reason software engineering works as a discipline. You can test it. You can trust that what ran on your laptop will run the same way in production (if you know what you’re doing, localhost).
An LLM breaks that contract. It's a non-deterministic abstraction layer sitting on top of your code, and non-determinism when you want it to regurgitate content to help you do boring stuff is one thing. Non-determinism in a decision — which architecture to implement, how to build a database, what a function assumes about its inputs — is a different problem entirely.
I started my career in process design at a manufacturing plant. If something went wrong on the line, you traced it back, step by step, and said exactly what happened and why. No hand-waving. That discipline didn't leave when I moved into software — it got sharper, because software fails in ways that are much easier to hide and increasingly more dangerous.
So: AI drafts. AI suggests. AI does not decide. If I can't explain why my code behaves the way it does — to another engineer, or to myself in six months — I haven't shipped something I can stand behind. I've shipped a black box with my name on it, and my sign-off on bad decisions. I wouldn’t dream of blaming AI for my poor design.
The discomfort a lot of people feel about AI in software isn't fear of automation. It's the loss of the thread of why. That instinct is correct. Wanting to know why your system did what it did isn't luddism. It's just being responsible.
Use the tools. I do, constantly. Just don't hand them the wheel.