Releasing sysdiff - AI Slop Challenge
I use projects to learn other things and build other things - in addition to the project itself. Introducing my new open source utility: sysdiff is a lightweight Linux utility that compares snapshots of a system’s configuration, packages, services, and files to quickly identify what has changed.
There were a number of things I was developing and using the systdiff project to accomplish.
1. Can I create an autonomous worker that will just build me useful linux utilities during the night as I sleep. I know LLM’s can code. This experiment was “can I give an autonomous AI employee a mission and have it pursue it based on its own initiative with minimal input or direction from me”.
1. I made wonderful progress doing that. My agent platform is still in development and a bit brittle, but using the sysdiff project to shake out the kinks in my AI platform was quite helpful.
2. Can I develop a “labor ladder” so that I distribute the work among highest intelligent and thus highest cost llm’s, and “workman” llm’s that are cost efficient but powerful enough for coding.
1. Yes. I used a couple of workman models (Grok Build, Grok 4.5 medium and high, and DeepSeek V4 Fast) at various times. As long as I had a quality designer and reviewer, the mistakes of the workman models were addressed without involvement from me. Well, other than developing the whole autonomous platform.
3. Can LLMs develop high quality code without a knowledgeable human in the loop. Mind you, I’m a strong advocate of human in the loop. But there needs to be understanding of the capability of AI to test and debug code. I’m not referring to vibe coding - one shot prompts. I’m speaking of developing a process that applies numerous quality measure as part of the process.
1. The autonomous process did NOT produce, by itself, publishable code.
2. It MAY have had I let it keep going, but once it had achieved “sysdiff works”, I took over and started directing quality passes with several llms in an adversarial review and repair operation
4. Can I use LLM tools to create quality c code not knowing a lick of c myself?
1. I submit to the good folks here to be the jury as I am not competent to judge.
5. Would I use this process for enterprise or customer work?
1. No, no I would not.
2. For development, yes.
3. Final technical accountability has to be in the hands of a qualified human.
4. My process should be able to create a lot of value and save time such that the quality, expert human is given code that’s already as high of quality as LLM’s can deliver
5. Keep in mind I have not philosophy of C code. If I were in a company that had its own best practices, I could have baked those into this process. Don’t judge on whether the code is exactly as you would write.
A taste of the testing gates the code had to pass
Dimension Checks
Build portability GCC, Clang
Source consistency and maintainability formatting, clang-tidy, cppcheck
Functional correctness fixtures, pytest
Runtime safety ASan, UBSan, Valgrind
That’s right, I don’t rely just on LLMs. We live in the agentic world that has tool calling. So have your LLMs use deterministic tools where it makes sense.
What’s next?
1. Continue maturing my autonomous platform. It’s already powerful but it’s not ready to ship
2. I’ll continue letting the autonomous worker use my sleeping hours to continuously come up and develop helpful little linux utilities until I come up with better missions to spend my AI subscriptions on.
Why did I call this the AI Slop challenge? I’ve long ago seen the real life benefits of AI as a tool for quality software development. But the sentiment remains strong — AI can’t be trusted to write quality code. Well, for SURE, the way some use AI ends up with slop. Slop isn’t an inevitable property of AI. It’s often the result of using AI without sufficient review, testing, or engineering discipline. I have given my current best effort to create a system that will reliably create quality code. If I have accomplished this in an area I have zero expertise (linux utilities, c coding) — then it’s the credit to my AI agentic platform development skills. That’s what I’ve been spending an enormous amount of time learning and working on. I’m not a c coder, do not aspire to be a C code. Nor am I expecting to get work doing C coding. But I do think I will have established that given the right effort, generative AI can be productive and write code of high quality.
For anyone interested in exposing my folly: https://github.com/leebase/linux-utilities
