Our consultant Dries Metten attended “The ‘human in the loop’ fallacy” by Corné Potgieter at Dataminds Connect in Mechelen. Below, he shares his takeaways.
A simple puzzle
Corné opened his session with a question. Try it yourself before reading on:
System 1, System 2 and AI
Corné Potgieter opened his session at Dataminds Connect with that question. He referred to Kahneman and Thinking, Fast and Slow. Kahneman describes two modes of thinking: System 1 is fast and intuitive, System 2 is slow and analytical. System 2 is lazy and doesn’t kick in automatically. With the bat-and-ball question, System 1 immediately says “10 cents” and System 2 accepts it without doing the math. Exactly the same thing happens with AI. An LLM gives a convincing answer, System 1 finds it plausible, and we approve it. But how much is the human in the loop still really checking?
What the numbers show
The figures from Faros were striking: 98% more merged pull requests (PRs), 91% more review time, PRs that are 154% larger and 9% more bugs per developer. More output, but also more code to review and more bugs. That raises the question of whether we’re checking AI output critically enough.
Quality over speed
In my view, this is one of the biggest challenges of AI in data engineering: not how fast we can produce code, but how we keep guaranteeing its quality.
Structure through data contracts
That’s why Corné argued for more structure through ODPS and ODCS, two open standards from Bitol. ODPS captures the ownership and intent of a data product. ODCS describes the contract for a dataset: schema, quality checks, SLAs and ownership. That way, you test AI output against concrete rules instead of relying on gut feeling alone.
From reassurance to real control
Data contracts can sometimes feel like extra admin, but the more we leave to AI, the more important clear agreements become. Without that structure, human in the loop remains more of a reassurance than a real control.
Dataminds Connect in pictures
Curious about AI in data engineering?
Want to spar about keeping humans meaningfully in the loop — with the right structure and data contracts? We’re happy to talk.
