AI-driven software engineering: The question is not ‘if’, but ‘how’

Hi Marc, an article has been doing the rounds here over the last few weeks: ‘LLM Driven Development’ by Kristian Köhntopp. What struck a chord?

Less of a surprise than an astonishingly accurate reflection. Köhntopp soberly describes AI as a lightning-fast colleague who takes what you give him at face value. Anyone who’s been in the industry for a while will be familiar with the ‘wizard’ type of developer, who delivers impressive results at an unrivalled pace – though only he knows how. A code agent is not so dissimilar to this – and requires the same approach: guidelines, reviews, and someone to document the unspoken assumptions.

Added to this is a memory that works differently from ours: a workbench, not an archive. Whilst the context windows are constantly expanding, and a great deal of research is being channelled into handling large amounts of context more efficiently – a larger workbench does not in itself mean order. What the agent needs to do a good job must be laid out on the workbench in a cleverly structured and appropriately organised manner. Gute Ergebnisse hängen also vor allem davon ab, ob man ihm brauchbares Arbeitsmaterial liefert – der Kontext ist das eigentliche Material. Good results therefore depend above all on whether you provide him with useful working material – the context is the actual material.

That’s exactly what we’d already learnt from our own experience: the naive request “Build me an application that …” works for small snippets, but as soon as things get serious, you need requirements, tests, reviews and clean-up phases – in other words, classic software engineering. The fact that someone put this so clearly into words sparked a lot of discussion here.

You say that the exciting part begins after the prompt. Köhntopp describes the discipline involved in managing individual repositories. Where do you start?

The article focuses – deliberately and quite rightly – on the workbench: how does one work in a disciplined manner with agents on a project? It is this precision that makes it so valuable. However, anyone familiar with our situation will already have an inkling of the questions that immediately follow: as a service provider, we develop for many clients simultaneously, each with very different requirements. This means that a whole series of decisions surrounds each of these workbenches: Where is this actually running? Is the code allowed to leave the premises? Which model for which purpose? Who verifies the quality? What are the running costs? There is no single switch you can simply flick. That is why we are deliberately taking a multi-pronged approach.

Let’s start with the day-to-day lives of developers. How exactly do you work with AI?

Our team chooses from a range of tools – Claude, ChatGPT, Cursor. We don’t prescribe any particular tool; every task has different requirements – and every tool has different strengths. In the same way, we work across the entire spectrum of methodologies: from spec-driven development at the more disciplined end to vibe-coding at the faster end. At its core, Köhntopp’s article is a plea for a disciplined approach to finishing, and rightly so – in real-world projects, simply saying ‘build me this’ often results in things that look plausible but are still not quite right.

But vibe-coding has its place: quick experiments, prototypes, figuring out what is actually wanted or needed. The trick is to choose the right approach for the task at hand. Incidentally, the downside of having a free choice of tools is precisely the problem Köhntopp describes when several agents are working on the same repository: Each tool reads a different instruction file, and it doesn’t take long for conflicting rules to arise. We therefore maintain a canonical source of truth, so that each tool does not, in its own way, plausibly do the wrong thing.

You mentioned “Can the code leave the house?”. That sounds like more than just a matter of convenience.

For a large part of our work, this is the very first question we ask. Those developing for their own use can send their repository to a cloud provider; many of our clients are not legally permitted to do so – or choose not to, for good reasons. We also develop software subject to government regulation – healthcare being the obvious example – and for clients whose contracts categorically prohibit any disclosure of source code. A public cloud service is therefore simply out of the question.

That is why we operate several AI systems in-house, powered by our proprietary middleware, the iSYS AI Hub – including some very powerful LLMs. We use it for experiments, for knowledge-based systems with RAG, for internal process automation – and, above all, to take AI-assisted development exactly where it would otherwise not be permitted. The Hub turns ‘we’d like to use that here’ into ‘we’re allowed to’.

That’s the quicker option. Köhntopp insists on tests and audits. How do you maintain your quality standards – and how do you demonstrate them?

That is the most important point for us, because ‘feels faster’ is meaningless if quality suffers. We set ourselves high standards in terms of usability, software quality and IT security – and our aim is not merely to maintain these standards using AI, but to raise them and demonstrate this robustly. Köhntopp’s approach fits in perfectly with this: prioritising testing gives the agent a clear objective, rather than allowing it to optimise for a response that merely looks plausible.

And audits as artefacts in the pipeline – security and refactoring reviews that generate findings, which in turn lead to tickets and then fixes – are precisely the way to make quality verifiable, rather than merely claiming it. We build processes around this that support the use of AI and make it traceable.

It is worth making one honest caveat: a repeatable agent audit is a tool, not a substitute for a professional security review of critical systems – Köhntopp says as much, and we agree entirely.

So you don’t just call an API.

No – and this is where our heritage in software engineering and data comes into its own. There are two levels. Firstly, operations: which architecture runs where, weighed against digital sovereignty, regulatory requirements in sensitive areas, and – quite simply – efficiency and costs. Running high-performance models effectively doesn’t come for free, so conscious trade-offs have to be made. Secondly, the models themselves: requirements-driven model selection, followed by techniques such as distillation, targeted fine-tuning and quantisation, and, in production, tricks such as speculative decoding – right through to bespoke context engineering.

The latter leads directly back to Köhntopp’s “Context is Material”: the context window is a workbench, not a rubbish tip, and a model is only as good as the material you put into it. We tailor the material and the model specifically to the task, budget and framework conditions.

That sounds like a major refurbishment. How far along are you with it?

Right in the thick of it – and we say that deliberately. Across the company, this is a transformation of the sort that many of our clients are currently undergoing themselves or are looking to initiate. The extent to which it is being implemented varies across projects and teams, and we are currently standardising the approaches that have proved successful. It is precisely this experience that we pass on: we do not recommend anything that we have not tried ourselves – including the detours that are part and parcel of the process.

If the agent takes on so much, does that mean the developers’ workload is reduced?

It isn’t shrinking; it’s shifting. What we’re seeing here is that the software engineer is becoming an environment builder. An increasing proportion of the work involves creating a suitable ‘virtual reality’ for the agents, in which they find optimal conditions. That sounds more abstract than it is: it’s about project rules, architectural decisions, security boundaries and tests. This sounds very familiar to me: as an engineering manager, I used to do exactly that for teams of developers – create an environment in which good work is possible. Today, we’re building that environment for people and agents.

And the wizard analogy from the beginning still holds true here: let the agent do the magic, but retain the judgement. AI raises the ceiling on what a team can build – yet someone has to decide what is worth building, what ‘good’ means in this context, and whether the result really works. That is engineering, and it is human. For us, this isn’t just a slogan: ‘Technology for the People’ includes the people who build it. So yes, AI-driven – but in an engineering way.

Thank you, Marc, for your time!

You’re welcome.