The Same Ticket, Four Times, One Year Apart

Same Polarion Task in November and last week. The ticket didn't change. Everything around it did, and not the way I predicted.

The ticket

"Implement the widget described in this Task. Configurable parameters, renders on a Live Report page. Document it. Tests to follow."

If you've built anything for Polarion, you know this ticket. It isn't hard. It's specific. The widget is Velocity, and it reaches into Polarion's Java API for everything that matters. That API has its own vocabulary, its own way of handing you a work item, its own opinion about how configuration arrives. None of it is guessable. You either know it or you have the docs open in another window.

And there's no compiler. A Velocity widget doesn't fail at build time. It fails when the page renders, in front of whoever opened it.

Which is the whole story. The question was never whether the model could write code. It was whether the model had the faintest idea what a Polarion widget is.

I've run this ticket four times in a year. After each run, two lines: what disappeared, and what still cost me. Watch the second one.

Run 1: I'm the transport layer

November 2025. ChatGPT in its own app, Polarion in the browser, VS Code on the side.

Open the Task. Read the spec. Copy it into the chat by hand. Then start explaining: this is Polarion, widgets are Velocity, they call this Java API, here's an existing widget from our repo so you can see the shape.

Then the part that actually took the afternoon. I had to describe the solution. Not the requirement, the solution: how it's structured, where the configuration lives, which of the two obvious approaches I wanted and why the other one bites us in a year. I worked that out in my head and typed it into a chat window as prose.

Back came Velocity that read beautifully and called three Java API methods that don't exist. No compiler to catch that. So: deploy, reload the Live Report page, watch it break, dig the real cause out of the log, paste that back, get a fix, reload again. Then write the docs myself, because by then I had nothing left. Then update the Task by hand.

One thing was already in place that November, and I'd forgotten how early it was until I checked my own archive: the Polarion Docker setup. I published it the same week I started this blog. A reproducible Polarion I could pull, run and deploy into, with remote debugging attached.

Note who it was for. It was for me. It let me reproduce, me deploy, me step through a breakpoint. The assistant had no idea it existed. Keep an eye on that container, because it's the only thing in this post that's present in all four runs.

The model was useful, in a way that's easy to misremember. It was good at everything that wasn't Polarion. Structure, naming, the sentence I couldn't phrase. It was confidently wrong about everything that was, because it had never seen it, and the only channel it could ever see it through was me, typing.

That's the transport layer. Not copy-paste as an annoyance. Copy-paste as the only way anything got in: the domain, the spec, and a design I'd already finished in my head. From zero, every session.

➜ Gone: the messiness of what I wrote. Even here, the docs came back tidier than what I produce at 6pm.
➜ Still costing me: the domain and the design, both by hand, retyped every session.

Same month, I published this prediction:

"But when I open my IDE on Monday morning to refactor a complex Java backend service, I'll likely be relying on GPT-5.1 and my VS Code setup."

Hold on to it. It's wrong in a more interesting way than it looks.

Run 2: it moves into the editor

Early 2026. Copilot in VS Code.

The assistant sees the file I have open. Better than that, it sees the other widgets in the repo, so it pattern-matches against my Velocity and my API calls instead of against the internet. For a domain this narrow, that's the difference between plausible and correct. Custom Instructions mean I stop re-explaining my stack.

The syntax stops being the hard part.

Now watch what didn't move. Copilot knows my repository. It has no idea the Task exists. The spec, the acceptance criteria, the linked requirement that says what this widget is for. All still in Polarion, still read with my eyes, still retyped with my hands.

Draw a line down the middle of my desk and it looks like this:

➜ Gone: copy-paste of code. Re-explaining my stack.
➜ Still costing me: the Task, the spec, the requirements. By hand.

The container got better in this stretch too. Proper repository, modular architecture, something I could hand to someone else. Still entirely my tool. The assistant wrote code, I ran it.

Run one repeated itself one level up. Distance to my code: zero. Distance to my systems: unchanged. I'd fixed the half that felt expensive and left the half that was.

Run 3: it stops asking me for context

June 2026. MCP, and later Playwright.

I give it a Task ID. It pulls the Task, follows the link to the requirement, reads the acceptance criteria in the document, finds the existing widgets, and writes implementation and docs against all of it.

Here is what one run actually looks like in the log:

Playwright is the part I underestimated, and it's the piece that matters most for Velocity. There's no compiler here. Nothing tells you the widget is right except the widget rendering. So the only real verification is loading the page and looking at it, and for a year that meant loading the page and me looking at it.

With a browser attached, the agent deploys, opens the Live Report page, sees the widget came up empty, and goes back to fix it. The check I was the bottleneck for is now part of the run.

And this is where the container quietly changes hands. It's the same Docker setup from November. Nobody rebuilt it for AI. The agent simply started deploying into it and driving a browser against it, and a thing I'd built for my own convenience turned into the place where an agent verifies its own work.

The copy-paste loop from run one isn't shorter. It's gone.

But here's the effect I didn't see coming, and it's the one I'd lead with now: my documentation and test coverage jumped a level.

The reason is boring. Docs and tests were never hard. They were what got cut when the implementation ran long, because they came last and I was done. Once the agent could read the Task and the acceptance criteria itself, the docs stopped being a separate chore I had to find energy for. They came out of the same context as the code, at the same time, against the same source of truth. Same for tests: the acceptance criteria were finally readable by the thing writing them.

The quality didn't come from a smarter model. It came from deleting the point where I used to give up. What I ship now is round as a whole product, not just correct in the part I found interesting. Delivering under your own name, that's worth more than the hours.

➜ Gone: context gathering, and thin documentation with it.
➜ Still costing me: my attention, the whole run. I watch every step.

It stopped being something I consult and became something that acts. But I still sat there for all of it.

Run 4: I'm not in the room

Today. Claude Code in a terminal, inside the repo, every tool attached. Subagents split what used to be sequential: one works the implementation while another drafts the docs against the same spec. Some of it runs as a nightly task.

And the container takes its last step. Every session now spawns its own. It deploys there, runs its tests there, drives Playwright against that instance, and attaches the debugger to it when something is off.

That isolation is the thing that makes an unattended run safe at all. Two sessions can work at once without stepping on each other. A run that goes sideways takes down its own container and nothing else. I can throw it away and start again in seconds.

Which means the most important piece of my agent setup is eight months older than the agent, and has nothing to do with AI. It's a pile of small puzzle pieces, and the pile is the actual product.

I give it the Task ID and go do something else. Sometimes I start it in the evening and read the result over coffee. End to end, the run now looks like this:

➜ Gone: my presence while it runs.
➜ Still costing me: the design, what to load, and whether the result is right.

Read that carefully, because it's easy to hear "faster again" and it isn't. Runs one to three made me faster. Run four meant the work stopped needing me at the moment it happened. Only the second one changes what your day looks like.

And here's the part that hasn't moved an inch. I still do the thinking.

I write the Tasks, or I have them generated and then correct them, because I'm the one who knows what good looks like here. That sounds like a small thing next to an agent that works overnight, and it's the opposite. In a year of this I have not once seen the model come up with a better solution than the one I had. Not a worse one either, on a good day. Just not a better one.

What it does do, extremely well, is take my idea and build it properly. Consistently, documented, tested, at 3am. That's a division of labour I'll take, and it's worth being precise about, because the marketing around this suggests the other split.

On "tests to follow": they still follow. What changed is that they now arrive, instead of quietly becoming a backlog line nobody reopens. The ordering didn't improve. The dropout rate did, and that was always the real problem.

Four runs, one ticket. Now the part that took me the whole year to see.

What actually changed

Put the four runs next to each other and read the second line of each one downward.

Same four runs, drawn as distance:

The tool list looks like the story. Split it properly and it stops being one.

Models: GPT, Gemini, Claude.
Tools: a chat window, Copilot in VS Code, Claude Code in a terminal.

The models changed and it barely mattered. The tools changed and it mattered every time:

Each new tool sat closer to the place my work lives. That's the whole line.

Which brings me to three sentences I'd have called nonsense in November.

I barely open an IDE. VS Code is where I read a diff, not where I work.

I don't write the code. On this widget I didn't type a line of Velocity.

The model doesn't matter. Not once did I move because something scored higher on a benchmark. Claude Code counted for where it runs: a terminal, inside my repo, with tools attached. That's what got me to the last row.

So if it isn't the IDE, isn't my typing and isn't the model, what's left?

The system around it. MCP so it reaches Polarion, Playwright so it sees the rendered page, a Docker setup so every session gets its own instance to deploy into and debug against, instructions so it knows my stack, subagents, a Second Brain holding what I used to hold in my head. Note the oldest piece on that list is the container, built in week one for entirely non-AI reasons. None of that is impressive alone. Wired together it's what made run four possible, and the wiring isn't a purchase. It's my knowledge of Polarion, my experience of where it breaks, and knowing which piece belongs where.

That's the part nobody can install. And it sits on top of the other thing nobody can install, which is knowing what should be built in the first place.

It also moved the bill. In November the expensive thing was my time. Today my time is near zero and the constraint is the context window: the agent can reach everything, so it can load everything, and the real question is what deserves to be in there. That's why token efficiency became my topic, and it has nothing to do with pricing.

One last thing. My November prediction was that I'd be refactoring Java in VS Code with GPT-5.1. Every noun wrong. But I closed that post with this:

"The tools are getting better, but the responsibility for the architecture still lies with us."

That held, and harder than at the start. Look at that list again: the design sits in the last row for the same reason it sat in the first. It never left. I still decide where configuration lives and how the widget is cut. In November that cost me an afternoon of typing. Last week it cost four sentences. Same decision, same responsibility, a fraction of the friction, which is a much better deal than the one usually promised, because the promise is normally that the thinking goes away too.

What I'm watching next

I got last year's prediction wrong in every noun, so no prediction this time. But three things are already visible, and none of them are speculation.

Agents are starting to coordinate without me. Run four uses subagents, and I define every handoff. As of this month, Claude Code sessions can pass information between themselves. That's a different shape: instead of me routing work between them, I describe the structure once and the work routes itself. If that holds, the thing I produce stops being a prompt and starts being an org chart for machines. It also means the failure mode changes. A bad prompt gives you a bad answer you can see. A bad structure gives you five agents confidently building the wrong thing in parallel.

The model is moving into Polarion itself. Polarion 2606 shipped a Copilot API, so reports, scripts, widgets and plugins can call an LLM from inside Polarion, and on-prem deployments can wire in their own via custom LLM connectors. That's the exact reverse of everything in this post. I spent a year pulling Polarion toward my terminal. Siemens is pushing the model out to the person who never opens one.

Both are real and they serve different people. Mine is for developers. Theirs is for the reviewer with four hundred requirements who will never install an MCP server, and that's the bigger group. I built for the smaller one all year, which is worth saying out loud.

And the ground may move under both. Siemens has signalled that Java plugins should become unnecessary over time, with apps on a modern stack much closer to the platform. The widget in this post lives off that API. If that lands, some of what I optimized this year is tooling for something on its way out, and I'd rather find that out early than defend it.

Three directions, and I don't know which one describes my next twelve months. What I notice is that they circle the same question, and it isn't a tooling question. Every run so far removed something I used to do by hand. The obvious continuation is that the work starts without me asking for it at all. Whether that's a good idea is a different question from whether it's possible, and that's the one I have no answer for:

How much should run without me asking?

Ask me again in November.