Team Perspectives

From point tools to closed-loop processes: what is real and what isn't

·5 min read

The future is already here, it's just not evenly distributed

Introduction: second thoughts after the AI competitions

Around Lunar New Year, I was invited to judge internal AI competitions at several companies. The last one ended last week, and it left me with a few thoughts.

The start of the year is an interesting moment. Everyone, without coordinating, is looking back at how they used AI over the past year and trying to set a direction for the new one. Across several AI competitions and sharing sessions, I got a close look at what many teams, from startups to departments in large companies, have achieved with AI in the software development lifecycle (SDLC).

Behind the busy tech demos, I saw three telling patterns. They show that the gap between leaders and followers is no longer "do you use AI" but a fundamental difference in "how you define the process".

Observation 1: tools are everywhere, and AI has moved from the chat box to the production line

Think back to this time last year. When we judged AI work then, most of it was still about technology that looked especially flashy. People were keen to discuss whether they were using Vibe Coding, which tool wrote better code and produced fewer bugs. This year, that has clearly changed.

Now, whether you are a PM (product manager), RD (software engineer) or QA (test engineer), these so-called coding tools have become an essential component of your workflow.

  • What we see: how deeply each role uses them varies. Some teams use them in sophisticated ways, others in plain ways. But using AI in daily work is now a shared norm.

  • Why it matters: AI has moved past the novelty stage. What we see is no longer the small win of "AI solved one problem in my work", but how AI is woven into development environments, project management tools and test frameworks.

The first warning from this spread is that once using AI becomes the passing grade, the dimension of competition shifts.

Observation 2: the limits of point use vs. closed-loop collaboration

This is the deepest divide I saw in the competitions. Delivery teams that were still patching up their processes and collaboration are still thinking in terms of point use.

What is point use?
For example, RD uses AI to generate code snippets and the PM uses AI to draft requirements. This does raise output speed, but if you step back and look at the whole system, overall development efficiency is still stuck in the handoffs between one stage and the next.

"AI glue" and the closed loop
The leading teams, especially the core units of major companies, have mostly already closed the loop on their processes. In a modern SDLC, AI doesn't just produce content. It connects the stages (The AI Glue).

  • Automated handoffs: going from a PRD (requirements document) to a technical design, and from the technical design to test cases, used to take a lot of manual handoffs, meetings and confirmations. Now AI connects these transitions.

  • Where competition moves: whoever uses AI to connect stages faster and more accurately, and better resolves cross-functional miscommunication, wins.

  • The core pain: if your thinking stays at point use, writing code gets faster, but if information is still moved between stages by hand, overall Lead Time won't shrink much. This is the most unfortunate gap between the major companies and everyone else right now.

Observation 3: the false boom in QA automation and the verification gap

This is the point I most want to dig into. In the competitions, we saw quite a few QA teams start using Claude, Codex or similar tools to build automated tests. On the surface, it gives QA staff who never had programming skills an automation toolkit. In practice, it hides serious technical risk.

The missing "translator" skill
A good automation QA used to need the ability to translate. Translating means understanding the core intent of a test case (which business logic it is meant to verify) and turning it efficiently into a script that runs. That human translation step is, at heart, a form of logic audit.

What is happening now:

  • Results nobody can verify: because AI makes generating scripts so easy, many QA staff without a development background lack the patience (or even the ability) to review line by line whether the automation code AI wrote actually matches what the original case expected.

  • Broken traceability: when QA asks AI to generate a test script, they can't confirm whether the logic AI wrote lines up with the requirements document. They think AI has done the automation for them, but the code may just run a meaningless check, in which case right or wrong doesn't even come into it.

  • False sense of security: this is what I consider most dangerous. When AI-produced test results say "Everything's fine", QA feels reassured by seeing Passed, while overlooking that AI may never have caught the real verification point.

Verification is the real test in deep water
The prevailing view right now is that whether verification is vision-based (operating on the visual interface) or text-based, when it is applied to verification in a complex SDLC, it is still some distance from fully trustworthy.

When a person checks a web feature, they can often tell right from wrong at a glance. But with automated test code, if you can't judge how deep its logic goes, you have no idea whether what AI verified for you is truly right or only looks right.

Summary and recommendations: think in strict conditions

Given these three observations, here are a few concrete things for any team driving an AI transformation to watch:

  1. Drop point thinking and invest in connecting the process: look at your SDLC again. Don't just ask "what can AI do for RD". Ask "how can AI cut the handoff losses across the whole delivery process".

  2. Redefine the QA role: in the AI era, QA's core strength is no longer writing scripts but auditing logic. If you can't understand AI-generated automated tests, you can't take responsibility for their results.

  3. Set a Strict Condition: don't let AI improvise under vague semantics. In an automated process, someone capable of logical judgement has to set clear measures and verification boundaries for AI. Only then, within those boundaries, should AI work hard to replace manual labour.

AI use has moved from "can chat" to "can collaborate". At this stage, what we need most is not only stronger models but a more rigorous attitude to verification. Only when we can verify AI output effectively does the efficiency gain have real value, instead of being a false boom built on sand.

What can we do next?

If you have also noticed an "AI handoff gap" or "QA verification problems" in your team, perhaps we can talk it through in depth: how do you build an effective system for auditing AI output?


This is not just a technical problem but a challenge of management and process design. I hope these observations give you some ideas, so that in AI's deep water we don't just swim fast, but also swim far.

Need Professional Assistance?

Our team are senior engineers and former executives with 20+ years in commercial product development — from apps serving hundreds of millions of users to B2B SaaS platforms serving tens of thousands of merchants. In recent years the focus has been enterprise AI adoption, from mapping the opportunity through to tuning what is live. Whether that means:

  • Mapping AI opportunities and sequencing them
  • Preparing data flows and integrating existing systems
  • Building forecasting and optimisation models
  • Monitoring and tuning after go-live
  • Test strategy and process review
  • Building and adopting test automation

We can provide professional advice and concrete solutions for teams of different scales and types. If you're facing similar challenges, feel free tocontact our consulting team.