SQA

When software testing meets the real world

·3 min read

Today we look at how to use Qaptly to test scenarios where software and hardware are integrated

In most automated software testing tools, it is clear where "testing" happens:

on the screen, in the API, inside the system.

As long as all of these return the right results,

the test is considered passed.

But once a system starts interacting with the real world, things are not so simple.

A small but very common IoT problem

Say you have an IoT device that can be controlled from an app or a web interface.

There is a button on the screen: "Turn on light"

After you tap it:

  • The UI shows "On"

  • The API returns success

  • The system state shows ON

Here is the question:  did the light actually come on?

For a human tester, this is easy to check.

For a testing tool, it is awkward.

Where traditional test automation gets stuck

For traditional tools, this test flow usually only gets this far:

  • Click the button

  • Verify the screen or the API response

What it can verify is:

"What the system says it did"

What it cannot answer is:

"What actually happened in the real world"

If the light did not come on but the system thinks it did,

the test still shows as passed.

Qaptly could already control many platforms from the start

Before talking about UPP, there is some background to make clear.

From the beginning, Qaptly has supported several Pilots:

  • Browser

  • Android

  • iOS

  • macOS

  • Windows

In other words, at the level of "operating system and screen", Qaptly could already automate everything.

But the problem is not at that level.

The problem: this way of dividing things never reaches the real world

However complete your platform Pilots are, you still run into the same problem:

The state of the real world does not belong to any operating system.

Whether an IoT device really carried out a command, whether a sensor really reported a value, whether the hardware really entered a given state:

none of these can be forced into being:

  • an Android feature

  • a Browser behaviour

  • or the result of some UI

If a test can only hang off a "platform Pilot",

it can only ever verify half the story.

That is why UPP exists

The starting point for UPP (Unified Pilot Protocol) is actually simple.

Instead of trying to cram all the hardware logic into Qaptly, we accepted one thing:

Some states can only be confirmed by "a role that lives in that world".

UPP was originally only for our project-based clients, since these scenarios need a lot of customisation.

With UPP, you can create a Pilot that represents an IoT device or sensor:

  • It knows the device's real current state

  • It can report whether the light really came on

  • It does not need to be part of any operating system

Qaptly does not need to know how the hardware is implemented. It only needs to let the Pilot that actually controls and understands the situation do the work.

This is where natural language matters

If UPP were just one more hardware verification mechanism, it would only be another engineering integration.

But Qaptly's core has never been just "connecting one more thing".

We let the test flow exist first in natural language.

For example:

"After I tap 'Turn on light' in the app, the device should light up within 2 seconds, and the sensor must report that the light is on."

This is not a specification document,

and it is not a script.

It is how a person would naturally describe the flow.

In this flow:

  • Qaptly understands the conditions described in natural language

  • The Platform Pilot operates the app or web interface

  • The IoT Pilot reports the actual hardware state through UPP

Whether the test passes

no longer depends only on the screen or the API,

but on whether the thing described actually happened.

Why we chose to open up UPP

Because this need will not show up in just one product.

Any system that starts touching the real world will run into this:

  • platform control works

  • but state verification does not

UPP is not there to show off capabilities. It is there so the testing model itself can be extended.

Qaptly understands how people describe the world, and Pilots tell us whether the world works that way.

We chose to open UPP up rather than keep it hidden inside the system.

Need Professional Assistance?

Our team are senior engineers and former executives with 20+ years in commercial product development — from apps serving hundreds of millions of users to B2B SaaS platforms serving tens of thousands of merchants. In recent years the focus has been enterprise AI adoption, from mapping the opportunity through to tuning what is live. Whether that means:

  • Mapping AI opportunities and sequencing them
  • Preparing data flows and integrating existing systems
  • Building forecasting and optimisation models
  • Monitoring and tuning after go-live
  • Test strategy and process review
  • Building and adopting test automation

We can provide professional advice and concrete solutions for teams of different scales and types. If you're facing similar challenges, feel free tocontact our consulting team.