Does a testing tool always have to stay within the test scenarios it was originally designed for?
Qaptly Auto for Android is an Android testing tool, and after v1.5 shipped we stopped updating it for quite a while. For Android app testing it already had pretty much everything it needed, and where it did hit limits, Qaptly's built-in automatic switch to adb covered them, so it was already very capable. Because we updated it so often early on, we were hardly getting bug reports from clients any more.
Requests started trickling in
Every so often we saw requests from clients come through Sentry.
Some asked us to move the Qaptly Auto GPS mock entry point onto the settings page.
Others asked us to let them turn off the Accessibility Service, which the app requires you to set up, or at least to skip it.
But that is the more formal way of doing Android automation. Skip it, and the only route left is adb.
For our users this should not matter, since they don't have to operate any of it themselves. Still, we genuinely found it hard to understand the scenarios behind these requests.
Until one day...
Small talk after a client interview
That day the interview finished early, and the client had no meeting afterwards, so we just started chatting.
Somehow the conversation came back to whether the Accessibility Setup could be skipped.
Half joking, half serious, I said:
"We could design it so you can skip it, but if you're driving the device remotely you might not be able to tell where an error came from. The tool assumes that once you reach the command history page, everything is set up and the device is ready to be tested at any time. If something does go wrong, it's hard to find the cause."
She replied straight away:
"We understand. Our test devices are always fully set up. We never skip it on those."
That only confused me more.
I had misunderstood from the start
I couldn't help asking:
"Then when do you need to skip the setup?"
She laughed a little and said:
"Because Qaptly Auto runs GPS mock more reliably than other apps. It doesn't jump around. Everyone in my family has it installed, but setting up Accessibility and checking USB debugging on every single phone is such a pain."
That was when I realised:
She wasn't talking about testing at all.
I checked again:
"Why does your whole family need a testing tool installed? And USB debugging does carry some risk."
She answered as if it were obvious:
"Because the whole family plays Pikmin Bloom. Some places are just too hard to get to, and in the city there are crowds fighting over mushrooms. Your interface is fast and reliable, so after I tried it I recommended it to my husband and daughter."
Then she added:
"While we're on the subject, can the Security Warning be skipped too?"
Clients are always one step ahead. I only install testing tools on my spare phone.
A tool never has just one use
In the end, we made a very simple decision.
We released a separate build just for GPS mock:
no Accessibility, no USB debugging, and none of the formal testing assumptions.
That keeps the phones a bit safer, and they can use it with peace of mind.
A tool designed for testing ended up as "reliable gear" on one family's Pikmin journey.
It also confirmed something for us once again:
A genuinely useful tool often ends up being used in places you never expected.
v1.8.0 really does add a Skip option
Which brings us back to the product: the upcoming v1.8 of Qaptly Auto for Android does formally add a way to skip some of the required setup.
This includes the Accessibility check and some of the preliminary environment checks, which can now be skipped in certain modes so you go straight to command execution and the history page. We did not make this change because we changed our view of the proper Android Automation workflow. We made it because we finally admitted something:
Some people use a tool for something other than testing, but they still need a stable, controllable and reproducible environment to run it in.
Of course, we should say something serious too:
Pikmin is meant to get you walking. It's better for your health.
As for uses like "stable replay tracks"...
let's just call that a very convenient side effect from the world of testing.
Don't tell anyone.