Emulator testing just isn’t enough

You’ve ticked all the QA boxes. The app is signed off. But it still crashes once it’s in users’ hands. What the heck?

It’s one of the most common stories we hear. A new release goes live and everything seems fine on paper, but the user complaints start to trickle in. Then they build. Your inbox is full of screenshots and bug reports. But, when you ask the team what went wrong, they’re just as surprised as you are.

The truth? It worked on your machine, but not anyone else’s.

Emulators are not real devices

If you’re not technical yourself, you probably won’t know how your app is tested. That’s okay. But it’s worth knowing this: emulators can only tell you so much. They simulate mobile devices on a computer, which helps with early testing and automation. But when it comes to the real world, they fall short. We’ve lost count of the number of apps that looked fine on an emulator but crumbled once real users got hold of them. If that’s happening to your app, this could be why.

– Performance bottlenecks don’t show up in emulators

Most emulator testing is done on high-spec development machines. Real users? They’re on a huge range of devices, many of which have low memory, limited storage or older chipsets. If your app hogs resources, those users will feel it first. You won’t see that in the lab.

– Real users don’t have perfect Wi-Fi

Most emulator environments are connected to fast, stable networks. But out in the wild, your users are on the move. They’re switching from 4G to 3G, they’re dropping signal on trains, they’re using your app in noisy, unpredictable conditions. Emulators can’t replicate that.

– Permissions behave differently on real devices

This is a big one. On an emulator, your app just works. On a real phone, it asks for location permissions. The user says no… then the app crashes. Why? Because your dev team never saw that happen during testing, so they didn’t plan for an alternative. Emulators often auto-approve permissions unless specifically told not to.

– You’re missing hardware-specific bugs

Cameras, GPS, motion sensors, different screen sizes – these are all features you can’t properly fake in an emulator. Some bugs only show up when the real hardware is in use, which is why app store reviews often reveal things no one saw coming.

– Android fragmentation is a real problem

The Android ecosystem is vast with hundreds of manufacturers, and thousands of devices. Emulator testing usually covers one or two setups, that’s just not enough. One user on a Samsung Galaxy might have a totally different experience to another on a Google Pixel. If you’re not testing for that, you’re leaving gaps. Prime opportunities for a poor user experience.

We’re not saying never use emulators

They’re great for development and great for rapid testing. But if your QA stops there, it’s not enough. Think of them as a rough guide, not a final check.

Here’s what you need to do next:

– Get real device coverage

You don’t need a lab full of phones, but you do need access to a meaningful spread of devices, OS versions and manufacturers. That might mean using in-house devices, or cloud services like BrowserStack or Firebase Test Lab.

– Test the full experience

Performance. Connectivity. Permissions. Hardware access. UI responsiveness. These all behave differently outside of a test rig, so build on your QA processes to reflect that.

– Track the right data

Make sure your app is logging crashes properly. Tools like Crashlytics or Sentry can show you where and why your app is failing, often long before users start complaining.

QA isn’t just about testing. It’s about learning what you don’t know

As the app owner, you don’t need to be across every testing detail. But you should have clarity on what’s being tested, and what isn’t. That’s where we come in.

At Indiespring, we don’t just run tests. We run audits. We highlight where your testing is solid, and where it’s blind, to identify the risks that won’t show up in the simulator. Risks that can cost you your users.

If you’ve been relying on emulator results alone, you’re missing the bigger picture. Let’s take a proper look…