What is it like chasing a tornado, and how did I find myself testing GPS app software on the border of Canada? With the car trunk packed to the brim and AC/DC playing at full power, you head central-east to Ontario. The rebellious sky changes color as you follow the route—blooming thunder signals that you are slowly approaching the epicenter. The storm is brewing, and you feel it coming.
It’s half an hour away at most.
After a sharp turn, you hear a flip sound out of the trunk—it seems like a tripod made a flight above the other parts of your video equipment. Hopefully, there’s no damage, but you don’t have time to figure it out—your adrenaline is high, and dozens of breathtaking images are at stake.
Apart from a couple of cameras, power supply units, and all the accessories you carry, you’ve got a special chaser’s mobile app that’s running in the background. It collects the logs of your route and shares your updates with other chasers by regularly changing your location on the map and uploading all the media files with recently caught pictures.
The app also allows you to chat with other chasers and remain connected as you’re moving on the map. Sharing your location keeps you safe and other storm chasers updated on the weather conditions and the road’s condition.
You’re heading another 200 miles towards the whirling gale when the app suddenly crashes—giving you no opportunity to continue your chase. All your efforts and your ride logs are wiped out from the live map and lost forever, both for you and the other chasers.
The latter can be described as “hardcore testing” of the app, which I also witnessed in my practice. At the same time, testing an app at its max before a big release remains any software testers’ main priority. And this can be achieved by reproducing the most realistic conditions and playing out several scenarios.
Testing Location-Based Apps
I’ve been into developing mobile apps and testing GPS navigation software for quite a while. And what I love most about testing location-based apps is how they behave in real conditions. Simply put, the main challenge is reproducing the real speed a chaser reaches when driving the expected distance.
But let’s start from the beginning.
Before the first version of the chaser app was deployed in production and went life, we spent hours testing it in several ways, including:
- Functionality testing: This includes testing the basic parameters of a location-based app, such as its responsiveness, UI, screen size, and orientation.
- Usability testing: Aims to test your app’s UX and answers the question of whether it’s user-friendly and comprehensive.
- Geo-location testing: Checking location-specific features and parameters. This includes ensuring that the date and time, speed, and distance on the map are displayed properly.
- Testing networks and airplane mode: With location-based apps, it is essential to test how the app performs with different network connections (Wi-Fi, 4G or 5G, LTE, etc.). Additionally, we checked the app’s operation in airplane mode as some of our chasers accomplish their distance by air.
- Interrupt testing: Aims at testing how the app reacts to incoming calls and messages, signal loss, or switching tracks in a music player. It also detects if the app is back to where it was before the interruption.
- Testing geolocation blocking: The usage of some apps may be restricted in certain areas for political or other reasons. That’s why it’s important to check that user access is not restricted in the regions where the app is expected to be used.
All this formed our software testing strategy. Once all the tests were completed and appeared successful, and the app remained stable during hours of continuous use—we officially released the beta version. Testing in real conditions was expected to detect more bugs to fix in the next release. And it actually did.
Once real chasers made a couple of rides using the app, they noticed that a crash occurred on an iOS device roughly 5-6 hours after the chase, making all of the data impossible to restore after re-launching the app. For other chasers, the user who experienced the crash was stuck on the map and shown as “offline.”
Conducting Lab Testing
Now, the goal was to investigate the bug, reproduce, and fix it. We sat down for another round of tests. One problem. All we knew about the crash was that it occurred on iOS version 13.6.
We launched the app on a similar device and conducted a couple of “chases” that lasted 10 and 20 hours with the app running in the background.
And guess what? No crash happened.
We understood that another ride should be longer and take place out of our lab conditions, and we needed to hit the road as if we were genuine storm chasers.
Simulating Real Conditions
Sometimes software testing in real conditions is impossible in the area where it takes place. In our case, we quickly understood that the speed at which real storm chasers are moving somewhere in Tornado Alley is impossible to reach within our location. The traffic laws of the country simply prohibit it.
Our average speed in the city was about 30 miles per hour, while other chasers experienced the crash when driving on the highway at 55 miles per hour. The sky-chasers who are piloting an aircraft are also quite difficult to reproduce by an average QA tester sitting in front of the computer at the office.
We needed to find a way to reproduce not just the crash but the exact conditions in which it appeared for the chasers by gradually increasing the speed of the ride, the distance, and the route.
That meant we needed to be able to react to the crash in an informative way. We saw that as the only way to reproduce the steps and conditions that caused the app to crash.
Using a Mock Location for Testing a Location-Based App
So, our goal was to simulate a continuous ride following real chasers’ routes while gradually increasing our speed. The expected result was catching the crash, getting the logs, and receiving the exact speed values, distances, and route coordinates that caused the app to become unstable.
GPS Navigation apps didn’t appear overnight. Testing software builds available with various apps for location simulation like iMyFone, AnyTo, EaseUS, MobiAnyGo, iSpoofer, Wondershare’s Dr. Fone, Xcode, and more.
We tried all of them and found that the main restriction was exactly the speed we were (or better) could not simulate. The limit was only 22 mph with iMyFone and EaseUS and about 31 mph with Wondershare’s Dr. Fone, which basically brought us back to the lab conditions and made it impossible to reproduce the crash.
Once this became clear, we switched to Xcode, where we could play by our rules and try chasing the crash at different speed values we set ourselves.
Simulating a chase with Xcode meant we needed to set the desired speed and geographic coordinates in a text file. This data format is known as .gpx and can be generated with a GPX Generator.
Uploading this type of file format to Xcode would allow us to simulate a continuous ride across the spots indicated on the map of the USA while being physically located in front of the PC. With Xcode, we also managed to simulate losing internet connection and changing connection types to 2G, 3G, EDGE, and more. After all, a GPX file with all the necessary information could be uploaded directly from the debug build of our app.
All that remained was to connect the iOS device to the PC, launch Xcode, and upload the GPX file.
This allowed us to use mock locations, simulate traveling the U.S., and test possible routes at different speeds similar to what tornado chasers could reach before the app crashed. In our case, the iOS build was down after 6 hours of continuous riding at 30 mph, which makes a total distance of only about 180 miles.
If the speed increased to 50 mph, the time of continuous app work decreased to 4 hours. The higher the speed, the quicker we reproduced the crash. It always appeared at a distance of 180 miles, no matter if the app was running in the foreground or background and if any media files had been uploaded to the app by the chaser.
Now it was time to fix the crash. We downloaded the container with files containing the information about the latest chase that was interrupted from the debug build using Xcode installed on macOS.
This allowed us to receive the most accurate information about the steps and conditions for reproducing the crash. In addition, with these files, it was possible to continue the chase on another device using Xcode at the developers’ side to investigate the app performance during the chase where the crash appeared.
Using Xcode for testing the GPS app appeared to be an effective and relatively easy way to gather sufficient data for applying the necessary fixes. My journey to Tornado Alley was completed when developers fixed the crash and minor bugs, and the final version of the app went live!
What’s Important to Know When Testing Location-Based Apps
Now, when storm chasers are getting ready for the tornado season, which is forecasted to pass with no crashes, it’s time to share the points we learned when testing a location-based app:
- The state of the app when receiving or sending the data (background/foreground): In our case, the state didn’t matter, as the app crashed both in the foreground and background.
- Type of Internet connection, its quality, and possible interruptions quality: Mobile data or WiFi—both are available for simulation in Developer Options. It’s important to simulate all the available connection types, including EDGE, 2G, or 3G, as the person who navigates with the map may stop at the petrol station and connect to the WiFi signal. At the same time, the app is expected to continue working smoothly in the background.
- The format of geolocation coordinates: Those should be precise and include latitude and longitude, altitude, speed, time, directions, and all the data that helps to reproduce the route as closely as possible.
- The time and frequency of sending geolocation data: Again, we need to reproduce the whole route. We tried increasing the distance (the interval of sending geolocation of the chaser) between each point of the map to decrease their number to offload the app. But this way causes the route to becoming less precise, which can prevent locating and reproducing the bug. In other words, “the line” you draw on the map should correspond to the route and the ride conditions.
- Continuous work of the app in the background mode: When collapsing the app or locking the device, the app is expected to continue working smoothly.
- App recovery after reboot: The app should withstand any interruptions and recover the whole route after it was closed, force closed, or if a crash occurs.
- Using real conditions in line with location simulation: Luckily, several tools for location simulation are available for software testers both on Android and iOS.
By going through all these tests, we finally reached a way to properly test the location-based app to ensure it will work for even the most daring tornado chasers.
And there you have it: Tornadoes, a QA engineer testing GPS apps, and a story for the ages. Feel free to check our quality assurance services and contact us to discuss your project.
The article was originally published on IoT For All.
