Augmented reality is no longer a futuristic fantasy. It is here and now, a viable part of the consumer market, available for millions of mobile users. Headsets, however, are a more complicated case, given all the limitations and issues with battery life, performance, and convenience for the user. Despite all that, businesses seize opportunities to build pioneering products and use augmented reality for engagement and retention of customers.
If your business is planning to launch an AR application project, you probably know that there are two crucial activities that will define user satisfaction with the final product. The first one is natural and immersive user experience. The second one—and the subject of today’s story—is quality assurance. That said, read on to learn all about the specifics and challenges in augmented reality app testing, illustrated with a case involving our recent Microsoft HoloLens demo.
Augmented Reality Testing: Preliminary Work
The QA team examines product requirements beforehand to see the conditions under which the product will be used. It includes selected devices and types of interaction with the product. The selected AR development environment—whether it is based on Apple ARKit, Vuforia, or Unity 3D—is scrutinized as well. This analysis helps to create an effective software testing strategy.
After that, a storyboard of use cases is created to be tested in real environment. Use cases help QA engineers cover all the potential scenarios and provide a holistic view of the product — far more thorough than a simple review of what wireframes can deliver.
The Specifics of AR Testing
Augmented reality software testing is far more time-consuming and less consistent than traditional testing. It is important to set up correct environments and test the app with different physical objects, in different scenes under different lighting. It fits well within the traditional testing pyramid—User Interface / Integration / Unit Testing; however, there are additional specifics that we believe you should know.
The environment might not be limited to rooms and lighting. We had a case of non-static scenes, where an AR-enhanced mobile application would allow passengers in moving transport augment the outside reality. Since the official ARKit documentation approved of work in static scenes only, we had to eliminate both vibration and the mismatch of visual and motion data with a custom algorithm. The application was naturally tested in moving vehicles, with proper comparison of calculations and achievement of the desired precision.
When it comes to mobile AR apps, we always consider specific checklists for testing: proper work of different screen orientations, Internet connection and its interruptions, as well as different levels of memory and battery consumption.
Owing to the diversity of AR-supporting smartphones, tablets, and AR headsets, testing on all devices listed in the product requirements is essential. Emulators obviously cannot substitute real devices when it comes to finding and eliminating possible issues within real physical environments.
A Case of Microsoft HoloLens Application Testing
The best known headset on the modern market is arguably Microsoft HoloLens. It is marketed as a mixed reality device that interacts with physical objects in the real environment. However, its testing specifics are quite similar to those of AR headsets. Upon launch, the HoloLens scans the environment and empowers the user to fill it with digital objects, control them with a special menu and make them interact. Testing basically helps all of these operations to run smoothly and precisely. Here is the procedure of HoloLens application testing that used in our own projects.
Scanning the environment
The device had to detect all meshes, including physical objects and surfaces, under different spaces, angles, distances, motion and lighting conditions, and levels of ambient noise. QA engineers tested the application in new environments and placed new physical objects in the room to see how the scanning went.
Placing objects
Scanning was followed with making sure that the placed digital objects (holograms) would move quickly and smoothly. QA checked how the holograms would interact with surfaces: for example, whether they would “fall” into surfaces, or they could be “hidden” behind such objects as walls or ceilings.
Controlling an object
It was critical to make sure that the user would be able to access the model menu at any time, under any conditions, regardless of the size and location of the model. For that purpose, QA scaled the model to maximum and minimum to see whether the menu buttons could be reached easily. It was also important to ensure adequate distance between the model and the menu. The user should be able to hit the buttons and catch them to control the model from the first or second try. QA checked whether the model reacted to the clicker and all the gestures.
It is worth noting that this approach to interaction is quite novel for people. That is why it is vital to make the overall experience of any augmented reality app easy and convenient for anyone.
It should be possible to move the model without any additional gestures simply using gaze, if this condition is mentioned in the product requirements.
Controlling several objects simultaneously
It sometimes happens that the user is launching several similar holograms, and then launches an animation (say, rotation effect) for one hologram, other holograms start to rotate as well. Our case covered this issue.
Making crashes visible
There are no visible messages about the system crash for the HoloLens. This issue should always be discussed before the development starts. In our case, crashes were made visible to the user, so that they would be able to relaunch the application.
Performing accessibility testing
Mobile accessinility testing is always important. Immersion in an augmented reality app sounds great in theory, yet it comes with health impacts and keeps causing headaches, eye strain, and motion sickness. Modern headsets are continuously perfected; however, it is up to the software team to create a product that can be used regardless of age and physical condition, reduce discomfort, and ensure quick and natural navigation. This must be kept in mind from the very start of the project and reflected in requirements and the actual code.
Additionally, it is worth noting that the native Admin Panel of the HoloLens is a useful tool that allows to stream MR experience in real time and record videos. Thus communication and troubleshooting becomes easier.
With headset-based AR and MR still very much in its infancy, research is more vital than ever in this continually developing field. Meanwhile mobile AR advances at a much faster pace, fronted by Apple and its ARKit technology. Our expertise covers both of these areas, and it’s well-described in corresponding case studies, guides, and AR demos on our blog.
Feel free to contact us with any questions about AR/MR or if you need Augmented reality services.
Why Choose MobiDev to Build and Test Your AR Project
If you’re ready to make your idea a reality, our AR development team can help you find the solutions you need to gain an edge in the market.
Having an extensive experience with innovative technologies, our augmented reality developers know how to overcome the limits of existing AR frameworks to create more effective solutions. Combining AR with advanced artificial intelligence and machine learning algorithms can help you provide more accurate and realistic AR experiences, improve app performance and increase customer satisfaction.
Rely on our cross-domain multi-platform AR expertise and quality assurance services to create an exciting solution that brings real value to your business. Can’t wait to hear about your project idea!
