All You Need To Know About Augmented Reality Testing
If your business is planning to launch an AR application project, you probably know that there are two crucial activities that will define user satisfaction with the final product. The first one is natural and immersive user experience. The second one—and the subject of today's story—is quality assurance. That said, read on to learn all about the specifics and challenges in augmented reality app testing, illustrated with a case involving our recent Microsoft HoloLens demo.
Augmented Reality Testing: Preliminary Work
The Specifics of AR Testing
Owing to the diversity of AR-supporting smartphones, tablets, and AR headsets, testing on all devices listed in the product requirements is essential. Emulators obviously cannot substitute real devices when it comes to finding and eliminating possible issues within real physical environments.
A Case of Microsoft HoloLens Application Testing
The best known headset on the modern market is arguably Microsoft HoloLens. It is marketed as a mixed reality device that interacts with physical objects in the real environment. However, its testing specifics are quite similar to those of AR headsets. Upon launch, the HoloLens scans the environment and empowers the user to fill it with digital objects, control them with a special menu and make them interact. Testing basically helps all of these operations to run smoothly and precisely. Here is the procedure of HoloLens application testing that used in our own projects.
Scanning the environment
The device had to detect all meshes, including physical objects and surfaces, under different spaces, angles, distances, motion and lighting conditions, and levels of ambient noise. QA engineers tested the application in new environments and placed new physical objects in the room to see how the scanning went.
Scanning was followed with making sure that the placed digital objects (holograms) would move quickly and smoothly. QA checked how the holograms would interact with surfaces: for example, whether they would "fall" into surfaces, or they could be "hidden" behind such objects as walls or ceilings.
Controlling an object
It was critical to make sure that the user would be able to access the model menu at any time, under any conditions, regardless of the size and location of the model. For that purpose, QA scaled the model to maximum and minimum to see whether the menu buttons could be reached easily. It was also important to ensure adequate distance between the model and the menu. The user should be able to hit the buttons and catch them to control the model from the first or second try. QA checked whether the model reacted to the clicker and all the gestures.
It is worth noting that this approach to interaction is quite novel for people. That is why it is vital to make the overall experience of any augmented reality app easy and convenient for anyone.
It should be possible to move the model without any additional gestures simply using gaze, if this condition is mentioned in the product requirements.
It sometimes happens that the user is launching several similar holograms, and then launches an animation (say, rotation effect) for one hologram, other holograms start to rotate as well. Our case covered this issue.
There are no visible messages about the system crash for the HoloLens. This issue should always be discussed before the development starts. In our case, crashes were made visible to the user, so that they would be able to relaunch the application.
Immersion in an augmented reality app sounds great in theory, yet it comes with health impacts and keeps causing headaches, eye strain, and motion sickness. Modern headsets are continuously perfected; however, it is up to the software team to create a product that can be used regardless of age and physical condition, reduce discomfort, and ensure quick and natural navigation. This must be kept in mind from the very start of the project and reflected in requirements and the actual code.
Additionally, it is worth noting that the native Admin Panel of the HoloLens is a useful tool that allows to stream MR experience in real time and record videos. Thus communication and troubleshooting becomes easier.
Daria Bondarieva, Quality Assurance Engineer
"With headset-based AR and MR still very much in its infancy, research is more vital than ever in this continually developing field. Meanwhile mobile AR advances at a much faster pace, fronted by Apple and its ARKit technology. Our expertise covers both of these areas, and it's well-described in corresponding case studies and guides on our blog.
June 21, 2018
We can name two recent events that pretty much defined the future of augmented reality for the rest of this year. Our team... more →
Digital transformation in healthcare disrupts the traditional methods of doing business. It replaces paper-based management with... more →
Augmented Reality is sweeping the globe with dynamic information delivery and innovative ways of making and selling products and... more →