Modern technologies make it possible to transform customer services by improving the user experience and reducing the burden on the staff at the same time. Virtual user manuals, for instance, can provide users with 3D visual instructions on how to use a product, while AR remote assistance apps allow professionals to provide customer support in real time from miles away.
But what if we combined these two solutions into one that facilitated efficient onboarding and troubleshooting without human intervention? In this demo, we will showcase how a fusion of artificial intelligence and augmented reality technology enables us to accomplish this.
Kindly take note that this demo is not a fully developed product. It serves as a demonstration of applying AR and AI capabilities for onboarding and troubleshooting and can be utilized as a starting point for creating custom software tailored to your needs.
Benefits of Using AR and AI Combination for Onboarding and Troubleshooting
Using a combination of AR and AI technologies not only allows us to provide users with instructions on how to utilize a product but they also allow users to diagnose current troubles and automatically provide recommendations for solving them. For example, our demo app guides users on how to set up a Wi-Fi router and helps them solve simple setup issues. The app is utilizing augmented reality to demonstrate configuration steps for users in a more engaging way.
On the one hand, such an app improves the user’s onboarding experience with a new device. On the other, it provides instant customer support in case of difficulties without the need to involve a human specialist. In 90-95% of cases, customers prefer to solve problems with devices on their own without the help of a support center. So this solution helps businesses save money by reducing the number of call center workers and improving the quality of support.
In addition to the router, the AR/AI troubleshooting app can also help with setting up other devices that can be diagnosed using a smartphone camera, such as a TV, computer, game console, etc.
It can also be used for some defect detection tasks. But it works with general user-oriented cases rather than complex industrial cases that require additional hardware and more complex machine learning models.
How to Implement a Smart Troubleshooting Application with AR/AI
Let’s dive into the flow of creating an AR/AI onboarding and troubleshooting solution with the example of our demo for router diagnostics.
This demo uses the state of the LED indicators to determine the state of the router based on indicator classification: on, off, or blinking. Using an AI model, the system detects the router and receives info about LEDs, ethernet, and power pins connections.
Based on this data the app then provides users with tips visualized in 3D with augmented reality.
Technology Stack and Workflow
The key technologies used in this project are ARKit and CoreML, an AI framework for iOS app development. For the PoC we have used only the CoreML iOS framework, but in production, we proceeded with the pre-trained YOLOv2 model for object detection. Our engineers collected a dataset based on images taken by iPhone. Images should contain the router, clearly visible indicators, and ports. Then the data is labeled.
The workflow of building such an app includes the following steps:
- Define a list of Wi-Fi routers
- Investigate router specifications and visual feature points
- Implement basic features
- Define a list of features and approve supported routers for MVP
- Development, testing and release
The development team will also need examples of devices to be tracked to study their specifications and train the models. AI feature development is the most complex part of this demo since AR is used only to visualize hints for users.
We can build such a solution as an iOS or Android mobile app, integrate it as a feature of an existing application, or turn it into SDK. It all depends on your business goals and development strategy.
Challenges of Building a Solution Like AR Router Diagnostics App
The current demo has certain limitations that can be addressed using new datasets. This solution works only with a given router model. So, if you want to implement something like this for your business you should take into account the following considerations:
- a list of device models supported
- the variety of device forms
- indicators, and the number of indicators
- setup and diagnostic flows for different models
So, you may need to create a separate AI model for each model line of a router or another device. Also, don’t forget that if we need support for modes other than the default, we need to develop a system that lets the user choose the device mode they want to configure.
For more video demos, check out our
Augmented Reality Demo Collection by MobiDevWhy Choose MobiDev for AR App Development
MobiDev augmented reality developers and AI engineers have extensive experience in adopting AR and AI across different platforms and helping companies solve real business problems with these technologies. We can successfully extend augmented reality capabilities with artificial intelligence to create truly innovative and original solutions.
If you need a skilled development team capable of creating a complex multifunctional application for your business, we will be happy to assist you on this journey. Tell us about your project and find out how our AR development services can meet your needs and goals.
