Updated: February 1, 2021 Augmented reality technology has seen unprecedented growth in 2020. Commercial use of the technology has exploded due to use by market leaders like Microsoft, Apple, Google, Facebook, and AmazonMore
Augmented Reality Development Guide: Devices, Tools, Platforms
Augmented reality’s success continues to power into the future despite last year’s numerous and persistent challenges. With the augmented reality market valuing at $8.9 billion USD and growing, this is an incredibly important segment to take advantage of to remain competitive in technology. With an audience of 810 million active AR users on mobile, the potential engagement that can be captured by AR features and products cannot be understated.
Augmented Reality Technology Overview
Indeed, the audience for AR is growing. ARtillery Intelligence reports that the mobile usage of AR features is up 29% for adults in the United States. This is mostly due to dedicated AR apps running on the native ARKit and ARCore architectures for iPhone and Android devices. ARtillery Intelligence also reports that there is much more engagement with AR features that are included within apps rather than apps based entirely around augmented reality.
Gaming, social, and visual search are the most popular consumer use cases for AR. It’s no surprise then that Pokemon Go brought in $1.92 billion USD in 2020 and is experimenting with Microsoft Hololens with the eventual aim of bringing wearable gaming experiences to a larger audience.
The technology doesn’t stop with general consumers. Manufacturers and smart cities are another important segment. By using AR technologies, remote assistance can be performed in real time. This allows for better inspection and maintenance, as well as technical support. Smart city solutions also include AR technologies like navigation and urban planning.
Let’s talk more about these groundbreaking and advancing technologies in our 2021 augmented reality technology guide.
Augmented reality encompasses many facets of different industries. One of the most useful ways of differentiating augmented reality technologies is via the hardware that is used as the platform. Generally speaking, these are mobile devices, smart glasses, head-mounted displays, and web based AR for desktop and mobile devices.
Augmented reality development is very accessible and simple applications are very easy to get into. To get started, a basic understanding of geometry and 3D graphics is all that’s needed. However, more complex projects can require much more expertise. Machine learning and AI can also play a major role in the development of more advanced AR.
Mobile Augmented Reality
Mobile devices synergize the processing power, hardware camera, and Internet connection to deliver augmented reality experiences for users on the device’s display. This method allows companies to reach the most users because so many people already own AR capable smartphones. In fact, according to AR Insider, 29% of U.S. adults have already tried mobile AR. Many brands take advantage of AR experiences to drive customer engagement. In 2020, Facebook saw 300 million active users of its AR features.
Developing augmented reality-capable apps for mobile requires an understanding of ARCore and ARKit development tools for Android and iOS. These two development platforms power nearly all app-based AR experiences on mobile. In 2020, ARKit had 1,185 million users compared to ARCore’s 633 million.
ARCore’s larger diversity of Android devices necessitates that it be more flexible for different smartphones. Last year we saw Google begin to crowdsource Google Maps Street View photos so long as they met certain guidelines. ARCore software made it possible for users to take photos with the quality Google was looking for to enhance their Street View experience.
ARKit seems to outpace ARCore by a significant margin, but both have grown at a significant pace since their initial introduction. From 2017 to 2018, 13 million downloads on the Apple App Store were AR-kit only apps.
A repository search on Github shown below suggests that developers prefer to build apps with ARKit over ARCore.
|Date||ARKit repository results||ARCore repository results|
AR Development for iOS Devices: ARKit 4
Let’s talk about a few new features that can be invaluable for the success of the development of your ARKit app.
ARKit 4 introduced location anchors, a crucial feature that allows developers to affix virtual objects in the real world by using geographic coordinates. This allows for further immersion through visual localization. With latitude, longitude, and elevation, virtual objects can be placed into the world with even more accuracy than before with geographic information.
One of the most groundbreaking elements of location anchors is how location is approximated. GPS is simply not precise enough to provide location anchors on a user’s screen. To assist, developers can look around with their camera to allow the device to scan for features on buildings around them. By using these architectural features in conjunction with Apple Maps Look Around data, a user’s location can be better approximated for location anchors.
Since location anchors are dependent on Apple Maps data, this means that if the city is not supported by Apple Maps, functionality for location anchors may be limited.
Depth API is another valuable feature of ARKit 4 which utilizes one of the most powerful hardware features for AR on a mobile device, the LiDAR scanner on the iPad Pro and iPhone 12 Pro. This enables much better analyzation of scenes and allows for real world objects to occlude virtual objects with much better accuracy.
Apple’s Raycasting API enables enhanced object placement in conjunction with depth data. This allows much more accurate placement of virtual objects on a variety of surfaces while taking into account their curvature and angle. For example, it’s possible to use this to place an object on the side of a wall or along the curves of a sofa rather than simply flat on the floor. The LiDAR sensor allows for the scanning of surfaces to take place much more quickly compared to traditional methods.
Advanced Face Tracking
Due to more powerful software, face tracking technology that was once limited to TrueDepth cameras can now be used with iPhones with an A12 Bionic processor or greater. This allows for tools such as face anchors to be incorporated into app features. A TrueDepth camera is still important for more complex tasks utilizing face tracking AR.
The video below shows how iPhone employs AR-based gaze-tracking technology that creates use cases for device controlling performed with eyes, from serving users with limited abilities to hands-free experience and monitoring. Combined with computer vision, face-based AR opens up further opportunities for facial recognition apps.
AR Development for Android Devices: ARCore
In an effort to remain competitive, Google has pushed ARCore far to remain one of the most versatile AR development platforms in the world. Let’s explore some of the features that are used by developers.
This tool allows users to place virtual objects in physical space that can be viewed by other users on their own devices. Google has made sure that Cloud Anchors can be seen by users on iOS devices as well.
The following video will demonstrate how ARCore object detection features for a virtual user manual.
ARCore enables developers to work with high-quality facial renderings by generating a 468-point 3D model. Masks and filters can be applied after the user’s face is identified. This is one of the most popular use cases for AR app features.
Virtual business cards and advertising posters are just a few of the applications that are possible with augmented reality images. 2D markers can be used to implement these features as well as more advanced solutions like AR indoor navigation.
The following video demonstrates an ARCore-based indoor navigation application.
ARKit vs ARCore
Apple and Google have both positioned their respective AR platforms competitively against each other. This means that both will likely be comparable in capability for the foreseeable future. Apple has positioned ARKit with a slight advantage given the better control over the software and hardware of their devices. The LiDAR sensor is one of the most advanced depth sensing technologies on a mobile device currently available. However, it won’t be long before Android developers start to pick up the pace.
Google and Apple are battling over platforms, installed users, and money. Apple’s lifestyle brand image has attracted a great deal of developer attention since it is perceived as a better source of revenue for startups needing funding. Although developers favor ARKit, Google’s ARCore’s market size is much larger and growing.
Since they aim to provide competitive functionality, both platforms offer similar features and performance such as motion sensors, understanding environments, and inspecting changes in lighting conditions.
In the following demo video, we demonstrate how ARKit and ARCore perform in recognizing and augmenting images.
By putting the Mona Lisa painting into the camera frame, the app is able to animate the subject by making it blink her eyes. The video demonstrates the differences in performance between ARKit and ARCore in executing this task.
Despite their differences, the way that AR scenes are calculated in ARCore and ARKit are nearly identical. However, it’s most likely that the reason why ARKit outpaces ARCore in performance is because Apple has much more control over the hardware and software of their devices. Since they develop both iPhones, iPads, as well as iOS, Apple can better optimize them for AR experiences.
It will be interesting to see how each platform’s competitive advantages play out. In other words, it is likely deeply unwise to pick a winner and a loser in this contest. Most likely, your business is going to have to create solutions that users can access on both platforms. That means it’s essential to understand the commonalities, strengths, and weaknesses of both ARKit and ARCore.
Augmented Reality Development For Wearables
When we talk about wearable technology for augmented reality, we typically are referring to display technology strapped to one’s head or a pair of glasses. Although the shape and form factor of this technology can vary, the most popular forms are gear like Microsoft HoloLens and more portable and comfortable glasses like Google Glass.
There is a big difference between these two types of devices, not only in purpose but in potential. Larger head-mounted displays like HoloLens are capable of much higher processing power. This enables them to provide experiences otherwise currently impossible for more compact devices like Google Glass.
Because of the smaller form factor and reduced processing power, AR glasses are more focused on providing two dimensional experiences similar to that of your smartphone. For example, displaying notifications, text, directions, photos, in your field of vision are solutions that AR glasses can reasonably provide. Voice-based Internet search and control are also feasible. Google Glass and the now discontinued Focals by North only project this information into one eye in two dimensions. Future glasses may be able to project this information into both eyes.
Hearables are also entering relevancy. Technologies like spatial audio are already making their way onto earbuds like the AirPods Pro, and more office-centric hearables like Microsoft’s Surface Earbuds.
Glasses and larger head-mounted displays have two different roles in the market at the moment with specific use cases. Let’s talk about both of them in more detail.
HoloLens was first announced in 2015, however its initial version has since been discontinued. Currently Microsoft is offering HoloLens 2 to business consumers only. It is battery powered and does not require a cabled connection to another device. Although HoloLens 2 has shown enough promise to be acquired by the military in a $22 billion dollar deal, the limitations and cost of the product are great barriers preventing it from being adopted more widely.
Organizations face large challenges when adopting technologies like HoloLens at scale. Incorporating new technologies can be difficult for businesses because they need time and effort to adjust and learn. HoloLens requires an entirely new way of working with others and information. This calls into question what the impact will be not only on the business itself but on the quality of its customer service.
Magic Leap, an AR headset positioned as a potential competitor to HoloLens, was discontinued in 2020.
Augmented Reality Development for Smart Glasses
Smart glasses have great potential to become what the iPhone was for consumers in the augmented reality space. Being able to have a comfortable to wear and fashionable platform for augmented reality experiences that could be mass produced and affordable for consumers would be game changing for many industries, especially retail.
AR experiences on smartphones are useful and helpful. However, viewing virtual objects through a smartphone as a portal can be inconvenient. It’s much more comfortable and realistic to see virtual objects through your natural field of vision via smart glasses. This could fundamentally change many industries, especially retail.
However, there are many challenges when attempting to make technology this sleek and this powerful. Fashionable AR glasses won’t be able to do nearly the same kinds of graphically challenging tasks that larger headsets can.
Focals by North were a notable AR Glasses startup recently acquired by Google. They were ambitious but limited, and have since been discontinued. This suggests that Google may be interested in bringing back smart glasses into the mainstream. Dream Glass 4K is another smart glasses project that can be paired with a smartphone.
Web-Based Augmented Reality Development
Although the majority of users are experiencing augmented reality through mobile apps, this is not the only way to experience the technology. Web-based AR is a more accessible avenue of development that does not require users to download apps. Instead, the AR experiences are loaded through a web browser on a device.
This is crucial due to app retention. Apptentive reports that after just one month, an app may lose on average about 35% of its users. If users don’t have to download an app to experience AR content, they will be much more driven toward engaging with it.
Example of WebAR: Dell Immersive Portals
Dell’s WebAR Immersive Portals are a great example of web AR. When stepping through these virtual ‘portals’ placed in physical space, users could immerse themselves in a new destination elsewhere in the world. AR 3D business cards are another useful application created by Aircards. These experiences are started by scanning a QR code on the back of the business card. When viewed through a device, the business card becomes fully animated in 3D space.
Limitations of WebAR
Since WebAR applications are extremely limited by the web browsers that they run in, WebAR is not a great solution for complex projects. WebAR may be sufficient for simple projects such as virtual business cards, simple face tracking and other solutions. However, for more complex operations, mobile AR is much more capable since it can unlock the full potential of the mobile device that it’s running on. Hopefully in the future technology will improve to make WebAR more capable to allow for more accessible AR experiences.
The key for success of Web-based AR will be a greater number of devices to support it and more websites to have it enabled. Although these standards have yet to be established, the implementation of AR in browsers is also performed by means of either porting existing libraries (e.g. AR.js) or developing new ones (e.g. A-Frame, React 360).
Augmented Reality Development in 2021: Main Takeaways
Getting past hardware and software limitations is one of the main challenges of augmented reality development. Although technology is advancing rapidly in 2021, there are still important obstacles to consider such as hardware limitations on mobile devices and operating system versions. One of these major challenges is the diversity of Android devices and their unique specifications that developers have to deal with.
Understanding these problems is key to coming up with a solution. When we work together and plan a path to success as a team, anything is possible in augmented reality development.
Want to get in touch?contact us
Thank you for subscribing