by Max von Übel ∙
Augmented reality apps attract much attention. We review the best software that will help you create your own custom AR apps and games.
The basic idea of augmented reality is to overlay digital content onto the real world giving users an unmediated access to information. Most extant devices like smartphones and smartwatches can be integrated into the AR ecosystem. Although the technology is only now taking off, some older handhelds like the PS Vita or the Nintendo 3DS already offered support for augmented reality apps.
The next stage in the augmented reality evolution will be dedicated AR Smart Glasses that aim to integrate digital content without the need to hold a phone in our hands. According to some experts this will promote the use of AR technology in our everyday life.
Smart Glasses are already available for business applications from Canon (MREAL) and Sony (Smart Eyeglass), and consumer-grade devices are expected to be released in 2018. The most notable among these is the Hololens by Microsoft.
Because AR is compatible with hardware most consumers already have they are not compelled to buy new gadgets. Unlike virtual reality, this makes augmented reality apps attractive for broader target groups.
For this reason, Google and Apple released their own AR SDKs in summer of 2017 for Android and iOS respectively.
Apart from games, augmented reality technology is already in use for a broad range of commercial, technical and scientific applications.
Already today, many businesses use augmented reality apps as a way to let their customers engage with their products. BMW, for instance, released a series of apps that allow consumers to preview a car before placing their orders. IKEA lets their customers redecorate their homes with their furniture using a recently released AR app. In short, AR apps can be used to broaden the appeal of a brand among prospective customers.
Also, AR experiences have also been successfully adapted for maintenance of technical equipment. Barcodes trigger animations to demonstrate how components can be replaced or how certain materials should be handled.
Jump to our selection of augmented reality CMSs!
In this article, we shall focus on making augmented reality apps that require no special hardware to display the augmented reality content. Therefore, we will not discuss Smart Glasses like the Hololens. Also, we will only brush dedicated tracking hardware, like Google’s Project Tango technology.
Instead, we deal with augmented reality apps made for smartphones, the bread-and-butter technology we all carry around with us. Rest assured that many of the concepts and tools mentioned here are also useful for dedicated devices like Smart Glasses.
Put simply, most augmented reality apps consist of two components: a game engine that handles the 3D assets and interaction with the user, and an AR SDK that supplies the tracking data needed to overlay the 3D content consistently in the physical world.
First, let’s take a look at game engines and then AR SDKs. Finally, we’ll show you two nifty programs that allow you to make an augmented reality app without writing a single line of code!
Because augmented reality apps display 3D content they are usually built on top of game engines like Unity or Unreal. These provide a framework to program custom-fit experiences using technologies that all skilled game designers are already familiar with.
So far, Unity is the leading game engine in AR, since this is the engine most AR SDKs support. Big SDKs like Vuforia have been criticized for not supporting Unreal thereby forcing studios to focus on only a single solution. Understandably, Unreal wants its share of this highly lucrative market.
However, the release of Apple’s ARKit marks the beginning of support for Unreal on a large scale. What is more, Google’s ARCore supports Unreal engine out of the box as well. To fully gauge the effects of these important changes in the augmented reality market we will have to wait until the first wave of ARCore apps is released.
At this point, it seems that Google is trying to use the added augmented reality capabilities as a selling point for bleeding edge Android phones. Whether this strategy will pan out or will discourage the distribution of AR technology on the consumer market remains to be seen.
AR SDKs are loaded into the game engines as packages. At the core, they provide sophisticated tracking technologies that allow the app to register the user’s movement in the physical space relative to the virtual content. The SDK’s feature set is integrated into the app’s user experience. This presupposes some familiarity with programming languages like C++ or Java.
Setting up the AR functionality will only take a fraction of the overall development time. Developers mostly spend their time creating the 3D content and game mechanics that are identical to non-augmented reality experiences.
Unfortunately, this is not all. In order to make your app available to the public, you need to distill it into an app for either Android or iOS. Unity and Unreal come with handy presets that ease most of the process. So brace yourself, as you may need to get your hands dirty in Xcode or Android Studio.
Jump to our selection of augmented reality SDKs!
Choose an augmented reality software development kit (SDK) that compliments the app you are designing. Many commercial and free options are available providing a wide range of tracking technologies.
Some AR SDKs require the consumer to point the camera to a predefined target object, usually a card carrying a barcode that must always in view to correctly place the 3D content. Others can scan discrete physical objects to create the illusion of interaction between the virtual and the real world. Imagine your living room table becoming the playing field that you can view from any angle.
The most advanced AR tracking systems (SLAM, simultaneous localization and mapping) are robust enough to track objects using the natural environment. SLAM solves the fundamental problem of all augmented reality apps: to map an unknown environment while tracking your own location within it. Apple’s ARKit even supports tracking the user’s facial expressions.
However, you should not feel discouraged from making your own augmented reality apps. There is a second kind of production software that is geared to the needs of non-specialists that allows laymen to produce immersive AR experiences quickly and without writing a single line of code.
Imagine these programs as content management systems where users can upload a target image and a 3D file onto a platform that takes care of the pesky tech side of AR.
Consumers download and install an augmented reality app by the software manufacturer, often referred to as an “AR browser”. When the phone’s camera scans the target object saved in the augmented reality CMS the linked AR content is displayed.
While this take on augmented reality experiences is sufficient for many purposes and easy enough that most people can design an AR experience, these apps sometimes limit the expressive possibilities.
AugmentedPro is published by a French company as an industrial grade AR solution. As such it brings impressive tracking capabilities to the table. In terms of its architecture this software is designed to be deployed on every platform imaginable. Apart from smartphones, this solution can be used to design augmented reality apps for eyewear, like Microsoft Hololens.
What makes AugmentedPro truly stand out from the other solutions on this list is its capability to anticipate the trajectory of moving target objects – even if that object has become occluded! Therefore, this bleeding edge motion tracking technology is perfectly suitable for the most advanced applications like autonomous vehicles.
This AR package is more comprehensive than most other solutions. However, the time you need to put into your application is well spent, as AR experiences made with AugmentedPro are more robust, than those made with Vuforia.
But AugmentedPro is not only suitable for experts. The Creator app allows you to develop entire apps without writing a single line of code. Multimedia assets that can be incorporated include text, images, 3D models, and videos.
Apple rolled out native support for augmented reality with iOS 11. This adds to the appeal AR already has. It allows more people to enjoy AR experiences on devices that people already have, provided it has the Apple A9, A10, or A11 processor.
ARKit relies on two inputs to place and track camera in a scene: Visual Inertial Odometry (VIO) and Core Motion data. The true technological breakthrough achieved by Apple has largely gone unnoticed: up until the release of ARKit, augmented reality apps depended on markers of known dimensions as a reference for scale. Apple combines the data from the built-in camera with sensor data from the gyroscope and accelerometer. To break it down in laymen’s terms: the camera provides the information of movement that is scaled according to the force measured by the sensors. This remarkable feat allows creating augmented reality apps that are even more accessible since they don’t require companies to send physical markers to their customers.
In addition, Apple has stressed a feature called “scene understanding” that enables iOS devices to find horizontal planes in the view to accurately track and place objects. Its light estimation feature allows 3D content to fit seamlessly into the physical environment by applying the correct amount of light available to a scene.
Apple also deserves praise for its advanced face tracking technology. It enables consumers to control the facial expressions of 3D characters using blending shapes.
Unity engine’s monopoly on the AR market was eventually broken by ARKit’s support for Unreal engine. Giving developers greater creative and economic freedom.
Just like Apple, Google has high stakes in the emerging AR market and does not want to leave it development in the hands of external developers. Therefore it will be offering native augmented reality support beginning in fall of 2017.
The demos available promise an experience that is at least equal to Apple’s ARKit. Under the hood, ARCore is equipped with markerless motion tracking and detection of horizontal surfaces. Its light estimation is instrumental in placing 3D content seamlessly into the physical environment.
In order to entice its customers, Google is going out of its way with a range of new apps. For instance, the company joined forces with entertainment studios for its new app “AR Stickers”. Here, you can pose with beloved characters from Star Wars, Stranger Things, and other high profile franchises in front of the camera.
Google is trying to make the development of augmented reality content easy, as ARCore supports not only Unity engine, but also the Unreal engine. But that’s not all, developers can use Android Studio Blocks, Google’s free VR 3D modeling software which could potentially heighten the appeal for non-specialists, where this app not currently limited to pricey high-end headsets (Vive and Oculus Rift). Google also offers a Chromium plugin for use of augmented reality in websites.
But ARCore is not Google’s first venture into AR. Already in 2014, the company released Project Tango, a sensor that was developed for tracking the physical environment of an Android phone. Since consumers shied away from buying a phone only to enjoy a technology that hadn’t proven itself Tango never really caught on. In ARCore, the job Tango did is taken over by smart programming.
Unity engine’s monopoly on the augmented reality market was eventually broken by ARCore’s support for Unreal engine. Giving developers greater creative and economic freedom.
The biggest drawback of ARCore is that it will apparently only support a small number of bleeding edge phones using Android 7 and above.
The AR software by Kudan is one of the pearls on this list, as it includes SLAM (simultaneous localization and mapping) – fundamental problem of all augmented reality solutions is to map an unknown environment while tracking its location.
This approach is by far more sophisticated than the run-off-the-mill rotational tracking many other solutions offer. For one, unlike Vuforia, Kudan’s SLAM does not require any markers to work correctly. Instead, the framework automatically tracks patterns on surfaces and edges of objects in the environment to deliver the data neccessary for tracking. On the flipside, SLAM has a hard time tracking on nondescript surfaces, like white tables.
As a plus, Kudan’s SLAM can be integrated modularly with other tracking technologies. To put it into perspective, Kudan’s tracking technology is more robust than Vuforia’s extended tracking.
The markerless tracking can be harnessed to develop augmented reality apps for a broad range of use cases. Apart from consumer AR experiences for mobile devices, Kudan is already in use to drones, robots and even autonomous driving.
Like the AR framework with the biggest market shares, Vuforia, Kudan currently only connects to Unity.
Beyond any doubt, Vuforia is the best known, mature AR SDK on the market.
Vuforia supports large range of tracking objects: flat images (print media, product packaging), “VuMarks” (custom made markers that contain data in many formats), and text recognition.
Another category are cylinder targets that enable tracking images on cylindrical objects like bottles, cups, and cans. Most interesting is object recognition which allows tracking based on scanned objects. This enables sophisticated interactions with physical objects such as maintenance of technical components or bringing toys to life in AR.
The newest addition to Vuforia tracking stable is branded “smart terrain”. This approach to tracking is based on the user’s environment. It allows exciting possibilities, as the reconstructed environment can be incorporated as geometry into the augmented reality experience.
More importantly, Vuforia’s recent addition of extended tracking capabilities allows more robust AR experiences. In case the tracking object is no longer in view of the camera, Vuforia is able to instead use the user’s environment.
Beyond any doubt, Vuforia is the most mature AR SDK. Unfortunately, Unity is the only game engine Vuforia supports. However, Unreal support can be implemented using 3rd-party plugins.
Augment is an ecosystem that is primarily designed to facilitate the creation of augmented reality apps for product presentations of manufacturers and retailers. The solution consists of an app used by the consumers, a web platform to create and maintain AR experiences, and Augment Desktop which allows you to preview and set up 3D models before uploading it to the platform.
This AR software runs both with and without markers. The markerless mode is recommended when the 3D model is standing on the ground, while marker mode is ideal when you want to, say, enhance a printed catalog or brochure with an augmented reality experience.
Augment can be set up by anyone, as it requires no programming skills. The only prerequisite for Augment to work is a 3D model. But this is not to say that Augment can only display trite static 3D models. This AR solution is adaptable to the level of complexity you feel is appropriate, products can come to life with triggered animations.
What is more, customers can configure the product with different materials. The user interface can be customized to display your company logo.
In case you prefer give your customers an AR experience that is tailored to your brand, Augment offers an SDK that will give you the freedom to program an app from scratch.
The pricing of Augment is quite unusual when compared to competitors, as it is not charged per app but on the monthly views of the model.
Wikitude Studio is an end-to-end solution that enables quick deployment of augmented reality content without the need for programming. The development app runs in the web browser, here you can upload your multimedia assets and connect them to trackers.
This AR solution supports markerless tracking, image tracking, and object tracking. Behind the scenes, Wikitude features high-end SLAM tracking (simultaneous localization and mapping). SLAM solves the fundamental problem of all augmented reality solutions: to map an unknown environment while tracking your location within it.
This approach is by far more sophisticated than the run-of-the-mill rotational tracking many other solutions offer. For one, unlike Vuforia, SLAM does not require any markers to work correctly. Instead, the framework automatically tracks patterns on surfaces and edges of objects in the environment to deliver the data necessary for tracking. On the flipside, SLAM has a hard time tracking in low-feature environments, like white tables.
Besides Wikitude Studio, the publisher offers a fully fledged augmented reality SDK that is a viable alternative to Vuforia.
License: The text of "How to Make Augmented Reality Apps, Games & More" by All3DP is licensed under a Creative Commons Attribution 4.0 International License.