Markerless Augmented Reality (AR) is the preferred image recognition method for AR applications. Learn how it works, the advantages of markerless AR, and how industries and companies benefit from incorporating this technology into their products and services.
In this article:
What Is Markerless AR?
Markerless Augmented Reality (AR) refers to a software application that doesn’t require prior knowledge of a user’s environment to overlay virtual 3D content into a scene and hold it to a fixed point in space.
Marker vs. Markerless Augmented Reality
Two of the most prevalent paths for experiencing AR are marker-based and markerless. Early-stage AR technologies were marker-based. Markerless AR is now the preferred image recognition method for AR applications.
Marker-based AR apps use markers (target images) to indicate things in a given space. These markers determine where the AR application places digital 3D content within the user’s visual field or through a camera feed.
Markerless AR places virtual 3D objects in the physical environment depending on the environment’s real features rather than identifying markers. This differentiation eliminates the need for object tracking systems. Markerless AR experiences are possible because of advancements in cameras, sensors, processors, and algorithms capable of accurately detecting and mapping the real-world.
How Does Markerless Augmented Reality Work?
Markerless AR merges digital data with input from real-time, real-world inputs registered to a physical space. The technology combines software, audio, and video graphics with a smartphone’s or headset’s cameras, gyroscope, accelerometer, haptic sensors, and location services to register 3D graphics in the real world.
Markerless AR detects objects or characteristic points of a scene without any prior knowledge of the environment, such as walls or intersection points. The technology is often associated with the visual effect that combines computer graphics with real-world imagery. The first markerless systems used a device’s location services and hardware to interact with available AR resources and define its location and orientation in space.
The development of simultaneous localization and mapping technology (SLAM) improved the accuracy of markerless AR image analysis. SLAM markerless image tracking scans the environment and creates maps of where to place virtual 3D objects. Even if the objects are not within a user’s field of vision, they do not move when the user moves, and the user does not have to scan new images. To learn more about SLAM markerless AR, read our guide to choosing an AR solution.
Location-Based Markerless Augmented Reality
Location-based AR merges virtual 3D objects into the user’s physical space. The technology uses a device’s location and sensors to unite the object to a point of interest.
Your immediate surroundings come alive, based on where you look. Niantic’s mobile game sensation Pokémon GO uses markerless, location-based AR technology. The game has more than 1 billion downloads worldwide.
Markerless AR is Everywhere
The emergence of advanced camera systems, mobile operating systems (OS), and sensor technology in ubiquitous mobile devices like the iPhone helped make advanced AR applications more accessible. Anyone with a modern smartphone or tablet running the latest Android OS or\ iOS can experience markerless AR.
Originally referred to as “dead reckoning,” markerless AR uses a combination of camera systems, dedicated sensors, and algorithms to accurately detect and map the real-world environment—such as the locations of walls and points of intersection. With a map of the area, an AR-enabled application can place virtual objects into a real context and have them remain stationary without the need for a target image.
Different Types of Markerless AR Systems
In its most basic form, markerless AR superposes virtual objects into a static, pre-captured 2D image. Of course, this is not the state-of-the-art method and straddles the line between AR and photo editing. That said, it’s straightforward and easy to implement for apps that want to offer offline AR instead of live experiences.
Markerless AR systems that use RGB-D SLAM and sensor fusion approaches are on the opposite end of the spectrum. Microsoft HoloLens is the most notable example. These systems integrate information from standard, red, green, and blue (RGB) cameras with state-of-the-art infrared time-of-flight cameras to construct a 3D map of the user’s surroundings while they use the application. This feature is a critical component of the SLAM tracking paradigm, as it enables apps running on these devices to place virtual content within the space concretely.
Advantages of Markerless AR
Markerless AR eliminates the need for capturing physical markers to trigger virtual interaction has the following advantages:
- You significantly increase the average range of motion while experiencing AR.
- You can initialize the application anywhere with a handheld device or a see-through headset like glasses or goggles.
- You can share the experience with others.
- You experience a wider field of view for AR content.
With a map of the area, an AR-enabled application makes it possible to place virtual objects into a real context and have them stay in place sans a target image. For example, with 3D Cloud’s Relative Tracking, users can walk around any open area (approx. 3-4 meters for most indoor locations)—far beyond the range of any extended tracking currently available.
Disadvantages of Markerless AR
Despite the benefits of markerless AR technology over earlier systems, challenges remain. The technology is mostly dependent on flat, textured surfaces to successfully render virtual images. Also, apps running on mobile devices use a lot of power.
Adoption is slow due to the lack of acceptance for AR wearables (glasses or headsets) and commercial investments. Still, AR headsets continue to grow as a segment of the gaming industry, with smaller markets developing in healthcare, academics, and industrial applications.
Pros and Cons of Markerless AR
| Benefits | Challenges |
| Increase range of motion with AR | Depends of flat, textured surfaces |
| Use a headset to initialize an AR app | Apps running mobile consume a lot of power |
| Share the experience | Slow adoption |
| Wider field of view for AR content |
Markerless AR Software Applications
User-centric markerless tracking in VR headsets will facilitate boundless virtual environments that can adapt to various commercial uses. Markerless AR software applications that track motion within a space has enormous potential to impact autonomous vehicle and robotics technology.
Imagine upgraded motorized wheelchairs with integrated markerless tracking technology. Wheelchair users could navigate the physical world with built-in obstacle avoidance and safe pathfinding features. They would use a headset with integrated eye-tracking and look at a place in front of them to activate. Autonomous vehicles could use the same innovations for navigation.
Top Markerless AR SDKs
Markerless AR solutions evolved from hardware-based products like Google Tango into software-based SDKs like Google’s ARCore and Apple’s ARKit. These SDKs produce the same effect without requiring specialized equipment.
ARKit and ARCore are responsible for making markerless AR available on hundreds of millions of smartphones and tablets. Here’s an overview of the top SDKs:
- Apple’s ARKit: Unveiled in 2017, the ARKit SDK brought core AR functionality (for example, plane tracking and anchoring) to the iOS operating system. Millions of people had an AR-capable device in their pocket overnight. The key features of ARKit are space recognition using SLAM, object detection, light estimation, and continuing the AR experience through several sessions. The SDK is free for developers to use.
- Google’s ARCore: In early 2018, Google came out with the free and open-source ARCore, essentially replicating the functionality of Apple’s ARKit. Key features include motion tracking, environmental understanding, light estimation, user interaction, oriented points, anchors and trackables, augmented images, and sharing. While ARKit and ARCore started life with feature parity, Google’s ARCore has suffered from the device fragmentation that hampers the Android platform. ARCore is on par with ARKit in terms of features, stability, and quality of the 3D images. They both offer development plugins for compatibility with Unity for Android and iOS and Unreal Engine 3D creation tool.
- 3D Cloud MxT Tracking: 3D Cloud’s proprietary AR tracking SDK, MxT Tracking, offers instant initialization compared to ARKit and ARCore. Those two SDKs are slow at establishing a tracking plane and rendering a 3D object into the scene. 3D Cloud’s MxT Tracking can also be paired with ARKit or ARCore, allowing developers to leverage each SDK’s best parts and deliver a premium AR experience to consumers. MxT uses a relative tracking approach. The tracking space scales based on estimates of how the user is standing or sitting with the application. Since it requires no action from the user to start placing objects on the ground, the experience affords rapid viewing of augmentations.
To learn more about top Augmented Reality SDKs, including a glossary of SDK terms and tracking specialties, see our AR SDK Guide.
Industries Using Markerless AR
Retailers, educators, and game developers are examples of industries adopting markerless AR technology. As more verticals adopt the technology, more data and feedback will empower designers and developers to create more consumer use cases.
Expanding the number of established use cases for markerless AR requires affordable and accessible devices and innovative software applications. Here’s a look at a few industries embracing the experience:
- Gaming: Markerless AR has seen its most significant impact in gaming, but we’re still at the beginning of its evolution. Since the release of Pokémon Go, a steady stream of AR games is hitting the market. ARKit/ARCore makes it possible to create multi-user real-world games with 3D content that remains fixed. Microsoft HoloLens and Oculus from Facebook lead the market for headset AR gaming and communication. These tech behemoths continue to develop new games and applications.
- Interior Design: Have you ever wanted to see how that new appliance might look in your kitchen? Or wondered whether you had room for a new sectional and a recliner? Markerless AR makes it possible to use a mobile device to scan a real-world environment such as a kitchen or living room, and virtually place an appliance or couch in the actual space. Shoppers can simulate furnishing an entire room virtually, swapping cabinet styles and flooring to see how they work before buying. Macy’s, Amazon, IKEA, and Ashley Furniture are just a few examples of retailers using AR design and product visualization to give consumers the freedom to shop and design inside their homes. This trend is accelerating with each new generation of hardware and updates to the AR SDKs.
- Retail: Amazon is developing AR to improve customer engagement and enhance the online shopping experience for home goods like furniture. In the future, markerless AR might display high-dollar purchases to buyers in 3D as an advanced version of the try-before-you-buy experience. The ability to interact and try-on products like jewelry, cosmetics, and clothing virtually before hitting the purchase button might bridge the gap for shoppers hesitant to buy these products online. Amazon is far from alone: Wayfair, Walmart, Made.com, and Alibaba are a few companies trying to beat Amazon to advanced AR experiences.
- Advertising: AR advertising started life as tracker-based print advertisements: The user would hold their device over the ad, and a 3D character would emerge. The technology freed AR marketing from its print prison, putting it into the real world. Companies already using AR advertising include Wayfair, the NBA, Sephora, and Gucci. The experience provides a differentiating factor for these big brands. The possibilities with AR advertising will become even more significant when wearables reach more mainstream acceptance. Attendees and live events like the Super Bowl could freely cycle between multiple virtual layers of advertising content and football-related experiences like on-field statistics and instant replays.
- Education: Markerless AR’s ability to overlay digital information on the real world is a boon to educators who use the tools to illustrate complex concepts. Apps like Anatomy 4D (for students learning about the human body) or Elements 4D (for those studying chemistry) are part of this early wave. Museums use AR to bolster their physical exhibits with useful digital content. Corporations and the U.S. military use Microsoft’s HoloLens to create training exercises and experiences on everything from repairing airplane engines to combat medicine and survival.
- Healthcare: Imagine an application that displays different color zones to indicate proximity to community flu and virus outbreaks. Health officials issue public health warnings to at-risk populations through mobile applications in real-time using their smartphone’s location. Surgeons can overlay 3D models of human anatomy on a patient to perform virtual-assisted surgeries using a powerful AI while remote attending surgeons supervise in real-time.
- Tourism: Location-based AR applications help travelers navigate foreign destinations. Travelers to Greece or Egypt might use AR to experience famous landmarks like the Parthenon and the pyramids. An app can display these structures as someone living thousands of years ago would see them. Users that prefer the confines of an upscale hotel can visit tourist hot spots and take in the sights without ever leaving their room. Online travel websites can lead you through a virtual tour of your next destination while you’re sitting in front of your computer.
- Entertainment: The availability of on-demand streaming content and HD audio and video already transformed how we consume movies and music. Markerless AR might create the next home entertainment evolution by reimagining live concerts and sporting events. Imagine having your favorite artist or athlete in the room while they perform at a live venue.
Companies Using Markerless AR
Markerless AR is a billion-dollar industry thanks to the success of Pokémon Go and more research and development. Here’s a list of companies that use the technology:
- Wayfair: Furniture and 3D interior design visualization
- National Basketball Association (NBA): Mobile NBA AR app provides unique on-demand content, including behind the scenes experiences, player intros, and post-game celebrations.
- Sephora: In-store AR smart mirrors so customers can try products.
- Microsoft: TeamViewer supports AR connections within Teams so users can collaborate on physical tasks in real-time.
- Alibaba: AR-infused shopping that features 3D holograms of products and virtual models to showcase apparel and accessories.
- Cirque du Soleil: The performing company partnered with HoloLens to design sets and plan show choreography remotely.
- Scope AR: The company creates code-free AR instruction manuals, including descriptive text, images, videos, 3D models, and animations for markerless, hands-free holographic directions.
The Future of Markerless AR
The future of AR is diverse and expected to grow beyond consumer-focused applications and video games. Industry-leading AR behemoths Apple (ARKit and RealityKit) and Google (ARCore) continue to update features and support for their AR developer platforms and devices.
With a market forecast of over $18 billion in 2020, merging immersive, digital content with the physical world has implications for all use cases. These opportunities are further enhanced with improved visualization, commentary, and storytelling. Consider the importance of these factors on remote teamwork. Imagine how markerless AR computing will impact the future of distributed work and remote collaboration in various sectors, especially healthcare and engineering.
Doctors, designers, engineers, marketers, and manufacturers will use AR computing platforms to host virtual project rooms and collaborate using virtual avatars in real-time. Conducting work across any time zone using 3D information to review and scope projects without travel requirements has tremendous implications in a post-pandemic economy. When you move beyond simple video feeds to virtual avatars that move, collaborate, and communicate in the same virtual space, you enhance the possibility of complicated workflows and synchronous collaboration previously reserved for in-person meetings.
Where Is the Sweet Spot for Markerless AR?
Looking for a markerless AR solution that will help you revolutionize the way consumers view your products? We’ve added markerless AR to our robust 3D Cloud platform. The stable, scalable, and extremely powerful will transform your commerce solution.
3D Cloud’s markerless AR technology doesn’t build a map of the surroundings. Instead, it provides an iterative update of the user’s position relative to wherever they started. This feature, known as relative tracking, yields a more immersive result than static offline alternatives.
Our system doesn’t require the initialization, calibration, or mapping overhead of SLAM, making it the ideal solution for virtual content visualization within small spaces and applications running on current mobile hardware.
Learn how 3D Cloud’s Markerless AR works for your project.
