Smartphone-based Augmented Reality in Indoor Settings: Challenges and Solutions
Augmented reality (AR) technology has grow to be increasingly popular, with quite a few apps allowing users to see digital information overlaid onto the actual world. However, in the case of indoor AR, the technology often struggles to operate effectively.
The Challenges of Indoor AR
Researchers at Osaka University have conducted extensive experiments to discover the explanations behind these issues and propose potential solutions. To do that, they used a mix of visual sensors, including cameras and lightweight detection and ranging (LiDAR) technology, in addition to the phone’s inertial measurement unit (IMU), which measures movement.
The Importance of Localization and Tracking
To successfully implement AR, the smartphone needs to find out its location (localization) and track its movement. The phone achieves this using visual sensors and the IMU.
Experimenting with AR in Real-World Environments
The research team conducted 113 hours of experiments across 316 patterns in real-world environments, disabling some sensors and changing the environment and lighting to isolate and examine the failure modes of AR.
Key Findings
The study found that virtual elements are likely to "drift" within the scene, resulting in motion sickness and reducing the sense of reality. The researchers also discovered that:
- Visual landmarks may be difficult to seek out from far-off, at extreme angles, or in dark rooms.
- LiDAR doesn’t at all times work well.
- The IMU has errors at high and low speeds that add up over time.
A Potential Solution: Radio-Frequency-Based Localization
To address these issues, the team recommends radio-frequency-based localization, corresponding to ultra-wideband (UWB) sensing, as a possible solution. UWB works similarly to WiFi or Bluetooth, and its most well-known applications are Apple AirTag and Galaxy SmartTag+. Radio-frequency localization is less affected by lighting, distance, or line of sight, avoiding the difficulties with vision-based QR codes or AprilTag landmarks.
Future Directions
The researchers consider that UWB or alternative sensing modalities, corresponding to ultra-sound, WiFi, BLE, or RFID, have the potential for integration with vision-based techniques, resulting in vastly improved augmented reality applications.
Conclusion
The study highlights the challenges of implementing AR in indoor settings and proposes a possible solution using radio-frequency-based localization. By understanding the constraints of current technology, researchers can develop simpler and practical AR applications, ultimately enhancing the user experience.
