Introduction to Diminished Reality
Diminished reality (DR) is a technology that enables users to remove objects from a scene and fill the resulting void in a plausible way. While its counterpart, augmented reality (AR), is well-known for putting digital objects in an actual scene, DR is less familiar but has quite a few potential applications. Researchers on the Institute of Visual Computing have made significant progress in developing DR technology, enabling the removal of objects from live recordings of three-dimensional environments without time delay, even when the camera is in motion.
How Diminished Reality Works
To achieve this, the researchers combined various technologies, including 2D inpainting, 3D scene understanding, and real-time processing. The process starts with user inputs on a 2D screen, that are projected right into a 3D scene to define the areas to be removed. A keyframe serves as the place to begin, and a plausible background is created by collecting and merging matching pixels from the environment of the article to be removed. The color information and depth data of the scanned scene are optimized concurrently to make sure a convincing result, even when the camera perspective is modified.
Applications of Diminished Reality
Diminished reality has quite a few potential applications in various fields. In the film industry, it might be used for efficient pre-visualization of film scenes at real locations. In the medical field, it might help remove distracting elements from the visual field of recording cameras during surgical procedures, providing undisturbed learning material for college kids. Additionally, DR might be utilized in the simulation of malfunctions in training data sets for autonomous vehicles. As Shohei Mori, considered one of the researchers, puts it, "Diminished reality is sort of a sort of Photoshop for 3D scenes."
The Technology Behind Diminished Reality
The latest technology, called InpaintFusion, uses 2D inpainting as a place to begin after which removes objects from a 3D scene. To achieve this in real-time, the researchers used two different technologies: fast patch matching and multithreading. With patch matching, essentially the most visually obvious surrounding pixels for the realm to be filled are looked for randomly to gather and merge the matching pixels. Multithreading utilizes the flexibility of computer processors to execute several processes concurrently on each processor core, allowing the computationally intensive 3D inpainting calculation to run within the background while the principal thread controls the visualization.
Making Diminished Reality Accessible
The biggest challenge was to make the processing steps possible in real-time. The researchers have succeeded in doing so, and now the duty is to develop the suitable toolkits to make these possibilities available to a wider audience. Another goal is the fast and efficient generation of 3D models from a small variety of individual images, which might give diminished reality a further dimension. As Mori says, "The real-time visualization of diminished reality is a vital step for this technology."
Conclusion
Diminished reality is a robust technology that might be utilized in various fields to remove objects from scenes and fill the resulting void in a plausible way. With its potential applications within the film industry, medical field, and autonomous vehicles, DR is an exciting development that may change the way in which we interact with and understand our surroundings. As the technology continues to evolve, we will expect to see more revolutionary uses of diminished reality in the longer term.
