(left) Projection mapping result under the proposed lighting condition.
(right) With and without projection mapping for a plaster bust under various illumination conditions.
Illumination reproduction results.
Reproducing mirror-like material using projection mapping under various illumination conditions.
Researchers from Osaka University can project images onto objects clearly in an illuminated environment by controlling both the image and illumination, enabling viewers to see not just the display but also their surroundings and each other
Osaka, Japan – Images projected onto objects in the real world create impressive displays that educate and entertain. However, current projection mapping systems all have one common limitation: they only work well in the dark. In a study recently published in IEEE Transactions on Visualization and Computer Graphics, researchers from Osaka University suggest a way to bring projection mapping “into the light.”
Conventional projection mapping, which turns any three-dimensional surface into an interactive display, requires darkness because any illumination in the surroundings also illuminates the surface of the target object used for display. This means that black and dark colors appear too bright and can't be displayed properly. In addition, the projected images always look like they are glowing, but not all real objects are luminous, which restricts the range of objects that can be displayed. Displays in dark environments have another disadvantage. Multiple viewers can interact with an illuminated scene, but they are less able to interact with each other in the dark environment.
“To get around this problem, we use projectors to reproduce normal illumination on every part of the room except the display object itself,” says Masaki Takeuchi, lead author of the study. “In essence, we create the illusion of global illumination without using actual global illumination.”
Projecting global illumination requires a set of techniques that differs from those of conventional projection mapping. The research team use several standard projectors to illuminate the room along with a projector with a wide aperture and large-format lens to soften the crisp edges of shadows. These luminaire projectors illuminate the environment, but the target object remains in shadow. Conventional texture projectors are then used to map the texture onto its shadowed surface.
The researchers built a prototype environment and evaluated the performance of their approach. One aspect they evaluated was whether the objects were perceived by humans in aperture-color mode (where the colors appear to radiate from the object itself) or surface-color mode (in which the light appears to be reflected from a colored surface).
“To our knowledge, we are the first to consider this,” says Daisuke Iwai, senior author of the study. “However, we believe it is fundamental for producing realistic environments.”
The researchers found that, using their method, they could project texture images onto objects without making the object appear to glow. Instead, the textures were perceived to be the true colors of the object’s surface.
In future, the researchers plan to add more projectors to handle the complex illumination in the areas next to the display object. Eventually, they aim to produce scenes that are indistinguishable from real-world three-dimensional scenes. They believe that this approach will enable visual design environments for industrial products or packaging, in which the participants can interact not only with their design under natural light but also with each other, facilitating communication and improving design performance.
###
The article, “Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous Projectors,” was published in IEEE Transactions on Visualization and Computer Graphics at DOI: https://doi.org/10.1109/TVCG.2024.3372031
About Osaka University
Osaka University was founded in 1931 as one of the seven imperial universities of Japan and is now one of Japan's leading comprehensive universities with a broad disciplinary spectrum. This strength is coupled with a singular drive for innovation that extends throughout the scientific process, from fundamental research to the creation of applied technology with positive economic impacts. Its commitment to innovation has been recognized in Japan and around the world, being named Japan's most innovative university in 2015 (Reuters 2015 Top 100) and one of the most innovative institutions in the world in 2017 (Innovative Universities and the Nature Index Innovation 2017). Now, Osaka University is leveraging its role as a Designated National University Corporation selected by the Ministry of Education, Culture, Sports, Science and Technology to contribute to innovation for human welfare, sustainable development of society, and social transformation.
Website: https://resou.osaka-u.ac.jp/en
