Ikeuchi and his team have addressed a great number of modeling and research issues that put them at the forefront of cyber-architecture as well as virtual tourism. The Digital Bayon Project, that has recorded the highly detailed Khmer temple at Angkor in Cambodia, provided a number of challenges with new sensors and software developed for large-scale data collection with both geometric (shape) and photometric (photo) information captured.
The central tower of this complex was a particular challenge as it is compromised, with good possibility it will collapse in the future. To capture this tall central structure with laser scanners the researchers deployed a balloon controlled from four wires to the ground. With the sensor moving in four dimensions, a television camera was used to rectify the scanner position, and algorithms post processed the scanned imagery to rectify and derive 3D coordinates for the model output.
The project captured detailed 3D moels of both the inside and outside of the temple, fitting scanners into tight areas and deploying new climbing sensor technology to reach into tall and narrow crevices. In addition to sensor advancements, the team made a number of photometric advances to match textures and colors to the 3D model. With illumination varying on the site over the course of the day due to the position and effect of sun, the team developed automated tools to strip the suns influence on color to provide a color balanced model.
The ultimate goal of these digital models is to provide a Cloud Museum to tell the history and stories of these areas and to aid in global cyber archaeology. While one outcome is the display of video in and models in theaters, there is also an interest in providing an augmented reality that mixes today’s reality with the historical scene.
In order to add to the realism of this virtual tour, the team is working to add shadows and anchor the models in reality with realism that includes a model that changes as the day changes with backgrounds that gets darker, and that accommodate the movement of people in the scene, with their shadows projected on the 3D image.
The team has also created a drivethrough to communicate ancient, but missing cities in a 3D view that can be seen from a moving platform. These moving tours of old city sites have been designed to offer a time machine effect, populated by virtual actors in period costumes. Riders on the moving 12-passenger bus each have head mounted displays with audio that diplay computer-generated images as well as the actual surroundings.
The website for the Computer Vision Laboratory at the University of Tokyo contains a number of galleries and videos that put these concepts into context.