Associate Professor
National Institute of Informatics
Served concurrently as Associate Professor at the Graduate university for Advanced Studies and
as Visiting Associate Professor at Department of Information Processing,
Tokyo Institute of Technology.
Recent News
- Internship opportunity at NII
- Paper on BRDF modeling accepted at ACCV 2012
- Paper on Spectral Denoising accepted at ICPR 2012
- Paper on Bispectral Photometric Stereo accepted at CVPR 2012
- Paper on Camera Spectral Sensitivity Estimation from a Single Image under Unknown Illumination accepted as oral presentation at CVPR 2012
Research
Fluorescent Analysis
Photometric Stereo: We propose a novel technique called bispectral photometric stereo that makes effective use of fluorescence for shape reconstruction. Fluorescence is a common phenomenon occurring in many objects. One of the important characteristics of fluorescence is its wavelengthshifting behavior: fluorescent materials absorb light at a certain wavelength and then reemit it at longer wavelengths. In this work we show that there is a strong similarity between fluorescence and ideal diffuse reflection and that fluorescence can provide distinct clues on how to estimate an object’s shape. Fluorescence’s wavelength-shifting property enables us to estimate the shape of an object by applying photometric stereo to emission-only images without suffering from specular reflection.
Camera Sensitivity Estimation: Camera spectral sensitivity plays an important role for various color-based computer vision tasks. In this work, we present a single-image estimation method using fluorescence with no requirement for a known illumination spectrum. Under different illuminations, the spectral distributions of fluorescence emitted from the same material remain unchanged up to a certain scale. Making use of this chromaticity invariance, the camera spectral sensitivity can be estimated under an arbitrary illumination whose spectrum is unknown.
Separation of reflective and fluorescent components: fluorescence is a very common phenomenon observed in many objects, from gems and corals, to different kinds of writing paper, and to our clothes. In this paper, we provide detailed theories of fluorescence phenomenon. In particular, we show that the color appearance of fluorescence is unaffected by illumination in which it differs from ordinary reflectance. Moreover, we show that the color appearance of objects with reflective and fluorescent components can be represented as a linear combination of the two components. A linear model allows us to separate the two components using images taken under two unknown illuminants using independent component analysis(ICA).
Shape Estimation
Shape from Similarity: we present a technique for determining an object’s shape based on the similarity of radiance changes observed at points on its surface under varying illumination. We show that the similarity between two observation vectors (a sequence of pixel intensities of a point on the surface under varying lighting) is closely related to the similarity between the surface normals of the corresponding points. This enables us to estimate the object’s surface normals solely from the similarity of radiance changes by using dimensionality reduction.
Attached shadow coding: We present a novel technique, termed attached shadow coding, for estimating surface normals from shadows when the reflectance and lighting conditions are unknown. Our key idea is encoding surface points via attached shadows observed under different light source directions and then estimating surface normals on the basis of the similarity of the attached shadow codes. Because shadows do not rely on reflectance properties, our method is applicable to surfaces with various complex reflectances such as anisotropic and composite materials.
Modeling and Rendering Appearance
Spectral Reflectance Recovery: we propose a novel and practical approach for fast spectral reflectance recovery by exploiting the temporally modulated light emitted from Digital Light Processing (DLP) projectors. Based on this approach, we build an imaging system in which an o-the-shelf DLP projector is used as light source, a high-speed camera as well as a diffuse white board is adopted to reverse-engineer the temporal illumination variation of the DLP projector. This temporal illumination variation, also termed as "color switch”, makes DLP projector an easily available light source for spectral measurement as fast as 100Hz. Based on the measurement, spectral reflectance of scene points can be recovered.
Appearance Sampling: The appearance of an object greatly changes under different lighting conditions. In this study, we present a novel method for analytically obtaining a set of basis images of an object for varying illumination (harmonic images)from input images of the object taken properly under a set of light sources, such as point light sources or extended light sources. Our proposed method incorporates the sampling theorem of spherical harmonics for determining a set of lighting directions to efficiently sample the appearance of an object.
- I. Sato, T. Okabe, Y. Sato, “Appearance Sampling of Real Objects for Variable Illumination,” Int. J. Computer Vision, 2007.
- I. Sato, T. Okabe, Y. Sato, K. Ikeuchi, “Using Extended Light Sources for Modeling Object Appearance under Varying Illumination”, Proc. IEEE ICCV05, pp. 325-332.
- I. Sato, T. Okabe, Y. Sato, and K. Ikeuchi, "Appearance Sampling for Obtaining a Set of Basis Images for Variable Illumination," Proc. IEEE ICCV03, pp. 800-807.
Reflectance Morphing: we consider a new approach for 3D shape and reflectance morphing of two real 3D objects. Our morphing method consists of two components: shape and reflectance property measurement, and smooth interpolation of those measured properties. The measured shape and reflectance parameters are used to compute intermediate shape and reflectance parameters. Finally, the computed shape and reflectance parameters are used to render intermediate images which represent a smooth transition between the two objects.
- Y. Sato, I. Sato, and K. Ikeuchi, "Object Shape Morphing with Intermediate Reflectance Properties," Int. J. Shape Modeling,, Vol. 3, Nos. 1&2, pp. 91-106, 1997.
- Y. Sato, I. Sato, and K. Ikeuchi, "3D shape and reflectance morphing" Proceedings of International Conference on Shape Modeling and Applications `97, pp. 234-242, Aizu, Japan, March 1997. [Best paper award]
Illumination Estimation
Illumination from Shadows: We demonstrate the effectiveness of using occluding information of incoming light in estimating an illumination distribution of a scene. Shadows in a real scene are caused by the occlusion of incoming light and, thus, analyzing the relationships between the image brightness and the occlusions of incoming light enables us to reliably estimate an illumination distribution of a scene even in a complex illumination environment. This study further concerns the following two issues that need to be addressed. First, the method combines the illumination analysis with an estimation of the reflectance properties of a shadow surface. Second, we introduce an adaptive sampling framework for efficient estimation of illumination distribution.
- I. Sato, Y. Sato, and K. Ikeuchi, "Illumination from shadows," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 25, No. 3, pp. 290-300, March 2003.
- I. Sato, Y. Sato, and K. Ikeuchi, "Stability Issues in Recovering Illumination Distribution from Brightness in Shadows," Proc. IEEE CVPR01, pp. II-400-407.
- I. Sato, Y. Sato, and K. Ikeuchi, "Illumination Distribution from Brightness in Shadows: Adaptive Estimation of Illumination Distribution with Unknown Reflectance Properties in Shadow Regions," Proc. IEEE ICCV99, pp. 875-882.
- I. Sato, Y. Sato, and K. Ikeuchi, "Ilumination Distribution from Shadows," Proc. IEEE CVPR99, pp. 306-312.
Frequency Analysis: We provides theoretical insights as to why the approach based on cast shadows works in a reliable manner, and discusses what kind of basis functions are appropriate to be used for recovering illumination from cast shadows. First, we formalize the approach based on cast shadows by using spherical harmonics and analyze the approach in the frequency domain . Second, we propose an efficient method using Haar wavelets that provide compact supports and sparsity of coefficients.
Acquiring Illumination for Image Synthesis
Using real illumination for rendering: we propose a method for measuring a radiance distribution of a real scene automatically and use it for superimposing virtual objects appropriately onto a real scene. First, a geometric model of the scene is constructed from a pair of omni-directional images by using an omnidirectional stereo algorithm. Then radiance of the scene is computed from a sequence of omni-directional images taken with different shutter speeds and mapped onto the constructed geometric model. The radiance distribution mapped onto the geometric model is used for rendering virtual objects superimposed onto the scene image.
Real time image synthesis: we present a method for superimposing synthetic objects with natural shadings onto a real scene whose illumination condition is dynamically changing. We take advantage of the linearity of the relationship between brightness changes observed on an object surface and change of illumination radiance in a scene and introduce an efficient rendering technique based on a linear combination of pre-rendered reference images of the scene.
- I. Sato, M. Hayashida, F. Kai, Y. Sato, and K. Ikeuchi, “Fast Image Synthesis of Virtual Objects in a Real Scene with Natural Shadings,” J. Systems and Computers in Japan, Vol. 36, Issue 14, pp. 102-111, December 2005.
Photometric Projector Compensation
We wish to compensate for irregularities in the output of digital projectors that occur when they are used in non-ideal situations, such as those with varying surface reflectance and ambient light. We transform the image to be displayed into a compensation image that will produce the desired appearance. In contrast to previous methods, the transformation is based on both a radiometric model of the system and the content of the image. Our method balances strict compensation against dynamic range in the final output, so we can produce a good result even when removal of all visible spatial variation is not possible.
- M. Ashdown, T. Okabe, I. Sato, Y. Sato, “Robust Content-Dependent Photometric Projector Compensation,” Proc. IEEE Int. Workshop on Projector-Camera Systems, pp. 60-67, June 2006. [Best paper award]
[Paper and Slides]
Image Processing
Aesthetic quality classification plays an important role in how people organize large photo collections. In particular, color harmony is a key factor in the various aspects that determine the perceived quality of a photo, and it should be taken into account to improve the performance of automatic aesthetic quality classification. In this work, we tackle the challenging problem of evaluating the color harmony of photos with a particular focus on aesthetic quality classification.
- M. Nishiyama, T. Okabe, I. Sato, Y. Sato, “Aesthetic Quality Classification of Photographs Based on Color Harmony,” Proc. IEEE CVPR 2011.
- M. Nishiyama, T. Okabe, Y. Sato, and I. Sato, “Sensation-based Photo Cropping,” Proc. the 17th ACM International Conference on Multimedia 2009, pp. 669-672, October 2009.
