%0 Journal Article %F J3 %A De Luca, A. %A Oriolo, G. %A Robuffo Giordano, P. %T {Feature Depth Observation for Image-based Visual Servoing: Theory and Experiments} %J The International Journal of Robotics Research %V 27 %N 10 %P 1093-1116 %X In the classical image-based visual servoing framework, error signals are directly computed from image feature parameters, allowing, in principle, control schemes to be obtained that need neither a complete three-dimensional (3D) model of the scene nor a perfect camera calibration. However, when the computation of control signals involves the interaction matrix, the current value of some 3D parameters is requiredfor each considered feature, and typically a rough approximation of this value is used. With reference to the case of a point feature, for which the relevant 3D parameter is the depth Z, we propose a visual servoing approach where Z is observed and made available for servoing. This is achieved by interpreting depth as an unmeasurable state with known dynamics, and by building a non-linear observer that asymptotically recovers the actual value of Z for the selected feature. A byproduct of our analysis is the rigorous characterization of camera motions that actually allow such observation. Moreover, in the case of a partially uncalibrated camera, it is possible to exploit complementary camera motions in order to preliminarily estimate the focal length without knowing Z. Simulations and experimental results are presented for a mobile robot with an on-board camera in order to illustrate the benefits of integrating the depth observation within classical visual servoing schemes %U http://rainbow-doc.irisa.fr/pdf/paolo/IJRR_2008.pdf %U http://dx.doi.org/10.1177/0278364908096706 %8 October %D 2008