Multiresolution reliability scheme for range image filtering

A large number of 3D cameras suffer from so-called holes in thedata, i.e. the measurement lattice is affected by invalidmeasurements and the range image has undefined values.Conventional image filters used for removing the holes perform notwell in presence of holes with large varying hole sizes. The novelhole-filling method presented in this paper operates onreliability attributed range images featuring unwanted holes withwide varying sizes. The method operates according to a multiresolution scheme where the image resolution is decreased at thesame time as the range reliability is successively increased untilsufficient confidence is reached. It builds on three maincomponents. First, the described process performs a weighted localneighbourhood filter where the contribution of each pixel standsfor its reliability. Second, the filtering combines filters withdifferent kernel sizes and implements therefore the multiresolution schema. Third, the processing requires a completetravel from high resolution down to the resolution of satisfactoryconfidence and back again to the highest resolution. The algorithmfor the described method was implemented in a efficient way andwas widely applied in the hole-filling of range images from adepth from focus process where reliability is obtainablenon-linearly from the local sharpness measurement. The method isvalid in a very general way for all range imagers providingreliability information. It seems therefore well suited to depthcameras like time-of-flight, stereo and other similar rangers.