Project: Ambiguous Orientation Estimation of Objects in RGB Images
Description
When estimating the orientation of an object in an image, several factors such as rotational symmetries, occlusion, and self-occlusion can create a set of possible orientations, either continuous or discrete. Consider a smooth cylinder where its orientation around one axis cannot be visually derived or a gear where the possible orientation around one axis will be a discrete set. Self-occlusion can cause ambiguity over an interval of orientations. Self-occlusion can cause ambiguity over an interval of orientations, such as when a mug's handle is occluded by the mug itself. Foreground objects and the view limits can also partially hide an object's features and result in ambiguity.
Traditional methods try to predict a single 'most-likely' orientation where ambiguity is treated like uncertainty. However, in contrast to situations involving uncertainty, there are cases where a model can be highly confident that multiple correct orientations exist, all of which induce the same RGB image.
To overcome this issue, more recent methods attempt to express the full set of possible orientations, which has numerous benefits. For instance, when multiple observations are available, intersecting the sets of possible orientations of each observation can result in an exact pose or at least a smaller set of orientations. This is useful when multiple cameras are present in a scene or when the camera is moving. It also opens the possibility to actively guide new observations based on the ambiguity in previous observations.
The goal of this project is to develop and experiment with novel models complex enough to understand a target object and express an orientation distribution. The models will be trained using large-scale synthetic image data. A content generation pipeline will be constructed where 3D models can be rendered with various poses, textures, materials, light conditions, and fore-/background objects. Although newer methods have attempted to overcome this issue, there is still room for improvement.
Details
- Student
-
Thimo Ferede
- Supervisor
-
Andrei Jalba
- External location
- Ambiguous Orientation Estimation of Objects in RGB Images