From BlenderWiki
Multiple View Reconstruction
Name
Matthias Fauconneau
Email / IRC / WWW
My email is matthias.fauconneau@gmail.com. I am MatthiasF on #blendercoders.
Synopsis
libmv/Ceres can be used to reconstruct motion using feature points seen from multiple cameras. Currently, Blender can only use a single camera which can fail to yield a solution in some cases. Using different views as seen from additional witness cameras helps resolve ambiguous projections and will improve the quality of the solution. This project will bring multiple view reconstruction to Blender..
Benefits to Blender
This contribution to Blender codebase will be essential in asserting the position of Blender as the best open source match moving tool. Indeed, match moving is not well developed in the open source world as it is an area which requires an unique combination of technical algorithms and effective user interface. Moreover, such applications is very dependent on users needs, which is a strong focus in the Blender community. The increased reconstruction capabilites will allow better visual effects production to compete with industry solutions.
Deliverables
- Improved multiple camera reconstruction
- Efficient API with Blender
- Support within Blender of the new capabilites
- Interface and code documentation
Project Details
The major goal of this project will be to improve reconstruction of the main camera path and 3D motions of scene objects by leveraging additional perspectives from witness cameras.
First, I will develop the existing support within libmv and make sure all provided features are properly exposed through the API to be easily leveraged by Blender. Then, I will integrate the new tools into Blender. Finally, I will focus on ensuring Blender is able to solve real test cases submitted by our users. Such additional feedback is necessary to improve and tweak the algorithms to obtain the best results.
Project Schedule
After discussing with users and previous developers, I'll start working on the solver backend. I will then complete the necessary interface to support an integrated multiview tracking experience in Blender. Any remaining time will be used to improve tracking accuracy, robustness and performance. The bulk of the work is divided in three milestones to avoid finishing with an incomplete project :
- Milestone 1 : Complete backend support for multiview tracking.
- Milestone 2 : Integration in Blender UI
- Milestone 3 : Production proven pipeline
The last weeks will be reserved for debugging, optimizing, cleaning and documenting the code.
Bio
I'm currently doing research towards a PhD thesis on "Computational simulation of wire-reinforced granular matter for jammed architecture" at ETH Zürich (Swiss Federal Institute of Technology).
I learned to become curious about programming and mathematics at a young age. I have >5 years practice of C++ software engineering through both research and personal projects demonstrating experience in technical/scientific computing. Source code publicly available on [GitHub]
For Google Summer of Code 2011, I developed tracking tools, 3D reconstruction and other algorithms to support visual effects production in Blender. Reports on the GSoC project : GSoC 2011
In 2010, I was contributing to libmv, creating the NView test UI for multi view reconstruction.
