From BlenderWiki

Multiview Reconstruction Internals Design

The following describes my original plans for the multicamera reconstruction. For a current description, see the technical overview.

C API

Currently, the libmv C API exposes reconstruction through the libmv_solve function. It takes a set of tracks, camera intrinsics options, reconstruction options, and a callback for progress updates. Depending on the libmv_reconstructionOptions::motion_flag flags, it forwards these arguments to either libmv_solveReconstruction or libmv_solveModal. It returns a libmv_Reconstruction which contains the reconstructed cameras and tracking points.

I've considered two options for adding multiview reconstruction to the API:

1. Expose a libmv_solveMultiview function in addition to libmv_solve.
2. Have libmv_solve determine whether an internal libmv_solveMultiview should be called based on the arguments passed.

I have chosen option 2, because it simplifies the interface and treats single view and multiview reconstruction uniformly (at least as far as the blenkern is concerned). The given Tracks arguments can store Markers associated with particular views, and libmv_Solve determine whether to perform multiview reconstruction based on the number of views associated with the given tracks. This means that libmv_Solve will call one of the following functions:

• libmv_solveReconstruction
• libmv_solveModal
• libmv_solveMultiview

Perhaps it is worth combining these functions in some way, as they all perform many steps in common. It's also worth thinking about what will happen when we have modal multiview reconstruction. In fact, for multiple views, only a subset of cameras may have fixed positions. This suggests that it might be possible to have a single libmv_solve function that:

1. Treats single view and multiview reconstruction uniformly.
2. Takes motion constraints associated with each view.

Tracks

To store tracks across multiple views, the Tracks class needs to be modified. Currently, Tracks just stores a vector<Marker>. The Markers themselves keep track of which track and image (frame of video) they belong to. For example, all of the markers for a single track will have the track value but different image values.

For multiview reconstruction, the obvious solution is to give Markers a view member, which associates them with a particular view. That is, all Markers with the same view value correspond to markers on the images from a single camera. So the Marker struct will simply be:

struct Marker {
  int view;
  int image;
  int track;
  double x, y;
};

Then the member functions of Tracks need to be updated to allow insertion and extraction of Markers in different views. For example, Tracks::MarkersInView will return all of the markers corresponding to a particular view. Attempting to insert or extract Markers giving only image and track values will assume a value of 0 for the view. This means that Tracks can continue to be used in a single view situation.

Solving

Coming soon...

Reconstruction result

Coming soon...