From BlenderWiki

Threaded Dependency Graph: Proposal

Project description

Main goal of this particular project is to achieve objects update in multiple threads, but it's an essential step towards new dependency graph.

For this project could foresee changes in both dependency and modifier stack.

Currently it seems dependency graph only defines order in which object's update happens (at least in it's external API, could be wrong tho). So first thing to be changes is to make it public API to return independent groups of objects, which could be handled simultaneously.

Not currently sure how much deep changes will touch existing graph, but just in case it's not so much changes i could always start making it more robust for update tags (currently tagging not always happens when it should, also, probably it'll worth looking into support or object relationship changes in time - like animated object layers).

Other thing to be changed is modifier stack. On the very first level, all the objects shall become thread-safe (metaballs code is not thread safe at all, good reason to get it finally fixed). This also seems to make it essential to change DerivedMesh to proper user-counter thing, and avoid pointers to base object's data which makes DM not thread-safe. It'll also solve old viewport crasher during animation render.

Also it brings interesting scheduling problems - for example currently multiresolution uses OpenMP for speed reasons. If you'll have multiple independent objects with multires, you'll need to make them not to use OpenMP. However, if you've got one huge object with multires, and other ones are simple cubes, you'd better handle all the simple objects in multiple threads and then handle multires object allowing OpenMP for it.

Not sure if detailed schedule could be written now - all this changes are related and probably would be solving simultaneously. Could only foresee taking blender apart, changing depsgraph/scene update and glueing all together again approach.

Schedule

Here's the rough schedule of the project.

• Milestone 1. Make curves/metaballs (and probably some other) thread safe.

Currently they're not safe enough to be be used from multiple threads and fixing this is essential thing for this project anyway. Also will help getting back familiar with what's happening in modifier stack and so (lots of changes were done there since my last active work there).

• Milestone 2. Threaded safe DerivedMesh which will use user-counter concept and wouldn't reference data outside of the DM.
• Milestone 3. Re-work scene-update cycle, so "groups" of object for which all the dependencies are computed will be updated in separate threads (using pool of threads for sure, no more than CPU cores threads are used).