Weekly Reports 2021

Week 1-2 4 - 15 January 2021

  • Fixed remaining known issues in optimization patch (D9555), to be landed in master
  • Started working on improvements to OpenSubDiv in Blender and Cycles, based on features present in Alembic
    • face holes (faces that are deleted after the subdivision was made, not sure how useful, will need user feedback)
    • vertex creasing
    • studied how rendering subdivision could be done
  • Reviewed D9961, to be landed
  • Started working on generating more complicated files (with higher resolution objects) for testing the optimization work, and perhaps do a second round of optimization.

Week 3 18 - 22 January 2021

Kept working on making tests files for the optimization work with more complicated scenes and higher polygon count objects, and continued working on improving subdivision tools and rendering.

  • Commits
  • Patches
    • Subdivision: add support for vertex creasing (D10145) (rework based on quick review in progress, but might need a redesign of how subdivision is handled in Blender to properly support such a feature)

Week 4 25 - 29 January 2021

  • General
    • Continued profiling and trying to optimize Cycles updates and the Alembic procedural, defined some areas to look at next week.
    • Continued work on subdivision a little bit.
    • Worked on proxies for the Alembic procedural (D10197, patch to be updated)
  • Bug fixes :
    • Fix T85144: Cycles crashes when editing render properties in viewport (d0f59d3842)
    • Fix T85010: Cycles viewport not showing certain material changes when using GPU (7a0281914e)

Week 5 1 - 5 February 2021

  • General
    • Cycles: investigated ways to reduce memory usage for socket and attribute data (reference counting, constant value optimization), but this made ccl::array complex and slower, might retry in a different way
    • Alembic procedural: added support for instancing
    • Alembic procedural: started working on a cache policy
    • Worked on documentation for the Cycles Node API, and the Cycles Scene in general
  • Bug fixes :
    • (unreported) Alembic procedural: fix crash when cancelling a render during synchronization (55c88e5582)

Week 5 8 - 12 February 2021

  • General
    • Cycles: investigated ways to reduce data transfer to the GPU
    • Alembic Procedural: continued work and testing for the cache policy
  • Patches
    • Finalized D10197: Cycles: experimental integration of Alembic procedural in viewport rendering
  • Bug fixes :
    • (unreported) Cycles: fix wrong flags used to tag device data as dirty (5213b18eb2)
    • Fix T85462: crash in render mode while removing instances (349c17cf54)

Week 6 15 - 19 February 2021

This week I continued working on memory usage and data transfers optimizations. Patches are mostly ready to be extracted for code reviews.

Patches will be for:

  • partial updates of the devices arrays to only retransfer to the GPU(s) parts of the buffer that have been modified (in my files data transfers were cut by 60% at best, more can be achieved depending on the file).
  • delta compression of the vertices and curve keys, this computes the difference in positions from the last update, stores this in 16-bits (instead of 32-bits) and sends that to the GPU where it is applied to the original memory (thus saving 50% of data transfer for those data)
  • octahedron compression of the normals, this projects the normals onto a unit octahedron, stores the results on 8-bits achieving a 75% memory saving. The normals are decompressed as needed in the kernel (this is a lossy compression, so slight (unnoticeable to the eye) render differences may be had, one test is failing so I am going to try a different approach).
  • lazy array allocation, this only allocates memory for arrays if we store more than one value in them. Arrays are used throughout the code base, however, the following arrays are allocated on the heap even when they may hold a single constant value:
    • Hair.radius (stores a float for every curve key)
    • Hair.curve_shader (stores an integer for every curve)
    • Hair.used_shaders (stores the list of shaders used by the object, useless heap allocation if only one shader is used)
    • Mesh.shader (stores an integer per triangle, redondant if all triangles use the same shader)
    • Mesh.smooth (stores a bool per triangle, redondant if all triangles are smooth)
    • Mesh.used_shaders (stores the list of shaders used by the object, useless heap allocation if only one shader is used)
    • Mesh.subd_shader (stores an integer per polygon, redondant if all polygons are using the same shader)
    • Mesh.subd_smooth (stores a bool per polygon, redondant if all polygons are smooth)
    • ImageHandle.tile_slots (stores the UDIM tiles used by the Image, allocating on the heap is a bit much if only using a single tile)

In spring.blend, this saves about 140 Mb of memory (I forgot to note down the total amount of memory, so I cannot give a percentage).

Week 7 22 - 25 February 2021

This week I mainly revised some of the patches I worked on last week before sending them to code review (although I haven't set any reviewers yet, so I can tackle some remaining todos).

The patch for normals compression I talked about in my last report was not revised nor sent to code review yet.

I decided to stop working on lazy allocation for the time being, as there are too many edge cases, especially when compiling shaders, using lazy allocation there makes shader execution fail as we miss some data since it will only allocate if the value added to the array does not change from the first one.

I also did some profiling and tried to optimize Blender a bit, as opening/loading large files takes a while. Some of the hotspot cannot be optimized easily or at all (like computing bézier curves, or tesselation), but inlining more maths functions does help.

  • Patches:
    • D10515 Cycles: partial array copy for reducing CPU/GPU data transfers
    • D10516 Cycles: delta compression for reducing CPU/GPU data transfers
  • Bug Fixes:
    • (unreported) Alembic procedural: fix crash when accessing data out of frame bounds (d9abcee47e)
    • Fix T85926: Cycles missing viewport updates when making materials single user (72370b92be)