User:Jon Denning/Reports/2022

Note: this document is broken up by week with a short update at the top and more details at the bottom of each week section. The updates are commit-like, while the details are more "talking out loud".

Links to My Work



Useful links

post dev grant, week 31: aug 1–aug 7

At this point, I'm officially done with my Blender dev grant. I do have a few more things I'd like to wrap up with the project, so I'll keep posting here as I work.

  • added quad only insert mode
  • some events are passed through to blender

the quad only insert mode follows this construction on desmos. some events (view changes, undo/redo, some selection) now pass through to blender when in sticky (modal) mode.

week 30: jul 25–jul 31


  • polishing retopology mode and geopen tool
  • created D15546
  • iterated over design with stakeholders

below is a summary of my takeaway from this project. (still wip!)

I ran into many road blocks while polishing and wrapping up the geopen tool. some of these road blocks are due to a few design decision in blender, but the vast majority is due to lack of documentation. some seem to be unknown by anyone I asked and required poking deep into blender code.

  • as of right now, add-ons cannot define modal keymaps (see rna_keymap_new). this means that either:
    • the add-on defines Blender keymap items for actions in modal. these keymaps could "bleed" into the rest of the ui (Blender doesn't know that certain actions are only handled when running modal, so the keymap item can show in statusbar), and that the user is required to keep the keymaps in sync (if pressing a modifier starts modal mode, the modal actions must either enable any or set the appropriate modifiers). furthermore, the add-on must then perform conversions of Event info into KeyMapItem info (or vice versa), and implement the logic to process the user's actions. Note: these modal keymaps can be hidden by attaching to a nop operator (poll always returns False).
    • the add-on implements its own keymap within the modal method. this effectively "hides" the keymap details from Blender's UI, but this also hides the details from the user, as it no longer shows up in keymap editor! (workaround: the add-on could create a keymap editor in the preferences or some other way, but that's an ad-hoc approach) this approach is taken by the modal operators that I surveyed within blender's scripts (ex: curvetools.fillet, node.nw_lazy_mix, gp.latticedeform, etc.).
    • workaround?: the modal operator could return {'PASS_THROUGH'} to allow Blender to process event+keymaps and then re-call the add-on with appropriate properties set, but this might lead to unintended issues (ex: knife modal is started).
  • it is impossible to control undo for invoked modal operators. operators can take an optional positional undo argument. this arg is undocumented, but C++ comments imply that it is possible (see wm_operator_finished). the implications are:
    • any invoked modal operator (ex: bpy.ops.transform.translate) will always push to undo, which means there is no way to create an operator that does two things in one undo step, making the operator always feel 2nd class. for example, the built-in extrude operator (E) will create geometry based on selection and then grab it. after committing the grab, undo will undo BOTH actions (create, grab). however, the python F2 add-on (F) will create geometry based on selection and then grab it (if autograb is enabled). However, this single action creates two different undo steps (must press undo 2x).
    • the only way to avoid having an undo state pushed is to duplicate the operator in C++ but without the OPTYPE_UNDO flag set in op->type->flag.
  • working with blender ui is cumbersome, which makes refactoring UI (exposing options, rearranging or reformatting ui, etc.) very slow. additionally, there are many "tricks" hidden throughout blender's own python, but documentation is way too scarce to really understand how to work with these tricks.
  • keymaps, WorkSpaceTool, gizmo and gizmogroups, properties, and ui are all still black boxes. the mysteriousness is mostly due to lack of documentation, but even the built-in templates are opaque.

other issues involve standard issue bugs.

  • bpy.context.edit_object will sometimes not reference the current edited object. this will happen right after calling undo, but sometimes it happens when an undo state is pushed. most of the time, the result is blender crashing.

many of my patches still remain without review. the devs that could review them are also core developers, so they are naturally busy. this means that any useful feedback typically takes 1--2wks, but often it takes longer because the issues i have run into are too complex for a single response.

the work I was tasked to do (bring retopology to Blender) required either expanding snapping features and bpy or duplicating code. clearly, the former adds complexity to an already complex product, and the latter is not great practice in general. as a developer, i understand both of these sides, and so i try to avoid unnecessary complication as much as possible. However, I also understand that new things cannot be done without picking one of these two directions.

week 29: jul 18–jul 24


  • copy-pasted mesh.relax into mesh.relax_snap
  • refactor and clean up retopotools add-on
  • reviewed feedback from Julien Kaspar and Daniel Bystedt
  • met with Daniel Bystedt

the relax operator seems like a useful tool for retopology work, so I copy-pasted it and added a call to bpy.ops.retopology.translate_facesnap right after it relaxes.

Started a Feedback doc to record feedback and responses.

week 28: jul 11–jul 17

discussed work with stakeholders and devs.

created a youtube video demonstrating the new retopo snapping options and tool. refactored the retopology tools add-on, getting it ready for publishing.

week 27: jul 4–jul 10

started the process of splitting many retopo-related changes into separate patches (D15398, D15406, ...) that are (hopefully) easier to review.

week 26: jun 27–jul 3

nearest face snapping is now in master!!

week 25: jun 20–jun 26

  • working retopo mode!
  • geopen: vertex and edge snapping is based on auto merge and auto split, respectively

now, snapping targets and methods are different when retopo mode is enabled! specifically, Vertex and Edge snapping only applies to Active and Edit source meshes (but only if the corresponding snapping method is enabled), and Face Raycast and Face Nearest snapping only applies to Non-Edit target meshes (again, only if corresponding snapping method is enabled). this works quite well (although Blender did crash on me once... still need to debug this).

the only con to this setup is if the artist wished to have the source verts snap to the target verts/edges (ex: when retopo'ing a hard surface model). but, they can always use the normal grab operator with the appropriate settings set, or a better implementation might be to allow vert/edge snapping if it is "sharp enough". meaning, target edge snapping is allowed if the edge is between two faces where their normals dotted is below some threshold (angle between normals is greater than some threshold angle). (the angle / dot product threshold will be adjustable) however, this then requires masking the target edges... a problem that is outside the scope of my project.

When using Geometry Pen, Vertex snapping is enabled whenever auto merge is enabled, and Edge snapping is enabled whenever auto split is enabled.

remaining issues:

  • there seems to be no way to prevent Invoked ops from pushing to undo stack. this is an issue as calling a single operator that wraps multiple operator calls will create multiple undo steps
  • calling operator that wraps another operator can have the redo menu show for a brief moment
  • sometimes auto merge will perform an edge split instead, even though visually and intuitively the snapped vert should merge with hovered vert. this issue happens very intermittently
  • using align argument will draw an extra normal vis when still grabbing a vertex.
  • face raycast should use mouse projected onto world rather than current offset position projected onto world. otherwise, verts will jump across the screen once mouse hovers another vertex

week 24: jun 13–jun 19

i started in on working through the hitches i ran into last week by implementing a retopo mode argument for transformation ops. however, while working in there, i realized that some of the refactoring/new features i had done in D14591 is not quite correct. in particular, the face nearest wasn't acting as expected with different settings. i took a few days off to allow myself to time to rest and forget in order approach it with fresh ideas.

week 23: jun 6–jun 12

  • diffs
  • continued testing and polishing geometry pen tool (it's very close)

Some questions / points to dig further:

  • should retopo tools have their own settings that override scene settings (ex: merge threshold, auto merge, etc.), or should they simply use scene settings?
    • if shared, what if it makes sense to have different settings for different tools? should the tool automatically change the scene settings to defaults (or user specified) when the tool becomes active?
    • if individual, how do we handle shortcut tool? ex: geometry pen uses face raycast and nearest snapping and sometimes vertex, and the user can press F at any time (calling F2 for example) which creates a face and grabs the inserted vertex. should we have an extensive keymap and operators that overload all reasonably useful operators? yuck.
  • should there be separate options for auto merging and auto splitting?
  • normals for vertices are read only for MeshVertex (891268aa824f), but they seemed to be ignored for BMVert too...? snapping is not setting the vertex's normal, which means new geometry can sometimes become flipped. this isn't much of an issue, unless displacement modifier is enabled.

the first hitch with snapping methods

I ran into a hitch with transform snapping. there currently is no way to use different methods against different snapping targets, but I have a couple of potential ways to solve this.

here is an example of why this is an issue: when creating new geometry or moving geometry with geometry pen, we want the grabbed geometry to snap to the target's surface with Face Raycast / Nearest, but we also want to allow the geometry to snap+merge with source geometry using Vertex snapping. If Face snapping and Vertex snapping are both enabled, and if the target geometry is highly tessellated, the grabbed geometry will snap to vertices of target very discretely rather than smoothly snapping to the faces of target. But without Vertex snapping, there is no feedback for when the grabbed geometry is going to merge into existing geometry.

below are a couple of ways to potentially solve this.

rather than using Vertex snapping and Auto Merge to cause grabbed geometry to snap+merge to existing geometry, i'll use a retopo flag (or maybe just simply use auto merge, but this starts to hijack and change existing behavior). if the retopo flag is enabled during invocation of transform op, then Vertex and Edge snapping applies only to self (not active, but actual self) while Face snapping applies to target (non-edit objects). This option could take away the option to snap grabbed geometry to target vertices and edges, but I don't know if that's really an issue.

another solution is to add options for Vertex and Edge snapping methods (similar to Face Raycast's option to project vertices individually or to treat as a group) that controls the objects to which snapping happens. I think this overly complicates the options.

both of these possible solutions complicate transform ops (tool settings are affected if we overload the auto merge), but i cannot think of another way to handle this without reworking the transform snapping code or reimplementing it.

the second hitch with snapping methods

There are some geometry pen edits that don't fit the current snapping methods approach. specifically, snapping happens only on the vertices. but as an example, one edit that would be ideal for geometry pen is the ability to snap a grabbed edge to a hovered edge (even if the two edges are oriented differently so both pairs of vertices don't overlap at the same time).

possibility: add a merge edge option to handle this special case?

week 22: may 30–jun 5

  • continued work on D15037
  • continued work on geometry pen

I pinged Campbell about how to proceed with submitting patches. most of the work i've done on Blender's side involves quite a large amount of change. breaking it into smaller patches is a lot of work, but also some of the changes don't make sense except in the context of the larger change. this has been a big learning point for me.

D15037 is ready! now, I will rework the nearest face snapping option based off this.

I continued working on geometry pen. I've found quite a lot of limitations, some of which I've been able to work around (to be submitted as patch(es)), but some I'm not so sure about. I've asked #python channel, but no one there seems to have an answer. will ping Campbell.

week 21: may 23–may 29

  • created D15037 as a (mostly) subset of D14591
  • added issues/bugs and limitations to To-Do

while Campbell was mostly positive of D14591, Germano had reasonable reservations about the size of the work. so, I took time to create D15037 as a (mostly) subset of D14591. i did notice and fix a few missing elements. next week, i will finalize D15037 and apply the create a new patch of D14591.

as i polished geopen, i discovered issues/bugs and limitations in current Blender code / API. these were added to a growing To-Do. i plan either to address these myself as patches (if reasonable for me to do) or to create some tasks.

my next tool is going to be a contours-like tool.

week 20: may 16–may 22

  • geometry pen prototype is working!
  • continued work on understanding gizmo-related classes (strengths, limitations)
  • started working on visual settings for retopology work

Earlier last week, I had a working prototype geometry pen (initial tweet). later in the week and over weekend I polished it quite a bit (update tweet), both on UX side and dev side. The tool is quite functional, but there are a few "hiccups" in the way it works. see experiments page for details.

I started adding options to modify 3D view settings to be friendlier to retopology work. so far the only options are to hide the 3D cursor and to hide the object origins. both of these options are temporary, meaning that switching into the retopology tool will adjust/override current view settings, but switching away from retopology tool will revert the view settings back to their original settings (before overriding). note: this work is just a workaround until we have an actual retopo mode in Blender.

week 19: may 9–may 15

  • continued experimenting with new geometry pen retopology tool
  • working to understand gizmo-related classes; started document

I continued working on different ways to create operators, and i have a very basic tool now for creating new geometry quickly. I tried to understand how to best handle Gizmo, GizmoGroup, and WorkSpaceTool, but documentation was scarce and examples are light, so i chatted with several on #blender-coders and searched through source. I plan to create a proper document with some examples later. in the end, i found many limitations to the various ways to accomplish this geometry pen tool. this work is outlined at Experiments doc. note: the doc is getting quite large, and it's a little unclear what was done when, but the details are organized by category rather than date (like this doc), so it's easier to see how things relate.

week 18: may 2–may 8

i explored several ways to create new retopo tools. i started with an equivalent to bpy.ops.transform.translate. i wanted to temporarily set the snapping settings based on the selected tool, to have the tool do its thing, then reset snapping settings back. (using snap=True argument enables snapping for the entire scene!) there are several ways to create new operators/tools in Python that are based on current operators. each of these are discussed in Experiment.

i plan to make public my work next week.

week 17: apr 25–may 1

week 16: apr 18–apr 24

This was a much slower week for development due to the traveling and patch review.

week 15: apr 11–apr 17

week 14: apr 4–apr 10

  • met with Campbell and Dalai to discuss next steps
  • cherry-picked the surface snap and new snap options
  • studying custom widgets and operators
  • fixed api docs (patch to be submitted)

the second half of this work will focus on the retopo tools. Campbell would like to take a different approach with custom / derived modes, and the retopo mode will be one use case example of this. the hope is that custom modes will be definable from Python, which means that retopo mode could be totally (or mostly) in Python! so, future dev on the mode is currently paused.

as the surface snapping and snap options are critical for retopo work, i pulled this out into its own diff (see Surface Snap and New Snap Options (D14591)).

started digging into creating custom widgets and operators. i fixed some of the api docs that were incorrect or basically empty. i plan to post a patch soon.

week 13: mar 28–apr 3

  • added a surface snap steps feature (experiment)
  • added a keep on same object feature
  • refactored/deduplicated edit/retopo mode code

when the surface snap steps is enabled, the surface snap action will be done iteratively over the specified number of steps rather than over a single step. setting to 1 step is equivalent to disabling this feature. this experimental feature tries to push surface snapping toward a geodesic snapping by solving the following two problems: prevent vertices from snapping to surfaces that are near in euclidean space but distant in geodesic space; prevent awkward snapping behavior when moving near the middle of U-shaped volumes. note: the issues could possibly be solved by labeling the geometry near the initial vertex position (alternative: only build BVH using geodesically nearby geometry), or somehow warping the mesh so euclidean distance approximates geodesic distance, but this could become expensive. overall, i'm not satisfied with this feature, yet.

when keep on same object feature is enabled, surface snapping will use only the closest object to the vertex's initial position for snapping. this feature solves the issue of changing which object a vertex snaps to when different objects intersect or are nearby in euclidean space. this feature works quite well.

when i started this project, i duplicated the edit mode toggle/enter/exit code for retopo mode, as both modes use similar data structures. this made it difficult to rebase or merge in master, as any change needed to be done to both sets. i have gone through the code and deduplicated that by adding a submode parameter to the low level functions.

This week, I will be working on a proposal for the future of this project. Campbell Barton gave a quick review, and he thought that this might be a good use case for custom / derived modes.

week 12: mar 21–mar 27

  • updated F2 add-on to work in retopo mode
  • update retopo mode toolbar
  • retopo icon now renders in outliner

Many of the edit mode toolbar menus are now available in retopo mode. Otherwise, this was a slower week.

week 11: mar 14–mar 20

  • fixed merge conflicts and updated the diff.
  • 3d snapping now respects target object's matrix and targets edit mesh and text objects
  • Hooray! Edit tools now work in Retopo Mode!
  • tab toggles between retopo and edit modes
  • transforming and snapping work in retopo mode

Some recent changes in the master branch overlapped with a lot of my work from the past few weeks. I'm hoping that I was able to merge in these changes successfully, but I haven't fully tested it.

I was finally able to get Edit Tools to work in Retopo Mode! Since the object's mode is a flag rather than a setting, I'm oring on an additional edit mode flag to signal that it is editing retopology. now, as long as the operators just test with (mode & OB_MODE_EDIT), then that operator can work in either Edit Mode or Retopo Mode. This needs more testing, especially with other edit modes (Curve, Text, etc.). To test that we're only in Edit Mode and not Retopo Mode, need to use mode == OB_MODE_EDIT. To test that we're in retopo mode and not just Edit Mode, need to use mode == OB_MODE_RETOPO, since the OB_MODE_EDIT flag will still make (mode & OB_MODE_RETOPO) true.

Question: does it make sense to be in retopo mode and another? (similar to edit + pose?)

Tabbing while in Retopo Mode will now toggle to Edit Mode, and another tab gets us back to Retopo Mode. This is similar to tabbing from in Sculpt Mode.

Translations and snapping now work correctly in Retopo Mode. Note: rotation and resize xforms was written to snap as a group only, not individual vertices... might need to change this.

Next week, I plan to take a look at LoopTools, F2, and Bsurfaces as retopo tools.

week 10: mar 7–mar 13

  • Hooray! I successfully implemented a 3d snapping!
  • refactored transformation snapping fn names

I was finally able to get 3d surface snapping working. i posted a video on twitter. There are still issues (does not account for target xforms, sometimes movement isn't very intuitive, ...), but it's functioning.

The biggest breakthrough come from realizing that some of the snapping functions had names that detailed what they were currently doing instead of what they were intended to be doing. The current names made it tricky to follow the flow or know how to modify, so I refactored the names and some of the code to be clearer.

I'm still fixing conflicts from rebasing off master, so I haven't yet updated my diff.

week 9: feb 28–mar 6


  • do BLI_ghash_* fns consider edits? i.e., if an edit mesh is operated on, does it's hash change?
    • is there anything that would act like that? to know if a mesh/edit mesh/bmesh changed?

I got stuck on a few functions around transformation snapping, where the code I copied/modified was segfaulting. Plus, the area was quite confusing in general. So I met with Campbell to try to get a high level view of that area of code. We discussed a few possible general ways to solve the snapping problem. I ended up diagramming the call tree of that area (see document linked above).

week 8: feb 21–27

  • met with Thomas Dinges and co.
  • continued work on snap/projection tool settings
  • refactored snap defines as an enum, cleaned up types around this
  • continued working on Function Naming doc

This week felt way less productive, as most of the time was spent trying to understand how the Translation Snap code worked. In that effort, I refactored a bunch of #defines related to snapping to an enum. Somewhere in the code, a char was used where short was used elsewhere, and therefore a new snap mode with value over 255 was getting truncated. I copied code over (with minor modification) from the calculate_mesh_proximity fn into snap, but Blender kept segfaulting. I believe it's due to creating a BVH tree of the original mesh in different threads, but I'm not entirely certain. It seems the needed BVH trees are already created (or at least the code to do so is available), so my next step is to try using that. I will ask Campbell to give me a walkthrough of the Translation Snap code, just to give me more context. My hope is that once this is working, new tools should be fairly quick to implement, even if they're written in Python, as the time consuming part (snapping/projecting) will be done automatically in Blender core.

Important Note: the snap code is critical for certain retopology tools. Without it, the retopo tools will be limited to screen space snapping only. This is why I chose to spend a significant amount of time here. I definitely underestimated the complexity of this code... but I'm not convinced that it needs to be so complex. It could just need a good clean-up pass. (switching from #define to enum, I touched 12+ files, and some of those touches required a lot of thread pulling to find)

week 7: feb 14–20

plan: continue implementing retopo tools into edit mode, update design document

The updated snap/projection tool settings code is working well, but it is still lacks the ability to snap to surface in 3D space or follow topology of another mesh. Presently it's not clear in tool settings that Face and Volume snapping use different code paths (with different behaviors) from the other snapping options (Vertex, Edge, Edge Center, Edge Perpendicular). Technically, Face and Volume "snapping" is really a projection. As it is, this is likely not an issue for the artist (except perhaps Volume snapping), but adding any additional snapping settings will require some better labels, descriptions, icons, etc.

Attempted to add a snap to surface option, which uses shrinkwrap code to snap vertices to the nearest surface point (not project). After chatting with Hans Goudey, started looking more at calculate_mesh_proximity instead.

In trying to understand function call hierarchy and general architecture, I created a document to capture function name prefix and suffix meanings. I posted this to #blender-coders to get some feedback and updated the document accordingly.

week 6: feb 7–13

plan: implement retopo tools into edit mode

This week the plan is to switch gears to work on the retopo tools instead. My hope is to gain some progress in this area, then revisit retopo mode and overlay later. To simplify my efforts, I will implement them into edit mode for the time being. This week I also plan to update the design doc and the retopo mode diff. So far, I have only copied the Rip-Move operator macro, but doing so gave me some insight into how to create new operator macros. I had thought that I would need to create a project operator and a snap operator, but it might be simpler to add these "clean up" operations as tool settings instead, because a lot of the code already flows touches the tool settings.

I ended up creating two new "snapping" (projection) constraints: Edit (non-active) and Non-edit. Presently, there are several settings for how the geometry is to be projected ("snapped") while transforming, but none were well-suited for retopo work, where typically the geometry is to be projected to the surface that's being retopologized. I added some additional complexity in the filtering, and I added a couple UI checkboxes in tool settings to allow the artist a bit more control over how the geometry is projected.

Finally, I started a new module for retopology operators (just a .h and .c with a single operator at the moment).

I still need to update the retopo mode design doc. I will work on that early next week.

week 5: jan 31–feb 6

  • met with thomas dinges, dalai felinto, and other devs
  • integrated jeroen bakker's retopo overlay into retopo mode
  • finding edit mode to be tightly integrated throughout blender, might need to switch approaches to retopo mode

I switched to bringing in Jeroen's retopo overlay code into my branch, thinking that maybe I could gain some insights with the draw engine by doing this. (especially the selection overlay system) His patch was straightforward to bring in, although I simplified it a bit (for now), but there still were some quirks to how it worked in my branch than what he posted in the task. Specifically, the depth buffer seemed to be blown away (or the "in front" option was getting set), because edit geometry that should be occluded was not. I spent a bit of time trying to understand the draw engine and chatting with Clément Foucault, but I was running into blocks.

week 4: jan 24–30

  • reviewed retopo overlay Retopo Overlay
  • continued work on basic retopo mode
  • started a design doc on wiki
  • (caught a second cold...)

Following my conversations with Campbell Barton, Daniel, and Julien, I've started with coding a retopo mode separate from edit mode but still with the hope that I wouldn't need to fork all of the tools and code or touch too much of the current code. The mode does look and feel like edit mode, with edit mesh and bmesh (exact same as edit mode's), overlays, etc., but I was unable to get selection overlays to work and/or the tools weren't behaving quite right. As far as I can tell by following in the debugger, the code paths are correctly followed, but there is still some levels of indirection that I'm not able to grok that keeps the edit mesh/bmesh from getting updated and/or the visualization of edit mesh/bmesh being correct.

observations: I'm finding that "Edit Mode" (whether editing meshes, text, curves, whatever) is _really_ tightly integrated throughout, but also there are about a dozen different ways that the code checks whether it's currently in edit mode. (also, I'm still not quite sure why the modes are masked rather than straight enumerations. I asked on #blender-coders, but the answers were not clear.) Adding a sculpt mode or the various gp modes was fine, as they were far enough away from the other modes that code sharing wasn't as big of a deal. This may complicate a separate Edit Mode that shares code with Edit Mode.

week 3: jan 17–23

  • continued coding on adding retopology mode
  • iterated on design doc with Daniel Bystedt and Julien Kaspar

week 2: jan 10–16

  • met with Dalai to discuss project, admin
  • started coding in a retopology mode
  • started design document

I started a Google Doc to capture and iterate over some ideas for a Blender retopo mode (including terms, workflow philosophies, etc.), and I shared it with Daniel Bystedt and Julien Kaspar, two stakeholders for this project. We iterated over much of the doc, trying to revisit all decisions that we made with RetopoFlow and where Blender's various modes are now. I will post a cleaned up version as a design doc to the wiki soon.

week 1: jan 3–9

  • (recovered from COVID...)