From BlenderWiki

Development of Blender's Non-Photorealistic Rendering Capabilities: Interim Report

Author

Tamito Kajiyama

Date

6 June 2014

Introduction

The present document is an interim report of the project granted to the Author by the Blender Foundation Development Fund. The primary objective of the funded project is to improve Blender’s non-photorealistic (NPR) rendering capabilities through the following three development targets:

1. Freestyle performance improvements in terms of rendering time and memory consumption.
2. Better artists experience during interactive line stylization exercises.
3. Technical support for the development of Blender NPR rendering features.

A full description of the project goals and planned work components is found in the project proposal.

Weekly reports of progresses in the project have been posted in the Author's Wiki page.

In the first half of the project duration (for one month and a half starting from the 1st of April, 2014), paid hours were spent for the three development targets approximately in the following proportions: 20% for Freestyle performance improvements, 10% for better Freestyle user interaction, and 70% for general Blender NPR development. Main results and contributions of the project are highlighted below.

Results and Deliverables

Freestyle performance improvements

A known performance bottleneck of the Freestyle line rendering is relatively high memory consumption, which makes Freestyle less applicable to large scene setups. To begin with the funded project work, an attempt was made to address the memory consumption issue through revisions of Freestyle internal data structures. Freestyle uses double-precision floating-point numbers to represent 3D coordinates of mesh vertices as well as 2D coordinates of stroke vertices. Blender on the other hand employs single-precision floating-point numbers for the mesh data representation. The idea was then to use single precision in Freestyle as well with a view to a reduction of the renderer’s memory footprint size.

Code revisions and testing with selected input 3D scenes showed that a 10% reduction of memory consumption was achieved without a major loss of visual quality. However, a closer look of the rendering results revealed that the use of single-precision floating-point numbers tends to increase a failure rate of a key algorithm in Freestyle (referred to as inverse perspective projection transformation; IPPT) used for view map construction. The failure of IPPT was caused by insufficient numerical precision. Specifically, the IPPT algorithm deals with image-space 2D coordinates whose extent is defined by users as X and Y resolutions of renders. Single-precision floating-point numbers has approximately six significant digits, so that high resolution rendering leads to a very few decimal digits in fractional parts. These observations suggest that double-precision arithmetic operations are necessary in the IPPT and subsequent data processing in Freestyle. This development direction was hence discontinued after a while.

A successful effort of performance optimization was then made toward faster and less memory-consuming rendering of stroke segments (Git commit 40520d549d50). Although only applicable to a limited set of line styles based on segmented strokes, the performance improvement is significant in some cases. For instance, the following render employs a line style based on line splitting by dashed line patterns and the rectangular “Blueprint” geometry modifier. Without the performance optimization, the rendering process did not finish (and sometimes led to a crash) on a machine having a Core i7 processor and 4 GB main memory due to a high memory load. After the documented performance tuning, the rendering now finishes in a few seconds without any hassle.

Better Freestyle user interaction

As part of bug hunting efforts, the functionality of animated edge/face marks was recovered. Animated edge/face marks were possible in pre-BMesh release versions of Blender but not working in more recent versions.

In the Datablock display mode of the Outliner, Freestyle edge marks are available in Meshes > (mesh name) > Edges > Mesh Edge, while Freestyle face marks are found in Meshes > (mesh name) > Polygons > Mesh Polygon. The right-click menu on the toggle button of an edge/face mark allows for adding the selected edge/face mark to a keying set.

The space-bar menu in the 3D view port also provides access to a couple of useful operators named “Add Edge Marks to Keying Set” and “Add Face Marks to Keying Set”. These operators help to add selected edge/face marks to a keying set.

Having defined a keying set of edge/face marks as seen in the Scene Properties window, keyframing of the edge/face marks can be done at once by a single click of the key-shaped button to the right of the active keying set name shown in the Timeline window.

The following experimental movie clip shows a few examples of animated edge marks.

General Blender NPR development

In view of the upcoming Blender 2.71 release, a couple of new Freestyle features were added.

Among long-awaited missing features that were available in the original stand-alone Freestyle program, a proper support of textured strokes has been brought back to Freestyle for Blender in close collaboration with Paolo Acampora (patch D246). Textures applied to Freestyle line drawing can be stacked one on top of another, mapped onto different coordinate systems, and mapped to diffuse color and/or alpha transparency through different blending options. A Freestyle-specific mapping called “Along Stroke” provides a mean to align UV coordinates with stroke directions, which allows for imitating textured marks of brush, pencil and other writing media. Textures can be still images (either a Freestyle-specific brush texture consisting of a head, body and tail, or just a plain left-to-right texture) as well as those made of texture nodes.

Another new feature is sorting of strokes. Line styles now have a set of sort options to rearrange the stacking order of lines. This gives artists more control to determine which lines should be drawn on top of others.

These new features have been documented in detail in the latest draft of Blender 2.71 release notes.

Plans for the rest of the project duration

Given the time frame of the granted work up to the end of June, a focus will be put on further attempts of memory consumption reduction, but only after the Blender 2.71 release, until which paid hours corresponding to the BCon4 phase will be specifically addressed to bug fixes and testing.