From BlenderWiki

Development of Blender's Non-Photorealistic Rendering Capabilities: Interim Report #2

Author

Tamito Kajiyama

Date

6 August 2014

Introduction

The present document is an interim report of work progresses by the project granted to the Author thanks to financial support by the Blender Foundation (BF) development fund. The initial three-month contract of the project from April 2014 has been extended by another three months up to the end of September. Hence this report covers project outcomes since the previous interim report (6 June 2014) instead of intending to serve as a final report of the initial contract. Recalling the main project components below as planned in the project proposal, the subsequent sections summarize major results and deliverables of the work done during the last two months.

1. Freestyle performance improvements in terms of rendering time and memory consumption.
2. Better artists experience during interactive line stylization exercises.
3. Technical support for the development of Blender NPR rendering features.

The report is then concluded with future plans for the rest of the project duration.

Results and deliverables

Freestyle performance improvements

During the last two weeks of July, development focus was put on Freestyle performance optimization for reduced memory consumption. It is recalled that the Freestyle rendering process consists of modular steps as listed below, which allows for step-by-step performance optimization for space:

1. Mesh loading: Input scene data is received from Blender and cleaned up.
2. Winged edge building: A graph data structure (called winged edges) for bidirectional traversing of edges is created.
3. Silhouette detection: Silhouette lines and other feature edges are identified based on winged edges.
4. View map building: A 2D representation of the input scene (called a view map) is created from winged edges.
5. Stroke drawing: A set of stylization options is applied to identified feature lines by Python scripting.
6. Stroke rendering: A raster image of stylized lines is rendered.

A concern related to the optimization efforts is to keep visual consistency between the pre- and post-optimization versions of the code. If code revisions for improved memory consumption are found “guilty” for major visual changes, then the code updates have to be either revised to address the differences or submitted for review by artists. For this reason, selected test cases (Figure 1) were used to evaluate performance improvements (in terms of peak memory consumption in Freestyle) and double-check if visual consistency is maintained.

	ID: ichiotsu 74641 vertices, 120592 faces
	ID: ramskull 370252 vertices, 740656 faces
	ID: lily 460330 vertices, 748683 faces

Figure 1. Test cases.

Memory optimization attempts so far have been made in the first four steps from mesh importing to view map building. Experimental results (Table 1) showed that performance gains were within the range from 6% to 9% with respect to the peak memory consumption in the non-optimized baseline version of the code.

Table 1. Freestyle peak memory consumption in megabytes [M].

Test case	Baseline	Optimized
ichiotsu	320.33M (100%)	296.75M (92.6%)
ramskull	1616.94M (100%)	1520.49M (90.8%)
lily	1550.79M (100%)	1408.10M (94.0%)

An idea of code tuning for space was further applied to a grid data structure used for view map building. Early test results of the code revisions including this idea indicated a memory consumption of 279.08M (87.1%) in the “ichiotsu” test case, namely a reduction of 12.9% from the baseline version. Experiments with the other two test cases are foreseen in a short time.

Code revisions have been made in a local branch. The optimized code will be proposed for inclusion in Blender 2.73 release.

Improved artists experience with Freestyle

In response to frequent requests from Blender artists, the Freestyle line renderer was made available in Cycles in addition to the Blender Internal (BI). Unlike BI, materials and textures in Cycles are configured only by means of shader nodes. This implies two issues that had to be properly addressed by the design and implementation of Freestyle for Cycles:

• There are no material color properties commonly used by both Cycles and BI (except for the diffuse color of BI materials, which is also used by Cycles when shader nodes are disabled).
• The conventional texture mapping and influence panels of the Textures properties window are not used for Cycles, while Freestyle for BI has been relying on these UI panels to set up textures.

To resolve the issue of material colors, a new Freestyle-specific color property named “Freestyle line” was introduced in material ID data blocks (Figure 2). This property provides users with a reliable way to specify line colors on a per-material basis, no matter what rendering engine is used for Freestyle renders. Another property named “line color priority” (shown next to the line color in the UI) was also introduced in order to specify the ordering of competing line colors at material boundaries. A use case of the line color priority is detailed in the following Freestyle development blog article.

http://freestyleintegration.wordpress.com/2014/07/07/line-color-priority/

The texture issue was instead addressed by two new shader nodes named “Line Style Output” and “UV Along Stroke” specifically designed for Freestyle line texturing (Figure 3). The Line Style Ouput node specifies how to map the colors and alpha transparency of textures on to the base color and base alpha transparency of line styles. The UV Along Stroke input node gives access to UV maps (with and without stroke tips) defined along strokes. These UV maps become available only during the Freestyle rendering process. Hence the UV Along Stroke node cannot be replaced by the conventional UV Map input node which takes an existing UV map already defined as part of mesh data.

It is noted that before the new support for Freestyle in Cycles was added, artists willing to use Freestyle together with Cycles had to rely on a complicated workflow outlined as follows. First a 3D scene is set up and rendered with Cycles. Second a full copy of the scene is created in order to render Freestyle lines using BI. Finally the Cycles and Freestyle renders are combined using the compositor. The availability of Freestyle in Cycles will greatly simplify NPR workflows of this kind, without having to create a copied scene for Freestyle.

The code revisions for Freestyle line rendering in Cycles (patch D632) have been under code review with a view to inclusion in the next Blender 2.72 release.

General Blender NPR development

In addition to the big development targets documented above, important code updates were made during the reported period of time with the aim of improved Blender NPR capabilities in general.

First, Freestyle line rendering in the 3D viewport was fixed so that changes in Freestyle settings will immediately take effect in the viewport preview (rB269b14f184bc). Freestyle preview rendering is likely to serve as an intuitive tool for artists to define a set of line stylization parameters in a shorter turn-around time.

Second, incoming reports of crashes in Freestyle were duly addressed, which contributed to improved Freestyle run-time stability:

• T40833: Crash when rendering with freestyle.
• T41180: Crash when using motion blur with Freestyle.
• T40307: Crash with freestyle and particle hair.
• Freestyle: Fix for a potential infinite loop in stroke resampling by vertex count (rB812515b623be).

Finally, last but not least, the implementation of the Freestyle Python API was significantly improved in close collaboration with flokkievids (Folkert de Vries). Freestyle comes with a number of predefined style modules written in Python that are intended to provide coder-artists with examples of Freestyle Python scripting. Hence the code readability of the existing Python scripts has been a major concern of Freestyle development. It is also shown that applying more Python-like coding practices to Freestyle style modules may lead to performance improvements in terms of rendering speed. These work elements were addressed by the following patch sets, each of which is a result of repeated code reviews and careful patch revisions:

• D319: Freestyle Python scripts update.
• D545: Freestyle Python API: new methods for Stroke and StrokeVertexIterator.
• D623: Freestyle Python API: Updates and speedups for the parameter editor.

Although not related to NPR stuff, several Windows and/or internationalization issues were also resolved through patch submissions by the Author:

• D594: Fix T40065: Pressing Esc in separate render result window does not focus main window.
• D595: Additional fix for T35176: Python failure with .blend files loaded from paths including non-ASCII characters.
• D605: Fixes for proper handling of wchar_t paths in MinGW.

Plans for the rest of the project duration

The general development targets listed in the Introduction section will keep serving as guidelines of the project during the remaining two months. Among the three project targets, a strong emphasis is put on Freestyle performance improvements. Specifically, tuning efforts for reduced memory consumption will be continued, while improved rendering speed will also be pursued as much as time permits.

A final report is planned at the end of the extended project duration. If the project contract is further extended for additional months from October, then another interim report will instead be delivered at that time.