User:Kazanbas/Gsoc2009/Blog

From BlenderWiki

Blog

May 7th

Current state: py export API is (almost?) ready, py import API is w.i.p.

May 10th

Baby-steps: bpy.meshes["Cube"].verts[0].co is a vector and is editable.

Why is bpy.meshes.verts readonly? Is this RNA limitation? Probably this has to do something with EditMesh conversion i.e. memory allocation effectiveness.

Most important: I have to learn RNA internals to answer these questions. Start with using RNA... then how RNA stores data, then... and finally how RNA gets exposed to python.

May 12th

Studying RNA. Will dump here new things I get to know.

FIXED: Tell brecht about typos here in set_vertex_selected in calls to RNA_pointer_create, RNA_id_pointer_create and RNA_property_identifier first arg is redundant, replace RNA_property_float_set_array with RNA_property_float_set_index.

What is "ID struct", "ID property"? The description of the ID struct RNA definition says: "Base type for datablocks, defining a unique name, linking from other libraries and garbage collection.". So "ID struct" means ID struct's derived struct?

May 14th

So, RNA allows to define RNA structs. RNA struct can then be associated with an SDNA struct. Each struct can have properties, functions (methods?).

RNA auto-generation

Curiously, RNA_blender_cpp.h and RNA_blender.h generated by makesrna are not used? They serve as demos of cool stuff that can be done with RNA?

For rna_mesh.c (compiled on preprocess and runtime), makesrna generates rna_mesh_gen.c which contains functions that operate on RNA structs and property RNAs that describe each property and refer to those functions. Functions include _get, _set, _length, _begin, _next, _end, etc. Property RNAs are named like "rna_[struct]_[property]", functions - like "[struct]_[property]_[function]" (e.g. rna_Mesh_verts, Mesh_verts_get).

In rna_mesh.c - the runtime part - manually written functions exist like "rna_[struct]_[property]_[function]", these are referred to by generated functions in rna_mesh_gen.c and replace them.

Summary

makesrna

So the preprocess works like this:

• generate RNA database by calling definition functions (like rna_mesh_define) specified in PROCESS_ITEMS. The functions define all structs and their properties.
  
• produce _gen.c files

_gen.c files can be built then.

Now I know that...

...RNA is an API to define, examine and manipulate structures at runtime.

May 16th

Collections can take form of arrays and dictionaries (associative array). To lookup values, use RNA_property_collection_lookup_int for arrays and RNA_property_collection_lookup_string for dicts.

Fixed this (diff).

Interesting excerpt from here, could help understand what ID struct is:

A pointer to an RNA struct is always wrapped in a PointerRNA struct. This contains the actual data pointer and the struct type. It also contains the data pointer and the type of the ID datablock that the data belongs to. This is necessary in some cases, as for example a vertex by itself does not provide enough information to update the dependency graph, we need to know the mesh this vertex came from.

IDEA: Why not, in RNA, allow collections to be editable? Methods like add, insert, remove, etc.?

Learnt what ID props and ID structs are. Thanks ideasman_42!

May 17th

If generic collection add is impossible then maybe

object = bpy.data.add_object(ob_type)
object.add_geometry(verts, faces, ...)

How about using operators for this? Find more about it.

Macro-style access?

May 17th - Sunday meeting

Discussed with brecht about add/remove functions on RNA collection. He thinks it's a good idea.

Hint: RNA already handles getting/setting array items - learn from this code to find out all the different kinds of lists used.

Come up with a proposal and send it to ML?

May 20th

According to the code in RNA_def_property_collection_sdna, RNA currently supports two types of lists: regular C arrays and ListBase linked lists.

So, to implement collection add, I need to find functions that add to listbase and find out how to reallocate an array.

May 23rd

Linked lists are modified with BLI_addhead, BLI_addtail and other funcs declared in source/blender/blenlib/BLI_listbase.h. Arrays, at least vertices, edges and faces in Mesh, are allocated with MEM_callocN (see load_editMesh).

EditMesh

EditMesh is created on enter edit mode (make_editMesh in ED_object_enter_editmode), all geometry modifications like extrude are done on EditMesh, once we exit edit mode (ED_object_exit_editmode), EditMesh is converted back to Mesh (load_editMesh). In EditMesh verts and faces are linked lists.

TODO

• EditMesh is efficient only when doing a series of operations? If so, RNA collection `add` on Mesh could directly add verts. Still need to figure out how to keep edges consistent with faces? I still don't know how these two are related?
• also see other examples of collection add, e.g. how new object is added, new scene, new mesh, new material. These do additional operations apart from listbase append?

svn

Oops, spoiled my svn branch with a wrong merge command :-)

Will read svn-book to fix this.

FIXED. When doing merge it's important to write down the right-hand revision and use it on next merge.

May 26th

tdd?

IDEA: use TDD to validate new RNA collection methods.

Haven't used C testing frameworks before.

So, check or Cmockery?

mainstream

• discussed with ideasman and brecht, got hints on what to do
• started obj exporter conversion
• renamed (in my branch) "exec" bpy operator method to "execu" for py2.5 compat.

NEXT: get operator stuff working, continue exporter conversion

May 27th

Operator are not getting context in exec, invoke or poll.

For exporter, I need context.

Poll isn't called yet?

Also, operator property definition doesn't work?

June 1st

Whew!

Spent lots of time to figure out that in bpy getattr on RNA_Context objects does some magic to provide props like "active_object", etc. (see pyrna_struct_getattro). The absense "active_object" in RNA_Context definition confused me.

So, to pass context to py operators I just need to pass it as is, pointer to bContext!

Oops!

According to my project schedule, by mid June basic data imp/export should be ready... :-)

June 2nd

Ok, now that context is passed to operators, I can continue with OBJ exporter conversion. I'll read PEP-8 first. Also, on todo:

• bring file selector back
• generic `extend` for RNA collection

June 9th

Painting over, fun begins!

Started OBJECT_OT_apply_transform...

Read again http://lists.blender.org/pipermail/bf-taskforce25/2009-March/000544.html that kaito pointed me to a long ago.

Interestingly the new python API is just a plain mapping to C!

Will dive into Python C API - need to know how object allocation/deallocation works to be able to create/delete meshes for temporary use.

June 10th

• Added Main.add_mesh and Mesh.copy.
• Rewrote Mesh.copy to copy data between meshes.
• Couldn't RNA-wrap obmat field of Object.

Now looking at operator property declaration. For file selector we need a string property declared. Then WM_event_add_fileselect(bContext*, wmOperator*) can be wrapped.

June 16th

What's been done:

• string property definable on operators - property definition code is hackish!
• imp/exp operator registration working. Implemented in Python.
• file selector working
• basic OBJ exporter working

Next:

• what other important things exporter needs?
• generic rna add, remove for importer

TODO

• apply PEP-8 in scripts
• learn how to use RNA
• quick brain-dump: RNA collection modification is difficult/impossible, better study prev py APIs, find patterns and design a new API based on custom RNA funcs  RNA collection add/remove is possible and desirable
• besides adding objects and object data any other things needed for importers? study existing import scripts to answer this
  
• refresh my memory on python syntax using this and other resources
• the result should go to `end-user spec`
• RNA (is said to be used?) in animation, animation is about creating keys, how RNA handles this? does it?
• aswer the question: is it possible to define a single generic `new` method that will work on meshes, objects, scenes and other collections? yep, indeed
• how uvs are combined/related with verts and faces, are there tfaces? need to refresh my memory on these (although there's probably not much :-) this could be helpful for Chingiz too
• how edges affect the topology? do they at all?