From BlenderWiki

Recently there has been a need to store per-library block data in blender. Python rendering scripts (such as Neqsis) need some way to store limited custom data in objects, meshs, cameras, etc. The to-be-written Render API also needs to store custom settings in materials, lamps, cameras, etc.

Imagine a a python script that linked blender to a 3D audio system, it'd need to attach sound info to objects. A rendering script to hook blender to renderman would need to store shader parameters in objects as well as settings in the scene itself. These properties would all be user-editable via a simple generic UI panel, that could be added to the render buttons, material buttons, and perhaps even a special Properties tab in the buttons window.

Common Features We Need

• We need supprot for float, int, string, and an ID reference type.
• We need support for storing vectors, matrices, etc.

Proposal

The IDProperty system will store properties as unique types (kindly packed into unions in a single IDProperty struct). There will be an Array type (which will support simple 2-dimensional arrays of basic types), Group type (a special linked-list container for grouping properties together) and float, int, ID and string types. Also there will be a Vector and Matrix type.

ID Type for properties

There will be a special idproperty type for referencing ID library block data. Properties of this type will increase and decrease the ID block usercount correctly.

Naturally, some hooks will have to be added to the existing unlink functions, but this shouldn't be a problem.

Here's a diagram of how id properties relate to stuff.

Group ID Properties

The Group type is just a container for storing other ID Properties. For example, a renderman script might store settings in a RMan group, with might have subgroups for displacement shaders, surface (light) shaders, volumetric shaders, etc).

Error System

Initially I wrote a very simple error stack implementation; however I'm going to either cut it out or just replace it with asserts, as it seems too complex and not really very useful.

Bypassing SDNA's Lack of Support for Unions

Unfortunately, SDNA does not support unions, so to bypass this in the implimentation I've had to basically use 2 structs, IDProperty (which is excluded by sdna using the /*##*/ comment) and IDPropertyDNA, which replaces the union with its largest member, ListBase.

Implementation So Far

Update: October the 27th, 2006

I've finished implementing Groups, Strings, Ints and Floats now. I've also worked on the Array type, which I've decided is going to need some rethinking to work.

So after Arrays are done, next up will be Vector and Matrix objects. And I need to start work on the general panel system to have an ID property editor panel in materials, object, editing, etc tabs in the buttons window.

Update: October 25th, 2006

Ok I now have ID properties working in a primitive state! Creating Group and String properties is now fully supported, including saving and the like. Currently only Materials have ID properties accessable, but other ID types won't be hard :)

To implement the file stuff, I typedef'd all the needed read/writefile.c functions and pass them around as function pointers. This way, most of the file code can reside inside the main ID property source file, rather then readfile/writefile.c (makes it much simpler and keeps with the internal hiarchal object-oriented design I'm going for).

I also made a little python ID properties browser; unfortunately the Draw.String button is crashing blender for me today. :S gerr.

Current Implementation

The Data Union

/*##*/
typedef union {
	int i;
	float f;
	char *str;
	void *array;
	float *floatdata;
	ListBase list;
	struct ID *id;
} IDPropertyData;

This is the all-important data union, used to store the data needed by all types. This is faily self-explanetory :)

The Template Data Union

/*##*/
typedef union {
	int i;
	float f;
	char *str;
	struct ID *id;
	struct {
		short type;
		short len;
	} array;
	struct {
		int matvec_size;
		float *example;
	} matrix_or_vector;
} IDPropertyTemplate;

This union is used to pass template values to the IDP_New function. the matrix_or_vector struct is just me wanted to combine the struct for mats and vectors, as they're exactly the same.

The IDProperty Struct

 /*##*/
typedef struct IDProperty {
	struct IDProperty *next, *prev;
	char name[32];
	char type, subtype;
	short flag;
	IDPropertyData data;
	int len; /* array length, also (this is important!) string length + 1. 
	            the idea is to be able to reuse array realloc functions on strings.*/
	/*totallen is total length of allocated array/string, including a buffer.
	  Note that the buffering is mild; the code comes from python's list implementation.*/
	int totallen; /*strings and arrays are both buffered, though the buffer isn't
				  saved.  at least it won't be when I write that code. :)*/
	int saved; /*saved is used to indicate if this struct has been saved yet.
	             seemed like a good idea as a pad var was needed anyway :)*/
} IDProperty;

/* initial idea to get around dna not supporting unions: just copy the struct and 
   replace the union with it's largest member! wahoo!!*/
typedef struct IDPropertyDNA {
	struct IDProperty *next, *prev;
	char name[32];
	char type, subtype;
	short flag;
	ListBase bleghreek; /*just specify largest member of union, should work for 32-bit
	                     systems, though I imagine it'll take some hacks in the file
	                     reading code for 64-bit systems to handle endian issues 
	                     properly.*/
	int len;
	int totallen;
	int saved;
} IDPropertyDNA;

The IDProperty struct is what stores data for all ID property types. It has a linked-list header for inserting in groups (which are just linked lists), a name, a type member to specify which type it is, a subtype to specify the data type for arrays (if I decide to keep them), a currently unused flag variable plus the data union. The rest should be self-explanetory from the comments.

Constants

//#define UNIONREF(Union, a) (*((Union *)a))

#define MAX_IDPROP_NAME	32
#define DEFAULT_ALLOC_FOR_NULL_STRINGS	64

/*IDProperties have a limited error stack system; these defines are to help
  python with figuring out what python error class to use.*/
  
#define IDP_TYPEERROR		1
#define IDP_RUNTIMEERROR	2
#define IDP_UNKNOWNERROR	3

/*->type*/
#define IDP_STRING	0
#define IDP_INT		1
#define IDP_FLOAT	2
#define IDP_VECTOR	3
#define IDP_MATRIX	4
#define IDP_ARRAY	5
#define IDP_GROUP	6
#define IDP_ID		7

/*special types for vector, matrices and arrays*/
/*this next one only has real meaning in arrays, I think.  so you can have
  arrays that can store any property type, rather like a python list.*/
//deprecated #define IDP_LISTARRAY	8
#define IDP_MATRIX4X4	9
#define IDP_MATRIX3X3	10
#define IDP_VECTOR2D	11
#define IDP_VECTOR3D	12
#define IDP_VECTOR4D	13

#define IDP_FILE	14

typedef struct ID {
	void *next, *prev;
	struct ID *newid;
	struct Library *lib;
	char name[24];
	short us;
	/**
	 * LIB_... flags report on status of the datablock this ID belongs
	 * to.
	 */
	short flag;
	int icon_id;
	IDPropertyDNA properties;
} ID;

The following functions have been written:

/* ---------- Error System ------- */
void IDP_RaiseError(int err);
int IDP_GetErrorClass(int err);
int IDP_TestError(void);
int IDP_GetAndRemoveError(void);
char *IDP_GetErrorString(int error);

/* ----------- Array Type ----------- */
/*this function works for strings too!*/
void IDP_ResizeArray(struct IDProperty *prop, int newlen);
void IDP_FreeArray(struct IDProperty *prop);
void IDP_UnlinkArray(struct IDProperty *prop);

/* ---------- String Type ------------ */
void IDP_AssignString(struct IDProperty *prop, char *st);
void IDP_ConcatStringC(struct IDProperty *prop, char *st);
void IDP_ConcatString(struct IDProperty *str1, struct IDProperty *append);
void IDP_FreeString(struct IDProperty *prop);

/*-------- ID Type -------*/
void IDP_LinkID(struct IDProperty *prop, ID *id);
void IDP_UnlinkID(struct IDProperty *prop);

/*-------- Group Functions -------*/
void IDP_AddToGroup(struct IDProperty *group, struct IDProperty *prop);
void IDP_RemFromGroup(struct IDProperty *group, struct IDProperty *prop);

/*-------- Main Functions --------*/
struct IDProperty *IDP_New(int type, IDPropertyTemplate val, char *name);
/*NOTE: this will free all child properties of list arrays and groups!
  Also, note that this does NOT unlink anything!  Plus it doesn't free
  the actual struct IDProperty struct either.*/
void IDP_FreeProperty(struct IDProperty *prop);
/*Unlinks any struct IDProperty<->ID linkage that might be going on.*/
void IDP_UnlinkProperty(struct IDProperty *prop);

/*These functions are used by blender's .blend system for file saving/loading.*/
void IDP_WriteProperty_OnlyData(IDProperty *prop, void *wd, WriteStructProc writestruct, WriteDataProc writedata);
void IDP_WriteProperty(IDProperty *prop, void *wd, WriteStructProc writestruct, WriteDataProc writedata);

void IDP_DirectLinkProperty(IDProperty *prop, void *fd, NewDataAddrProc newdataadr, 
                      NewGlobAdrProc newglobadr, NewLibAdr newlibadr,
                      NewLibAdrUs newlibadr_us);
void IDP_LibLinkProperty(IDProperty *prop, void *fd, NewDataAddrProc newdataadr, 
                      NewGlobAdrProc newglobadr, NewLibAdr newlibadr,
                      NewLibAdrUs newlibadr_us);