From BlenderWiki

Data Management

This section follows a bottom-up order, starting with the description of single particle elements and going up to the higher access level at the end.

Some open questions are noted on the discussions page!

Constant Properties

A few properties of a particle are assigned to it at creation time and don't ever change after that. These are

ID

Each particle gets a running number assigned to it, starting with 0 for the first particle emitted and increased for each new particle. This can be used to uniquely identify a particle. A common usage is also as a random number generator (RNG) seed value to ensure repeatability of randomized effects.

Birth Time

When the particle is emitted, the current frame value (or a fraction in case of subframes) is assigned to it as it's birth time. A common usage is a general age limit for particles by subtracting birth time from current frame.

Particle State

The state of a particle describes the properties that can change over time. A particle state always contains the time value at which it was in effect.

In addition to the current state of the particle, a second state instance for the new state is stored for each particle. See Simulation for details.

Attributes

The state of a particle is made up of different attributes.

On the lowest level, a particle defines a position in space together with the velocity. In addition a particle can carry several attributes. Here are a few examples:

• Orientation (used for direction-dependent particles)
• Mass
• Size
• Colour
• Temperature
• Charge
• Strand (hair/grass particles)
• Mode/Type (to define different types of particles in the same system)
• ...

Obviously each attribute can have very different data type and size, ranging from simple floats to arrays (as in hair strands).

Having all attributes defined in the particle data struct is not feasible because

1. Usually most attributes are not needed for a simulation. It would be a waste of space to store them all. Attributes should be enabled if needed by simulation elements or explicitly set by the user.
2. The list of attributes should be extensible to allow more simulation effects and display methods in the future.
3. Particle attributes are a part of the concrete implementation and should therefore be excluded from the core code on principle.

For this reason the enabled attributes in a point cloud are stored in a dynamic list. The size of a particle is calculated from this list of attributes (instead of simply using sizeof(Particle)). When allocating the buffers additional space of this size is added to the base Particle struct.

Code accessing the attributes must take the byte offset into account and should store it if possible to avoid slow lookups by iterating the attribute list. Also simulation code should not access such attributes directly but use generic data sockets and get/set nodes instead (more information on attributes in ICE trees can be found here).

Interpolation

When storing keyframes or cache frames of particle data, only the state needs to be stored. Particle states can generally be interpolated. This is useful for keyframe animation but also for subframes and for loading from caches that skip frames. Each attribute must therefore implement an interpolation function.

Dynamic Attributes

In addition to predefined attributes that come as part of an existing simulation feature, the user should be able to define custom attributes for particles.

Particle Buffer

Some requirements distinguish the particle buffer from a typical mesh buffer:

1. The particle buffer must be dynamically extensible, because new particles can be added to the simulation at any time.
2. In order to keep memory requirements small, dead particles should be removed from memory as soon as possible.

For these reasons the particle buffer will implement a paging system. This means that particles are not allocated in one contiguous memory block, but rather in pages containing a fixed amount of particles.

Newly allocated pages are empty (their fill counter is 0). New particles are added to the last allocated page as long as it has "unborn" particles, i.e. particles that are allocated but not born yet (fill < page_size). As soon as this last page is completely filled, a new page is allocated.

Killing a particle just means setting its life status to "dead". Particles are only removed from memory when all particles on a page are dead, in which case the whole page is freed.

There are some rules-of-thumb to selecting the optimal page size:

• When emitting a lot of particles each frame and/or particle lifetimes are short, the page size should be relatively large. This prevents all too frequent allocations of new pages.
• When emitting only few particles and/or lifetimes are long and vary greatly, small page sizes are generally preferable. The reason is that in this case there can be a lot of pages with just a few living particles, thereby increasing memory wastage and reducing performance because each page has to be searched for just a few particles (while dead pages can be skipped).

It may be possible to implement automatic page size estimation in addition to allowing the user to set it manually. This is however a non-trivial task and could cause more harm then help if it does not work well.

Access layer

An important improvement over the current particles implementation must be the encapsulation of accessing particle data. The more complex inner structure of particle buffers prohibits the use of simple pointer arithmetic. Several different ways of access are made available, according to the situation they are needed in:

1. Access by ID

   This is a very straight-forward method and allows the access of a particle by it's ID.

2. Iterators

   A common case is that one needs to iterate over all particles or a specific subset of particles (e.g. all visible particles). For this there is the convenient method of using an iterator. Iterators are small structs and should be created as local stack data. They should not be stored permanently!

Good:

void my_func()
{
    ParticleIterator pit;
    ...
    for (pit_init(&pit, psys); pit.pa; pit_next(&pit)) {
        do_stuff_with_particle(pit.pa);
        do_stuff_with_particle_index(pit.index);
    }
}

BAD:

ParticleIterator* my_evil_func()
{
    return MEM_callocN(sizeof(ParticleIterator), "ParticleIterator");
}

Iterators have the possibility to automatically skip unwanted particles by an internal skip function. By default this only includes dead particles (which should never be accessed from outside the system, since they are not guaranteed to be in memory anyway).