Infrastructure/BuildBot

BuildBot

Buildbot automatically builds and tests Blender on all supported platforms.

The buildbot administration interface is located at builder.blender.org/admin, with information on build progress and logs to investigate build and test failures.

Master Builds

Buildbot automatically builds the master branch every night, which is then made available for download on builder.blender.org/download.

Branch Builds

Developers can create builds of branches they are working in, to verify that it builds correctly and tests pass on all platforms. The resulting builds can also be used for users to test new functionality before it goes to master.

These branch builds remain available for 100 days, or until a newer revision of the branch is built.

Steps

Release Builds

Making official release builds is done by building the appropriate blender-v2.8x-release branch with the steps described above.

BKE_blender_version.h in this branch must have the cycle set to release (or rc for release candidates), which will trigger buildbot to build all the necessary release packages.

Currently, code signing is not yet supported on buildbot and must be done manually afterwards.

Automated Tests

Buildbot runs the automated tests. If tests fail, the build still continues currently. In the future this will fail the build, once test are running reliably.

Logs for failed tests can be found in the administration interface for every build.

Setup

Builders

Builders are a configuration for how to build for a particular platform. They consist of a number of steps:

  • blender git update: update Blender git repository to latest master or other branch
  • libraries and submodules update: update precompiled libraries, tests and submodules to appropriate version for the branch
  • compile: rebuild blender
  • test: run tests
  • package: create package for upload
  • upload: transfer package to the server
  • unpack: server side updates for the new package

The update, compile, test and package steps are python scripts in the Blender repository. This is also the place to add any build configurations.

build_files/buildbot/slave_update.py
build_files/buildbot/slave_compile.py
build_files/buildbot/slave_test.py
build_files/buildbot/slave_pack.py

Local Testing

Buildbot slave scripts can be run locally for testing. Ensure these directories exist side-by-side.

blender.git/   # Blender source repository checkout
build/         # Empty directory where build files will go.

Then run slave scripts like this:

cd build
python3 ../blender.git/build_files/buildbot/slave_<step>.py <builder> <branch>

Where <builder> is for example win64_cmake_vs2017 as found on the buildbot admin page, and <branch> is the branch name to build (e.g. master).

Master

Buildbot master configuration files are located in the blender-buildbot repository.

To actually update the server configuration, these are manually pulled from the repository by an administrator. The buildbot master is (re)started with

service buildbot restart

The code for the download pages is located in the blender-buildbot-www repository.

Adding Build Workers

  • Download and install the Buildbot Worker (not master), see the Buildbot manual for instructions.
  • On Windows, install pywin32.
  • Next, a number of changes have to be done on the server:
    • A build worker account must be added in master_private.py.
    • If no relevant builder for the given platform exists yet, one must be created in master.cfg.
  • Once this is set up, the build worker can be set up. A directory must be specified, where buildbot will keep its configuration files, download the source code, and write build system output.
buildbot-worker create-worker <directory> builder.blender.org:9989 <workername> <password>
  • After this, the worker can be started and stopped:
buildbot-worker start <directory>
buildbot-worker stop <directory>