User:Jbakker/Blender docker

From BlenderWiki

Jump to: navigation, search
Note: This is an archived version of the Blender Developer Wiki. The current and active wiki is available on wiki.blender.org.

Project page: Blender docker

This is the projectpage for Blender Docker. Docker.io is a framework that can solve software configuration issues by using chroot, linux namespaces and disk images. It is widely used in the web domain.

Problem statement

install_deps.sh is a shell-script within the blender source code to install all dependencies (external projects) easily in a Linux based system. It does it work good but has some limitations when we introduce new libraries/dependencies. The does not work correctly when the system is not clean. When this is the case it is hard to detect what is the cause of the error. Which part of the system configuration is failing causing the build error.

As install_deps.sh is for Linux and Docker.io only runs on a Linux host system, this project only creates a solution for Linux. Apple and Windows systems are out of scope.

Project solution

Docker.io is a system that can solve software configuration issues by having an own file system (called image) for compiling. These images can be rebuild and distributed among developers.

These images contain all software that is needed to compile blender. It will be easy for new Linux developers to get into developing as they only need to install Docker.io which is available through most package managers.

When the software of the image is outdated (we decide to use a new version of whatever external library) this image can be build, committed and push just like you do using git. The image will not be stored in our git repository, but on a public available docker repository.

This has no impact on our CMake building files. only the command that is needed to start building, configure blender will be different.

Alternatives

In stead of using Docker.io we could also use chroot with the limitation of distributing the file systems.

Deliverables

The deliverables of this project will be:

  • scripts and documentation to build blender using docker.
  • scripts and documentation to recreate a docker image for building blender.
  • a distributed docker image that can be downloaded and used directly.

Requirements

  • As a developer it must be possible to build blender using a docker image
  • As a developer it must be possible to start blender on your host machine
  • As a developer it must be possible to modify the building configuration (ccmake)
  • As a developer it must be possible to recreate the docker image
  • As a developer it must be possible to use this building mechanism inside QTCreator and other IDEs
  • The solution must have no impact on the build files of blender it should be a different way to trigger the building. All current methods of building blender must all be available when not using Docker.
  • The solution must be tested on at lease the next Linux systems (ubuntu 14.04-14.10-15.04, TBC)

Project log

This project is currently under discussion. The proposal is being discussed with the other developers.

  • 27 April 2015: Initiated the project.

Prototype

The next commands will show you how you can already build blender using Docker.io The method is tested on Ubuntu 14.04 and 15.04. The steps differ a bit between the versions. This section describes the steps for Ubuntu 15.04. Using ubuntu 14.04 is quite easy, getting to run under ubuntu 15.04 is harder as more version conflicts exists (boost, llvm).

Ubuntu 15.04

install dependencies

On your host system the minimum needed packages can be installed using: 

> sudo apt-get install docker.io libjemalloc

Note1: I needed to libjemalloc in order to run Blender. Not sure if this is something that needs to be fixed in master

Create docker image

For the next step I assume you used the normal folder structure as described by the development documentation. Modify the next statement when needed. 

> sudo docker run --name blender_build -ti -v ~/blender-git/:/root/blender-git ubuntu:15.04 bash

This will download an ubuntu 15.04 image from the repository and start a bash in this image. The ubuntu version must be the same as your host system, otherwise the right libraries will not be found when starting blender.

Next statements can be executed inside the new started bash 

apt-get update
apt-get -y upgrade
export HOME=/root
apt-get install -y git build-essential sudo libjemalloc-dev
cd /root/blender-git/blender
echo Y | ./build_files/build_environment/install_deps.sh
make -j4 BUILD_CMAKE_ARGS=" -D OPENCOLORIO_ROOT_DIR=/opt/lib/ocio -D OPENEXR_ROOT_DIR=/opt/lib/openexr -D OPENIMAGEIO_ROOT_DIR=/opt/lib/oiio -D WITH_CYCLES_OSL=ON -D WITH_LLVM=ON -D LLVM_VERSION=3.4 -D CYCLES_OSL=/opt/lib/osl -D WITH_CODEC_FFMPEG=ON -D FFMPEG_LIBRARIES='avformat;avcodec;avutil;avdevice;swscale;rt;theoradec;theoraenc;theora;vorbisenc;vorbis;vorbisfile;ogg;x264;openjpeg;openjpeg_JPWL' -D FFMPEG=/opt/lib/ffmpeg"

The make line I copied from the install_deps.sh output. Using static linking is a benefit so you can run blender on your own host. (UNDER DEVELOPMENT)

If building blender fails with LLVM not found you need to set the LLVM library + root dir manual. reason is that during compiling multiple llvm are installed, and the latest is selected. 

cd ../build_linux
ccmake ../blender
make install -j 4

Note. this is fixed since last time I tried this.

Now the docker process can be exited by quitting the bash shell: 

exit

Save the state of the container to be reused next time. On your host system 

> sudo docker ps -a
CONTAINER ID        IMAGE                         COMMAND                CREATED             STATUS                        PORTS               NAMES
b0c4f2a12889        ubuntu:15.04                  "bash"                 15 minutes ago      Exited (0) About a minute ago                     blender_build

Commit the changes to a new Docker image, referring to the container by the name you gave it above. Alternatively you could use the container ID (hash). The resulting image can be reused, shared and altered. 

> sudo docker commit blender_build blender-build:2.75

Rebuild blender

You can now rebuild blender using the next statements 

> sudo docker run -t -i -v ~/blender-git/:/root/blender-git blender-build:2.75 bash
> cd /root/blender-git/build_linux
> make install

Notes

  • note2: install_deps should get an option to not prompt for confirmation (-y) in order to auto generate these images
  • note3: we need an different image for every os version.
  • note4: the commands to build blender should be easier.
  • note5: It is also possible to run blender inside a container by tunneling the X protocol.
Retrieved from "https://wiki.blender.org/index.php/User:Jbakker/Blender_docker"