Org:Institute/SystemsPeach

From BlenderWiki

[edit] Peach System Setup

[edit] Problems!

If the graphical display dosnt start - Your kernel maybe updated but not the nvidia drivers.

...press Ctrl+Alt+F1
...Login under your own username
sudo apt-get install linux-restricted-modules
sudo /etc/init.d/gdm restart

If you cant log in to your gnome desktop - bug https://bugs.launchpad.net/ubuntu/+bug/164804

...press Ctrl+Alt+F1
...Login under your own username 
sudo rm /tmp/*
sudo /etc/init.d/gdm restart

[edit] Server

The server is a simple install of ubuntu gutsy 64bit server addition

hostname: biserver

username: user

password <secret>

[edit] User Setup

All users are added on the server, as normal unprivilidged users, accessed via NIS, The users will have sudo access on their own computers to be able to install software and edit configuration files.

These users will be added using the following commands on the server

Adding the groups

sudo addgroup biusers
sudo addgroup biadmins # not used yet

Adding the users

sudo adduser --ingroup biusers $BIUSER

Where $BIUSER are...

andy
cessen
sago
env
william
margreet
ton
brecht
ideasman42

[edit] NFS Server

Use for sharing home files

sudo apt-get install nfs-kernel-server nfs-common portmap
sudo dpkg-reconfigure portmap
sudo /etc/init.d/portmap restart

Add a home share so NIS can have the users home directories mounted remotely.

# run the command 
sudo nano /etc/exports
# Add the line at the bottom of the file
/home 192.168.1.0/255.255.255.0(async,no_subtree_check,rw,insecure)
/media/data/shared 192.168.1.0/255.255.255.0(async,no_subtree_check,rw,insecure)

[edit] Samba Server

This is used for the shared directory - so windows users can also access it.

smb.conf

[global]
security = user
hosts allow = ALL
null passwords = yes

[shared]
comment = Documentation Samba Server
path = /media/data/shared
read only = No
writable = Yes
guest ok = Yes
guest only = Yes

after changing settings run...

sudo /etc/init.d/samba restart

[edit] SVN Server

Following this guide.

sudo apt-get install subversion
mkdir /media/data/svnroot
svnadmin create /media/data/svnroot/peach
sudo useradd subversion
chown -R root:subversion /media/data/svnroot
chmod -R ug+rw /media/data/svnroot

Add /etc/init.d/svnserve with this contents.

sudo chmod +x /etc/init.d/svnserve
sudo /etc/init.d/svnserve start
sudo update-rc.d svnserve defaults

Uncomment the "password-db = passwd" line in /media/data/svnroot/peach/conf/svnserve.conf.

Add some users in /media/data/svnroot/peach/conf/passwd.

[edit] Backup Script

#!/bin/sh

TARGET="/media/backup"
mount /dev/sdb1 $TARGET

TEST="0"
for i in `cat /proc/mounts | cut -d' ' -f2`; do
  if [ "$i" == "$TARGET" ]; then
    echo "found mount..."
    TEST="1"
  fi
done

if [ "$TEST" == "0" ]; then
  echo "drive not mounted"
  exit 1
fi

cd $TARGET

mkdir home
mkdir system
mkdir data

cd ~

umount $TARGET


rsync -avx --delete --exclude "*_nobackup/" /                   /media/backup/system
rsync -avx --delete --exclude "*_nobackup/" /home/              /media/backup/home
rsync -avx --delete --exclude "*_nobackup/" /media/data/        /media/backup/data

[edit] Build Blender

This script (build.sh) can be configured to run as a cron job.

PREFIX="/media/data/shared/blender_nobackup"
cd $PREFIX/blender
svn update
svn revert -R .

REV=$(svnversion)
if [ -f $PREFIX/install/linux2/blender_$REV ]; then
	echo "Nothing to build"
else
	LOG_DEBUG=$PREFIX/build_log_debug_$REV.txt
	LOG=$PREFIX/build_log_$REV.txt


	### Remove all unneeded files??

	## IFS="
	## "
	## for f in $(svn stat | grep '?\t*')
	## do
	## 	f=$(echo $f | cut -c8-) # remove the ? and tab
	## 	echo "Removing unknown file "$f
	## 	rm -rf $f
	## done

	# Debug
	## nice -n19 scons clean BF_DEBUG=1 
	nice -n19 scons BF_FANCY=0 WITH_BF_YAFRAY=0 WITH_BF_GAMEENGINE=0 WITH_BF_FFMPEG=1 BF_DEBUG=1 \
		BF_BUILDDIR="../build_debug/linux2" \
		BF_INSTALLDIR="../install_debug/linux2"    1> $LOG_DEBUG 2>&1

	mv $PREFIX/install_debug/linux2/blender $PREFIX/blender/blender_debug_$REV
	rm blender_debug_current 2> /dev/null
	ln -s blender_debug_$REV blender_debug_current

	# Optimized
	## nice -n19 scons clean
        nice -n19 scons BF_FANCY=0 WITH_BF_YAFRAY=0 WITH_BF_GAMEENGINE=0 WITH_BF_FFMPEG=1    1> $LOG 2>&1

        mv $PREFIX/install/linux2/blender $PREFIX/blender/blender_$REV
	strip -s $PREFIX/blender/blender_$REV # make it a bit smaller
        rm blender_current 2> /dev/null
        ln -s blender_$REV blender_current

fi

[edit] Client

For the client we used Ubuntu gutsy 64bit edition.

The client can be created an mirrored across to many others. as sell as network authentication.

The following steps are required to setup the client systems.

[edit] Bios

the computers have USB Keyboard disabled by default, this needs to be enabled so the USB keyboards that came with the computer work.
The CDROM needs to be set as the first boot device.

[edit] OS (Ubuntu)

when the boot CD shows its menu, press F6 and remove the argument "splash" and press enter.
OS Ubuntu Gutsy (mostly a default install), US keyboard + language

[edit] Workaround for "end of partition error"

- dont partition the drive from ubuntu, when the asking for guided or manual partition, open a terminal instead (leave the installer running)
- type "sudo fdisk /dev/sda"
- remove all existing partitions
- add an extended partition the size of the disk (use default values)
- add the following logical partitions...
  [+8000M : mount /] (os)
  [+8000M] (spare for dual boot)
  [+4000M] (swap)
  [+300000M : mount /media/data] (remaining space, work files)
- set the type of partition 7 (the 2 gig partition) to swap type (82)
- save the changes and close the terminal.
- Now select manual in the ubuntu installer.
- Make sure all ext3 partitions have the formatted option enabled.
set the username and password to both be "guest"
The OS should install now.
Once installed youll be prompted to restart your system

[edit] Workaround for blank screen on load (disable splash)

hold Esc key (as the system boots) to open the grub menu, and select recovery mode
once logged in edit "sudo nano /etc/boot/grub/menu.lst"
remove the argument "splash" from the kernel /boot/..... lines (at the bottom of the file) and from the commented line "# kopts....." so the splash will not show. removing from the commented line means the splash will not be added back after running update-grub.
save, exit then reboot.

[edit] Remove CDROM from the sources.list

use the interface to do this, System, Administration, Software Sources

untick the CD as a source.

[edit] NVidia drivers

Install the restricted drivers AFTER upgrading all packages.

The quadro fx 1700 cards would flicker and run slow sometimes,
to fix this remove the nvidia drivers...

sudo apt-get purge linux-restricted-modules
sudo apt-get purge nvidia-glx-new

Installing the beta drivers from nvidia's site.

[edit] NIS Server

install portmap and nis

sudo apt-get install portmap nis

[edit] hosts

add this line to /etc/hosts

192.168.1.76 biserver

The server does not have a static IP so if it ever changes, update the IP in this file.

[edit] hosts.allow

add this line to /etc/hosts.allow

portmap : ALL

This means that the computer will accept a NIS connection from the server, if the server had a static IP it would be better to use that.

[edit] Set up name services to use NIS

Edit /etc/passwd to add a line at the end saying:

+::::::

Edit /etc/group to add a line at the end saying:

+:::

Edit /etc/shadow to add a line at the end saying:

+::::::::

[edit] yp.conf

Edit /etc/yp.conf and add the line:

ypserver biserver

Restart NIS

/etc/init.d/nis restart

[edit] Changing Users/Passwords

sudo make -C /var/yp

[edit] NFS Client

This is used to mount the remote home directory. (coupled with NIS)

sudo apt-get install portmap nfs-common

[edit] fstab

Add this to the fstab

biserver:/home /home nfs rsize=8192,wsize=8192
biserver:/media/data/shared /shared nfs rsize=8192,wsize=8192

Explanation for Shared

biserver - is the server hostname
/media/data/shared - is the share on the server (use the servers home dir)
/shared - is also where we mount the filesystem
nfs - is the filesystem type
rsize=8192,wsize=8192 - recommended settings for good performance.

Now you may need to restart services using the following command

sudo /etc/init.d/portmap restart
sudo /etc/init.d/nfs-common restart

[edit] SVN

Installation

sudo apt-get install subversion rapidsvn

GUI

open RapidSVN in the Applications > Programming menu
open Repository menu, choose Checkout ...
URL: svn://biserver/peach (or svn://biserver/apricot)
Destination Directory: /media/data/peach (or /media/data/apricot)
click OK, enter username and password

Command Line

cd /media/data
svn co --username <username> svn://biserver/peach (or svn://biserver/apricot)
enter password
cd peach

[edit] Samba

Samba is used for shared directory for everyone to access.

Most of samba is configured on the server however an entry to the fstab is needed for automounting the shared directory.

install samba fs support

sudo apt-get install smbfs

[edit] mount samba (removed! hangs gnome after sleep)

This will mount the samba share but it would not work after waking up so its better to use NFS fstab.

//biserver/shared /shared smbfs rw,guest 0 0

[edit] User Setup

[edit] User access to audio

By default the new users cant access the system autio to play music. here are the steps to solve this.

add this line to /etc/pam.d/common-account

auth optional pam_group.so

add this line to /etc/security/group.conf

*;*;*;Al0000-2400; audio,floppy,video,cdrom,plugdev,irc,scanner,src

[edit] Sudo Users

As unprivileged users, installing applications and configuring the system is not possible. however users may have privileges to configure their own workstation.

do this by adding the group "biusers" to the allowed sudo group.

run this on the client...

sudo visudo

add this line to the bottom of the file.

%biusers ALL = ALL

[edit] Default permissions

Once the systems are running, edit ~/.profile and uncomment the line

# umask 022

and set to

umask 000

Also add the file ~/.gnomerc and add the line above to it. this means gnome apps will write files everyone can read on shared dirs.

[edit] Blender Compile

These commands can be copied into a terminal to compile blender on ubuntu gutsy.

sudo apt-get install scons subversion openexr libopenexr-dev build-essential libjpeg-dev \
libpng12-dev libopenal-dev libalut-dev libsdl-dev libfreetype6-dev libtiff-dev python-dev \
gettext libxi-dev; \
svn checkout https://svn.blender.org/svnroot/bf-blender/trunk/blender blender; \
cd blender; \
echo "BF_OPENAL_LIB = 'openal alut'" > user-config.py; \
scons -j4;

[edit] Server Build Script

PREFIX="/media/data/shared/blender_nobackup"
cd $PREFIX/blender
svn update
svn revert -R .

REV=$(svnversion)
if [ -f $PREFIX/install/linux2/blender_$REV ]; then
	echo "Nothing to build"
else
	LOG=$PREFIX/build_log_$REV.txt

	IFS="
	"
	for f in $(svn stat | grep '?\t*')
	do
		f=$(echo $f | cut -c8-) # remove the ? and tab
		echo "Removing unknown file "$f
		rm -rf $f
	done

	nice -n19 scons BF_FANCY=0 BF_DEBUG=1 WITH_BF_YAFRAY=0 WITH_BF_GAMEENGINE=0 WITH_BF_FFMPEG=1 1> $LOG 2>&1

	mv $PREFIX/install/linux2/blender $PREFIX/install/linux2/blender_$REV
	rm $PREFIX/install/linux2/blender_current # remove the symlink
	ln -s blender_$REV $PREFIX/install/linux2/blender_current
fi

[edit] Crash Reporting

Automatically report crashes from applications in /shared/bin to /shared/bugs/crash_reports/. Client installation:

/shared/bugs/crash_reports/install_script/install.sh

The patch to /usr/share/apport/apport:

--- original_apport 2007-10-16 18:16:56.000000000 +0200
+++ new_apport  2007-10-16 19:53:36.000000000 +0200
@@ -14,7 +14,7 @@
 # the full text of the license.

 import sys, os, os.path, subprocess, time, traceback, tempfile, glob
-import signal, inspect, grp, fcntl
+import signal, inspect, grp, fcntl, time

 import apport, apport.fileutils

@@ -285,9 +285,11 @@
     else:
         error_log('executable: %s (command line "%s")' %
             (info['ExecutablePath'], info['ProcCmdline']))
+   
+    blendercrash = info['ExecutablePath'].startswith("/shared/bin")

     # ignore non-package binaries
-    if not apport.fileutils.likely_packaged(info['ExecutablePath']):
+    if not (apport.fileutils.likely_packaged(info['ExecutablePath']) or blendercrash):
         error_log('executable does not belong to a package, ignoring')
         # check if the user wants a core dump
         drop_privileges(pid)
@@ -312,9 +314,15 @@
     # Create crash report file descriptor. We prefer to create the report in
     # report_dir if we can create a file there; if not, we just use stderr.
     try:
-        report = '%s/%s.%i.crash' % (apport.fileutils.report_dir, info['ExecutablePath'].replace('/', '_'), pidstat.st_uid)
+        if blendercrash:
+            report_dir = "/shared/bugs/crash_reports" # "/media/data/crash"
+        else:
+            report_dir = apport.fileutils.report_dir
+
+        timestr = time.strftime("%a_%d_%b_%H:%M:%S", time.localtime())
+        report = '%s/%s.%i.%s.crash' % (report_dir, info['ExecutablePath'].replace('/', '_'), pidstat.st_uid, timestr)
         if os.path.exists(report):
-            if apport.fileutils.seen_report(report):
+            if blendercrash or apport.fileutils.seen_report(report):
                 # do not flood the logs and the user with repeated crashes
                 crash_counter = apport.fileutils.get_recent_crashes(open(report))
                 crash_counter += 1
@@ -330,8 +338,13 @@
                     write_user_coredump(pid, cwd, core_ulimit)
                 sys.exit(1)
         reportfile = open(report, 'w')
-        os.chmod(report, 0000)
-        os.chown(report, pidstat.st_uid, pidstat.st_gid)
+        if not blendercrash:
+            os.chmod(report, 0000)
+            os.chown(report, pidstat.st_uid, pidstat.st_gid)
+        else:
+            # dirstat = os.stat(report_dir)
+            # os.chown(report, dirstat[4], dirstat[5])
+           pass
     except (OSError, IOError):
         report = None
         reportfile = sys.stderr

[edit] Video Settings - Compressing DV to OGG Theora (for the blog)

When exporting blender movies use ffmpeg codec.

Format MPEG-2
Bitrate 10000
Min Bitrate 0
Max Bitrate 10000

Audio

Enable Multiplex 
Format PCM

...leave other settings as is

ffmpeg2theora converts from almost any video format (DV, MOV files etc) into ogg.

ffmpeg2theora can be installed

sudo apt-get install ffmpeg2theora

From the cheap and nasty video camera

ffmpeg2theora clip.dv --audiobitrate 32 --channels 1 --samplerate 22050 --videobitrate 600 --optimize -x 360 -y 288

For DV on Andys camera we used this...

ffmpeg2theora weeklyNov16.avi --audiobitrate 32 --channels 1 --samplerate 22050 --videobitrate 800 --optimize --deinterlace -x 720 -y 400

To upload to google, we need a small MPEG file...

mencoder weeklyNov16.avi -o weeklyNov16_small.avi -vop scale=720:400 -ovc lavc -oac lavc lavcopts=vcodec=mpeg4:vbitrate=900:abitrate=32

Letterbox so google video plays properly

mencoder weeklyNov16.avi -o weeklyNov16_small2.avi -vf scale=720:400,expand=720:576 round=2 -ovc lavc -oac lavc lavcopts=vcodec=mpeg4:vbitrate=900:abitrate=32

[edit] Video compression, (high quality content)

For the peach trailer and high quality files.

For trailers and high definition content First make an avi with lossless jpeg compression and no sound from rendered PNG's

mencoder mf://*.png -mf w=1920:h=1080:fps=25:type=png -ovc lavc -lavcopts vcodec=ljpeg -oac copy -o test.avi

Now mix the audio with the avi

mencoder -oac copy -ovc copy -o test.avi -audiofile test.wav output.avi

From this we can make a high and low quality ogg

ffmpeg2theora output.avi --optimize -v 8 -o output_big.ogg
ffmpeg2theora output.avi -x 720 -y 400 --optimize -v 8 -o output_small.ogg

For an avi that can play on windows xp without downloading any codecs

mencoder test.avi -oac lavc -ovc lavc -lavcopts vcodec=msmpeg4:vbitrate=6000:vhq:acodec=ac3 -o test-win-compat.avi

[edit] Making a DVD

This is a test to make the dvd from the trailer

Followed the instructions from http://www.linux.com/articles/53702

[edit] Unessential's

add the repositry - ref http://phorolinux.com/how-to-install-non-free-multimedia-codecs-on-ubuntu-710-gutsy-gibbon.html

Codecs

sudo apt-get install \
gstreamer0.10-alsa \
gstreamer0.10-esd \
gstreamer0.10-ffmpeg \
gstreamer0.10-gnomevfs \
gstreamer0.10-pitfdll \
gstreamer0.10-plugins-bad \
gstreamer0.10-plugins-bad-multiverse \
gstreamer0.10-plugins-base \
gstreamer0.10-plugins-base-apps \
gstreamer0.10-plugins-good \
gstreamer0.10-plugins-ugly \
gstreamer0.10-plugins-ugly-multiverse \
gstreamer0.10-tools \
gstreamer0.10-x \
libgstreamer-plugins-base0.10-0 \
libgstreamer0.10-0 \
totem-gstreamer

Flash

sudo apt-get install flashplugin-nonfree

MS Fonts

sudo apt-get install msttcorefonts

[edit] Cloning the client disk

First remove all cached packages so the disk image is smaller.

sudo apt-get clean

Edit the /etc/fstab and replace all device unique ID's (UUID's) since they will break the system when copying. Also replace UUID's in /boot/grub/menu.lst with the boot device (/dev/sda5 in our case)

reboot with a ubuntu install CD - you cant backup on a live system because of udev, proc etc.

cd /media/sda5 # where this is the OS partition
sudo tar cvpjf /media/storage/linux2.tar.bz2 ./ # where /media/storage is a spare filesystem, in our case /dev/sda8

Now this can be copied to the shared samba drive

Use the ubuntu install CD's to boot up another PC and partition it exactly the same way as the imaged PC's. now copy the archive onto the large partition of the fresh systems local drive (the 300gig partition) and then extract the files onto the first partition where the OS is installed.

# format and mount the OS partition, and large free space and then extract the archive.
sudo mkfs.ext3 /dev/sda5 # small os partition
sudo mkfs.ext3 /dev/sda8 # large free space
# ... mount the drives, youll need to create directories in /media for this.
cd /media/sda5
sudo tar xvpfj linux2.tar.bz2

Org:Institute/SystemsPeach

From BlenderWiki

[edit] Peach System Setup

[edit] Problems!

[edit] Server

[edit] User Setup

[edit] NFS Server

[edit] Samba Server

[edit] SVN Server

[edit] Backup Script

[edit] Build Blender

[edit] Client

[edit] Bios

[edit] OS (Ubuntu)

[edit] Workaround for "end of partition error"

[edit] Workaround for blank screen on load (disable splash)

[edit] Remove CDROM from the sources.list

[edit] NVidia drivers

[edit] NIS Server

[edit] hosts

[edit] hosts.allow

[edit] Set up name services to use NIS

[edit] yp.conf

[edit] Changing Users/Passwords

[edit] NFS Client

[edit] fstab

[edit] SVN

[edit] Samba

[edit] mount samba (removed! hangs gnome after sleep)

[edit] User Setup

[edit] User access to audio

[edit] Sudo Users

[edit] Default permissions

[edit] Blender Compile

[edit] Server Build Script

[edit] Crash Reporting

[edit] Video Settings - Compressing DV to OGG Theora (for the blog)

[edit] Video compression, (high quality content)

[edit] Making a DVD

[edit] Unessential's

[edit] Cloning the client disk

Contents