User:Sergey/DevTools

Developer Tools

Here's some developer tools used by me. Some of them are ported from here to fork with Git.

bachangelog-generator

Here's a script I'm using to help generating bug fixes section of release logs.

#!/bin/bash

#echo -n "From revision: "
#read revision_from
revision_from=56521

#echo -n "Up to revision: "
#read revision_to
revision_to=56521

for r in `seq "${revision_from}" "${revision_to}"`; do
    h=`git svn find-rev "r${r}"`

    if [ -z "${h}" ]; then
        # commit was done to another branch
        continue
    fi

    msg=`git log --pretty="%s%n%n%b" "${h}^".."${h}" | grep -v "git-svn-id"`

    # Skip svn maintaince commits
    maint=`echo "${msg}" | grep -i "SVN maintenance"`
    if [ ! -z "${maint}" ]; then
        continue
    fi

    # Skip style and code cleanup commits
    cleanup=`echo "${msg}" | grep -i "^style cleanup"`
    if [ ! -z "${cleanup}" ]; then
        continue
    fi

    cleanup=`echo "${msg}" | grep -i "^code cleanup"`
    if [ ! -z "${cleanup}" ]; then
        continue
    fi

    git log --pretty="-- {{Commit|${r}}} by %an ----------------%n" "${h}^".."${h}"

    # todo:
    # - skip "committed by accident"
    # - support "Bugfix for"
    fix=`echo "${msg}" | grep -Ei "fix( for| bug|ed|es bug|es )?:?\s+\[?#[0-9]+\]?"`
    if [ ! -z "${fix}" ]; then
        bug=`echo "${fix}" | sed -r 's/.*#([0-9]+).*/\1/'`
        title=`echo "${fix}" | sed -r 's/.*fix( for| bug|ed|es bug)?:?\s+\[?#[0-9]+\]?:?\s*//i'`
        echo "Fix {{BugReport|$bug}}: ${title} ({{Commit|$r}})"
        echo
    fi

    echo "${msg}"
    echo
    echo
done

It could be optimized further, like ignoring style/code cleanup commits, but it's already helps a lot.

bcheck

Script makes basic check for code style and it checks only lines being changed in current patch.

#!/bin/bash

DIFF=`mktemp`

git diff-index -p -M HEAD -- > $DIFF

Lines=`wc -l < $DIFF`

count=0
while [ $count -lt $Lines ]
do
    count=$(($count+1))
    LINE=`head -n $count $DIFF | tail -1`

    if echo "$LINE" | grep -cqP '^diff --git a/.* b/(.*)$'; then
        filename=`echo "$LINE" | sed -r 's/^diff --git a\/.* b\/(.*)$/\1/'`
        continue
    fi

    if echo "$LINE" | grep -cqP '^@@ -\S+ \+(\d+)'; then
        lineno=`echo "$LINE" | sed -r 's/^@@ -\S+ \+([0-9]+).*/\1/'`
        continue
    fi

    if echo "$LINE" | grep -cqP '^ '; then
        lineno=$(($lineno+1))
        continue
    fi

    if echo "$LINE" | grep -cqP '^\+'; then
        newline=`echo "$LINE" | sed -r 's/^\+//'`

        if echo "$newline" | grep -cqP '\s$'; then
            echo "$filename:$lineno: Trailing whitespace"
        fi

        if echo "$newline" | grep -cqP '^[<>=]{7}'; then
            echo "$filename:$lineno: Unresolved merge conflict"
        fi

        if echo "$filename" | grep -cqP '\.py$'; then
            if echo "$newline" | grep -cqP '^ *\t'; then
                echo "$filename:$lineno: TAB used for indentation"
            fi
        else
            if echo "$newline" | grep -cqP '^\t* '; then
                if echo "$newline" | grep -cqP '^ {1,2}'; then
                    # do nothing -- most probably indentation for comment
                    echo "" > /dev/null
                else
                    echo "$filename:$lineno: SPACE used for indentation"
                fi
            fi
        fi

        lineno=$(($lineno+1))
    fi

done

rm $DIFF

bcheck-files

This script checks style in all currently changed files and would report all issues met in that files.

#!/bin/bash

d=$PWD

# get root directory of a repository
root=`git rev-parse --show-toplevel 2>/dev/null`
if [[ -z "$root" ]]; then
  root=`pwd`
  while [[ -d "`dirname $parent`/.svn" ]]; do
    root=`dirname $parent`
  done
fi

cd $root

# TODO: port to svn too
for f in `git diff --name-only -r HEAD`; do
  python3.2 $root/source/tools/check_style_c.py $f
done

cd $d

bpropcheck

This script checks SVN properties assigned to source files and prints files and properties which are missing.

#!/bin/sh

for f in `find . -type f`; do
    if echo "$f" | grep -cqP '\.[cpp|c|cc|h|hpp|cxx|py]$'; then
        # Checking source file
        keywords=`svn propget svn:keywords ${f}`
        eolstyle=`svn propget svn:eol-style ${f}`

        if [ -z "$keywords" ]; then
            echo "$f: missing property svn:keywords"
        else
            echo "Author\nDate\nId\nRevision" | while read x; do
                if ! echo "$keywords" | grep -cqP "$x"; then
                    echo "$f: missing svn:keyword '$x'"
                fi
            done
        fi

        if [ -z "$eolstyle" ]; then
            echo "$f: missing property svn:eol-style"
        else
            if [ "$eolstyle" != "native" ]; then
                echo "$f: svn:eol-style expected 'native' but '$eolstyle' found"
            fi
        fi
    fi
done

bsrcprop

This script resets required SVN properties for list of source files.

#!/bin/sh

for x in $@; do
    svn propset svn:keywords 'Author Date Id Revision' $x
    svn propset svn:eol-style native $x
done

bsymlink-cycles

I've got WITH_PYTHON_INSTALL disabled in my CMake configuration which makes it a bit difficult to check issues appearing in Cycles.

Here's a small utility script which allows to use Cycles with all the features like GPU rendering and OpenShadingLanguage. It basically initializes Cycle addon's directory structure with a symbolic links to source tree and compiler OSL shaders.

Simply running it from ~/.config/blender/2.69/scripts/addons/cycles will do all the job done and you'll be ready to use Cycles (tweak blender version in the path if needed)

#!/bin/bash

SOURCE_TREE="$HOME/src/blender/blender/"
BUILD_DIR="$HOME/src/blender/build/cmake-debug/"
ADDON_PATH=$PWD

if [ $# -ne 0 ]; then
  echo "Usage: $0"
  exit 1
fi

echo "Clear previous symbolic links..."
for file_path in "$ADDON_PATH"/* "$ADDON_PATH"/kernel/* "$ADDON_PATH"/shader/*; do
  file_name=`basename $file_path`
  if [ "$file_name" == "init.sh" ]; then
    continue
  elif [ -h "$file_path" ]; then
    echo "  Unlinking $file_path"
    unlink $file_path
  fi
done

echo "Creating symbolic links for addon..."
for file_path in "$SOURCE_TREE"/intern/cycles/blender/addon/*; do
  file_name=`basename $file_path`
  if [ "$file_name" == "__pycache__" ]; then
    continue
  fi
  echo "  Linking to $file_name"
  ln -s "$file_path" "$ADDON_PATH/$file_name"
done

echo "Creating symbolic links for kernel..."
mkdir -p "$ADDON_PATH"/kernel
for file_path in "$SOURCE_TREE"/intern/cycles/kernel/*; do
  file_name=`basename $file_path`
  if [ "$file_name" == "shaders" ]; then
    continue
  fi
  echo "  Linking to $file_name"
  ln -s "$file_path" "$ADDON_PATH/kernel/$file_name"
done

echo "Creating symbolic links for OSL shaders..."
mkdir -p "$ADDON_PATH"/shader
# TODO(sergey): Sypport SCons
if [ -f "$BUILD_DIR"/CMakeCache.txt ]; then
  for file_path in "$BUILD_DIR"/intern/cycles/kernel/osl/nodes/*; do
    file_name=`basename $file_path`
    file_extension="${file_name##*.}"
    if [ "$file_extension" == "oso" ]; then
      echo "  Linking to $file_name"
      ln -s "$file_path" "$ADDON_PATH/shader/$file_name"
    fi
  done
fi

echo "Creating symbolic links to utility headers..."
for header in util_color.h util_math.h util_transform.h util_types.h; do
  echo "  Linking to $header"
  ln -s "$SOURCE_TREE/intern/cycles/util/$header" "$ADDON_PATH/kernel/$header"
done

for file_path in "$SOURCE_TREE"/intern/cycles/kernel/shaders/*; do
  file_name=`basename $file_path`
  file_extension="${file_name##*.}"
  if [ "$file_extension" == "h" ]; then
    echo "  Linking to $file_name"
    ln -s "$file_path" "$ADDON_PATH/shader/$file_name"
  fi
done

svn-diff

Script creates svn-compatible patch for changes made in git-svn repo. Ininitally it was found in yandex or Git mailing list, not sure..

#!/bin/bash

# Generate an SVN-compatible diff against the tip of the tracking branch

TRACKING_BRANCH=`git config --get svn-remote.svn.fetch | sed -e 's/.*:refs\/remotes\///'`
REV=`git svn find-rev $(git rev-list --date-order --max-count=1 $TRACKING_BRANCH)`
git diff --no-prefix $(git rev-list --date-order --max-count=1 $TRACKING_BRANCH) $* |
  sed -e "s/^+++ .*/&\t(working copy)/" -e "s/^--- .*/&\t(revision $REV)/" \
      -e "s/^diff --git [^[:space:]]*/Index:/" \
      -e "s/^index.*/===================================================================/" \
   -r -e "s/^(@@ \-[0-9]+,[0-9]+ \+[0-9]+,[0-9]+ @@).*/\1/" |
  sed -r -e 'N;N; s/\n---\s\/dev\/null\s\(revision [0-9]+\)\n\+\+\+ (.*)?\s\(working copy\)/\n--- \1\t(revision 0)\n+++ \1\t(revision 0)/' \
         -e 's/\nnew file mode [0-9]+//' \
         -e 's/\ndeleted file mode [0-9]+//' \
         -e 'N;N; s/\n\-\-\- (.*)?\s\(revision ([0-9]+)\)\n\+\+\+\s\/dev\/null\s\(working copy\)/\n--- \1\t(revision \2)\n+++ \1\t(working copy)/'

touchdiff

Ported from script by Campbell Barton: Sometimes you want to know if any of the files you edited caused errors. Rebuilds you may have missed some so touching all modified files, then rebuilding is useful for this.

Add this to your ~/.bashrc file.

alias touchdiff='git status > /dev/null 2>&1 && touch `git diff --name-only -r HEAD --relative .` || touch `svn st | grep "^M" | cut -f4-8 -d" "`'

Show branch in bash prompt

Adding this to ~/.bashrc will make it so command prompt contains name of current branch in git repository.

Black='\[\033[00;30m\]';
Blue='\[\033[00;34m\]';
Green='\[\033[00;32m\]';
Cyan='\[\033[00;36m\]';
Red='\[\033[00;31m\]';
Purple='\[\033[00;35m\]';
Brown='\[\033[00;33m\]';
LGray='\[\033[00;37m\]';
DGray='\[\033[01;30m\]';
LBlue='\[\033[01;34m\]';
LGreen='\[\033[01;32m\]';
LCyan='\[\033[01;36m\]';
LRed='\[\033[01;31m\]';
LPurple='\[\033[01;35m\]';
Yellow='\[\033[01;33m\]';
White='\[\033[01;37m\]';
Normal='\[\033[00m\]';

__bashrc_git_ps1()
{
  if git rev-parse --git-dir > /dev/null 2>&1;  then
    #local b="$(git symbolic-ref HEAD 2>/dev/null)";
    git_dir=`git rev-parse --git-dir`
    if [ "`basename \"$git_dir\"`" == '.git' ]; then
        GIT="git -C ${git_dir}/.."
    else
        GIT="git"
    fi
    local b="$($GIT branch | sed -n '/\* /s///p' 2>/dev/null)";
    if [ -n "$b" ]; then
        local status=""
        local changed_files="$($GIT diff-index --name-only HEAD -- | wc -l)"
        local new_commits="$($GIT log --oneline @{u}.. 2>/dev/null | wc -l)"

        if [ ! "$new_commits" -eq "0" ]; then
            status="$status($new_commits)"
        fi

        if [ ! "$changed_files" -eq "0" ]; then
            status="$status[M]"
        fi

        if [ ! -z "$status" ]; then
            status=" $status"
        fi

        printf " (${b##refs/heads/}${status})";
    fi
  fi
}

PS1='${debian_chroot:+($debian_chroot)}'"${Green}\u${Normal}@${Yellow}\h${Normal}:${LBlue}\w${Green}"'$(__bashrc_git_ps1)'"${Normal}\$ "

Git hooks

pre-commit

Disallows git commit when there're staged changes to the submodules hash.

#!/bin/sh

ROOT=`git rev-parse --show-toplevel`
SUBMODULES=`grep path ${ROOT}/.gitmodules | sed 's/.*= //'`

have_changed_submodules=false
for submodule in $SUBMODULES; do
  status=`git diff --ignore-submodules=none --cached --name-only $ROOT/$submodule`
  if [ ! -z ${status} ]; then
    echo "Changed and staged submodule: $submodule"
    have_changed_submodules=true
  fi
done

if $have_changed_submodules; then
  echo ""
  echo "Some of the submodules are changed. This should not normally happen."

  if [ -z ${FORCE_SUBMODULES} ]; then
  echo ""
    echo "If you REALLY need to commit changes to submodule, use FORCE_SUBMODULES"
    echo "environment variable like this:"
    echo ""
    echo "  FORCE_SUBMODULES=1 git commit"
    exit 1
  else
    echo "Commit of staged submodules was forced."
  fi
fi

If you actually need to commit changes to hash use:

 FORCE_SUBMODULES=1 git commit

prepare-commit-msg

Put information about submodules changes to be added to the commit.

#!/bin/sh

ROOT=`git rev-parse --show-toplevel`
SUBMODULES=`grep path ${ROOT}/.gitmodules | sed 's/.*= //'`

first_time=true
for submodule in $SUBMODULES; do
  status=`git diff --ignore-submodules=none --cached --name-only $ROOT/$submodule`
  if [ ! -z ${status} ]; then
    if $first_time; then
      echo "# Submodules to be committed:" >> $1
    fi
    echo "#\tmodified:   $submodule" >> $1
    first_time=false
  fi
done