From BlenderWiki

Jump to: navigation, search
Note: This is an archived version of the Blender Developer Wiki. The current and active wiki is available on wiki.blender.org.

HOWTO transplant to git.blender.org developpment branches intertwinned with git-svn

Introduction

This HOWTO aims to salvage developement branches made with SVN mirrors, that are impossible to rebase. To know if this is for you, check the following:

  • MANDATORY Did you use to develop for Blender in a Git repository either cloned from jesterking's SVN mirror or directly from git-svn ? (other SVN mirrorring solutions might also be handled with modifications, but not manual periodic dumps of SVN)

and

  • Is the branch that much intertwinned with trunk from git-svn (several merges with conflict resolutions) that rebasing is too painful ? (Note that while rebasing you'd have to know how to use the -Xsubtree=blender option, unless your SVN clone doesn't have the leading blender/ directory)

or

  • Did you commit to files with CR-LF line endings so that conflicts happen all the time with git.blender.org while rebasing and the -Xrenormalize rebase option doesn't help ?

or

  • Do you want to try a wicked smart way to transplant history

Then read the following.

Prerequisites

  • Of course, you'll need a git repository with the work you want to transplant. For ease of use I suggest to only keep two branches: the upstream generated with git-svn or cloned from jesterking's mirror, and the branch you want to transplant.

DON'T USE YOUR REGULAR REPOSITORY but a clone of it, it will undergo heavy surgery.

  • Add git.blender.org as a remote to your repository:
git remote add blender git://git.blender.org/blender.git
git fetch blender

In the following I'll suppose that blender trunk from git-svn is the branch master and that your developement branch is mybranch. The goal is to replace commits of master with commits of blender/master (the trunk from git.blender.org).

Initial work in git

Anytime you can (and should) check if things are good by looking at git log --graph --decorate --oneline --all or gitg or gitk or equivalent. That's invaluable.

Finding the root(s) of your branch

We need to find all commits in your branch whose first parent is in master, because they'll undergo specific treatment from the reposurgeon tool.

To do that, you can use the following (it needs git >= 1.8.0 but there are other ways) :

for c in $(git rev-list master..mybranch); do git merge-base --is-ancestor $c^1 master && echo $c; done

Create branches root1, root2, ... for all of them. Note that most of the time you should only get one result here.

Marking the branch point(s) of your branch

Also, to save work in reposurgeon, create a branch base1 at root1^1 (its first and probably only parent) and similar for the other roots.

Now you have to find in blender/master the exact same commit as base1 in master. Of course it won't have the same SHA1, not even the same commit message (the one from git-svn has a git-svn-id: line appended). But it will have the same date and author (though from git-svn you only get logins, not full names -- that will be fixed later).

The following code can help find it for you, it should return only one result:

date=$(git show -s base1 --format="format:%aD" 2>/dev/null)
log=$(git show -s base1 --format="format:%B" | head -n 1 2>/dev/null)
git log blender/master --before="$date" --after="$date" -F --grep="$log"

Create a newbase1 branch at that commit, and do the same for the other branch points.

Export relevant history for consumption by reposurgeon

Now you need to find the "oldest" branch point, because we will not send to reposurgeon history older than it. By "older" here I really mean "deepest in the ancestry graph", to avoid problems with commits that are children (or descendants), but have an older date (you can put the date you want in a git commit).

The following command might help you:

git log newbase1^! newbase2^! ... newbaseN^!

Let's create a branch newbase0 at that oldest base. Also create a base0 similarly.

Now we can export with

git fast-export --reverse ^newbase0^@ newbase1 ... newbaseN \
                          ^base0^@ base1 ... baseN \
                          root1 ... rootN \
                          master mybranch > ../unfiltered.fi

Surgery in the reposurgeon

Importing and cleaning up

reposurgeon will run quite a lot of cleaning up commands; to avoid typing them you can fetch a blender.lift script here that we'll use next. This file contains the same cleanup commands as were used for the official git.blender.org conversion.

Also you need to get the authors.txt file from the git.blender.org conversion process.

Now you can do:

/path/to/reposurgeon
reposurgeon% script blender.lift

The tool will load unfiltered and perform some cleanups, then will give you back the reposurgeon% prompt.

Adoption of your branch root(s)

The goal now is to change root1 first parent to be newbase1 instead of base1. Because of the reposurgeon (and git-fast-import) storage model, that means root1 will now inherit newbase1 tree contents instead of those of base1. There shouldn't be a lot of differences but that will ensure that your branch will no longer contain cruft from the old SVN mirror (some of this cruft might still not have been cleaned up).

Normally, root1 wasn't a merge, so the command is simply:

reposurgeon% reparent <newbase1>,<root1> rebase # the rebase modifier is crucial here

If that's not the case, you need to:

reposurgeon% inspect <root1> # look at "merge :xxxx" lines and note them
reposurgeon% reparent <newbase1>,<root1> rebase # the rebase modifier is crucial here
reposurgeon% merge :xxxx,<root1> # for each line you noted before

Do the same work for root2, ..., if you have them.

Exporting back

reposurgeon% prefer git
reposurgeon% write filtered.fi

Use CTRL-D to quit reposurgeon.

Final surgery in git

Import filtered commits

Return in your git repository, and run:

git fast-import --force < ../filtered.fi

Replace git-svn commits by git.blender.org ones

Now get the surgery script here. That script will search for commits from master in the ancestry of your branch, and then make git use the same commit from blender/master instead of all other instances of the same commit that it can find in your history. This uses the git replace mechanism.

The script will continue to try as long as it finds commits to replace in your branch.

Last replacement(s)

There is a last commit to replace: when the commits passed through reposurgeon, the newbase1 branch was changed to be an identical commit but without parents (because we cut history at that point to speed the import/filtering/export phases). It isn't replaced by the previous script because it doesn't belong to git-svn master. You should search for the equivalent commit of blender/master and call:

git replace newbase1 sha1_of_equivalent_commit

See above for a command to search for equivalent commits. You should do the same for all other branch points.

Making all of it final

Before you can push and work with your history, you should make all replacement final by using:

git filter-branch mybranch

To speed it up you can refine the revision spec you give to filter-branch to avoid doing spending time doing nothing on all of git.blender.org commits. Maybe add a "--since=" argument with a good date; it seems that git filter-branch ^newbase0^@ mybranch does not stop filtering before newbase0 but I might be wrong.

Now your mybranch branch should be tied to git.blender.org and pull in no commit of your old git-svn mirror; push it to a public repository (e.g. a clone of git.blender.org, maybe a fork of the official GitHub or Gitorious mirror if/when they exist) and enjoy !