View source for Migration to Git

{{TOCright}}

The source files of the Rodin platform and associated plug-ins are under
version control on SourceForge.  Initially, we used CVS and we migrated to
Subversion in March 2009.  This page describes the migration to another
version control system: Git.

The migration from CVS to Subversion was far from perfect.  But then, there
was not much time available for polishing it. In this new migration to Git,
we would like to start again from scratch and first migrate the CVS repository
to Git, then all the subsequent commits in Subversion.

Another concern is that the Subversion repository has grown very large.  We
would like to split it in several smaller repositories, which is now possible
with Git.  The new repositories should each correspond to a set of plug-ins
which are delivered together.

<strong>Important Note:</strong> Make sure that all conversions are performed in a case sensitive filesystem. For instance, on Mac OS X, create a special Volume with a different filesystem than the default (which is case insensitive).

==Migration of the SMT plug-in==

To begin the migration to Git, I started with a very simple set of plug-ins: the SMT Solver plug-ins. Their history is linear, with just some renaming from time to time. All the development was performed in Subversion.  The conversion was performed with <tt>git-svn</tt> with no special constraint using [https://sourceforge.net/p/rodin-b-sharp/rodincore/ci/master/tree/org.rodinp.releng/gitMigration/migrate-svn-smt.sh migrate-svn-smt.sh].

==Migration of the Core Platform==

For the Core Platform, the migration is more intricate.  As stated in the introduction, the initial conversion from CVS to Subversion was of average quality. Moreover, when one includes the sources in the <tt>_exploratory</tt> directory, the files to migrate are a bit scattered all over the place.

The migration is then performed in three steps:
- Migrate the CVS repository
- Migrate the Subversion repository
- Merge the two together

===From CVS===

This is done with the ZSH script [https://sourceforge.net/p/rodin-b-sharp/rodincore/ci/master/tree/org.rodinp.releng/gitMigration/migrate-core-from-cvs.sh migrate-core-from-cvs.sh].  This script
creates a fresh Git repository from a CVS archive (67 MB). To run it, one
needs the CVS archive be named <tt>cvs-Rodin-b-sharp.tgz</tt> and
an author-mapping file named [https://sourceforge.net/p/rodin-b-sharp/rodincore/ci/master/tree/org.rodinp.releng/gitMigration/Authors.txt Authors.txt].

After running the script, the git repository needs some massaging:
* Fix three commit messages that are not UTF-8:
      939bacf56436084c5fb58b041a26c9fe4dd7d876 (2008-01-16T18:57:44Z)
      d2eca7a9ec561c2c7bfbc9c5d57ca9558964ee87 (2008-10-02T16:46:07Z)
      dccf0b77d7d20538a44df5aac719646eb2b62fa4 (2008-10-09T13:06:44Z)
* Merge the branch UNDO_DEV back into <tt>master</tt> as described below.
* Rename tags into a hierarchy.

====Merging a Branch Back in Main Branch====

After importation from CVS, some development branches are dangling.  To make history easier to browse, one needs to merge them back into the main development branch. This is done by grafting.

Suppose that the development branch <tt>mybranch</tt> needs to be merged back into the main development. First, identify the commit where the branch has ceased to be useful. Let's name this commit <tt>mergepoint</tt>. Then, type:
      echo $(git log -1 --format=format:'%H %P' mergepoint) \
            $(git show-ref -s branch) >> .git/info/grafts

Then check that this change works fine and type the following command to make it permanent:
      git filter-branch -- --all

The branch names can now be removed with
      git branch -d branch
      git branch -d mergepoint

====Renaming Tags into a Hierarchy====

After importation from CVS, the tags were named:
 I20060720
 …
 fr.systerel.explorer-0.0.1
 fr.systerel.explorer-1.0.0
 org.eventb.core-0.5.1
 org.eventb.core-0.5.3

This is a long list and it is not very practical. However, Git supports hierarchical tags, like a filesystem. To benefit from this, type the following commands:
 cd .git/refs/tags
 mkdir RodinCore
 mv I* RodinCore/
 ls | grep -e '-' | cut -d- -f1 | sort -u |
     for f in $(cat /tmp/a); do
         mkdir $f
         for g in ${f}-*; do
             mv $g $f/$(echo $g | sed "s:^${f}-::")
         done
     done

The list of tags becomes much shorter, as they are now neatly organized by plug-ins.

===From Subversion===

The Subversion commit that corresponds to the last commit done in CVS is <tt>r6896</tt>.  However, the first interesting Subversion commit is <tt>r6903</tt>.  In between, lie some Subversion commits for massaging the result of porting from CVS, which can be ignored.

====Extracting All Paths====

The first task is to identify the interesting paths in the Subversion repository, that is the ones to extract to the new Git repository.  For this, I have installed the tool [git://git.goodpoint.de/svneverever.git svneverever] and run it like that (assuming that I have a full copy of the Subversion repository from SourceForge in <tt>/tmp/svn</tt>):
    svneverever --tags --branches --no-dots file:///tmp/svn > svn-paths.txt

The resulting file contains all the directory that have ever been committed to the Subversion repository.

====Writing Rules====

From the contents of the previous file, I have written rules for [git://gitorious.org/svn2git/svn2git.git svn2git from KDE]. They are available in file [https://sourceforge.net/p/rodin-b-sharp/rodincore/ci/master/tree/org.rodinp.releng/gitMigration/RodinCore.rules RodinCore.rules]. 

The rules are organized by pattern matching and tell what to do with the files that match the pattern.

====Performing the Migration====

The migration from SVN is performed from a local copy of the SVN repository (it would take too long to access to the remote SVN repository on SourceForge). The local copy was obtained by rsync. The migration was completed by running the <tt>do-migrate</tt> of the script [https://sourceforge.net/p/rodin-b-sharp/rodincore/ci/master/tree/org.rodinp.releng/gitMigration/migrate-core-from-svn.sh migrate-core-from-svn.sh].


[[Category:Developer documentation]]
[[Category:Rodin Platform]]