Posts: Git

How to merge multiple Git repositories into one

Are you looking for a method to merge multiple Git repositories into a single one? If so, you have reached the right tutorial!

Please bear with me for a second while I provide you with background information and introduce the subject of our experiments. We’ll get to the actual procedure soon and you will be able to apply it to any repository of your choice.

In the Kyua project, and with the introduction of the kyua-atf-compat component in the Summer of 2012, I decided to create independent Git repositories for each component. The rationale was that, because each component would be shipped as a standalone distfile, they ought to live in their own repositories.

February 15, 2014 · Tags: featured, git, kyua
Continue reading (about 4 minutes)

Projects migrated to Git

I finally took the plunge. Yesterday night, I migrated the Kyua and Lutok repositories from Subversion to Git. And this morning I migrated ATF from Monotone and custom hosting to Git and Google Code; oh, and this took way longer than expected.

Migration of Kyua and Lutok

Migrating these two projects was straightforward. After preparing a fresh local Git repository following the instructions posted yesterday, pushing to Google Code is a simple matter:

$ git remote add googlecode https://code.google.com/p/your-project
$ git push googlecode --all
$ git push googlecode --tags

February 26, 2012 · Tags: atf, git, kyua, lutok
Continue reading (about 3 minutes)

Converting a Subversion repository to Git

As discussed over a week ago, I have been pondering the idea of migrating my projects from Subversion to Git. One of the prerequisites of such a migration is the preparation of a process to cleanly migrate the revision history from the old system to the new one. Of course, such process should attempt to preserve the revision history as close to reality as possible (regardless of what some other big projects have done by just throwing away their history; shrug).

The projects I am interested in migrating from Subversion to Git all live in Google Project Hosting. This is irrelevant for all but one detail I will mention later. However, the project hosting's site is the obvious place to look for help and, as expected, there is a little document titled Convert your project from Subversion to Git that provides a good start.

Summarizing, there are two well-suited tools to convert a Subversion repository to Git:

• The built-in git svn command: works pretty well and has a good amount of customization features. The only little problem is that git svn does not recognize Subversion tags as such and therefore converts them to Git branches instead of tags.
• The third-party svn2git: while this is actually built on top of git svn, it properly handles Subversion tags and converts them to native Git tags.

your-address@example.net = Your Name 
(no author) = Your Name 

#! /bin/sh

if grep git-svn-id "${1}"; then
    # We are editing a commit message.
    new="This revision was r\1 in Subversion."
    sed -r -i -e "s,git-svn-id[^@]+@([0-9]+).*$,${new}," "${1}"
else
    # We are editing the git-rebase interactive control file.
    sed -i -e 's,^pick,reword,' "${1}"
fi

first=$(git log | grep '^commit' | tail -n 1 | cut -d ' ' -f 2)
EDITOR="${convert}/editor1.sh" 
    git rebase --interactive "${first}" || true
touch ../control/empty
while [ -f .git/MERGE_MSG ]; do
    head -n 1 .git/COMMIT_EDITMSG >>../control/empty
    EDITOR="${convert}/editor1.sh" git commit --allow-empty
    EDITOR="${convert}/editor1.sh" git rebase --continue || true
done

#! /bin/sh

echo "Empty revisions to be deleted:"
cat ../control/empty | while read line; do
    grep "${line}" "${1}"
    sed -i -e "/^pick ${line}$/s,^pick,fixup," "${1}"
done

#! /bin/sh

set -e -x

[ ${#} -eq 1 ] || exit 1
convert=$(dirname ${0})
project=${1}

rm -rf git control
mkdir git control
cd git

# Start at revision 2 to skip the initial empty revision.
svn2git -v --revision=2 -m "http://${project}.googlecode.com/svn"

# Tag git revisions with the original Subversion revision id.
first=$(git log | grep '^commit' | tail -n 1 | cut -d ' ' -f 2)
EDITOR="${convert}/editor1.sh" 
    git rebase --interactive "${first}" || true
touch ../control/empty
while [ -f .git/MERGE_MSG ]; do
    head -n 1 .git/COMMIT_EDITMSG >>../control/empty
    EDITOR="${convert}/editor1.sh" git commit --allow-empty
    EDITOR="${convert}/editor1.sh" git rebase --continue || true
done

# Drop empty revisions recorded in the previous step.
# The list is in the 'empty' file and is read by editor2.sh.
EDITOR="${convert}/editor2.sh" 
    git rebase --interactive "${first}" || true

# Tag the root revision with the original Subversion revision id.
git tag root $(git rev-list HEAD | tail -1)
git checkout -b new-root root
EDITOR="${convert}/editor1.sh" git commit --amend
git checkout @{-1}
git rebase --onto new-root root
git branch -d new-root
git tag -d root

cd -
rm -rf control

February 25, 2012 · Tags: git, google-code, subversion
Continue reading (about 8 minutes)

Switching projects to Git

The purpose of this post is to tell you the story of the Version Control System (VCS) choices I have made while maintaining my open source projects ATF, Kyua and Lutok. It also details where my thoughts are headed to these days.

This is not a description of centralized vs. distributed VCSs, and it does not intend to be one. This does not intend to compare Monotone to Git either, although you'll probably feel like it while reading the text. Note that I have fully known the advantages of DVCSs over centralized systems for many years, but for some reason or another I have been "forced" to use centralized systems on and off. The Subversion hiccup explained below is... well... regrettable, but it's all part of the story!

Hope you enjoy the read.


Looking back at Monotone (and ATF)

I still remember the moment I discovered Monotone in 2004: simply put, it blew my mind. It was clear to me that Distributed Version Control Systems (DVCSs) were going to be the future, and I eagerly adopted Monotone for my own projects. A year later, Git appeared and it took all the praise for DVCSs: developers all around started migrating en masse to Git, leaving behind other (D)VCSs. Many of these developers then went on to make Git usable (it certainly wasn't at first) and well-documented. (Note: I really dislike Git's origins... but I won't get into details; it has been many years since that happened.)

One of the projects in which I chose to use Monotone was ATF. That might have been a good choice at the time despite being very biased, but it has caused problems over time. These have been:

• Difficulty to get Monotone installed: While most Linux distributions come with a Monotone binary package these days, it was not the case years ago. But even nowadays if all Linux distributions have binary packages, the main consumers of ATF are NetBSD users, and their only choice is to build their own binaries. This generates discomfort because there is a lot of FUD surrounding C++ and Boost.
• High entry barrier to potential contributors: It is a fact that Monotone is not popular, which means that nobody is familiar with it. Monotone's CLI is very similar to CVS, and I'd say the knowledge transition for basic usage is trivial, but the process of cloning a remote project was really convoluted until "recently". The lack of binary packages, combined with complex instructions on just how to fetch the sources of a project only help in scaring people away.
• Missing features: Despite years have passed, Monotone still lacks some important features that impact its usability. For example, to my knowledge, it's still not possible to do work-directory merges and, while the interactive merges offered by the tool seem like a cool idea, they are not really practical as you get no chance to validate the merge. It is also not possible, for example, to reference the parent commit of any given commit without looking at the parent's ID. (Yeah, yeah, in a DAG there may be more than one parent, but that's not the common case.) Or know what a push/pull operation is going to change on both sides of the connection. And key management and trust has been broken since day one and is still not fixed. Etc, etc, etc.
• No hosting: None of the major project hosting sites support Monotone. While there are some playground hosting sites, they are toys. I have also maintained my own servers sometimes, but it's certainly inconvenient and annoying.
• No tools support: Pretty much no major development tools support Monotone as a VCS backend. Consider Ohloh, your favorite bug tracking system or your editor/IDE. (I attempted to install Trac with some alpha plugin to add Monotone support and it was a huge mess.)
• No more active development: This is the drop that spills the cup. The developers of Monotone that created the foundations of the project left years ago. While the rest of the developers did a good job in coming up with a 1.0 release by March 2011, nothing else has happened since then. To me, it looks like a dead project at this point :-(

• ATF continues to be hosted in a Monotone repository, and Kyua depends on ATF. You can spot the problem, can't you? It's a nightmare to check out all the dependencies of Kyua, using different tools, just to get the thing working.
• As of today, Git is as popular, if not more, than Subversion. All the major operating systems have binary packages for Git and/or bundle Git in their base installation (hello, OS X!). Installing Git on NetBSD is arguably easier (at least faster!) than Subversion. Developers are used to Git. Or let me fix that: developers love Git.
• Subversion gets on the way more than it helps; it really does once you have experienced what other VCSs have to offer. I currently maintain independent checkouts of the repository (appropriately named 1, 2 and 3) so that I can develop different patches on each before committing the changes. This gets old really quickly. Not to mention when I have to fly for hours, as being stuck without an internet connection and plain-old Subversion... is suboptimal. Disconnected operation is key.

February 11, 2012 · Tags: atf, git, kyua, lutok, monotone, vcs
Continue reading (about 8 minutes)

Talk about Git

I've been using Git (or better said Cogito) recently as part of my PFC and, although I don't like the way Git was started, I must confess I like it a lot. In some ways it is very similar to Monotone (the version control system I prefer now) but it has its own features that make it very interesting. One of these is the difference between local and remote branches, something I'll talk about in a future post.

For now I would just like to point you to a talk about Git by Linus given at Google. He focuses more on general concepts of distributed version systems than on Git itself, so most of the ideas given there apply to many other systems as well. If you still don't see the advantages of distributed VCSs over centralized ones, you must watch this. Really. Oh, and it is quite "funny" too ;-)

May 19, 2007 · Tags: cogito, dvcs, git, monotone
Continue reading (about 1 minute)