Two way git mirror

This articles describes how to setup a two way mirror of git repositories. I hope this article will be helpful as it works around a number of caveats for two way git mirrors.

Setup

We create a clone from either of the upstream repositories. It is important that the repository is bare.

cd /srv/gitsync
git clone --bare git@github.com:[account]/[repository].git
mv [repository].git [repository]

Now we delete remote "origin" and configure a remote setting for each upstream repository instead:

cd [repository]
git remote remove origin
git remote add github  git@github.com:[account]/[repository].git
git remote add sourceforge  ssh://[account]@git.code.sf.net/p/[repository]/code

The mirroring should happen right after changes have been pushed to one of the repositories. Therefore a webhook is required to trigger the mirror script.

On Sourceforge a project admin needs to enable it at Admin -> Tools -> Repository -> Webhooks. On Github it is at Settings -> Webhooks & Services -> Add Webhook.

Mirror scripts

We use the following script to do the actually mirroring. It can be invoked with the name of a known repository as parameter or with the "--all" flag.

#!/bin/bash

function sync_repo {
    cd /srv/gitsync
    cd $1
    echo $1
    # fetch all known remotes
    git fetch --all -p
    # push branches from sourceforge to github and via versa.
    git push github "refs/remotes/sourceforge/*:refs/heads/*" "refs/tags/*:refs/tags/*"
    git push sourceforge "refs/remotes/github/*:refs/heads/*" "refs/tags/*:refs/tags/*"
}

cd /srv/gitsync
if [ "$1" == "" ]; then
    # no command line parameters, print help message
    echo "gitsync [report]|--all"
elif [ "$1" == "--all" ]; then
    # "--all": for all known repositories
    for D in *; do
       if [ -d "${D}" ]; then
             sync_repo $D
             cd /srv/gitsync
       fi
    done
elif [ -d "$1" ]; then
    # sync only the specified repository
    sync_repo $1
else echo "gitsync [report]|--all"
fi

In function sync_repo the branches from remotes named github and sourceforge are pushed to each other. You can more mappings here.

As discussed in the previous section, we use webhooks to initiate a mirroring run on every push. This is achieved by the following php script:

if ($_SERVER['REQUEST_METHOD'] !== 'POST') {
	die('POST required');
}

if (!isset($_REQUEST['repository'])) {
	die('repository not specified');
}

// validate repository name to prevent injection and traversing attacks
$repo = $_REQUEST['repository'];
if (!preg_match('/^[a-zA-Z0-9]+$/', $repo)) {
	die('invalid repository name');
}

header('HTTP/1.0 204 Found');
system('sudo -Hu gitsync /usr/local/bin/gitsync '.$repo);

Security

• It is important to validate the repository name in order to prevent shell command injection, git parameter injection and directory traversing
• We use sudo to execute gitsync as user gitsync. Thus the webserver process does not require write access to the repositories
• Both Sourceforge and Github support a secret to authorize webhook invocations. In the above example this is not verified, so anyone can trigger a sync.

Deleting branches

There is one caveat: Deletion of branches is not mirrored, but deleted branches are resurrected by the mirror script.

To delete a branch for good, the following commands need to be executed in quick succession.

git branch -d [branchName]
git push github --delete [branchName]
git push sourceforge --delete [branchName]