Incorporating a remote Subversion repository into a local spin off

Posted on by

To render and display the OpenSocial apps available on our platform we are using the Apache Shindig gadget server, currently with the version 1.0.incubating
Up until now we had just exported this Shindig version and committed this into our local in-house subversion repository. From there on we added all kinds of patches, bug fixes and extensions to it. Over the month this accumulated to about 50 different changes to the original shindig codebase, which made it very hard to upgrade to a newer Shindig version or to apply any patches for the original version.

Now that the time is coming to migrate to the OpenSocial 0.9 spec with Shindig 2.0 we rethought our Shindig development strategy and came up with a different process:

At first we refactored the changes we did heavily to be as separated as possible from the original Shindig codebase.
What we couldn’t strip out, we extracted as diff patches into separated patch files.
This left us with the original codebase, some 20 patch files and some extra php classes and JavaScript packages.
Now this allowed us to just commit the patches and extra files into our own subversion repository. Then we added a small build script which will export the rest of the folders and files we need from the Apache Shindig Subversion server. This means that these files are not under version control anymore and subversion does not want to commit them to the Apache Shindig Subversion (for which we have no write access) if some files have changed. For this files and folders there is a svn:ignore property set in our subversion, so that these files are not committed accidentally.
After the export the build script applies all patches and we have a working shindig server in our file system.


#!/bin/bash
apache_svn_base="https://svn.apache.org/repos/asf/"
shindig_svn_base="shindig/trunk"


paths[0]="config/OSML_library.xml"
paths[1]="config/container.js"
paths[2]="config/oauth.json"
paths[3]="content"
paths[4]="extras/src/main/javascript/features-extras"
paths[5]="features"
paths[6]="php/config/container.php"
paths[7]="php/docs"
paths[8]="php/external"
paths[9]="php/src"
paths[10]="php/test"
paths[11]="php/.htaccess"
paths[12]="php/index.php"


revision[0]="965715"
revision[1]="965715"
revision[2]="965715"
revision[3]="965715"
revision[4]="965715"
revision[5]="965715"
revision[6]="965715"
revision[7]="965715"
revision[8]="965715"
revision[9]="965715"
revision[10]="965715"
revision[11]="965715"
revision[12]="965715"


ELEMENTS=${#paths[@]}


for (( i=0;i<$ELEMENTS;i++)); do
# remove local files
rm -rf ${paths[${i}]}
# export remote files to local copy
svn export -r ${revision[${i}]} $apache_svn_base$shindig_svn_base/${paths[${i}]} ${paths[${i}]}
done


# apply patches
patches="patches/*.patch"


for f in $patches
do
echo "********** load patch $f"
patch -p0 -N -i $f
done

In this solution it is necessary for us to be able to see what we changed in the original files while developing, so that we can add or modify new patch files. For this we wrote a small diff script which downloads the version of a file from the Apache Shindig SVN and diffs it to its local copy.


#!/bin/bash
apache_svn_base="https://svn.apache.org/repos/asf/"
shindig_svn_base="shindig/trunk"


paths[0]="config/OSML_library.xml"
paths[1]="config/container.js"
paths[2]="config/oauth.json"
paths[3]="content"
paths[4]="extras/src/main/javascript/features-extras"
paths[5]="features"
paths[6]="php/config/container.php"
paths[7]="php/docs"
paths[8]="php/external"
paths[9]="php/src"
paths[10]="php/test"
paths[11]="php/.htaccess"
paths[12]="php/index.php"


revision[0]="965715"
revision[1]="965715"
revision[2]="965715"
revision[3]="965715"
revision[4]="965715"
revision[5]="965715"
revision[6]="965715"
revision[7]="965715"
revision[8]="965715"
revision[9]="965715"
revision[10]="965715"
revision[11]="965715"
revision[12]="965715"


ELEMENTS=${#paths[@]}
line="==================================================================="


for (( i=0;i<$ELEMENTS;i++)); do
find ${paths[${i}]} -type f -exec sh -c '(curl -s $2!svn/bc/$3/$4/$1 | diff -w - $1 | sed -e "1i\\
Index: $1\\
$5")' {} {} $apache_svn_base ${revision[${i}]} $shindig_svn_base $line \;
done

For all folders we traverse through all files, fetch the file from the remote Shindig SVN with CURL, diff the fetched content to the local file, and print out a ready diff patch with leading Index entry and separator line like its generated by the svn diff command.

This entry was posted in Uncategorized by admin. Bookmark the permalink.

3 thoughts on “Incorporating a remote Subversion repository into a local spin off

  1. Or you could use git/hg/… to handle patch queues for you. No assembly required.

  2. I agree with Peritus. You could handle such a workflow much better with Mercurial. You can use the mentioned patch-queues, which is especially nice if you want to develop patches that you might send upstream.

    Or hgsvn to have a local copy of the shindig svn in mercurial. From that copy you can clone your internal master and develop your changes in there, you can always pull more changes from the SVN and merge them painlessly (most times).

    Of course git should do the trick too, but I only know mercurial ;-)

  3. Sorry, I meant hgsubversion. hgsvn is nice, but hgsubversion is just better.

Leave a Reply

Your email address will not be published. Required fields are marked *


*