Using git subtree to manage shared custom code

As owners of multiple apps or sites, we often have custom code libraries that we want to share with one of more of those sites, without duplicating the code. This also makes it easier to change the code once and have it deployed to all sites that use that code. Examples of shared code like this include site footers, API backends and standard contact forms.

This post shows you how to manage the sharing of custom code using git subtree.

For the examples below, assume:

• We have 2 sites, Site A and Site B
• Both sites share one custom contact-form module called LibContact
• We store all shared code for each site under lib/shared
• Site A’s repo is git@github.com:farez/SiteA.git
• Site B’s repo is git@github.com:farez/SiteB.git
• LibContact’s repo is git@github.com:farez/LibContact.git
• You work on git feature branches before committing them into your master branch

Creating a new shared module from an existing custom module

Goal: You have an existing custom module, LibContact, in one of your sites (Site B), and now you want to put it into its own repo, and share the module with other sites.

First, create a remote repo to hold the shared LibContact module, e.g. git@github.com:farez/LibContact.git.

Create a local directory to hold the repo just for the LibContact module. In this example, we want to put the module into a local directory, ~/shared/lib_contact. Create this directory outside of your Site B root directory.

> mkdir -p ~/shared/lib_contact

Go into the directory you just created.

> cd ~/shared/lib_contact

Create a new git bare repo in that new directory.

> git init --bare

Go into the top level of your site’s (Site B’s) git working tree (where the .git directory is). This would be the site that contains the custom module you want to share.

> cd /path/to/site_a

Assume you want to share the copy of LibContact that’s in Site B’s master branch. So checkout the master branch.

> git checkout master

Now put a copy of just the LibContact module into a new branch, called lib_contact_split. This branch will contain only the LibContact module. We assume here that the custom module LibContact is sitting in /path/to/site_a/lib/custom/lib_contact.

> git subtree split --prefix=lib/custom/lib_contact -b lib_contact_split

Push the contents of the lib_contact_split branch into the local bare repo you created above.

> git push ~/shared/lib_contact lib_contact_split:master

Go into the local shared repo for LibContact.

> cd ~/shared/lib_contact

Create a remote called origin, and point it to the repo for LibContact git@github.com:farez/LibContact.git

> git remote add origin git@github.com:farez/LibContact.git

Push the LibContact module up to the remote repo, into its master branch.

> git push origin master

Now remove the custom module’s folder from the site’s repo, commit the removal, and re-add it using instructions from the “Using the shared LibContact module in your site for the first time” example below.

> cd /path/to/site_b/lib/custom 
> git rm -r lib_contact 
> git commit -m “Removed LibContact module”

… and continue from the first step in the “Using the shared LibContact module in your site for the first time” example below…

Using the shared LibContact module in your site for the first time

Goal: You have a new feature to build on Site A and want to use the shared LibContact module in the project.

Make sure you have committed any local changes first before continuing. Then check out your feature branch to work on.

> git checkout feature_branch

Go into the top level of your site’s git working tree (where the .git directory is).

> cd /path/to/site_a

If the lib/shared directory does not yet exist, create it.

> mkdir -p lib/shared

Add a git remote called lib_contact_repo, pointing to the shared module LibContact’s repo.

> git remote add lib_contact_repo git@github.com:farez/LibContact.git

Now add the LibContact module to your project as a git subtree. You are going to pull it from the LibContact module’s master branch, and place the module at lib/shared/lib_contact.

> git subtree add --prefix=lib/shared/lib_contact — squash lib_contact_repo master

If you do a git status at this point, you will not see any new or untracked files. This is to be expected, because the LibContact module is already being tracked as a subtree to its own repo.

> git status
On branch feature_branch
Your branch is ahead of ‘origin/feature_branch’ by 2 commits.
(use “git push” to publish your local commits)
nothing to commit, working directory clean

Site A now has the LibContact module for you to use.

Updating your local copy of the shared LibContact module

Goal: You want to pull from the remote copy of LibContact and update your local copy.

Check out the branch where you want to update the LibContact module.

> git checkout feature_branch

Go into the top level of your site’s git working tree (where the .git directory is).

> cd /path/to/site_a

If you haven’t done so, add a git remote called lib_contact_repo, pointing to the shared module LibContact’s repo.

> git remote add lib_contact_repo git@github.com:farez/LibContact.git

Pull from the LibContact module’s repo into your local LibContact module subtree.

> git subtree pull --prefix=lib/shared/lib_contact — squash lib_contact_repo master
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), done.
From github.com:farez/LibContact
* branch            master -> FETCH_HEAD
  6323c96..a19e684  master -> lib_contact_repo/master
Merge made by the ‘recursive’ strategy.
lib/shared/lib_contact/README.md | 1 +
1 file changed, 1 insertion(+)
create mode 100644 lib/shared/lib_contact/README.md

Make a change to the shared LibContact module and push upstream to share it

Goal: You want to edit the LibContact module and push it back up to its shared repo, so that other sites can pull the edited version down and test it. You want to push into a new branch, not the master branch.

Go ahead and make your changes in the module.

> git checkout feature_branch
> cd /path/to/site_a/lib/shared/lib_contact
… make your changes to the LibContact module …
> git add *
> git commit -m “Changes made”

Go into the top level of your site’s git working tree (where the .git directory is).

> cd /path/to/site_a

Push the modified LibContact code up to the shared repo, into a branch called new_updates. This branch can be a new or existing branch.

> git subtree push --prefix=lib/shared/lib_contact — squash lib_contact_repo new_updates

Merge changes in LibContact into master branch of LibContact repo

Goal: You have fnished testing the new changes you made in the LibContact module, and now want to merge those changes into the LibContact module’s master branch.

Go into the top level of your site’s git working tree (where the .git directory is).

> cd /path/to/site_a

Check out your site’s master branch, and merge in the branch that contains the amended LibContact code. This branch may also contain other changes to your site code, which are not related to LibContact.

> git checkout master
> git merge feature_branch

Push the changes specific to the LibContact module, into its repo’s master branch.

> git subtree push --prefix=lib/shared/lib_contact — squash lib_contact_repo master

References

The best external reference I’ve found so far on git subtree is https://makingsoftware.wordpress.com/2013/02/16/using-git-subtrees-for-repository-separation/