Working with Git Submodules

Why Using Submodules?

Modular software: (really) hard questions

Large (and even smaller) projects are often composed of smaller, self-contained, independent software modules. Managing versions of highly modular software raises lots of difficult questions, like:

1. If we've just found a bug in one specific version of one module, what versions of what products are affected?
2. If we've just found a bug in one specific product version, what version of what modules are built into this? And what other products are using the same modules, and should be re-tested now?
3. If we release a new version of a module and we have a wide product portfolio, what products have to rebuilt?

Things even more difficult when you use modules in multiple levels. For instance, this is an example from one of Intland Software's customers:

• there are a huge number of products built from same code base,
• products may contain multiple components,
• components may contain multiple modules,
• modules may depend on external libraries.

How to manage modular source code effectively?

Modelling this network of software packages and dependencies is immensely hard with tradiotional Version Control systems.

Most of the time you have to resort to statically copying source code from the module repository to all the dependant ones (push). Then, you need maintain the lifecycle manually, upgrade to new versions of modules manually, and so on.

Git, however, introduces a feature called Git submodules as the right tool to handle these situations effectively. CodeBeamer provides additional functionality on the top of plain vanilla submodules to support building modular software, like a convenient web interface, a unified look of all CodeBeamer-managed repositories and Configuration Management Database to support flexible versioning of separate modules.

What is a Git submodule?

Submodules are regular Git repositories referenced from your "main" repository (supermodule), that have their own release cycles.

Typically you are using multiple branches during development and you are marking your releases with tags. This is true both for your project (supermodule) and for the submodules used by your project. Different versions of your project probably depend on different versions of the submodules. The more submodules are used in a project the more complicated to understand the dependencies between specific versions of submodules and the project.

Especially if a bug was introduced in a specific version of a submodule, it is very important to know which versions of your project are impacted. Similarly, a single bug may impact multiple projects and multiple project versions.

CodeBeamer efficiently visualizes project (supermodule) and submodule dependencies.

Working with supermodules and submodules

First we need to create:

1. a new codeBeamer project called my project,
2. two initially empty managed Git repositories in this project, called main and xmllibrary, respectively.

Adding submodules to your project

First we create a working directory:

$ mkdir $HOME/work
$ cd $HOME/work

We first initialize the xmllibrary repository (the module used by the main project) by importing a new file to it:

$ git clone http://localhost:8080/cb/git/xmllibrary
$ cd xmllibrary
$ echo Hello > xml-readme.txt
$ git add xml-readme.txt
$ git commit -m "New readme file of xmllibrary"
$ git push origin master

We initialize the main repository similarly:

$ cd $HOME/work
$ git clone http://localhost:8080/cb/git/main
$ cd main
$ echo World > main-readme.txt
$ git add main-readme.txt

This is where it starts getting more interesting. Now we add the xmllibrary repository to main as submodule, using the master branch of the submodule.

$ git submodule add -b master http://localhost:8080/cb/git/xmllibrary

Finally we commit and push all changes:

$ git commit -m "Added a new file and a submodule"
$ git push origin master

The screenshot below shows that now the master branch of main repository uses (depends on) the master branch of the xmllibrary repository.

Now if you click on link xmllibrary the new page will show now that the master branch of xmllibrary repository is used by master branch of supermodule main.

Adding tags to supermodules and submodules

Now we create tags x_v1 in the xmllibrary repository and m_v1 in main, and push them:

$ cd $HOME/work/main/xmllibrary
$ git tag x_v1
$ git push --tags

$ cd $HOME/work/main
$ git tag m_v1
$ git push --tags

The screenshot below shows that both the main/m_v1 tag and the main/master branch depend on the xmllibrary/x_v1 tag and the xmllibrary/master branch.

Working on the supermodule

Now we make a change in the file main-readme.txt of main, and tag the new state with m_v2:

$ cd $HOME/work/main
$ echo World2 > main-readme.txt
$ git commit -a -m "A change in main"
$ git tag m_v2
$ git push --tags

The screenshot below shows that the main/m_v2 tag still depends on the xmllibrary/x_v1 tag, as we made changes only in main and not in xmllibrary.

Updating submodules

Let's assume that a new version of the xml library has been released in the meanwhile. We want to update our project to use this latest version, without loosing the previous history.

First we go back to the xmllibrary repository cloned at the beginning of the examples, and pull the new version into the clone (should be up-to-date):

$ cd $HOME/work/xmllibrary
$ git pull

Please note that the xmllibrary repository was cloned outside of main, and can be used totally independently of main. This is how normally third party libraries are developed.

Now we add a new file to the update xmllibrary repository and create the tag x_v2 to reflect that it now contains the new version of the library:

$ echo Simple > INSTALL.txt
$ git add INSTALL.txt
$ git commit -m "New installation document"
$ git tag x_v2
$ git push --tags

The screenshot below shows that no supermodule depends on x_v2 tag of xmllibrary.

Analyzing the impact of changes in modules

While working with modules, the most frequrent question you are going to ask yourself is: I have this change here (and I know that it introduced a new bug), but what does this impact in this repository, and in its superrepositories? More accurately, what tags and branches contain this change in the current repository and its superrepositories (supermodules)?

First, navigate to the Changeset Details screen by clicking on the link with the SHA code of the change either in the Changes tab of the repository, or in the Changes tab of an affected issue.

Which tags and branches of the enclosing repository and superrepositories are impacted by a change?

It is important understanding on which tags and branches a changeset appears in the repository and which superrepositories (supermodules) and on which tags and branches are impacted.

This is what the next screen shows: