Submodule management (git submodule)

Submodule Management (git submodule)

Git submodules allow one Git repository to be included as a subdirectory within another Git repository. They enable you to clone other repositories into your project while maintaining independent commits.

Why Submodules Are Needed

In large projects, it is often necessary to reference code from other projects. For example:

• Using third-party libraries
• Sharing common components across multiple projects
• Separating core code from plugin systems

Copying code directly makes it difficult to synchronize updates, whereas submodules provide a better solution.

Adding a Submodule

The basic command to add a submodule is:

git submodule add <repository> <path>

For example, to add a third-party library named lib to your project:

git submodule add https://github.com/example/lib.git libs/external

This does three things in the project:

1. Clones the repository at the specified path
2. Creates a .gitmodules file to record submodule information
3. Tracks the submodule's commit in the parent repository

The .gitmodules file will look something like this:

[submodule "libs/external"]
    path = libs/external
    url = https://github.com/example/lib.git

Cloning a Project with Submodules

When cloning a project containing submodules, the submodule directories will be empty. Execute the following:

git submodule init
git submodule update

Or use the combined command:

git clone --recurse-submodules <repository>

Updating Submodules

Submodules do not update automatically by default. To fetch the latest commits for a submodule:

git submodule update --remote

This pulls the latest changes from the submodule's remote repository but does not automatically merge them. To switch to a specific branch in the submodule:

git config -f .gitmodules submodule.libs/external.branch main

Modifying and Committing in Submodules

To make changes in a submodule:

1. Enter the submodule directory
2. Create or switch to a branch
3. Make changes and commit
4. Push changes to the submodule's remote repository
5. Return to the parent project and commit the new submodule reference

cd libs/external
git checkout -b feature-branch
# Modify files
git commit -am "Add new feature"
git push origin feature-branch
cd ../..
git add libs/external
git commit -m "Update submodule reference"

Removing a Submodule

Removing a submodule requires multiple steps:

1. Delete the corresponding entry in .gitmodules
2. Remove the submodule configuration from .git/config
3. Execute git rm --cached <path>
4. Delete the submodule directory

git submodule deinit libs/external
git rm libs/external
rm -rf .git/modules/libs/external
git commit -m "Remove external submodule"

Common Submodule Issues

Outdated Submodule Pointers

When other collaborators update the submodule reference, synchronize locally with:

git pull
git submodule update

Recursive Submodules

Projects may contain nested submodules. Use the --recursive option:

git submodule update --init --recursive

Submodules and Branches

By default, submodules are in a "detached HEAD" state. To track a branch:

cd submodule_dir
git checkout main
cd ..
git config submodule.submodule_dir.branch main

Advanced Submodule Usage

Partial Submodule Checkout

You can check out only a specific branch or tag of a submodule:

git submodule add -b release https://github.com/example/lib.git

Submodule foreach Command

Execute the same command for all submodules:

git submodule foreach 'git checkout main'

Submodule Status Check

Check the modification status of submodules:

git submodule status

Submodule Alternatives

In some cases, consider other solutions:

• Git subtree: Merge subproject code into the main project
• Package managers (npm, yarn, etc.): Better suited for frontend dependency management
• Monorepo: Use tools like Lerna to manage multiple projects

Real-World Submodule Example

Assume a web project with the following structure:

my-project/
├── .gitmodules
├── package.json
├── src/
│   └── index.js
└── libs/
    ├── ui-components/  # Submodule
    └── utils/          # Submodule

.gitmodules file content:

[submodule "libs/ui-components"]
    path = libs/ui-components
    url = git@github.com:myteam/ui-components.git
    branch = main

[submodule "libs/utils"]
    path = libs/utils
    url = git@github.com:myteam/utils.git
    branch = dev

Referencing submodules in frontend code:

// src/index.js
import { Button } from '../libs/ui-components';
import { formatDate } from '../libs/utils';

console.log(Button, formatDate);

Submodule Workflow Best Practices

1. Assign dedicated teams to maintain submodules
2. Submodules should have clear version release strategies
3. The main project should lock submodules to specific commits
4. Regularly update submodules and test compatibility
5. Clearly document submodule usage in the project

Automating Submodule Management

Include submodule update checks in CI/CD pipelines:

# .gitlab-ci.yml
stages:
  - submodules

update_submodules:
  stage: submodules
  script:
    - git submodule sync --recursive
    - git submodule update --init --recursive
  only:
    - schedules

Or add scripts to package.json:

{
  "scripts": {
    "update:submodules": "git submodule update --remote --merge",
    "init:submodules": "git submodule update --init --recursive"
  }
}

Submodules and Git Hooks

Use Git hooks to automate submodule management. For example, add to .git/hooks/post-checkout:

#!/bin/sh
git submodule update --init --recursive

Or check submodule status in a pre-commit hook:

#!/bin/sh
if ! git submodule status --recursive | grep -q '^[+]'; then
  echo "ERROR: Submodules have uncommitted changes"
  exit 1
fi

Submodule Permission Management

When submodules require different permissions:

1. Use SSH keys to manage access to different repositories
2. Configure multiple sets of credentials in CI systems
3. For public submodules, use the HTTPS protocol

# .gitmodules using SSH protocol
[submodule "private-lib"]
    url = git@github.com:myorg/private-lib.git

# Public library using HTTPS
[submodule "public-lib"]
    url = https://github.com/example/public-lib.git