Subtree merging strategy

Basic Concepts of Subtree Merge Strategy

Subtree merging is a special merge strategy in Git that allows embedding one repository as a subdirectory within another repository while preserving their respective commit histories. Unlike submodules, subtree merging does not require additional metadata files (.gitmodules), as all content is stored directly in the parent repository.

The core idea of subtree merging is to "graft" one repository's codebase into a specific subdirectory of another repository. This approach is particularly suitable for the following scenarios:

• Reusing code from another project without introducing complex submodule management
• Embedding dependency projects directly into the main project
• Modifying third-party libraries while retaining the modification history

Basic Operations of Subtree Merging

Adding a Subtree

To add an external repository as a subtree, use the following commands:

git remote add -f <subtree-name> <repository-url>
git subtree add --prefix=<prefix-path> <subtree-name> <branch> --squash

For example, adding the React library to the vendor/react directory of a project:

git remote add -f react https://github.com/facebook/react.git
git subtree add --prefix=vendor/react react main --squash

Updating a Subtree

When the upstream repository has updates, use the following commands to pull changes:

git fetch <subtree-name> <branch>
git subtree pull --prefix=<prefix-path> <subtree-name> <branch> --squash

For the React example:

git fetch react main
git subtree pull --prefix=vendor/react react main --squash

Advanced Usage of Subtree Merging

Splitting a Subtree

Sometimes, you may need to extract a subdirectory from the project as an independent subtree:

git subtree split --prefix=<prefix-path> -b <new-branch-name>

For example, extracting src/utils as a utils subtree:

git subtree split --prefix=src/utils -b utils-subtree

Pushing Changes Upstream

If you modify subtree content and want to contribute it back to the original project:

git subtree push --prefix=<prefix-path> <repository-url> <branch>

For the React example:

git subtree push --prefix=vendor/react https://github.com/facebook/react.git my-feature-branch

Pros and Cons of Subtree Merge Strategy

Pros

1. Simplified dependency management: All code is in the main repository, no additional initialization is needed.
2. Complete commit history: Option to retain the full history of the subtree project.
3. Easy modification: Directly modify subtree code within the project.
4. Simplified deployment: Cloning the main repository includes all dependencies.

Cons

1. Increased repository size: Especially when retaining full history.
2. Merge conflicts: May occur when the subtree and main project modify the same files.
3. Complex update process: Requires manually tracking upstream changes.

Practical Examples of Subtree Merging

Example 1: Embedding a UI Component Library in a Frontend Project

Assume a main project needs to embed an internal UI component library:

# Add UI component library
git remote add -f ui-components git@internal.com:ui/components.git
git subtree add --prefix=src/components ui-components main --squash

# Modify components during development
# Then push changes back to the component library
git subtree push --prefix=src/components git@internal.com:ui/components.git feature/new-button

Example 2: Subtree Management in a Monorepo

Managing multiple related projects in a Monorepo:

// package.json
{
  "scripts": {
    "update:shared": "git subtree pull --prefix=packages/shared git@internal.com:shared.git main --squash",
    "push:shared": "git subtree push --prefix=packages/shared git@internal.com:shared.git main"
  }
}

Comparison of Subtree Merging with Alternatives

Subtree Merging vs. Submodules

Subtree Merging vs. Package Managers

For JavaScript projects, you can also use package managers like npm/yarn:

# Using npm
npm install <package>

# Using subtree
git subtree add --prefix=node_modules/<package> <package-repo> main

The advantage of subtree merging is the ability to directly modify code and contribute back to the original project, whereas npm packages typically require version updates.

Best Practices for Subtree Merging

1. Clear directory structure: Create a unified directory for all subtrees, such as vendor/ or third-party/.
2. Documentation: Record all subtrees and their sources in the README.
3. Regular updates: Establish a subtree update mechanism to avoid difficulties with major version upgrades due to long-term neglect.
4. Selective squashing: Use --squash for frequently updated subtrees to reduce history.
5. Automation scripts: Create scripts to simplify subtree operations.

Example automation script:

#!/bin/bash
# update_subtrees.sh

# Define subtree list
declare -A subtrees=(
  ["vendor/react"]="https://github.com/facebook/react.git main"
  ["src/components"]="git@internal.com:ui/components.git develop"
)

# Update all subtrees
for path in "${!subtrees[@]}"; do
  IFS=' ' read -r repo branch <<< "${subtrees[$path]}"
  echo "Updating $path from $repo $branch"
  git subtree pull --prefix="$path" "$repo" "$branch" --squash
done

Troubleshooting Common Subtree Merging Issues

Issue 1: Merge Conflicts

Conflicts occur when the subtree and main project modify the same files. Solutions:

1. Clearly define responsibilities to avoid modifying the same files in the main project and subtree.
2. Prioritize retaining the subtree version during conflicts, then adapt the main project.
3. Use git rerere to record conflict resolutions.

Issue 2: Messy History

Prevention methods:

1. Use the --squash option when merging commits.
2. Regularly rebase subtree branches.
3. Add a uniform prefix to subtree commits.

Issue 3: Incorrect Subtree Removal

To remove a subtree:

# 1. First split out the subtree history
git subtree split --prefix=path/to/subtree -b subtree-branch

# 2. Then delete the subtree directory
git rm -r path/to/subtree
git commit -m "Remove subtree"

Example Workflow for Subtree Merging

Developing a New Feature Involving Subtree Modifications

1. Create a feature branch from the main branch.
2. Modify subtree code in the feature branch.
3. Test integration between the main project and subtree.
4. Push subtree modifications back to the upstream repository.
5. Update the subtree reference in the main project.

# 1. Create branch
git checkout -b feature/new-form

# 2. Modify components
# Edit vendor/react/src/components/Form.js

# 3. Commit changes
git add .
git commit -m "Improve form component"

# 4. Push back to React repository
git subtree push --prefix=vendor/react https://github.com/facebook/react.git feature/new-form

# 5. Create PR to React main repository
# Wait for merge...

# 6. Update React reference in main project
git fetch react main
git subtree pull --prefix=vendor/react react main --squash

Performance Optimization for Subtree Merging

For large subtrees, consider the following optimizations:

1. Shallow cloning: Use the --depth option when adding subtrees.
2. Partial cloning: Clone only the necessary subtree directories.
3. Regular cleanup: Use git gc to optimize the repository.
4. Selective history: Retain only essential commit history.

Example of shallow cloning when adding a subtree:

git remote add -f react --depth=1 https://github.com/facebook/react.git
git subtree add --prefix=vendor/react react main --squash