Packaging and archiving

Basic Concepts of Packing and Archiving

In Git, packing and archiving primarily involve converting repository content into compressed formats or standalone file packages. The git archive command is the core tool, allowing the creation of archive files from specific commits or branches, supporting various formats such as zip and tar.gz. Unlike regular compression, Git archiving preserves file permissions and Git metadata, making it suitable for code distribution or deployment.

# Create a zip archive of the current branch
git archive --format=zip --output=project.zip HEAD

Creating Archives of Specific Versions

By specifying commit hashes, branches, or tags, you can precisely control the version of the content being archived. This is particularly useful in continuous integration environments, such as when delivering a specific version of code to a testing team:

# Archive the contents of the v1.0.0 tag as tar.gz
git archive --format=tar.gz v1.0.0 | gzip > release-v1.0.0.tar.gz

For projects containing submodules, additional handling is required. The following script demonstrates how to create a complete archive including submodules:

#!/bin/sh
git archive --prefix=main/ -o main.tar HEAD
git submodule foreach --recursive 'git archive --prefix=main/$path/ HEAD > sub.tar && tar --concatenate --file=main.tar sub.tar'
gzip main.tar

Incremental Packing Techniques

The git bundle command creates incremental packages, suitable for offline environments or transferring large repositories. It generates a binary file containing data for a specific range of commits:

# Create a bundle containing the last 5 commits
git bundle create updates.bundle HEAD~5..HEAD

The recipient can verify and apply the updates using the following commands:

git bundle verify updates.bundle
git fetch updates.bundle main:new-feature

Path Control in Archiving

The --prefix parameter allows adding custom directory hierarchies within the archive, which is helpful for deployment standardization:

# Add an app/ prefix to all files
git archive --prefix=app/ -o deploy.tar HEAD

Combined with path filtering, you can archive only specific directories:

# Only archive the contents of the src directory
git archive HEAD -- src/ | tar -x -C ./dist

Automation Deployment Integration

In CI/CD workflows, archiving is often combined with deployment scripts. Here’s an example deployment script for a Node.js project:

const { execSync } = require('child_process');
const fs = require('fs');

// Generate an archive file with a timestamp
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
const archiveName = `build-${timestamp}.zip`;

execSync(`git archive --format=zip --output=${archiveName} HEAD`);

// Add the built assets directory
execSync(`zip -ur ${archiveName} public/assets`);

// Example upload to AWS S3
const AWS = require('aws-sdk');
const s3 = new AWS.S3();
const fileContent = fs.readFileSync(archiveName);

const params = {
  Bucket: 'deployment-bucket',
  Key: `builds/${archiveName}`,
  Body: fileContent
};

s3.upload(params, (err, data) => {
  if (err) throw err;
  console.log(`Archive uploaded at ${data.Location}`);
});

Binary File Handling Strategies

For repositories containing binary files, regular archiving may result in excessively large packages. In such cases, you can combine git-lfs with partial checkouts:

# First archive code files
git archive --format=zip HEAD $(git ls-files | grep -v '\.psd$') > code.zip

# Handle large files separately
git lfs ls-files -n | xargs zip assets.zip

Archive Signature Verification

To ensure archive integrity, you can add a GPG signature:

git archive HEAD | gpg --detach-sign > archive.tar.sig

Verification is performed using:

gpg --verify archive.tar.sig archive.tar

Cross-Platform Packaging Differences

Windows and Unix systems handle paths differently, requiring special attention for cross-platform projects:

# Handling archives with Chinese paths in PowerShell
git archive --output=archive.zip HEAD | Out-Null
[System.IO.Path]::GetFullPath('archive.zip')

Batch Archiving of Historical Versions

The following script can create standalone archives for the last 10 commits in bulk:

for i in {0..9}; do
  commit=$(git rev-parse HEAD~$i)
  git archive --format=zip --output=patch-$i.zip $commit
done

Integration with CI Systems

Example GitLab CI configuration to automatically create an archive during merge requests:

build_artifact:
  stage: package
  script:
    - git archive --format=zip --output=artifacts.zip $CI_COMMIT_SHA
  artifacts:
    paths:
      - artifacts.zip

Archive Content Preprocessing

The export-ignore attribute in .gitattributes can control which files are included in the archive:

# Example .gitattributes
docs/* export-ignore
*.test.js export-ignore

Performance Optimization Tips

For large repositories, these methods can improve efficiency:

1. Use --worktree-attributes to skip processing the .git directory.
2. Operate on SSD storage.
3. Archive in stages:

# First create a temporary index
TEMP_INDEX=$(mktemp)
GIT_INDEX_FILE=$TEMP_INDEX git read-tree HEAD

# Create an archive from the temporary index
GIT_INDEX_FILE=$TEMP_INDEX git archive --format=zip HEAD > optimized.zip
rm $TEMP_INDEX

Archive Metadata Retention

By default, Git archiving does not preserve all file attributes. To retain UNIX permissions and symbolic links:

git archive --format=tar --prefix=project/ HEAD | (cd /target && tar --preserve-permissions -xf -)

Error Handling Patterns

Common issues during archiving and their solutions:

1. Filename Encoding Issues:

# Force UTF-8 encoding
git -c core.quotepath=false archive HEAD

2. Out-of-Memory Errors:

# Use streaming processing
git archive HEAD | pigz -9 > large-repo.tar.gz

3. Path Length Limitations (Windows):

# Enable long path support
git config --global core.longpaths true