Index file parsing

Index File Parsing

Git's index file (.git/index) is the core data structure of the staging area, recording the current state and metadata of staged files. This binary file is updated every time the git add command is executed and contains a snapshot of files ready to be committed.

Index File Structure

The index file uses a fixed binary format, primarily consisting of the following parts:

1. Header Information: A 12-byte fixed header
2. Index Entries: Metadata for each tracked file
3. Extension Data (optional): Additional functional extensions
4. SHA-1 Checksum: A 20-byte file checksum

Example header structure (pseudocode representation):

struct IndexHeader {
  char[4] signature;  // "DIRC"
  uint32 version;     // Version number (2, 3, 4)
  uint32 entries;     // Number of index entries
}

Detailed Index Entries

Each index entry contains the following key information:

interface IndexEntry {
  ctime: [number, number];  // Creation time (seconds + nanoseconds)
  mtime: [number, number];  // Modification time (seconds + nanoseconds)
  dev: number;              // Device number
  ino: number;              // Inode number
  mode: number;             // File mode (type + permissions)
  uid: number;              // User ID
  gid: number;              // Group ID
  size: number;             // File size
  sha: string;              // 40-byte SHA-1 hash
  flags: number;            // Flags
  path: string;             // Relative path
}

Practical Parsing Example

Here is a code snippet for parsing the index file using Node.js:

const fs = require('fs');

function parseIndex(indexPath) {
  const buffer = fs.readFileSync(indexPath);
  let offset = 0;
  
  // Parse header
  const header = {
    signature: buffer.toString('utf8', offset, offset + 4),
    version: buffer.readUInt32BE(offset + 4),
    entries: buffer.readUInt32BE(offset + 8)
  };
  offset += 12;

  // Parse entries
  const entries = [];
  for (let i = 0; i < header.entries; i++) {
    const entry = {};
    entry.ctime = [buffer.readUInt32BE(offset), buffer.readUInt32BE(offset + 4)];
    entry.mtime = [buffer.readUInt32BE(offset + 8), buffer.readUInt32BE(offset + 12)];
    entry.dev = buffer.readUInt32BE(offset + 16);
    // ...Continue parsing other fields
    
    // Handle variable-length path names
    const pathStart = offset + 62;
    const nullPos = buffer.indexOf(0x00, pathStart);
    entry.path = buffer.toString('utf8', pathStart, nullPos);
    
    entries.push(entry);
    offset = Math.ceil((nullPos + 1) / 8) * 8; // 8-byte alignment
  }
  
  return { header, entries };
}

Version Differences

Git index files come in multiple version formats:

• Version 2: Basic format supporting regular files
• Version 3: Added the "assume valid" flag for deleted paths
• Version 4: Supports unmerged paths and sparse checkouts

Example of new extension data in Version 4:

IEOT: Index Entry Offset Table  
UNTR: Untracked file cache  

Advanced Use Cases

Index State During Conflict Resolution

When a merge conflict occurs, the index contains multiple stages:

$ git ls-files --stage
100644 78981922613b2afb6025042ff6bd878ac1994e85 1	file.txt
100644 2abd5c1c08ca5b8d6d4c7d31551e9a287241b0f2 2	file.txt
100644 cb1d2fd071c6ae9c08969b5a7c8e5f1e64d02f52 3	file.txt

Interaction Between Index and Worktree

Git determines modification states by comparing the index and worktree:

function getStatusChanges() {
  // Get index SHA1
  const indexSHA = getIndexSHA();
  // Get actual SHA1 of worktree files
  const worktreeSHA = calculateWorktreeSHA();
  
  return {
    modified: indexSHA !== worktreeSHA,
    newFiles: /* Files present in worktree but not in index */,
    deleted: /* Files present in index but not in worktree */
  };
}

Performance Optimization Tips

For large repositories, index performance can be optimized in the following ways:

1. Use FSMmonitor: Enable in .git/config:

   [core]
   fsmonitor = true
   

2. Split Index: Use splitIndex configuration:

   git config feature.splitIndex true
   

3. Preload Index: Speed up with preloadindex:

   git config core.preloadindex true
   

Debugging Index Issues

When index issues arise, use low-level commands to inspect:

# View raw index content
git ls-files --stage --debug

# Verify index consistency
git fsck --cache

# Dump index tree structure
git ls-tree -r --name-only HEAD

Index and Sparse Checkout

Sparse checkout dynamically updates the index:

# Set up sparse checkout mode
git config core.sparseCheckout true
echo "src/" > .git/info/sparse-checkout
git read-tree -mu HEAD

The corresponding index changes are reflected in the $GIT_DIR/info/sparse-checkout file.