The Domain Sharding strategy

What is Domain Sharding

Domain sharding is a technique to improve webpage loading performance by distributing resources across multiple subdomains. Browsers impose a limit on the number of concurrent requests to the same domain. By spreading static resources across different domains, this limit can be bypassed, enabling parallel downloads of more resources.

Browser Concurrent Connection Limits

Modern browsers enforce limits on concurrent connections per domain:

• Chrome/Firefox: 6
• IE11: 13
• Edge: 6
• Safari: 6

This means if all resources come from a single domain, the browser can download at most 6 resources simultaneously (using Chrome as an example). When a page contains numerous static resources, this limitation creates significant performance bottlenecks.

How Domain Sharding Works

Domain sharding circumvents browser concurrency limits by creating multiple subdomains. For example:

• static1.example.com
• static2.example.com
• static3.example.com

Browsers treat these subdomains as distinct "origins," allowing more concurrent connections. Theoretically, using 3 subdomains could increase concurrent connections to 18 (in Chrome).

Methods to Implement Domain Sharding

1. DNS Configuration

First, configure multiple subdomains in DNS, all pointing to the same server IP:

static1 IN A 192.0.2.1
static2 IN A 192.0.2.1
static3 IN A 192.0.2.1

2. Server Configuration

Ensure the server can handle requests from these subdomains. Example Nginx configuration:

server {
    listen 80;
    server_name static1.example.com static2.example.com static3.example.com;
    root /var/www/static;
    # Other configurations...
}

3. Frontend Resource References

Distribute resource references across subdomains in HTML:

<!-- Primary domain -->
<script src="https://www.example.com/main.js"></script>

<!-- Sharded subdomains -->
<link rel="stylesheet" href="https://static1.example.com/styles.css">
<script src="https://static2.example.com/library.js"></script>
<img src="https://static3.example.com/logo.png" alt="Logo">

Automated Sharding Strategies

Manual management of sharded domains is inefficient. Use build tools for automation:

Webpack Configuration Example

// webpack.config.js
module.exports = {
  output: {
    filename: '[name].[contenthash].js',
    publicPath: (resourcePath, context) => {
      // Determine subdomain based on file type or path
      const shard = hashPath(resourcePath) % 3 + 1;
      return `https://static${shard}.example.com/`;
    }
  }
};

function hashPath(path) {
  // Simple hash function example
  let hash = 0;
  for (let i = 0; i < path.length; i++) {
    hash = ((hash << 5) - hash) + path.charCodeAt(i);
    hash |= 0; // Convert to 32-bit integer
  }
  return Math.abs(hash);
}

Best Practices for Domain Sharding

1. Optimal Number of Subdomains: Typically 2-4 subdomains suffice; too many increase DNS lookup overhead.
2. Session Consistency: Ensure session cookies are shared across all subdomains for the same user.
3. Resource Categorization Strategies:
   • By file type: static1 for CSS, static2 for JS, static3 for images.
   • By functionality: static1 for core resources, static2 for third-party libraries, static3 for non-critical resources.
4. Preconnect Optimization: Use <link rel="preconnect"> to establish early connections with subdomains.

<link rel="preconnect" href="https://static1.example.com">
<link rel="preconnect" href="https://static2.example.com">
<link rel="preconnect" href="https://static3.example.com">

Impact of HTTP/2

HTTP/2's multiplexing reduces the need for domain sharding:

• Single connections can handle multiple parallel requests.
• Fewer TCP connection setups are required.
• Header compression minimizes redundant header transmissions.

However, sharding may still be necessary in these cases:

1. Supporting legacy HTTP/1.1 clients.
2. Extremely complex pages requiring high concurrency.
3. Isolating critical vs. non-critical resources.

Trade-offs of Domain Sharding

Advantages

• Bypasses browser concurrency limits.
• Potentially reduces page load times.
• Enables isolation of critical resources.

Disadvantages

• Increases DNS lookup times.
• Requires more TCP connections (under HTTP/1.1).
• May undermine HTTP/2 multiplexing benefits.
• Adds configuration and maintenance complexity.

Modern Alternatives

With technological advancements, consider these alternatives or supplements:

1. Resource Bundling: Combine multiple small files into single files.
2. CDN Utilization: Distribute requests via CDN edge nodes.
3. HTTP/2 Server Push: Proactively push critical resources.
4. Preloading: Use <link rel="preload"> to fetch important resources early.

<link rel="preload" href="critical.css" as="style">
<link rel="preload" href="main.js" as="script">

Monitoring and Optimization

After implementing domain sharding, validate effectiveness through performance monitoring:

// Measure resource load times using Performance API
const resources = performance.getEntriesByType('resource');
resources.forEach(resource => {
  console.log(`${resource.name} load time: ${resource.duration}ms`);
});

// Calculate average load times per subdomain
const shardTimings = {};
resources.forEach(({name, duration}) => {
  const shard = name.match(/static(\d)/)?.[1] || 'main';
  shardTimings[shard] = (shardTimings[shard] || 0) + duration;
});

Case Study

An e-commerce homepage contains:

• 15 JavaScript files
• 8 CSS files
• 25 images

Without domain sharding, Chrome's waterfall chart showed severe resource queuing, with full page load taking 4.2 seconds. After implementing 3 subdomains:

• Core JS/CSS on static1
• Non-critical JS on static2
• Images on static3

Load time dropped to 2.8 seconds (33% improvement). However, increasing to 5 subdomains degraded performance to 3.1 seconds due to added DNS lookups and TCP connection overhead outweighing concurrency benefits.

Domain Sharding and Caching Strategies

Sharded domains should maintain consistent caching policies to avoid redundant downloads:

# Nginx configuration example
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
    expires 1y;
    add_header Cache-Control "public, immutable";
    # Apply same configuration to other subdomains
}

Ensure versioned filenames for long-term caching:

<script src="https://static1.example.com/app.3a7b9c.js"></script>
<link href="https://static2.example.com/styles.5d8e2f.css" rel="stylesheet">