Data synchronization and conflict resolution

Data synchronization and conflict resolution are core issues that cannot be avoided when building modern web applications. With the widespread adoption of HTML5 technology, front-end applications have become increasingly complex. Scenarios where multiple devices and users simultaneously operate on the same data source have become common. Mastering the skills to efficiently synchronize data and properly handle conflicts has become essential for developers.

Basic Principles of Data Synchronization

The essence of data synchronization is maintaining consistency in data states across different clients and the server. HTML5 provides multiple mechanisms to achieve this goal:

1. Polling: Periodically requesting updates from the server

function pollUpdates() {
  setInterval(async () => {
    const response = await fetch('/api/updates');
    const data = await response.json();
    // Process updates
  }, 5000); // Poll every 5 seconds
}

2. Long Polling: The server holds the connection until there is an update

async function longPoll() {
  const response = await fetch('/api/long-poll');
  const data = await response.json();
  // Process updates
  longPoll(); // Immediately initiate the next request
}

3. WebSocket: Full-duplex communication channel

const socket = new WebSocket('wss://example.com/sync');

socket.onmessage = (event) => {
  const update = JSON.parse(event.data);
  // Process real-time data updates
};

Types and Detection of Conflicts

Data conflicts mainly fall into three categories:

Write-Write Conflicts

Occur when two clients modify the same data item simultaneously. For example:

// Client A modification
{"id": 1, "title": "New Title A", "version": 2}

// Client B modification at nearly the same time
{"id": 1, "title": "New Title B", "version": 2}

Order Conflicts

Different operation sequences lead to inconsistent final states. For example:

1. Client A increases inventory → Client B reads inventory
2. Client B decreases inventory → Client A reads inventory

Logical Conflicts

Business rule conflicts, such as allowing a bank transaction when the balance is insufficient.

Common methods for conflict detection:

function hasConflict(localData, serverData) {
  return localData.version !== serverData.version ||
         new Date(localData.updatedAt) < new Date(serverData.updatedAt);
}

Conflict Resolution Strategies

Last Write Wins (LWW)

The simplest strategy but may result in data loss:

function resolveConflict(local, remote) {
  return local.updatedAt > remote.updatedAt ? local : remote;
}

Operational Transformation (OT)

An algorithm suitable for text collaboration:

function transform(op1, op2) {
  // Example: Text operation transformation
  if (op1.type === 'insert' && op2.type === 'insert') {
    if (op1.position < op2.position) {
      return op2;
    } else {
      return {...op2, position: op2.position + op1.text.length};
    }
  }
  // Other transformation rules...
}

Merge Strategy

Manually merge conflicting fields:

function mergeUserProfiles(local, remote) {
  return {
    ...remote,
    avatar: local.avatarUpdated > remote.avatarUpdated 
            ? local.avatar 
            : remote.avatar,
    bio: local.bio || remote.bio
  };
}

Offline-First Architecture

HTML5's Service Worker and IndexedDB support offline operations:

// Cache strategy in Service Worker
self.addEventListener('fetch', (event) => {
  event.respondWith(
    caches.match(event.request)
      .then(response => response || fetch(event.request))
  );
});

// Offline storage with IndexedDB
const dbPromise = idb.open('sync-db', 1, upgradeDB => {
  upgradeDB.createObjectStore('todos', {keyPath: 'id'});
});

async function syncLocalChanges() {
  const db = await dbPromise;
  const tx = db.transaction('todos', 'readwrite');
  const store = tx.objectStore('todos');
  // Get all local modifications
  const changes = await store.getAll();
  // Sync to the server
  await fetch('/api/sync', {
    method: 'POST',
    body: JSON.stringify(changes)
  });
}

Version Control and Vector Clocks

Distributed systems commonly use vector clocks to track causality:

class VectorClock {
  constructor(clientId) {
    this.timestamps = { [clientId]: 1 };
  }

  increment(clientId) {
    this.timestamps[clientId] = (this.timestamps[clientId] || 0) + 1;
  }

  compare(other) {
    // Compare the relationship between two vector clocks
    let less = false, greater = false;
    for (const clientId in this.timestamps) {
      if ((other.timestamps[clientId] || 0) < this.timestamps[clientId]) {
        greater = true;
      } else if ((other.timestamps[clientId] || 0) > this.timestamps[clientId]) {
        less = true;
      }
    }
    return { less, greater };
  }
}

Practical Application Example

Shopping cart synchronization scenario:

class ShoppingCartSynchronizer {
  constructor(userId) {
    this.userId = userId;
    this.pendingOperations = [];
  }

  async addItem(item) {
    const op = {
      type: 'add',
      item,
      timestamp: Date.now(),
      clientId: this.userId
    };
    this.pendingOperations.push(op);
    await this.trySync();
  }

  async trySync() {
    if (navigator.onLine) {
      const opsToSend = [...this.pendingOperations];
      this.pendingOperations = [];
      
      try {
        await fetch('/cart/sync', {
          method: 'POST',
          body: JSON.stringify(opsToSend)
        });
      } catch (error) {
        // Re-add to queue if failed
        this.pendingOperations.unshift(...opsToSend);
      }
    }
  }
}

Performance Optimization Techniques

1. Delta Synchronization: Only send changed parts

function generatePatch(oldDoc, newDoc) {
  const diff = {};
  for (const key in newDoc) {
    if (!deepEqual(oldDoc[key], newDoc[key])) {
      diff[key] = newDoc[key];
    }
  }
  return Object.keys(diff).length ? diff : null;
}

2. Batching: Combine multiple updates

let batchTimer;
const BATCH_DELAY = 200;

function queueUpdate(update) {
  clearTimeout(batchTimer);
  pendingUpdates.push(update);
  batchTimer = setTimeout(() => {
    sendBatch(pendingUpdates);
    pendingUpdates = [];
  }, BATCH_DELAY);
}

3. Compression: Use binary protocols

const encoder = new TextEncoder();
const data = encoder.encode(JSON.stringify(payload));
const compressed = await window.pako.deflate(data);

Testing and Debugging

Methods for simulating network issues:

// Testing conflict resolution with Cypress
describe('Conflict Resolution Test', () => {
  it('Should correctly merge parallel edits', () => {
    cy.intercept('POST', '/api/save', (req) => {
      req.reply({ delay: 1000, body: { conflict: true } });
    }).as('delayedSave');

    // Simulate parallel edits in two tabs
    cy.window().then((win) => {
      win.open('/', '_blank');
    });
    
    // Verify merge result
    cy.get('.content').should('contain', 'Merged text');
  });
});

Useful features in Chrome DevTools:

• Application → Service Workers to view offline caches
• Network → Throttling to simulate slow networks
• Application → IndexedDB to inspect local data

Security Considerations

1. Validate Synchronization Requests:

async function validateSyncOperation(operation) {
  if (operation.type === 'delete' && !operation.userId) {
    throw new Error('Missing user identity');
  }
  // Check data scope permissions
  if (!userHasAccess(operation.resourceId, currentUser)) {
    return false;
  }
  return true;
}

2. Encrypt Sensitive Data:

async function encryptBeforeSync(data) {
  const key = await crypto.subtle.importKey(
    'raw',
    new TextEncoder().encode(ENCRYPTION_KEY),
    { name: 'AES-GCM' },
    false,
    ['encrypt']
  );
  
  const iv = crypto.getRandomValues(new Uint8Array(12));
  const encrypted = await crypto.subtle.encrypt(
    { name: 'AES-GCM', iv },
    key,
    new TextEncoder().encode(JSON.stringify(data))
  );
  
  return { iv: Array.from(iv), data: Array.from(new Uint8Array(encrypted)) };
}

Framework Integration Examples

Reactive synchronization with Vue:

// As a Vue plugin
const SyncPlugin = {
  install(app, options) {
    const store = reactive({
      data: {},
      pending: false,
      error: null
    });

    async function sync() {
      store.pending = true;
      try {
        const response = await fetch(options.url);
        store.data = await response.json();
      } catch (err) {
        store.error = err;
      } finally {
        store.pending = false;
      }
    }

    // Auto-sync
    setInterval(sync, options.interval || 30000);
    app.provide('sync', { store, sync });
  }
};

// Usage
app.use(SyncPlugin, { url: '/api/data' });

React Hook implementation:

function useSyncState(initialState, syncUrl) {
  const [state, setState] = useState(initialState);
  const [version, setVersion] = useState(0);

  useEffffect(() => {
    const ws = new WebSocket(syncUrl);
    ws.onmessage = (e) => {
      const { data, v } = JSON.parse(e.data);
      if (v > version) {
        setState(data);
        setVersion(v);
      }
    };
    return () => ws.close();
  }, [version]);

  const updateState = async (newState) => {
    const response = await fetch(syncUrl, {
      method: 'POST',
      body: JSON.stringify({ 
        data: newState,
        clientVersion: version 
      })
    });
    const result = await response.json();
    if (result.conflict) {
      // Handle conflict
    }
  };

  return [state, updateState];
}