Limitations of Multi-Site Replication

The GemFire XD multi-site implementation has the following limitations and restrictions.

  • Only SQL insert, update, and delete operations are distributed using configured gateway senders. Queries and DDL operations are not distributed to remote GemFire XD systems.
  • Table expiration actions are not distributed to remote GemFire XD systems.
  • GemFire XD does not extend transaction boundaries to remote systems that are configured for WAN replication. If you use a transaction to update a table that is configured for WAN replication, the transaction is only honored within the local GemFire XD cluster where you initiated the transaction. Any DML operations that are replicated to a remote table are applied outside of a transaction
  • GemFire XD provides no conflict detection to address simultaneous updates to table data on multiple GemFire XD systems. If multiple sites in a WAN configuration update the same table data at the same time, then the table data can become inconsistent between sites.
    For example, consider a situation where two GemFire XD sites replicate the table "employees" to each other over a WAN, and the employees data is currently consistent between both sites. Site 1 executes the DML operation:
    update employees set bonus = bonus + 500 where department = 'sales'
    At the same time, Site 2 executes the DML operation, moving the employee with ID 12345 from the Sales department to the QA department:
    update employees set department = 'qa' where employee_id = '12345'

    In this case the DML is successfully applied to each GemFire XD site and propagated to the other. However, in Site 1, the employees row for employee_id 12345 has a higher bonus value than the same row in Site 2.

    Similar problems can occur if each site tries to add a row using the same primary key, if one site deletes a parent row required by an insert on the other site, and so forth.

    To avoid these kinds of inconsistencies, enable WAN replication only for tables that do not require simultaneous updates of the data on different GemFire XD systems.

  • GemFire XD preserves the order of DML statements applied to the distributed system (and queued to remote WAN sites) only for a single thread of execution. See No Ordering Guarantee for DML in Separate Threads.

See also the Pivotal GemFire XD 1.4.0 Release Notes for information about known issues associated with multi-site deployments.