Sequence of Events for a Distributed Transaction

Here is a step-by-step description of events that occur before and during the transaction commit sequence.

Before a transaction is committed, the following events occur:
  1. When a transaction is started the transaction coordinator creates a globally unique ID (TXID) and creates a work space (TXState) to track operations. If the transaction is started from a thin client connection, the coordination happens on the server to which the client is connected. Note that transactions are started implicitly in JDBC: the end of one transaction implicitly starts a new one.
  2. All updates in the scope of a transaction are immediately propagated to all replicas in parallel and are partially coordinated on each data store (cohort). Each member that owns the data involved in the transaction manages the state in a local TXState. When updates are received by a cohort, it attempts to obtain local write locks on the rows.

    A cohort fails immediately if the rows are already locked by another transaction. Failure to obtain locks results in the coordinator implicitly rolling back the entire transaction and releasing all the locks on the data hosts. These types of failures may be exacerbated by triggers, because triggers are executed on different members and may attempt to update the same data rows concurrently. Batched updates can also compound these failures because GemFire XD can lazily perform conflict detection at any point before the the transaction is committed (instead of at the time of the update, for non-batched transactions).

  3. GemFire XD detects only Write-Write conflicts while in READ_COMMITTED isolation level. It detects both Write-Write and Read-Write conflicts while in REPEATABLE_READ isolation level. Detecting Read-Write conflicts occurs at commit time in order to reduce the window of potential conflicts. For example, if a reader begins and ends its read before the writer starts to commit, then there is no conflict. Also, in most cases the writer transaction will receive the conflict exception.

    To prevent rows fetched in the transaction from being modified before transaction commit, GemFire XD supports select for update where the selected rows are locked before the result set can be returned.

  4. While the transaction is in progress the updates are maintained only in the TXState on the data stores, completely isolated from other concurrent connections.
  5. Any constraint checks on the rows are immediately applied, and failures result in a constraint violation exception.
    Note: In this release of GemFire XD, constraint violations also implicitly roll back the transaction.
  6. Readers do not normally acquire any locks unless the rows being read are about to be committed. Transactional read operations are applied to the TXState first, before they are applied to the committed state.
  7. When using REPEATABLE_READ isolation level, readers also acquire read locks on the affected rows. However, this does not immediately cause a conflict with ongoing writers. When a writer tries to commit a row, it attempts to upgrade the write lock to an exclusive mode, and this can result in a conflict if read locks are held. In rare cases readers may receive conflict exceptions that are caused by deadlocks.

    Using REPEATABLE_READ isolation level guarantees atomicity for a distributed commit because after readers have locked a row for read, then any transaction that changes the row cannot commit those changes (or associated changes that are part of another transaction) across the cluster.

These events occur during the transaction commit sequence:
  1. When a commit message is received by the coordinator, it dispatches a single commit message to all of the cohorts. Because the rows are already locked and constraints applied during the transaction, it is certain that the transaction will not fail due to conflicts.
  2. Each cohort guarantees transaction atomicity by making sure that no concurrent transaction can see partially-committed state on the data store. However, with READ_COMMITTED isolation level the commit phase does not guarantee that to be the case across all of the cohorts that are involved in the transaction. Commit atomicity is guaranteed only for REPEATABLE_READ isolation level, as described in Step 3 of the events occurring before transaction commit. Keep in mind, however, that using REAPEATABLE_READ isolation has higher chance of receiving deadlocks because there are much greater overlaps between readers and writers. Deadlocks cause one or both transactions to fail after a timeout period (see gemfire.LOCK_MAX_TIMEOUT).
  3. Each cohort applies the TXState changes to the tables in memory, and releases all the locks before acknowledging to the coordinator.
Note: Commits are performed in a background thread, and it is possible that other threads do not immediately see the results of the transaction. Because the outcome of the transaction is assured at commit time, the coordinator does not wait for individual commit replies from the cohorts before returning the committed transaction to the initiating thread.

If the same connection immediately initiates another operation on the same data, then the cohorts wait for pending replies from the previous transaction (as described in Step 3) before applying the change. Also, a commit or rollback operation takes place even if the initiating client process becomes unavailable while the commit or rollback is performed.