GemFire XD Cache Strategies

You can deploy GemFire XD as a distributed cache for data that is managed in a traditional relational database system. GemFire XD offers several strategies for using the product as a database cache.

Considerations for Auto-Generated Keys

If cached tables require automatic generation of identity column values, then you should use GemFire XD to automatically generate the key values but disable generated keys in the backend database. If the backend database also generates key values, then inserts can fail if GemFire XD has already generated the key.

Note: GemFire XD supports automatic generation of identity column values, but it does not support generating a sequence of key values. If a sequence is required, then consider using a stored procedure to obtain the next sequence value from the backend database before inserting the row in GemFire XD.
Note: When you use the GemFire XD DBSynchronizer implementation to synchronize tables that have an identity column, by default identity column values are not automatically applied to the synchronized database. DBSynchronizer expects that the back-end database table also has an identity column, and that the database generates its own identity value. If you would prefer that DBSynchronizer apply the same GemFire XD-generated identity column value to the underlying database, then set skipIdentityColumns=false in the DBSynchronizer properties file. See Configuring and Using DBSynchronizer.

Lazy-Loading the Cache

Application query requests are first attempted in GemFire XD. On a miss, a loader that implements lazy loading is invoked that can fetch the data from a backend database. GemFire XD only supports primary key-based queries when the table is lazily loaded. It is impractical (too expensive) to determine the subset of the rows missing in GemFire XD and construct a query to the backend database that only fetches the missing rows. It is also typical for applications to configure least-recently-used (LRU) caching where the cache optimizes the available memory by evicting the least frequently used rows. Applications can only issue primary key-based queries on tables that are configured for eviction. For more information, see Using a RowLoader to Load Existing Data.

All updates can be synchronously or asynchronously written back to the database using a 'writer' or the built in DBSynchronizer. For tables that are identical in structure to the backend database, applications can asynchronously "write behind" if you configure the database URL for the backend database.

Preloading Strategies

GemFire XD offers several caching strategies that involve preloading data.
  • Preload database. Load the entire database at bootstrap time into GemFire XD. If all updates are going through GemFire XD, it is unnecessary to resynchronize the state from the backend database. The data bootstrapping process is outlined below. With this choice, the tables are loaded with the entire data set from the backend database and all queries can directly be executed on GemFire XD data stores.
  • Preload "hot" data and lazy-load the rest. If the data set in the backend database is very large, a better option would be to maintain all the "hot" tables in GemFire XD and lazy load the historical data into "historical" tables. With this design, the entire supported SQL syntax can be used on fully populated "hot" tables, but, only primary key based queries can be issued on historical tables. Typically, these historical tables will be configured for LRU eviction.

Integration with Hibernate

GemFire XD can also be integrated in object relational mapping products such as Hibernate and provide an L2 cache - entire query result sets can be cached in GemFire XD. Updates done using the Hibernate API will be synchronously propagated to the database and query result sets will be invalidated in GemFire XD.

See GemFire XD Hibernate Dialect for information about using the GemFire XD dialect file with your Hibernate projects.

Exporting and Transforming an Existing Schema

Application developers can use a tool such as DdlUtils to export an existing relational database schema, transform it to GemFire XD, and import the new schema along with data into a GemFire XD cluster.