Overview of Pivotal GemFire XD

Pivotal GemFire XD is a memory-optimized, distributed data store that is designed for applications that have demanding scalability and availability requirements. With GemFire XD you can manage data entirely using in-memory tables, or you can persist very large tables to local disk store files or to a Hadoop Distributed File System (HDFS) for big data deployments. In this initial release, GemFire XD provides a low-latency SQL interface to in-memory table data, while seamlessly integrating data that is persisted in HDFS. A single GemFire XD distributed system can be easily scaled out using commodity hardware to support thousands of concurrent clients, and you can also replicate data between distinct GemFire XD clusters over a WAN interface. GemFire XD also provides easy access to persisted HDFS data using tools in the Pivotal HD ecosystem such as MapReduce or HAWQ (using the GemFire XD PXF driver that is installed with HAWQ).

Note: GemFire XD is a new product that builds upon the previous vFabric SQLFire product, adding features such as:
  • HDFS-persistence
  • Off-heap memory storage for table data
  • Federated JMX MBean architecture
  • Monitoring using the GemFire XD Pulse web application.

If you are familiar with the vFabric SQLFire product, you will recognize that GemFire XD shares many internal features with SQLFire, such as the SQL language extensions and query capabilities, system tables, local disk store persistence files, product directories, and the gfxd command-line utility.