Tuning Disk I/O

Most GemFire XD applications that access the hard disk work well with synchronous disk stores and reasonably fast disks. If disk I/O becomes a bottleneck, you can configure the system to minimize seek time.

The degree of tuning depends on your application’s data access patterns:
  • Place the disk store directories used by a GemFire XD server on local drives or on a high-performance Storage Area Network (SAN).
  • If you have limited disk space, use smaller oplog sizes for disk stores to avoid expensive file deletes. A value of 512 MB for MAXLOGSIZE is a good starting point.
  • If space allows, turn off automatic oplog compaction in disk stores by setting AUTOCOMPACT to false. This prevents latency-sensitive writes to new oplogs from competing with seeks and writes related to compaction. Oplogs are still removed when they no longer contain current data. When compaction is unavoidable, set ALLOWFORCECOMPACTION to true and use gfxd to do manual compaction at a time when system activity is low or the system is offline.
  • Run backups with gfxd when system activity is low or the system is offline.
  • ASYNCHRONOUS disk stores can give some latency benefit for applications with infrequent writes that flush using a TIMEINTERVAL. They can also reduce disk I/O for applications that do very frequent writes to the same data, because these are conflated in the buffer. But asynchronous disk stores offer insignificant benefit in most cases and use heap space that can be better utilized elsewhere. Instead, use SYNCHRONOUS disk stores and let the operating system handle buffering. This simplifies application design and performs well with modern operating systems and disk storage.
  • Avoid configuring tables for expiration with overflow to disk, especially when expired data is frequently read. This increases disk seek times and latency, in addition to fragmenting the heap.
  • Use different disk stores for persistence and overflow, and map them to different physical disks. This practice isolates the fast sequential writes used in persistence from the higher latency random access caused by faulting data from disk into memory.
  • A disk store that supports overflow or persistence with overflow can benefit from using multiple directories mapped to different disks.
  • Isolate disk I/O for your application from that of other applications, including the operating system.