Monitoring Off-Heap Memory Usage

This topic describes how to monitor off-heap memory usage and the statistics that are available in GemFire XD.

The GemFire MemberMXBean interface exposes several statistics related to off-heap memory usage, including the amount of off-heap memory used and available, and the number of objects stored in off-heap memory. You can also observe the time that GemFire XD spends compacting objects in off-heap memory, as well as the degree of fragmentation of off-heap data. See JMX Manager MBeans.

You can view some of these statistics in the Pulse interface or view all of them by using a standard JMX monitoring tool such as JConsole. In addition, there are several APIs to obtain statistics (MBean attributes) of the MemberMXBean.

List of Available Off-Heap Statistics

Statistic	Description
compactions	The total number of times off-heap memory has been compacted. If this statistic is a non-zero value, then the member has had to compact off-heap memory. This may indicate that the member is close to running out of off-heap memory.
compactionTime	The total time spent compacting off-heap memory. This value will be zero unless you enable time statistics by using the enable-time-statistics boot property.
fragmentation	The percentage of off-heap memory fragmentation. Updated every time a compaction is performed. An excessively high Fragmentation percentage indicates that the off-heap memory is sufficiently fragmented to potentially make it difficult to fill up or utilize.
fragments	The number of fragments of free off-heap memory. Updated every time a compaction is done.
freeMemory	The amount of off-heap memory, in bytes, that is not being used.
largestFragment	The largest fragment of memory found by the last compaction of off heap memory. Updated every time a compaction is done.
maxMemory	The maximum amount of off-heap memory, in bytes. This is the amount of memory allocated at startup and does not change.
objects	The number of objects stored in off-heap memory. Each row in a GemFire XD table will have one offheap Object for the row itself and one additional off-heap Object for each LOB in the row.
reads	The total number of reads of off-heap memory. This statistic is incremented only by reads of a full object. If an object is only partially read, the statistic is not incremented.
usedMemory	The amount of off-heap memory, in bytes, that is being used to store data.

MemberMXBean Off-Heap Attributes

MemberMXBean has new attributes which expose several of the off-heap memory statistics for the member. You can use the following APIs to obtain values for off-heap statistics:

getOffHeapCompactionTime provides the value of the compactionTime statistic.
getOffHeapFragmentation provides the value of the fragmentation statistic.
getOffHeapFreeSize provides the value of the freeSize (freeMemory) statistic.
getOffHeapObjects provides the value of the objects statistic.
getOffHeapUsedSize provides the value of the usedSize (usedMemory) statistic.

To obtain the value of the off-heap maxMemory (the amount of allocated off-heap memory) statistic, combine the values for freeMemory (getOffHeapFreeSize) and usedMemory (getOffHeapUsedSize).

Note: The MemberMXBean operation listGemFireProperties includes the property off-heap-memory-size, which contains the same value as maxMemory.

RegionMXBean Off-Heap Attributes

The RegionMXBean listRegionAttributes API includes a new boolean attribute that indicates if the region is configured to use off-heap memory or regular Java heap memory. One of the attributes returned by the listRegionAttributes operation is enableOffHeapMemory which will have a value of true if that region is configured to use off-heap memory

Viewing Off-Heap Usage and Table Status in Pulse

To view how much off-heap memory is being used on a member, select Cluster View. Then select the List View icon and double-click on the member you wish to view. See Using Pulse Views for additional information.

The following screenshot depicts how Pulse reports off-heap memory usage for a member:

To view whether a table is storing its data in off-heap memory, select Data View and then select the table you wish to examine.

The following screenshot indicates how off-heap tables are represented in Pulse:

Errors

The following exceptions are associated with off-heap memory:

OutOfOffHeapMemoryException. When this exception is thrown, the cache will close in order to maintain consistency across the cluster. Each member should be configured to use off-heap eviction and critical off-heap percentage thresholds on the ResourceManager to avoid getting this exception and to prevent data inconsistency or member loss.
LowMemoryException. This exception is thrown during local and client put operations when the amount of off-heap memory being used has surpassed the configured critical-off-heap percentage threshold. You can specify this threshold using the gfxd start server command or by using the SYS.SET_CRITICAL_OFFHEAP_PERCENTAGE or SYS.SET_CRITICAL_OFFHEAP_PERCENTAGE_SGsystem procedure.