Supported Configurations and System Requirements

This topic describes the supported configurations and system requirements for GemFire XD.

Standalone Product Support

GemFire XD 1.4 is certified only for stand alone use. It is not certified for installation or interoperability with Pivotal HD 2.1. Pivotal is working to certify GemFire XD 1.4 with an upcoming version of Pivotal HD for HDFS table support.

If you are currently using GemFire XD 1.3.x with Pivotal HD for HDFS table support, continue to use your installed version of the product. Pivotal will provide maintenance releases for version 1.3.x until GemFire XD 1.4 is certified with an upcoming version of Pivotal HD.

Supported Configurations

The following table shows all supported configurations for GemFire XD.

Note: Pivotal GemFire XD 1.4.0 no longer supports Java SE 6. If your current GemFire XD installation uses Java SE 6, you will need to update to Java SE 7 before starting GemFire XD 1.4 members.

Also keep in mind that applications that are ported from JDK 1.6 to JDK 1.7 may need to adjust their JVM configuration. Specifically, the default sizes for generations in the Java heap have changed significantly between JDK 1.6 and JDK 1.7. For more information, see Java Virtual Machine Enhancements in JDK 7 in the Oracle Java documentation.

Operating System Processor Architecture JVM Production or Developer Support
Red Hat EL 5, 6.2, 6.4 x86 (64bit)

Java SE 1.7.0_72

Production
CentOS 6.2, 6.4, 7 x86 (64bit)

Java SE 1.7.0_72

Production
Ubuntu 14.04 x86 (64bit and 32 bit)

Java SE 1.7.0_72

Production
SUSE Linux Enterprise Server 11 x86 (64bit and 32 bit)

Java SE 1.7.0_72

Production
Windows Server 2008 R2 (Standard)*‡ x86 (64bit and 32 bit)

Java SE 1.7.0_72

Production
Windows 7 SP1 (Professional or Enterprise)* x86 (64bit and 32 bit)

Java SE 1.7.0_72

Developer
Windows 8, Windows 8.1 x86 (64bit and 32 bit)

Java SE 1.7.0_72

Developer
Ubuntu 10.11 x86 (64 bit)

Java SE 1.7.0_72

Developer
Mac OS X 10.8, 10.9 x86 (64 bit)

Java SE 1.7.0_72

Developer

*The Microsoft Loopback Adapter is not supported.

‡Windows Server users should use the Resource Monitor to verify the amount of RAM that is available to the operating system. If a large amount of RAM is marked as "Hardware Reserved," then check the computer's BIOS settings to ensure that features such as "Redundant Memory" or "RAS Memory Mirroring" are not accidentally enabled.

Note: The GemFire XD product download does not include Java; you must download and install a supported JDK for your system.

Host Machine Requirements

Requirements for each host:
  • A supported Java SE installation.
  • File system that supports long file names.
  • Adequate per-user quota of file handles (ulimit for Linux). Large deployments with HDFS-persistent tables can require several thousand file descriptors to manage the HDFS log files for partitioned table buckets. Ensure that you set a high enough ulimit (up to the hard limit of 81,920) for your system. See How GemFire XD Manages HDFS Data and Compaction for HDFS Log Files.
  • TCP/IP.
  • System clock set to the correct time.
  • For each Linux host, the hostname and host files must be properly configured. See the system manpages for hostname and hosts.
  • For each Linux host, configure the swap volume to be the same size as the physical RAM installed in the computer.
  • Time synchronization service such as Network Time Protocol (NTP).
Note: For troubleshooting, you must run a time synchronization service on all hosts. Synchronized time stamps allow you to merge log messages from different hosts, for an accurate chronological history of a distributed run.
Note: If you deploy GemFire XD to a virtualized host, see also Running GemFire XD in Virtualized Environments.

External Load Balancing, HA Solutions Not Supported

GemFire XD does not support using external load balancing techniques between clients and the GemFire XD distributed system, or between distributed system members. This includes software and hardware load balancers, Virtual IP Addresses (VIPs), and proxies. Similarly, external HA techniques such as Linux-HA and Windows Clustering are not supported.

Increase Unicast Buffer Size on Linux Platforms

On Linux platforms, execute the following commands as the root user to increase the unicast buffer size:
  1. Edit the /etc/sysctl.conf file to include the following lines:
    net.core.rmem_max=1048576
    net.core.wmem_max=1048576
  2. Reload sysctl.conf:
    sysctl -p

Disable SYN Cookies on Linux Platforms

Many default Linux installations use SYN cookies to protect the system against malicious attacks that flood TCP SYN packets. The use of SYN cookies dramatically reduces network bandwidth, and can be triggered by a running GemFire XD distributed system.

If your GemFire XD distributed system is otherwise protected against such attacks, disable SYN cookies to ensure that GemFire XD network throughput is not affected.

To disable SYN cookies permanently:
  1. Edit the /etc/sysctl.conf file to include the following line:
    net.ipv4.tcp_syncookies = 0

    Setting this value to zero disables SYN cookies.

  2. Reload sysctl.conf:
    sysctl -p

Filesystem Type for Linux Platforms

For optimum disk-store performance, Pivotal recommends the use of ext4 filesystems when operating on Linux or Solaris platforms. See Optimizing a System with Disk Stores.

Requirements for Running GemFire XD and Hadoop on the Same Machine

Severe performance degradation can occur for insert-heavy workloads when GemFire XD and Hadoop run on the same machine and GemFire XD is configured with very large memory sizes (either off-heap or on-heap memory). The performance degradation occurs when the operating system pages out GemFire XD memory to disk.

To prevent this problem, perform these steps when deploying GemFire XD and Hadoop on the same machine:
  1. Increase the operating system's ulimit -l value (the maximum size that may be locked in memory) from the default of 32 KB to at least the total amount of memory used by GemFire XD for on-heap or off-heap storage.

    To permanently change the default limit for all users, edit the /etc/security/limits.conf file and add the line:

    * hard memlock limit-value-in-kb
  2. When you start each GemFire XD data store, include the gfxd -lock-memory option to prevent the operating system from paging out heap or off-heap memory.

Note: If you do not increase the "ulimit -l" value to at least the amount of memory used by GemFire XD, then GemFire XD will fail to start when you specify -lock-memory.

Note: Using locked memory in this manner increases the time required to start GemFire XD. The additional time required to start GemFire XD depends on the total amount of memory used, and can range from several seconds to 10 minutes or more. To improve startup time and reduce the potential of member timeouts, execute the following command before starting a GemFire XD member:
$ echo 1 > /proc/sys/vm/drop_caches

Requirements when using Hadoop without High Availability

If you do not configure your Hadoop system for high availability with redundant failover, then it is important that you set Hadoop timeout properties to these recommended values:
Table 1. Hadoop Client Timeout Properties
Client Timeout Property Recommended Value
ipc.client.ping false
ipc.ping.interval 5000
ipc.client.connect.timeout 5000
ipc.client.connect.max.retries 2
ipc.client.connect.max.retries.on.timeouts 0
dfs.client.socket-timeout 30000

Using these values ensures that GemFire XD read operations to Hadoop quickly fail if the Hadoop node is unavailable (instead of hanging for a long period of time due to the default timeout intervals).

Restriction for Deploying HAWQ with GemFire XD

Due to the memory requirements and memory configuration required for each product, Pivotal does not support production deployments of GemFire XD and HAWQ on the same machine. Deploy GemFire XD and HAWQ on the same machine or virtual machine only for development or testing purposes.

Note: When you deploy GemFire XD with HAWQ on the same machine, you must set the swap volume to the same size as physical RAM. See Adding Swap Space in the RedHat or CentOS documentation for details.

Client Requirements

GemFire XD supports two JDBC drivers: a thin client JDBC driver and a peer JDBC driver. GemFire XD server instances and the peer driver are supported only with Java SE 6. You can download Java from http://www.oracle.com/technetwork/java/javase/downloads/index.htm.

GemFire XD provides a wire protocol ADO.NET driver that you can use for developing non-Java client applications. Requirements for using the ADO.NET Client distribution are documented in the GemFire XD ODBC distribution. See Using the GemFire XD ADO.NET Client.

GemFire XD provides an ODBC driver that responds to ODBC API function calls in client applications. The ODBC driver supports both 32-bit and 64-bit versions of Linux and Windows and is ODBC Level 1 compliant (supports all ODBC Core and Level 1 functions). It also supports a subset of Level 2 functions. See ODBC Driver Supported Configurations for more information on driver installation requirements.