RHEL: Upgrade GemFire XD from RPM

If your guest operating system is Red Hat Enterprise Linux (RHEL) and you have installed a previous version of the product using yum or the Pivotal HD ICM utility, use the RPM distribution to upgrade GemFire XD. You complete the upgrade procedure on every virtual and physical machine that runs GemFire XD.

Note: Thoroughly test your development systems with the new version before moving into production.

When you upgrade GemFire XD from RPM, the new software is installed by default into /opt/pivotal/gemfirexd/GemFireXD_XX where XX corresponds to the version of GemFire XD (for example, GemFireXD_130) that you have installed. No files are overwritten during the upgrade process.

Prerequisites

Procedure

  1. If your system uses any stored procedures, listeners, or functions that reference the SQLFire package names (either com.vmware.sqlfire or com.pivotal.sqlfire), then you must stop and remove these objects from the system before you begin the upgrade process. For example, if you are upgrading from SQLFire and you configured DBSynchronizer using the built-in com.vmware.sqlfire.callbacks.DBSynchronizer implementation, then you must remove this listener implementation before proceeding.
    Perform the following steps for any database objects that use older SQLFire package names:
    1. Ensure that you have the DDL commands necessary to recreate the procedure or listener. Typically these commands are stored in a SQL script used to create your database schema. For example, an existing DBSynchronizer might have been created using the statement:
      gfxd> create asynceventlistener testlistener
      > (
      > listenerclass 'com.vmware.sqlfire.callbacks.DBSynchronizer' 
      > initparams 
      >   'com.mysql.jdbc.Driver,jdbc:mysql://localhost:3306/gfxddb,SkipIdentityColumns=true,user=sqlfuser,secret=25325ffc3345be8888eda8156bd1c313' 
      > )
      > server groups (dbsync);

      If you do not have the DDL commands readily available, use the sqlf write-schema-to-sql command to write the existing schema's DDL commands to a SQL file, and verify that the necessary commands are present.

    2. Use SYS.STOP_ASYNC_EVENT_LISTENER to stop any listener or DBSynchronizer implementation that uses the older API. For example:
      gfxd> call sys.stop_async_event_listener('TESTLISTENER');
    3. Alter any tables that use a stopped listener or DBSynchronizer to remove the listener from the table. For example:
      gfxd> alter table mytable set asynceventlistener ();
    4. Use the appropriate DROP command (DROP PROCEDURE, DROP ASYNCEVENTLISTENER, DROP FUNCTION) or system procedure (SYS.REMOVE_LISTENER, SYS.REMOVE_LOADER, SYS.REMOVE_WRITER) to remove procedure and listener configurations that use the older API. For example:
      gfxd> drop asynceventlistener testlistener;
    5. Upgrade your procedure or listener code to use the newer com.pivotal.gemfirexd package prefixes, and recompile as necessary. If you are using the built-in DBSynchronizer implementation, you need only modify the DDL SQL script to specify the newer implementation in the CREATE ASYNCEVENTLISTENER command. For example:
      -- Upgraded DBSynchronizer DDL
      	
      create asynceventlistener testlistener
       (
       listenerclass 'com.pivotal.gemfirexd.callbacks.DBSynchronizer' 
       initparams 
         'com.mysql.jdbc.Driver,jdbc:mysql://localhost:3306/gfxddb,SkipIdentityColumns=true,user=sqlfuser,secret=25325ffc3345be8888eda8156bd1c313' 
       )
       server groups (dbsync);
    6. Continue with the distributed system upgrade before you attempt to recreate updated procedure or listener implementations.
    Note: If you fail to remove a procedure, listener, or function that uses the older SQLFire API, you will receive a ClassNotFoundException when you start the upgraded GemFire XD member. The exception will reference the com.vmware.sqlfire or com.pivotal.sqlfire package, which is not available in GemFire XD.
  2. Use the shut-down-all and locator commands to stop all members. For example:
    $ sqlf shut-down-all -locators=localhost[10101]
    $ sqlf locator stop -dir=$HOME/locator
    In the sample command, substitute the address, port, and working directory of a locator for your SQLFire distributed system. Ensure that you stop all locators as well as data stores.
  3. Make backups of all disk store files.
  4. Beginning with machines that host locator members, execute the following rpm command to install the new version of the GemFire XD software. If necessary, use sudo to run the command if you are not logged in as root:
    RHEL 5:
    sudo rpm -Uvh pivotal-gemfirexd-1.3.0-48613.el5.noarch.rpm
    RHEL 6:
    sudo rpm -Uvh pivotal-gemfirexd-1.3.0-48613.el6.noarch.rpm

    The rpm command begins the install process, resolves dependencies, and displays the packages it plans to install.

  5. If you use a startup script or a sqlfire.properties file to configure the local member at startup, note that the product prefix that is included in certain member boot properties has changed from sqlfire. to gemfirexd. Update your script or properties file to use the gemfirexd. prefix as necessary. You may also consider renaming sqlfire.properties to gemfirexd.properties. See Configuration Properties.
  6. Modify any member startup scripts to use the single -heap-size property instead of -initial-heap and/or -max-heap. GemFire XD no longer support thes -initial-heap and -max-heap properties. See gfxd Launcher Commands.
    Note: Although GemFire XD introduces the gfxd utility to replace the earlier sqlf utility, "sqlf" is still provided and supported as an optional syntax in this release for convenience.
  7. GemFire XD requires libraries from Pivotal HD Enterprise in order to create HDFS tables or use the MapReduce API. If you installed GemFire XD on the same machine as Pivotal HD, then GemFire XD will automatically locate the required libraries in the default locations:
    • /usr/lib/gphd/hadoop
    • /usr/lib/gphd/hadoop-mapreduce
    • /usr/lib/gphd/hadoop-hdfs/
    If you installed GemFire XD on a machine that does not have a Pivotal HD installation, then you can use the create_hdfsclient_dist.sh script to create a ZIP file of the necessary libraries, which you can then add to the local GemFire XD installation. Follow these steps to use the script:
    1. Copy the create_hdfsclient_dist.sh script to a machine that has a Pivotal HD installation. The script is installed with GemFire XD as /opt/pivotal/gemfirexd/bin/create_hdfsclient_dist.sh.
    2. Execute the create_hdfsclient_dist.sh on the Pivotal HD machine:
      $ ./create_hdfsclient_dist.sh

      The script packages the necessary library files into a ZIP file named gfxd_hdfsclient_dist.zip in the directory where you executed the script.

    3. Copy the gfxd_hdfsclient_dist.zip back to the machine where you installed GemFire XD.
    4. Unzip the gfxd_hdfsclient_dist.zip file's contents into the /opt/pivotal/gemfirexd/ext-lib directory. GemFire XD will now automatically load the necessary Pivotal HD libraries when you start the local member.
  8. Start the GemFire XD member process on the local machine or VM using the new GemFire XD software. This process automatically upgrades the disk store files as necessary.
  9. Repeat these steps as necessary to upgrade other members running on other machines or VMs (locators first, followed by data stores and accessors).
  10. After the distributed system is running, use your modified DDL SQL scripts to recreate stored procedure and listener implementations, using the new com.pivotal.gemfirexd package names. (See Step 1 above.) For example, to srestore the sample DBSynchronizer configuration described in Step 1:
    gfxd> create asynceventlistener testlistener
    > (
    > listenerclass 'com.pivotal.gemfirexd.callbacks.DBSynchronizer' 
    > initparams 
    >   'com.mysql.jdbc.Driver,jdbc:mysql://localhost:3306/gfxddb,SkipIdentityColumns=true,user=gfxduser,secret=25325ffc3345be8888eda8156bd1c313' 
    > )
    > server groups (dbsync);
    gfxd> alter table mytable set asynceventlistener (testlistener);
    gfxd> call sys.start_async_event_listener('TESTLISTENER');

Post-Upgrade Instructions

After you upgrade the individual GemFire XD members, perform these steps if you upgraded from SQLFire 1.1.1 or 1.1.2:
  1. Update all client applications to use the new GemFire XD package names and libraries, then recompile and redeploy those applications:
    • Java package names beginning with com.vmware.sqlfire or com.pivotal.sqlfire were changed to com.pivotal.gemfirexd (see the Javadocs).
    • ADO.NET application interfaces were changed to begin with Pivotal.Data.GemFireXD (see ADO.NET API).
    • JDBC thin client and peer client applications must use the new JAR file names, gemfirexd-client.jar and gemfirexd.jar, to access an upgraded distributed system.
    • The ADO.NET client DLL is now installed using an MSI file. See Installing GemFire XD ADO.NET Components.
  2. The JDBC connection string prefix has changed from jdbc:sqlfire to jdbc:gemfirexd. Update any custom or third-party JDBC clients to use the correct connection string format.
  3. Modify any member startup scripts to use the single -heap-size property instead of -initial-heap and/or -max-heap. The gfxd member startup script no longer support thes -initial-heap and -max-heap properties. See gfxd Launcher Commands.
  4. GemFire XD now uses a JMX Manager node to provide access to a federated MBean architecture. Update your GemFire XD member configuration to use the new architecture, and modify JMX clients as necessary to access the new GemFire XD MBeans. See Using Java Management Extensions (JMX).
  5. The --SQLFIRE-PROPERTIES query hint has changed to --GEMFIREXD-PROPERTIES. Modify any scripts that use the older hint.

What to Do Next