Persist Tables to Disk

By default, a GemFire XD distributed system persists only the data dictionary for the tables and indexes you create. These persistence files are stored in the datadictionary subdirectory of each locator and data store that joins the distributed system. Table data, however, is not persisted by default; if you shut down all GemFire XD members, the tables are empty on the next startup. In this tutorial you will persist table data to GemFire XD disk store files.

Procedure
  1. In a separate terminal window or GUI editor, examine the contents of the create_persistent_schema.sql script. Notice that this script uses the PERSISTENT keyword in each CREATE TABLE statement. For example, in the script the statement for creating the COUNTRIES table reads:

    CREATE TABLE COUNTRIES
       (
          COUNTRY VARCHAR(26) NOT NULL CONSTRAINT COUNTRIES_UNQ_NM Unique,
          COUNTRY_ISO_CODE CHAR(2) NOT NULL CONSTRAINT COUNTRIES_PK PRIMARY KEY,
          REGION VARCHAR(26),
          CONSTRAINT COUNTRIES_UC
            CHECK (country_ISO_code = upper(country_ISO_code) )
    
       ) REPLICATE PERSISTENT;
    

    When you execute the script in the next step, the existing schema will be dropped and replaced with a schema that GemFire XD persists to operational log files in the server member directories.

  2. In the gfxd session, execute the create_persistent_schema.sql script, then load table data:
    gfxd> run 'create_persistent_schema.sql';
    gfxd> run 'loadTables.sql';
  3. Exit the gfxd session:
    gfxd> exit;
  4. Now list the contents of each GemFire XD server directory:
    $ ls -l ~/server1 ~/server2
    total 384
    -rw-r--r-- 1 gpadmin gpadmin 966367641 Oct 29 06:49 BACKUPSQLF-DEFAULT-DISKSTORE_1.crf
    -rw-r--r-- 1 gpadmin gpadmin 107374183 Oct 29 06:21 BACKUPSQLF-DEFAULT-DISKSTORE_1.drf
    -rw-r--r-- 1 gpadmin gpadmin     55293 Oct 29 06:49 BACKUPSQLF-DEFAULT-DISKSTORE.if
    drwxr-xr-x 2 gpadmin gpadmin      4096 Oct 29 06:21 datadictionary
    -rw-r--r-- 1 gpadmin gpadmin         0 Oct 29 06:21 DRLK_IFSQLF-DEFAULT-DISKSTORE.lk
    -rw-r--r-- 1 gpadmin gpadmin    255088 Oct 29 06:49 gfxdserver.log
    -rw-rw-r-- 1 gpadmin gpadmin         0 Oct 29 06:21 start_gfxdserver.log
    
    /home/gpadmin/server2:
    total 364
    -rw-r--r-- 1 gpadmin gpadmin 966367641 Oct 29 06:49 BACKUPSQLF-DEFAULT-DISKSTORE_1.crf
    -rw-r--r-- 1 gpadmin gpadmin 107374183 Oct 29 06:22 BACKUPSQLF-DEFAULT-DISKSTORE_1.drf
    -rw-r--r-- 1 gpadmin gpadmin     55295 Oct 29 06:49 BACKUPSQLF-DEFAULT-DISKSTORE.if
    drwxr-xr-x 2 gpadmin gpadmin      4096 Oct 29 06:22 datadictionary
    -rw-r--r-- 1 gpadmin gpadmin         0 Oct 29 06:22 DRLK_IFSQLF-DEFAULT-DISKSTORE.lk
    -rw-r--r-- 1 gpadmin gpadmin    230842 Oct 29 06:49 gfxdserver.log
    -rw-rw-r-- 1 gpadmin gpadmin         0 Oct 29 06:22 start_gfxdserver.log

    Notice that each GemFire XD directory now contains several default disk store files to manage the table data on disk.

  5. Because the data is persisted to disk, GemFire XD will recover data from disk even if you shut down the data stores. Use the shut-down-all command to shut down all data stores in the cluster, and then restart the servers:
    $ gfxd shut-down-all -locators=localhost[10101]
    Connecting to distributed system: locators=localhost[10101]
    Successfully shut down 2 members
    $ gfxd server start -dir=$HOME/server1 -locators=localhost[10101] -client-port=1528 -sync=false
    Starting GemFireXD Server using locators for peer discovery: localhost[10101]
    Starting network server for GemFireXD Server at address localhost/127.0.0.1[1528]
    Logs generated in /home/gpadmin/server1/gfxdserver.log
    GemFireXD Server pid: 40612 status: waiting
    Waiting for table APP.FLIGHTS, buckets [1] (DiskId: 6cd1dff5-b6a8-4165-bc01-9a3555721f6e, Location: /home/gpadmin/server1/.) on: 
     [DiskId: 01e17e2d-6f7f-427d-b66f-6ea5dde49f52, Location: /home/gpadmin/server2/.]
    $ gfxd server start -dir=$HOME/server2 -locators=localhost[10101] -client-port=1529 -sync=false
    Starting GemFireXD Server using locators for peer discovery: localhost[10101]
    Starting network server for GemFireXD Server at address localhost/127.0.0.1[1529]
    Logs generated in /home/gpadmin/server2/gfxdserver.log
    GemFireXD Server pid: 40776 status: running
      Distributed system now has 3 members.
      Other members: 192.168.125.147(39381:locator)<v0>:50344, 192.168.125.147(40612:datastore)<v5>:11337
    Note: By default, GemFire XD servers and locators start to a "waiting" state if the member depends on another server or locator for disk store synchronization. The server automatically continues booting after any dependent servers or locators start. If you start servers from within a shell script, you can use the gfxd locator wait or gfxd server wait command to wait for members to reach the running state before continuing. As an alternative, you can monitor the current status of GemFire XD members using STATUS column in the MEMBERS system table.
    Notice the startup message:
    Waiting for table APP.FLIGHTS, buckets [1] (DiskId: 6cd1dff5-b6a8-4165-bc01-9a3555721f6e, Location: /home/gpadmin/server1/.) on: 
     [DiskId: 01e17e2d-6f7f-427d-b66f-6ea5dde49f52, Location: /home/gpadmin/server2/.]

    This is not an error message. It indicates that the GemFire XD member you are starting is waiting for another member to become available online, in order to ensure that the distributed system uses the very latest version of any persisted data.

  6. Now verify that the persistent tables were reloaded:
    $ gfxd
    gfxd> connect client 'localhost:1527';
    gfxd> select id from sys.systables s, sys.members where s.tablename='FLIGHTS';
    ID                                                                             
    -------------------------------------------------------------------------------
    192.168.125.147(40776)<v6>:31019                                               
    192.168.125.147(40612)<v5>:11337                                               
    
    2 rows selected
    gfxd> select count(*) memberRowCount, dsid() from flights group by dsid();
    MEMBERROWCOUNT|2                                                               
    -------------------------------------------------------------------------------
    276           |192.168.125.147(40612)<v5>:11337                                
    266           |192.168.125.147(40776)<v6>:31019                                
    
    2 rows selected
    Note: The exact row count may be different for your GemFire XD distributed system.