Clustertest Test Framework

6.7. Clustertest Test Framework

6.7.1. Introduction and Overview

The clustertest framework is implemented in Java, where tests are implemented in the interpreted JavaScript language. The use of Java made it much easier to implement tests involving concurrent activities, both in terms of inducing test load, and in, concurrently changing configuration of the replication cluster.

It consists of two physical portions:

  • A framework, implemented in Java

    This software is available at clustertest-framework @ GitHub .

    This framework makes use of libraries from several other open source projects:

    • js.jar

      This is for org.mozilla.javascript, the Mozilla JavaScript interpreter

    • junit-4.8.1.jar

      JUnit, a unit test framework.

    • log4j-1.2.15.jar

      Log4J is a popular Java-based framework for generating event logs.

    • postgresql-8.4-701.jdbc3.jar

      This is the PostgreSQL JDBC driver.

    To build the framework, it is necessary to have a Java compiler and the build tool, Ant, installed. To build all the .jar files used by the framework, one will run the command, with output similar to the following:

    % ant jar
    Buildfile: /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml
    
    compile-common:
        [mkdir] Created dir: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes
        [javac] /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml:23: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
    
    compile-testcoordinator:
        [javac] /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml:44: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
        [javac] Compiling 25 source files to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes
        [javac] Note: /var/lib/postgresql/PostgreSQL/clustertest-framework/src/info/slony/clustertest/testcoordinator/script/ClientScript.java uses or overrides a deprecated API.
        [javac] Note: Recompile with -Xlint:deprecation for details.
         [copy] Copying 1 file to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes/info/slony/clustertest/testcoordinator
    
    jar-common:
        [mkdir] Created dir: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar
          [jar] Building MANIFEST-only jar: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar/clustertest-common.jar
    
    compile-client:
        [javac] /var/lib/postgresql/PostgreSQL/clustertest-framework/build.xml:30: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
        [javac] Compiling 1 source file to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes
         [copy] Copying 2 files to /var/lib/postgresql/PostgreSQL/clustertest-framework/build/classes/info/slony/clustertest/client
    
    jar-client:
          [jar] Building jar: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar/clustertest-client.jar
    
    jar-coordinator:
          [jar] Building jar: /var/lib/postgresql/PostgreSQL/clustertest-framework/build/jar/clustertest-coordinator.jar
    
    jar:
    
    BUILD SUCCESSFUL
    Total time: 2 seconds

    At this time, there is no "regression test" for the test framework; to validate that it works requires running tests that use it.

    It includes classes supporting PostgreSQL- and Slony-I-specific functionality such as:

    • CreateDbScript

      Creates a database

    • DropDbScript

      Drops a database

    • LogShippingDaemon

      Starts up Slony-I logshipping daemon

    • LogShippingDumpScript

      Dumps and loads logshipping-based schema

    • PgCommand

      Run a PostgreSQL shell command (such as psql, createdb, and such)

    • PgDumpCommand

      Dump a PostgreSQL database

    • PsqlCommandExec

      Run SQL

    • ShellExecScript

      Run a shell script/command

    • SlonLauncher

      Start up a slon process

    • SlonikScript

      Run a slonik script

  • Tests integrated into the Slony-I software distribution, that consist of a combination of shell scripts, JavaScript, and SQL scripts.

    See the directory clustertest in the Slony-I software distribution, which has two sets of tests:

6.7.2. DISORDER - DIStributed ORDER test

The DISORDER or DIStributed ORDER test is intended to provide concurrency tests involving a reasonably sophisticated schema to validate various aspects of Slony-I behavior under concurrent load.

It consists of:

  • A schema for an inventory management application.

    Major objects include customers, inventory items, orders, order lines, and shipments.

    There are foreign key relationships between the various items, as well as triggers that maintain inventory and customer balances. Some of these relationships involve ON DELETE CASCADE, and so some actions may induce large numbers of cascaded updates.

  • Stored procedures to induce creation of the various sorts of objects, purchases, shipments, and additions and removals of customers and products.

  • Some tests are intended to be run against replicas, validating that balances add up. We believe that PostgreSQL applies changes in a transactional fashion such that they will always COMMIT leaving the visible state consistent; certain of the tests look for inconsistencies.

  • There are JavaScript test scripts that induce all sorts of manipulations of replication clusters to validate that replication configuration changes succeed and fail as expected.

6.7.2.1. Configuring DISORDER

DISORDER test configuration may be found in the following files:

  • conf/disorder.properties.sample

    This file contains Java style properties indicating how to connect to the various databases used by the DISORDER tests, including paths to tools such as slon and slonik

    The sample file is to be copied to conf/disorder.properties, and customized to indicate your local configuration. By using a .sample file, a developer may run tests within a Git tree, and not need to worry about their customizations interfering with the "canonical" sample configuration provided.

  • conf/java.conf.sample

    This is a shell script containing a path indicating where the clustertest Java code (e.g. - the clustertest-coordinator.jar file) may be found. This is also used, indirectly to determine where additional Java .jar files such as the JDBC driver are located.

    As with the disorder properties, above, this needs to be copied to conf/java.conf, and customized to indicate one's own local configuration.

  • conf/log4j.properties

    See documentation for Log4J for more details as to how this is configured; the defaults provided likely do not need to be altered.

6.7.3. Regression Tests

These tests represent a re-coding of the tests previously implemented as shell scripts using the clustertest framework.

These tests have gradually been enhanced to provide coverage of scenarios with which Slony-I has had problems; it is to be expected that new bugs may lead to the addition of further tests.

6.7.3.1. Configuring Regression Tests

Similar to the Section 6.7.2.1 for DISORDER tests, there are three configuration parameters:

  • conf/slonyregress.properties.sample

    This file contains Java style properties indicating how to connect to the various databases used by the regression tests, including paths to tools such as slon and slonik

    The sample file is to be copied to conf/slonyregress.properties, and customized to indicate your local configuration. By using a .sample file, a developer may run tests within a Git tree, and not need to worry about their customizations interfering with the "canonical" sample configuration provided.

  • conf/java.conf.sample

    This is a shell script containing a path indicating where the clustertest Java code (e.g. - the clustertest-coordinator.jar file) may be found. This is also used, indirectly to determine where additional Java .jar files such as the JDBC driver are located.

  • conf/log4j.properties

    Identical to configuration for DISORDER.