Mon Mar 7 19:31:08 PST 2005
- Previous message: [Slony1-commit] By cbbrowne: There's a minor typo in src/slon/confoptions.h that
- Next message: [Slony1-commit] By cbbrowne: Major changes to log shipping, allowing it to support
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Log Message: ----------- Added scanner support for include and define Flex scanner code as per Korry <korryd at hotmail.com> cbbrowne added a ducttape script to test this facility as well as documentation. Due to the addition of this facility, the documentation describing the use of M4 for similar purpose has been dropped. Modified Files: -------------- slony1-engine/doc/adminguide: failover.sgml (r1.12 -> r1.13) help.sgml (r1.12 -> r1.13) slonik_ref.sgml (r1.17 -> r1.18) usingslonik.sgml (r1.7 -> r1.8) slony1-engine/src/slonik: scan.l (r1.22 -> r1.23) Added Files: ----------- slony1-engine/src/ducttape: test_7_defines (r1.1) -------------- next part -------------- Index: slonik_ref.sgml =================================================================== RCS file: /usr/local/cvsroot/slony1/slony1-engine/doc/adminguide/slonik_ref.sgml,v retrieving revision 1.17 retrieving revision 1.18 diff -Ldoc/adminguide/slonik_ref.sgml -Ldoc/adminguide/slonik_ref.sgml -u -w -r1.17 -r1.18 --- doc/adminguide/slonik_ref.sgml +++ doc/adminguide/slonik_ref.sgml @@ -83,17 +83,120 @@ <para> Those commands are grouped together into one transaction per participating node. </para> -<!-- ************************************************************ --></sect3></sect2></sect1></article> +<!-- ************************************************************ --> + + <reference id="metacmds"> + <title>Slonik Meta Commands</title> + <partintro> + <para> + The following commands may be used to somewhat abstract the + definitions of components of Slonik scripts; <xref + linkend="stmtinclude"> grouping configuration into central files + that may be reused, and <xref linkend="stmtdefine"> allowing + mnemonic identifiers to replace cryptic numeric object IDs. + </para> + </partintro> + <!-- **************************************** --> + <refentry id ="stmtinclude"><refmeta><refentrytitle>INCLUDE</refentrytitle> </refmeta> + + <refnamediv><refname>INCLUDE</refname> + + <refpurpose> pulling in slonik code from another file </refpurpose> + </refnamediv> + <refsynopsisdiv> + <cmdsynopsis> + <command>include </command> + <arg><replaceable class="parameter"> <pathname></replaceable></arg> + </cmdsynopsis> + </refsynopsisdiv> + <refsect1> + <title>Description</title> + <para> + This draws the specified slonik script inline into the present + script. If the <option>pathname</option> specifies a relative + path, <xref linkend="slonik"> will search relative to the + current working directory. + </para> + + <para> + Nested include files are supported. The scanner and parser + report the proper file names and line numbers when they run into + an error. </para> + </refsect1> + <refsect1><title>Example</title> + <programlisting> + include </tmp/preamble.slonik>; + </programlisting> + </refsect1> + </refentry> + <!-- **************************************** --> + <refentry id ="stmtdefine"><refmeta><refentrytitle>DEFINE</refentrytitle> </refmeta> + + <refnamediv><refname>DEFINE</refname> + + <refpurpose> Defining a named symbol </refpurpose> + </refnamediv> + <refsynopsisdiv> + <cmdsynopsis> + <command>define </command> + <arg><replaceable class="parameter"> name </replaceable></arg> + <arg><replaceable class="parameter"> value </replaceable></arg> + </cmdsynopsis> + </refsynopsisdiv> + <refsect1> + <title>Description</title> + <para> + This defines a named symbol. Symbol names must follow the + slonik rules for constructing identifiers, by starting with a + letter, followed by letters, numbers, and underscores. + </para> + + <para> + Symbol values may contain spaces and may recursively contain + symbol references. + </para> + + <para> + Symbols are referenced by using a <quote>@</quote> followed by + the symbol name. Note that symbol referencing is suppressed + inside string literals. + </para> + </refsect1> + <refsect1><title>Example</title> + <programlisting> +define cluster movies; +define sakai 1; +define chen 2; +define fqn fully qualified name; + +cluster name = @cluster; +node @sakai admin conninfo = 'service=sakai-replication'; +node @chen admin conninfo = 'service=chen-replication'; +define setMovies id = 1; +define sakaiMovies @setMovies, origin = @sakai; + +create set ( @sakaiMovies, comment = 'movies' ); + +set add table( set @sakaiMovies, id = 1, @fqn = 'public.customers', comment = 'sakai customers' ); +set add table( set @sakaiMovies, id = 2, @fqn = 'public.tapes', comment = 'sakai tapes' ); +echo 'But @sakaiMovies will display as a string, and is not expanded'; + </programlisting> + </refsect1> + </refentry> + + </reference> + +<!-- **************************************** --> <reference id="hdrcmds"> <title>Slonik Preamble Commands</title> <partintro> <para> - The following commands must appear as a - <quote>preamble</quote> at the very top of every - <application>slonik</application> command script. They do not - cause any direct action on any of the nodes in the replication - system, but affect the execution of the entire script. + The following commands must appear as a <quote>preamble</quote> at + the beginning of each <application>slonik</application> command + script. They do not cause any direct action on any of the nodes in + the replication system, but affect the execution of the entire + script. </para> </partintro> <!-- **************************************** --> Index: help.sgml =================================================================== RCS file: /usr/local/cvsroot/slony1/slony1-engine/doc/adminguide/help.sgml,v retrieving revision 1.12 retrieving revision 1.13 diff -Ldoc/adminguide/help.sgml -Ldoc/adminguide/help.sgml -u -w -r1.12 -r1.13 --- doc/adminguide/help.sgml +++ doc/adminguide/help.sgml @@ -38,7 +38,22 @@ <listitem><para> If your Russian is much better than your English, then <ulink url="http://kirov.lug.ru/wiki/Slony"> KirovOpenSourceCommunity: Slony</ulink> may be the place to -go.</para></listitem> </itemizedlist></para> +go.</para></listitem> + +<listitem><para> <ulink url="http://pgpool.projects.postgresql.org/" +id="pgpool"> <application> pgpool </application> </ulink> </para> + +<para> <application>pgpool</application> is a connection pool server +for &postgres;; it allows an application to connect to it as if it +were a standard &postgres; server. It caches connections, which +reduces the overhead involved in establishing them. It supports a +<quote>scheduled switchover</quote> feature, which would allow +dynamically switching over from one server to another. That would be +very useful when doing a <xref linkend="stmtmoveset">, as it would +allow applications to be switched to point to the new origin without +needing to update their configuration. </para> </listitem> + +</itemizedlist></para> <sect2><title> Other Information Sources</title> <itemizedlist> Index: usingslonik.sgml =================================================================== RCS file: /usr/local/cvsroot/slony1/slony1-engine/doc/adminguide/usingslonik.sgml,v retrieving revision 1.7 retrieving revision 1.8 diff -Ldoc/adminguide/usingslonik.sgml -Ldoc/adminguide/usingslonik.sgml -u -w -r1.7 -r1.8 --- doc/adminguide/usingslonik.sgml +++ doc/adminguide/usingslonik.sgml @@ -32,8 +32,9 @@ address.</para></listitem> <listitem><para> Users seem interested in wrapping everything possible -in <command>TRY</command> blocks, which is regrettably -<emphasis>less</emphasis> useful than might be imagined...</para></listitem> +in <command>TRY</command> blocks, which is regrettably somewhat +<emphasis>less</emphasis> useful than might be +hoped...</para></listitem> </itemizedlist></para> @@ -43,11 +44,9 @@ <itemizedlist> <listitem><para> Named nodes, named sets</para> -<para> Unfortunately, the use of naming nearly turns into a need for -an <quote>ESP protocol</quote>, as <application>slonik</application> -would need to <emphasis>start</emphasis> by determining the (possibly -in flux) set of mappings between node names and node -IDs. </para></listitem> +<para> This is supported by the (new in 1.1) <xref +linkend="stmtdefine"> and <xref linkend="stmtinclude"> statements. +</para></listitem> <listitem><para> Looping and control constructs</para> @@ -64,10 +63,6 @@ <itemizedlist> -<listitem><para> Some sort of text rewriting system such as M4 may be -used to map mnemonic object names onto the perhaps-less-intuitive -numeric arrangement.</para></listitem> - <listitem><para> Embedding generation of slonik inside shell scripts</para> @@ -83,111 +78,6 @@ </itemizedlist> -<sect1 id="slonikm4"><title> Using m4 to rewrite slonik scripts </title> - -<para> This needs to be prefaced with something of a warning, from the -GNU M4 documentation: - -<warning><para> Some people found `m4' to be fairly addictive. They -first use `m4' for simple problems, then take bigger and bigger -challenges, learning how to write complex `m4' sets of macros along -the way. Once really addicted, users pursue writing of sophisticated -`m4' applications even to solve simple problems, devoting more time -debugging their `m4' scripts than doing real work. Beware that `m4' -may be dangerous for the health of compulsive -programmers.</para></warning></para> - -<para> This being said, <application>m4</application> has three -significant merits over other text rewriting systems (such as -<application>cpp</application>, the C preprocessor): - -<itemizedlist> - -<listitem><para> Like slonik, m4 uses <quote>#</quote> to indicate -comments, with the result that it may be quietly used to do rewrites -on slonik scripts.</para> - -<para> Using <application>cpp</application> would require changing -over to use C or C++ style comments.</para></listitem> - -<listitem><para> <application> m4 </application> is reasonably -ubiquitous, being available in environments like -<productname>Solaris</productname> and <productname>AIX</productname> -even when they do not have compiler tools for C available. Its -presence is commonly mandated by the presence of -<productname>Sendmail</productname>. -</para></listitem> - -<listitem><para> A <emphasis>potential</emphasis> merit over -<application>cpp</application> is that <application> m4</application> -can do more than just rewrite symbols. It has control structures, can -store data in variables, and can loop.</para> - -<para> Of course, down that road lies the addictions warned of above, -as well as the complexity challenges of -<filename>sendmail.cf</filename>. As soon as you discover you need -things like loops and variables, it is quite likely that you want to -write a slonik generator in your favorite scripting language, whether -that be Bourne Shell, Perl, or Tcl. Fans of more esoteric languages like -Icon, Snobol, or Scheme will have to fight their own battles to get -those deemed to be reasonable choices for <quote>best -practices.</quote></para></listitem> -</itemizedlist></para> - -<sect2><title> An m4 example </title> - -<para> Without further ado, here is an example where you set up a -central file, <filename> cluster.m4 </filename> containing some M4 -rewrite rules: -<programlisting> -define(`node_srvrds005', `1') -define(`node_srvrds004', `4') -define(`node_srvrds003', `3') -define(`node_srvrds007', `78') -define(`ds501', `501') -</programlisting></para> - -<para> In view of those node name definitions, you may write a Slonik -script to initialize the cluster as follows, <filename>setup_cluster.slonik</filename>: - -<programlisting> -node node_srvrds005 admin conninfo 'dsn=foo'; -node node_srvrds004 admin conninfo 'dsn=bar'; -node node_srvrds003 admin conninfo 'dsn=foo'; -node node_srvrds007 admin conninfo 'dsn=foo'; -node ds501 admin conninfo 'dsn=foo'; - -create cluster info (id=node_srvrds005); -store node (id=node_srvrds004, comment='Node on ds004', spool='f'); -store node (id=node_srvrds003, comment='Node on ds003', spool='f'); -store node (id=node_srvrds007, comment='Node on ds007', spool='t'); -store node (id=ds501, comment='Node on ds-501', spool='f'); -</programlisting></para> - -<para> You then run the rewrite rules on the script, thus:</para> -<para> -<command> % m4 cluster.m4 setup_cluster.slonik </command></para> -<para> And receive the following output: -<programlisting> -node 1 admin conninfo 'dsn=foo'; -node 4 admin conninfo 'dsn=bar'; -node 3 admin conninfo 'dsn=foo'; -node 78 admin conninfo 'dsn=foo'; -node 501 admin conninfo 'dsn=foo'; - -create cluster info (id=1); -store node (id=4, comment='Node on ds004', spool='f'); -store node (id=3, comment='Node on ds003', spool='f'); -store node (id=78, comment='Node on ds007', spool='t'); -store node (id=501, comment='Node on ds-501', spool='f'); -</programlisting></para> - -<para> This makes no attempt to do anything <quote>smarter</quote>, -such as to try to create the nodes via a loop that maps across a list -of nodes. As mentioned earlier, if you wish to do such things, it is -highly preferable to do this by using a scripting language like the -Bourne Shell or Perl. - <sect1 id="slonikshell"><title> Embedding Slonik in Shell Scripts </title> <para> As mentioned earlier, there are numerous &slony1; test scripts @@ -251,9 +141,13 @@ <para> The <envar>PREAMBLE</envar> value could then be reused over and over again if the shell script invokes <command>slonik</command> -multiple times. </para> +multiple times. You might also consider using <xref +linkend="stmtinclude"> and place the preamble in a file that is +<command>include</command>d. + </para> -<para> It also becomes simple to assign names to sets and nodes: +<para> Shell variables provide a simple way to assign names to sets +and nodes: <programlisting> origin=1 @@ -331,11 +225,18 @@ scripting language supports things like: <itemizedlist> -<listitem><para> <quote>Record</quote> data structures that allow assigning things in parallel</para></listitem> -<listitem><para> Functions, procedures, or subroutines, allowing you to implement -useful functionality once, and then refer to it multiple times within a script</para></listitem> -<listitem><para> Some sort of <quote>module import</quote> system so that common functionality -can be shared across many scripts</para></listitem> + +<listitem><para> <quote>Record</quote> data structures that allow +assigning things in parallel</para></listitem> + +<listitem><para> Functions, procedures, or subroutines, allowing you +to implement useful functionality once, and then refer to it multiple +times within a script</para></listitem> + +<listitem><para> Some sort of <quote>module import</quote> system so +that common functionality can be shared across many +scripts</para></listitem> + </itemizedlist></para> <para> If you can depend on having <ulink @@ -344,9 +245,9 @@ url="http://www.kornshell.com/"> Korn shell</ulink> available, well, those are all shells with extensions supporting reasonably sophisticated data structures and module systems. On Linux, Bash is -fairly ubiquitous; on commercial <trademark>UNIX</trademark>, Korn shell is -fairly ubiquitous; on BSD, <quote>sophisticated</quote> shells are an -option rather than a default.</para> +fairly ubiquitous; on commercial <trademark>UNIX</trademark>, Korn +shell is fairly ubiquitous; on BSD, <quote>sophisticated</quote> +shells are an option rather than a default.</para> <para> At that point, it makes sense to start looking at other scripting languages, of which Perl is the most ubiquitous, being Index: failover.sgml =================================================================== RCS file: /usr/local/cvsroot/slony1/slony1-engine/doc/adminguide/failover.sgml,v retrieving revision 1.12 retrieving revision 1.13 diff -Ldoc/adminguide/failover.sgml -Ldoc/adminguide/failover.sgml -u -w -r1.12 -r1.13 --- doc/adminguide/failover.sgml +++ doc/adminguide/failover.sgml @@ -60,8 +60,9 @@ switched roles completely.</para></listitem> <listitem><para> After reconfiguring the web application (or -<application>pgpool</application>) to connect to the database on node2, the web -server is restarted and resumes normal operation.</para> +<application><link linkend="pgpool"> pgpool </link></application>) to +connect to the database on node2, the web server is restarted and +resumes normal operation.</para> <para> Done in one shell script, that does the application shutdown, <application>slonik</application>, move config files and startup all --- /dev/null +++ src/ducttape/test_7_defines @@ -0,0 +1,308 @@ +#!/bin/sh + +# ********** +# test_7_defines +# +# This test script creates a standalone pgbench database +# as slony_test1 and then: +# +# - initializes a primary node and starts the node daemon +# - creates a set containing all 4 pgbench tables +# - creates a second database as slony_test2 +# - adds database slony_test2 to the system +# - starts the second replication daemon +# - creates the pgbench tables (schema only) +# - subscribes the replication set from the primary node +# +# The nature of the test has to do with the use of the new slonik +# "define" and "include" commands. +# ********** + +export PATH +TMPOUT=/tmp/output.$$ +DB1=slony_test1 +DB2=slony_test2 + +PGBENCH_SCALE=1 +PGBENCH_CLIENTS=5 +PGBENCH_TRANS=`expr 30000 / $PGBENCH_CLIENTS` + +trap ' + echo "" + echo "**** user abort" + if [ ! -z $pgbench_pid ] ; then + echo "**** killing pgbench" + kill -15 $pgbench_pid + fi + if [ ! -z $slon1_pid ] ; then + echo "**** killing node daemon 1" + kill -15 $slon1_pid + fi + if [ ! -z $slon2_pid ] ; then + echo "**** killing node daemon 2" + kill -15 $slon2_pid + fi + exit 1 +' 2 15 + +###################################################################### +# Preparations ... create a standalone pgbench database and +# have the "application" (pgbench) running. +###################################################################### + +##### +# Make sure the install is up to date +##### +WGM=`which gmake` +if [ -z $WGM ] ; then + MAKE=make + CGNU=`make -v | grep GNU` + if [ -z $CGNU ] ; then + echo "GNU Make not found - please install GNU Make" + exit 1 + fi +else + MAKE=gmake +fi +echo -n "**** running 'make install' in src directory ... " +if ! ${MAKE} -C .. install >$TMPOUT 2>&1 ; then + echo "failed"; cat $TMPOUT; rm $TMPOUT; exit 1 +fi +echo "done" +rm $TMPOUT + +PREAMBLE_FILE=/tmp/preamble.$$ +cat <<EOF > $PREAMBLE_FILE +define origin 11; +define sub1 22; +cluster name = T1; +node @origin admin conninfo='dbname=$DB1'; +node @sub1 admin conninfo='dbname=$DB2'; +EOF + + + +##### +# Remove old databases, if they exist +##### +echo "**** remove old test databases" +dropdb $DB1 || echo "**** ignored" +sleep 1 +dropdb $DB2 || echo "**** ignored" +sleep 1 + +##### +# Create the "Primary Node" +##### +echo "**** creating database for Node 11" + +createdb $DB1 || exit 1 +pgbench -i -s $PGBENCH_SCALE $DB1 +pg_dump -s $DB1 >pgbench_schema.sql + +##### +# Start pgbench in the background and give it rampup time +##### +pgbench -n -s $PGBENCH_SCALE -c $PGBENCH_CLIENTS -t $PGBENCH_TRANS $DB1 & +pgbench_pid=$! +echo "**** pgbench is running in background with pid $pgbench_pid" +echo -n "**** sleeping 10 seconds to give pgbench time for rampup ... " +sleep 10 +echo "done" + +echo "" +echo "**********************************************************************" +echo "**** $DB1 is now a standalone database with a running pgbench" +echo "**********************************************************************" +echo "" + +###################################################################### +# Setup DB1 as the primary cluster T1 node, start the node daemon, +# and create a replication set containing the pgbench tables. +###################################################################### + +echo "**** initializing $DB1 as Primary Node for Slony-I cluster T1" +slonik <<_EOF_ + include <$PREAMBLE_FILE>; + init cluster (id = @origin, comment = 'Node @origin'); + echo 'Database $DB1 initialized as Node 11'; +_EOF_ +if [ $? -ne 0 ] ; then + kill $pgbench_pid; + exit 1 +fi + +echo "**** starting the Slony-I node deamon for $DB1" +xterm -title "Slon node 11" -e sh -c "slon -d2 -s500 -g10 T1 dbname=$DB1; echo -n 'Enter>'; read line" & +slon1_pid=$! +echo "slon[$slon1_pid] on dbname=$DB1" + +echo "**** creating a replication set containing the 4 pgbench tables ... " +slonik <<_EOF_ + include <$PREAMBLE_FILE>; + try { + table add key (node id = @origin, fully qualified name = 'public.history'); + } + on error { + exit 1; + } + + try { + create set (id = 1, origin = @origin, comment = 'Set 1 - pgbench tables'); + set add table (set id = 1, origin = @origin, + id = 1, fully qualified name = 'public.accounts', + comment = 'Table accounts'); + set add table (set id = 1, origin = @origin, + id = 2, fully qualified name = 'public.branches', + comment = 'Table branches'); + set add table (set id = 1, origin = @origin, + id = 3, fully qualified name = 'public.tellers', + comment = 'Table tellers'); + set add table (set id = 1, origin = @origin, + id = 4, fully qualified name = 'public.history', + key = serial, comment = 'Table accounts'); + } + on error { + exit 1; + } +_EOF_ + +if [ $? -ne 0 ] ; then + echo "failed" + kill $pgbench_pid 2>/dev/null + kill $slon1_pid 2>/dev/null + cat $TMPOUT + rm $TMPOUT + exit 1 +fi +echo "**** set created" + +##### +# Check that pgbench is still running +##### +if ! kill -0 $pgbench_pid 2>/dev/null ; then + echo "**** pgbench terminated ???" + kill $slon1_pid 2>/dev/null + exit 1 +fi + +echo "" +echo "**********************************************************************" +echo "**** $DB1 is now the Slony-I origin for set 1" +echo "**********************************************************************" +echo "" + +###################################################################### +# Setup DB2 as a subscriber node and let it subscribe the replication +# set of the running pgbench +###################################################################### +echo "**** creating database for node 22" +if ! createdb $DB2 ; then + kill $pgbench_pid 2>/dev/null + kill $slon1_pid 2>/dev/null + exit 1 +fi + +echo "**** initializing $DB2 as node 22 of Slony-I cluster T1" +slonik <<_EOF_ + include <$PREAMBLE_FILE>; + echo 'Creating node 22'; + try { + store node (id = @sub1, comment = 'node @sub1', event node = @origin); + } on error { + echo 'could not establish node @sub1'; + exit -1; + } + try { + store path (server = @origin, client = @sub1, conninfo = 'dbname=$DB1'); + store path (server = @sub1, client = @origin, conninfo = 'dbname=$DB2'); + } + on error { + echo 'could not establish paths between @origin and @sub1'; + exit -1; + } + echo 'Database $DB2 added as node @sub1'; +_EOF_ +if [ $? -ne 0 ] ; then + kill $pgbench_pid 2>/dev/null + kill $slon1_pid 2>/dev/null + exit 1 +fi + +echo "**** starting the Slony-I node deamon for $DB1" +xterm -title "Slon node 22" -e sh -c "slon -d2 -s10000 -o10000 -g10 T1 dbname=$DB2; echo -n 'Enter>'; read line" & +slon2_pid=$! +echo "slon[$slon2_pid] on dbname=$DB2" + +##### +# Check that pgbench is still running +##### +if ! kill -0 $pgbench_pid 2>/dev/null ; then + echo "**** pgbench terminated ???" + kill $slon1_pid 2>/dev/null + exit 1 +fi + +###################################################################### +# And now comes the moment where the big elephant starts to pee +# and the attendants in the first row climb on their chairs ... +###################################################################### +echo "**** creating pgbench tables and subscribing node 22 to set 1" +( + cat pgbench_schema.sql +) | psql -q $DB2 +slonik <<_EOF_ + include <$PREAMBLE_FILE>; + subscribe set ( id = 1, provider = @origin, receiver = @sub1, forward = yes ); +_EOF_ + +echo "" +echo "**********************************************************************" +echo "**** $DB2 should now be copying data and attempting to catch up." +echo "**********************************************************************" +echo "" + +echo -n "**** waiting for pgbench to finish " +while kill -0 $pgbench_pid 2>/dev/null ; do + echo -n "." + sleep 10 +done +echo "**** pgbench finished" +echo "**** please terminate the replication engines when caught up." +wait $slon1_pid +wait $slon2_pid + +kill $pgbench_pid 2>/dev/null +kill $slon1_pid 2>/dev/null +kill $slon2_pid 2>/dev/null + +echo -n "**** comparing databases ... " +psql $DB1 >dump.tmp.1.$$ <<_EOF_ + select 'accounts:'::text, aid, bid, abalance, filler + from accounts order by aid; + select 'branches:'::text, bid, bbalance, filler + from branches order by bid; + select 'tellers:'::text, tid, bid, tbalance, filler + from tellers order by tid; + select 'history:'::text, tid, bid, aid, delta, mtime, filler, + "_Slony-I_T1_rowID" from history order by "_Slony-I_T1_rowID"; +_EOF_ +psql $DB2 >dump.tmp.2.$$ <<_EOF_ + select 'accounts:'::text, aid, bid, abalance, filler + from accounts order by aid; + select 'branches:'::text, bid, bbalance, filler + from branches order by bid; + select 'tellers:'::text, tid, bid, tbalance, filler + from tellers order by tid; + select 'history:'::text, tid, bid, aid, delta, mtime, filler, + "_Slony-I_T1_rowID" from history order by "_Slony-I_T1_rowID"; +_EOF_ + +if diff dump.tmp.1.$$ dump.tmp.2.$$ >test_1.diff ; then + echo "success - databases are equal." + rm dump.tmp.?.$$ + rm test_1.diff +else + echo "FAILED - see test_1.diff for database differences" +fi +rm $PREAMBLE_FILE Index: scan.l =================================================================== RCS file: /usr/local/cvsroot/slony1/slony1-engine/src/slonik/scan.l,v retrieving revision 1.22 retrieving revision 1.23 diff -Lsrc/slonik/scan.l -Lsrc/slonik/scan.l -u -w -r1.22 -r1.23 --- src/slonik/scan.l +++ src/slonik/scan.l @@ -10,6 +10,30 @@ * $Id$ *------------------------------------------------------------------------- */ +struct __yy_buffer +{ + YY_BUFFER_STATE buffer; /* lexer buffer to restore on pop */ + long lineno; /* line number to restore on pop */ + char * fileName; /* file name to restore on pop */ + struct __yy_buffer * prev; /* pointer to previous stack frame */ +} * yy_buffer = NULL; + +typedef struct _symbol +{ + char * name; /* Name of symbol with % prepended */ + char * value; /* Value of symbol */ + struct _symbol * next; /* Pointer to next symbol */ +} symbol; + +static symbol * symbols; /* Head of list of symbols */ + +static char * getSymbol( const char * name ); /* Return a symbol's value (or NULL)*/ +static void addSymbol( char * str ); /* Add a new symbol */ +static void freeSymbols( void ); /* Free all symbols */ +static void pushBuffer( char * context ); /* Push a new lexer buffer on stack */ +static void popBuffer( void ); /* Pop previous lexer buffer */ + +extern char * current_file; %} %option 8bit @@ -18,12 +42,14 @@ %option yylineno %option case-insensitive +%x incl define %x IN_STRING digit [0-9] ident_start [A-Za-z\200-\377_] ident_cont [A-Za-z\200-\377_0-9\$] space [ \t\n\r\f] +quote ' /* quoted_ident (\"[^\"]*\")+ @@ -37,6 +63,9 @@ BEGIN(INITIAL); +include{space}* { BEGIN(incl); } +define{space}* { BEGIN(define); } + add { return K_ADD; } admin { return K_ADMIN; } all { return K_ALL; } @@ -126,6 +155,44 @@ } <IN_STRING>[^'\\]+ {} +<define>{identifier}{space}+.*";" { addSymbol( yytext ); BEGIN(INITIAL); } + +@{identifier} { + char * value = getSymbol( yytext ); + + if( value ) + { + pushBuffer( strdup( current_file )); + yy_scan_string( value ); + } + } + +<incl>\<[^\>]+\>{space}*";"? { + + char * fileName = strdup( yytext + 1 ); /* Skip '<' */ + + *strchr( fileName, '>' ) = '\0'; /* Trim '>' */ + + pushBuffer( fileName ); + + if(( yyin = fopen( fileName, "r" )) == NULL ) + { + fprintf( stderr, "Include file (%s) not found\n", fileName ); + exit( 1 ); + } + + yy_switch_to_buffer( yy_create_buffer( yyin, YY_BUF_SIZE )); + + BEGIN(INITIAL); + } + +<<EOF>> { + if( yy_buffer == NULL ) + yyterminate(); + else + popBuffer(); + } + #[^\r\n]* ; . { return yytext[0]; } @@ -151,8 +218,108 @@ yy_switch_to_buffer(yy_scan_string(dstring_data(&ds))); dstring_free(&ds); + + freeSymbols(); } +static void pushBuffer( char * context ) +{ + struct __yy_buffer * yb = malloc( sizeof( *yb )); + + yb->buffer = YY_CURRENT_BUFFER; + yb->lineno = yylineno; + yb->fileName = strdup( current_file ); + yb->prev = yy_buffer; + + yy_buffer = yb; + + current_file = context; + yylineno = 1; +} + +static void popBuffer( void ) +{ + struct __yy_buffer * yb = yy_buffer; + + if( yyin != NULL ) + fclose( yyin ); + + yy_delete_buffer( YY_CURRENT_BUFFER ); + yy_switch_to_buffer( yy_buffer->buffer ); + + free( current_file ); + + current_file = yy_buffer->fileName; + yylineno = yy_buffer->lineno; + + yy_buffer = yy_buffer->prev; + + free( yb ); +} + +static void addSymbol( char * str ) +{ + char * name = str; + char * value = str; + symbol * sym = NULL; + + while( *value != ' ' && *value != '\t' ) + value++; + + *(value++) = '\0'; + + while( *value == ' ' || *value == '\t' ) + value++; + + value[strlen(value) - 1 ] = '\0'; + + sym = malloc( sizeof( *sym )); + sym->value = strdup( value ); + sym->next = NULL; + + /* Store the symbol name in searchable form with a leading @ */ + + sym->name = malloc( strlen( name ) + 1 + 1 ); + + sym->name[0] = '@'; + strcpy( sym->name+1, name ); + + if( symbols != NULL ) + sym->next = symbols; + + symbols = sym; +} + +static char * getSymbol( const char * name ) +{ + symbol * sym; + + for( sym = symbols; sym; sym = sym->next ) + { + if( strcmp( name, sym->name ) == 0 ) + return( sym->value ); + } + + return( NULL ); +} + +static void freeSymbols( void ) +{ + symbol * sym = symbols; + + while( sym ) + { + symbol * victim = sym; + + sym = sym->next; + + free( victim->name ); + free( victim->value ); + free( victim ); + } + + symbols = NULL; +} /* * Local Variables: * tab-width: 4
- Previous message: [Slony1-commit] By cbbrowne: There's a minor typo in src/slon/confoptions.h that
- Next message: [Slony1-commit] By cbbrowne: Major changes to log shipping, allowing it to support
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the Slony1-commit mailing list