Fri Apr 29 14:50:56 PDT 2011
- Previous message: [Slony1-general] Slony I download & INstllation
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
I decided to start taking a poke at this, as time seems permitting, at
least for now. I don't anticipate this being ready for 2.1, but
hopefully some strategy can be arrived at, and problems worked thru
now, rather than having to wait until later.
The problem noticed was that, with the enormously decreased locking,
in 2.0, of DDL, some race conditions have emerged. They wouldn't
generally bite us at Afilias, as we have the practice of taking
outages before DDL goes in, so that we prevent the "race" by virtue of
outage. But one of the regression tests is rather more aggressive,
having data changes trying to go in until the last possible instant.
And so you can have the scenario where the DDL "sync" has I/U/D
changes interspersed with DDL changes, and no way to get the right
ordering.
Two "bits" of change seem apropos to rectify this:
a) We allow the DDL script to specify what tables it needs to lock.
Before applying the DDL against the origin, the connection submits the
lock request. The tables being locked, no more data is going to get
in edge-wise into those tables until after we apply the DDL. This
still hasn't guaranteed ordering, though, hence part b)...
b) Rather than stowing the DDL inside the event, the DDL gets
dribbled, statement by statement, into sl_log_{1|2}.
Now, as I'm reviewing the code, it looks a whole lot like this
strategy makes the notion of a "DDL event" go away.
As I reviewed bits of the implementation, more and more sections of
the DDL handling code seemed to become irrelevant, to the dual points
that...
1. I'm wondering if I'm crazy in thinking that the DDL_SCRIPT event
might be able to be eliminated.
2. If I'm not crazy, then boy, this seems to simplify the
implementation of, well, "everything about DDL."
What seems to disappear are:
a) The code for the DDL_SCRIPT event
b) Capturing anything at all in sl_event, except just as an informational thing.
c) pl/pgsql functions ddl_prepare_script_int(), ddlscript_complete()
d) Running the parser in src/parsestatements/scanner.c as part of
slon. It's still needed in slonik to split apart the statements, but
once split, and stowed in sl_log_*, no more splitting needed.
What remains...
slonik needs to:
a) Lock the tables
b) Run ddlscript_prepare(), which just does some validation, and sets
replication mode to 'local'
c) Split the DDL into statements
d) Prepare a statement to drop DDL into sl_log_*
e) For each statement
e.1) Run the DDL on the origin
e.2) Save DDL into sl_log_*
f) Restore replication mode (probably to 'origin')
slon, today, does a great deal of work, but it basically all gets dropped.
- The DDL event doesn't need to do anything.
- No need to split statements up.
Basically, when processing sl_log_*, we'll sometimes have DDL
statements, so that today's (Insert,Update,Delete,Truncate) augments
to (Insert,Update,Delete,Truncate,Script), and we basically just need
to enclose the Script items with set/restore replication mode.
Still apparent complications:
- ONLY ON NODE functionality basically just connects to the specified
node, and omits saving the DDL in sl_log_*, right? That's a small
change to slonik, and requires *nothing* of slon, right?
- Log shipping... Nothing special; we're capturing sl_log_* output;
DDL is just an addition to this.
This *looks* like it turns into a considerable code simplification,
which would be mighty nice, if so.
- Previous message: [Slony1-general] Slony I download & INstllation
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the Slony1-general mailing list