<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi all,<br>
<br>
PostgreSQL v 9.1.5 - 9.1.6<br>
Slony version 2.1.0<br>
<br>
I'm having an issue that's occurred twice now. I have 4 node slony
cluster, and one of the operations is to drop a node from
replication, do maintenance on it, then add it back to replication.
<br>
<br>
Node 1 = master<br>
Node 2 = slave <br>
Node 3 = slave -> dropped then readded<br>
Node 4 = slave<br>
<br>
I dropped node 2 from replication, then a while later added it back.
I have my own set of scripts that add it to replication all
automatically. They have been used dozens of times over the past few
months without issue. But recently, I've had the issue where after
adding back the node (in this example, node 3), node 4 suddenly
stops syncing. I've attached a snip of the log for node 4 in this
email. The main thing I'm seeing is this:<br>
<br>
Node 4 (the one that stopped syncing) was proccessing the following
SYNC events<br>
5000030355<br>
5000030357<br>
5000030358<br>
5000030359<br>
<br>
Then at this point it receives the event recognizing the new node
addition to the slony cluster.<br>
<br>
then we have the error where it's attempting to insert the event
5000000032 (see log attached).<br>
<br>
Are these sequential event id's supposed to be related? <br>
<br>
I looked at the sl_event on the node 4 box that's erroring, and did
the following query:<br>
<br>
select now() - ev_timestamp, ev_seqno from _slony.sl_event where
ev_origin = 4 order by 1;<br>
(how long ago was this event <br>
<font face="monospace"> ?column? | ev_seqno<br>
-----------------+------------<br>
01:48:30.7277 | 5000000031<br>
01:48:40.731095 | 5000000030<br>
01:48:50.734454 | 5000000029<br>
01:49:00.735438 | 5000000028<br>
01:49:10.728117 | 5000000027<br>
01:49:20.731387 | 5000000026<br>
01:49:30.734199 | 5000000025<br>
01:49:40.726768 | 5000000024<br>
01:49:50.730152 | 5000000023<br>
01:50:00.732747 | 5000000022<br>
01:50:10.736126 | 5000000021<br>
01:50:20.737394 | 5000000020<br>
01:50:30.733839 | 5000000019<br>
01:50:40.736928 | 5000000018<br>
01:50:50.739931 | 5000000017<br>
01:51:00.742365 | 5000000016<br>
01:51:10.745724 | 5000000015<br>
01:51:20.748913 | 5000000014<br>
01:51:30.751388 | 5000000013<br>
01:51:40.754439 | 5000000012<br>
01:51:50.757796 | 5000000011<br>
01:52:00.760079 | 5000000010<br>
01:52:10.763595 | 5000000009<br>
01:52:20.762106 | 5000000008<br>
01:52:30.764838 | 5000000007<br>
01:52:40.768154 | 5000000006<br>
01:52:50.771847 | 5000000005<br>
01:53:00.775925 | 5000000004<br>
01:53:38.2839 | 5000000003<br>
01:53:38.422047 | 5000000002<br>
01:53:38.435689 | 5000000001<br>
01:55:29.173397 | 5000000032</font><br>
<br>
NOTE: At the moment I took this snapshot, the st_lag_time for this
node was '01:55:28.411043', which falls between the last two
timestamps above.<br>
<br>
What strikes me as 'interesting' is that 5000000032 (the one it's
erroring on) happened way before the rest of them timestamp wise. So
I'm thinking that perhaps it was leftover from the previous 'node'
before I dropped it, and it was never cleaned up? <br>
<br>
The last time this happened, I deleted that offending row myself
(desperate, i know to usually never touch the slony schema data),
and it actually did completely break slony replication. It's a
smaller set so wasn't too hard to set up again from scratch.<br>
<br>
So I'm going to do some testing to see if it's due to lack of
cleanup, but thought I'd throw this out there. Is there anything
else that could possibly cause this?<br>
<br>
thanks in advance,<br>
- Brian F<br>
<br>
</body>
</html>