Hi,<br><br>I have attempted to investigate further into why the failover/drop node is not being picked up on node 3. Here is the actual output of the slonik script in my original post:<br><br>[oper@backup slonik]$ slonik forceProviderChangeToBackup.sk <br>
INFO: calling failedNode(1,2) on node 1<br>forceProviderChangeToBackup.sk:9: NOTICE: failedNode: set 1 has other direct receivers - change providers only<br>forceProviderChangeToBackup.sk:9: NOTICE: failedNode: set 2 has no other direct receivers - move now<br>
forceProviderChangeToBackup.sk:9: NOTICE: failedNode: set 3 has no other direct receivers - move now<br>INFO: calling failedNode(1,2) on node 3<br>forceProviderChangeToBackup.sk:9: NOTICE: failedNode: set 1 has other direct receivers - change providers only<br>
forceProviderChangeToBackup.sk:9: NOTICE: failedNode: set 2 has no other direct receivers - move now<br>forceProviderChangeToBackup.sk:9: NOTICE: failedNode: set 3 has no other direct receivers - move now<br>INFO: Waiting for slon engines to restart<br>
IMPORTANT: Last known SYNC for set 1 = 383<br>INFO: Node with highest sync for set 1 is 2<br>INFO: Node with highest sync for set 2 is 2<br>INFO: Node with highest sync for set 3 is 2<br><br>After the inspecting the logfile generated by the slon process at node 3 and it seems to pick up on the fact that the set has been moved to node 2, but it does not remove node 1.<br>
<br>DEBUG2 remoteWorkerThread_2: Received event 2,180 ACCEPT_SET<br>DEBUG2 start processing ACCEPT_SET<br>DEBUG2 ACCEPT: set=1<br>DEBUG2 ACCEPT: old origin=1<br>DEBUG2 ACCEPT: new origin=2<br>DEBUG2 ACCEPT: move set seq=384<br>
DEBUG2 got parms ACCEPT_SET<br>DEBUG2 ACCEPT_SET - node not origin<br>DEBUG2 remoteListenThread_2: queue event 2,183 SYNC<br>DEBUG2 remoteListenThread_2: queue event 2,184 DROP_NODE<br>DEBUG2 remoteListenThread_2: queue event 2,185 SYNC<br>
DEBUG2 ACCEPT_SET - MOVE_SET or FAILOVER_SET not received yet - sleep<br>ERROR slon_connectdb: PQconnectdb("dbname=db host=node1 port=5432 user=postgres") failed - could not connect to server: Connection refused<br>
Is the server running on host "node 1" and accepting<br> TCP/IP connections on port 5432?<br>WARN remoteListenThread_1: DB connection failed - sleep 10 seconds<br>DEBUG2 syncThread: new sl_action_seq 1 - SYNC 181<br>
DEBUG2 remoteListenThread_2: LISTEN<br>DEBUG2 remoteListenThread_2: queue event 2,186 SYNC<br>DEBUG2 remoteListenThread_2: UNLISTEN<br>DEBUG2 ACCEPT_SET - MOVE_SET or FAILOVER_SET not received yet - sleep<br>DEBUG2 localListenThread: Received event 3,181 SYNC<br>
ERROR slon_connectdb: PQconnectdb("dbname=db host=node1 port=5432 user=postgres") failed - could not connect to server: Connection refused<br> Is the server running on host "node 1" and accepting<br>
TCP/IP connections on port 5432?<br><br><br>Does the below line mean it is waiting for some kind of notification from somewhere? : <br> DEBUG2 ACCEPT_SET - MOVE_SET or FAILOVER_SET not received yet - sleep<br><br>
Additionally, does anyone know how to make the slon logs contain a timestamp (e.g. DEBUG2 [2009-01-14 12:12] syncThread), as I find it pretty hard to follow what is going on when comparing the log files at multiple nodes.<br>
<br>Cheers,<br><br>Andy<br><br><div class="gmail_quote">2010/1/13 Andy Dale <span dir="ltr"><<a href="mailto:andy.dale@gmail.com">andy.dale@gmail.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi,<br><br>I have set up a simple 3 node slony cluster, and every thing works pretty much as I would expect, however I am running into a few issue when using the drop node (in a failover scenario).<br><br>I have a simple slonik script to perform a failover as follows (node 1 is the old master node to removed):<span><br>
<br><span style="font-family: courier new,monospace;">#!/usr/bin/slonik</span><br style="font-family: courier new,monospace;"><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">include <<a href="http://preamble.sk" target="_blank">preamble.sk</a>>;</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"># hard failover to the backup system</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">failover (id = 1, backup node = 2);</span><br style="font-family: courier new,monospace;">
<br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"># purge out the opersystem node</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">drop node (id = 1, event node = 2);</span><br>
<br><br>The purges node 1 from the current cluster, the updated cluster is the correct at node 2 (sl_node has node 2 and 3), but node 3 does not get/apply the drop node command (sl_node still has nodes 1, 2, 3).<br><br>Looking at the slon log file on node 3 it is still trying to connect to node 1, and I do not understand why the cluster topology change has not been detected on node 3.<br>
<br>Does anyone have any suggestions as to what the problem might be ? I am using Slony 1.2.16<br><br>Many thanks,<br><br>Andy<br></span>
</blockquote></div><br>