<br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 29, 2012 at 1:31 PM, Scott Marlowe <span dir="ltr"><<a href="mailto:scott.marlowe@gmail.com" target="_blank">scott.marlowe@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Nov 29, 2012 at 2:27 PM, Tory M Blue <<a href="mailto:tmblue@gmail.com">tmblue@gmail.com</a>> wrote:<br>
<br>
>>> > From: Tory M Blue <<a href="mailto:tmblue@gmail.com">tmblue@gmail.com</a> <mailto:<a href="mailto:tmblue@gmail.com">tmblue@gmail.com</a>>><br>
>>><br>
>>> >To: slony1-general <<a href="mailto:slony1-general@lists.slony.info">slony1-general@lists.slony.info</a><br>
>>> <mailto:<a href="mailto:slony1-general@lists.slony.info">slony1-general@lists.slony.info</a>>><br>
>>> >Sent: Wednesday, 28 November 2012, 18:35<br>
>>> >Subject: [Slony1-general] data copy for set 1 failed 3 times -<br>
>>> sleep 60 seconds<br>
>>> ><br>
>>> ><br>
>>> >Greetings<br>
>>> ><br>
>>> >I've just brought up a replication node across the state, we have<br>
>>> gig circuits, but still going over the net over a vpn tunnel. So<br>
>>> there is some delay,<br>
>>> ><br>
>>> >I'm getting these errors and can't get the initial replication to<br>
>>> finish<br>
>>> ><br>
>>> >4273435:2012-11-27 16:45:17 PST WARN remoteWorkerThread_1: data<br>
>>> copy for set 1 failed 3 times - sleep 60 seconds<br>
>>> ><br>
>>><br>
>>> Is that the only line you get in your logs? If so use a higher<br>
>>> setting for log_level (like log_level=4).<br>
>>><br>
>>> Also is there anything in the postgresql log to indicate the problem?<br>
>>><br>
>>><br>
>>> I get the following<br>
>>><br>
>>> 2012-11-29 08:34:38 PST CONFIG remoteWorkerThread_1: 3858.988 seconds to<br>
>>> copy table "cls"."listings"<br>
>>> 2012-11-29 08:34:38 PST CONFIG remoteWorkerThread_1: copy table<br>
>>> "cls"."customers"<br>
>>> 2012-11-29 08:34:38 PST CONFIG remoteWorkerThread_1: Begin COPY of table<br>
>>> "cls"."customers"<br>
>>> 2012-11-29 08:34:38 PST ERROR remoteWorkerThread_1: "select<br>
>>> "_admissioncls".copyFields(8);"<br>
>>><br>
>><br>
>> What is the structure of the table with table_id=8 (ie set add table(id=8,<br>
>> fully qualified name=?????)<br>
>><br>
>> If you do<br>
>> manually run<br>
>> select _admissioncls".copyFields(8);<br>
>> from psql, what does it come back with?<br>
>><br>
>><br>
> So ran it again with debug = 4, same failure same spot. Should be be dying<br>
> due to the logswitch, but I guess it could be?<br>
><br>
> Failed again, this time with heavier debug, but not showing me much (level<br>
> 4)<br>
> 1235574-2012-11-29 12:22:12 PST CONFIG remoteWorkerThread_1: Begin COPY of<br>
> table "cls"."customers"<br>
> 1235665-2012-11-29 12:22:12 PST ERROR remoteWorkerThread_1: "select<br>
> "_admissioncls".copyFields(8);"<br>
> 1235759:2012-11-29 12:22:12 PST WARN remoteWorkerThread_1: data copy for<br>
> set 1 failed 1 times - sleep 15 seconds<br>
> Followed sometime later by this<br>
> 2012-11-29 12:22:28 PST DEBUG2 remoteWorkerThread_2: forward confirm<br>
> 3,5001168772 received by 4<br>
> 2012-11-29 12:22:28 PST INFO copy_set 1 - omit=f - bool=0<br>
> 2012-11-29 12:22:28 PST INFO omit is FALSE<br>
> And it starts all over again.<br>
><br>
> Postgres logs<br>
><br>
> 2012-11-29 12:19:40 PST HINT: Consider increasing the configuration<br>
> paramete<br>
> r "checkpoint_segments".<br>
> 2012-11-29 12:22:13 PST admissionclsdb postgres [local] NOTICE: Slony-I:<br>
> Logswi<br>
> tch to sl_log_2 initiated<br>
> 2012-11-29 12:22:13 PST admissionclsdb postgres [local] CONTEXT: SQL<br>
> statement<br>
> "SELECT "_admissioncls".logswitch_start()"<br>
> PL/pgSQL function "cleanupevent" line 96 at PERFORM<br>
><br>
> Nothing in /var/log/messages or dmesg, this is not a system thing it doesn't<br>
> appear. Slony configurations sync rate, or something is putting the kabosh<br>
> on this.<br>
<br>
Is there anything useful in the postgresql logs from the same time?<br>
Like maybe lost connections or crashing backends?<br></blockquote><div><br>Nothing, the postgres logs are clean, other than the checkpoint warnings, nothing, really absolutely nothing. This appears to be slony centric.<br>
<br>The only other thought on this, is that this remote db is 9.1.6 and the local (master) is 9.1.4, but same slony versions 2.1.1. However I and the world seem to upgrade our databases using this method, so that can't be it.<br>
<br>I don't know, not sure if there are some slony tuning I should be making due to the extra delay.<br><br>Tory <br></div></div><br></div>