Hi Steve,<br><br>After more troubleshooting and several times reconfiguration whole system, it seems the issue should be in master node configuration. However, I still haven't figured out the root cause and reproduce step. This is not reproduced every time. <br>
<br>In the normal correct case, after configure master node, there has consistent SYNC events which will be generated. However, in the error case, looks like master node slon has hung and there is no new SYNC events generated. <br>
<br>Do you have any more insight on the potential issue which might happen during master node configuration? <br><br>Thanks,<br>Jason<br><br><br><div class="gmail_quote">On Fri, Oct 29, 2010 at 11:05 AM, Jason Chen <span dir="ltr"><<a href="mailto:yunfeng82@gmail.com">yunfeng82@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hi Steve,<br><br>Thanks for your response. During those 7 hours, actually I have done nothing but just leave it there. The sl_path will be correctly updated after that time. Another workaround to solve this issue is restart master node slon service which will also update sl_path to the correct value in pa_conninfo. <br>
<br>I have turned on the highest level log on both slon and postgresql but didn't find more useful information. In the normal case, pa_conninfo in sl_path will be event pending only for a short time and then will be updated correctly after master generates STORE_PATH event. However, in the error case, looks like master will not generates STORE_PATH event. <br>
<br>What will be case that cause master node hangs and cannot generate STORE_PATH event? Is there any more debug information I can check in gdb for the slon process? <br><br>Thanks,<br><font color="#888888">Jason</font><div>
<div></div><div class="h5"><br><br><br><div class="gmail_quote">
On Fri, Oct 29, 2010 at 2:05 AM, Steve Singer <span dir="ltr"><<a href="mailto:ssinger@ca.afilias.info" target="_blank">ssinger@ca.afilias.info</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
My question is what were the slons doing during those 7 hours.<br>
<br>
If you configure your slon processes to log at the debug level they should print a fair amount of stuff.<br>
<br>
You get the <event pending> entries in sl_path when you subscribe the set before processing the STORE_PATH message on the other node.<br>
<br>
What maybe want to do is move define_replication_set to come after you've started up the slons.<br>
<br>
Having said that even if you do things in the order you described things still should have worked and I don't see why it took 7 hours to update sl_path. What were the slons doing during those 7 hours.<br>
<br></blockquote></div><br>
</div></div></blockquote></div><br>