<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Apr 2, 2014 at 4:33 PM, Jeff Frost <span dir="ltr"><<a href="mailto:jeff@pgexperts.com" target="_blank">jeff@pgexperts.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=""><br>
On Apr 2, 2014, at 1:28 PM, Christopher Browne <<a href="mailto:cbbrowne@afilias.info">cbbrowne@afilias.info</a>> wrote:<br>
<br>
> It's taking a thousand-ish seconds to process ~200K inserts/updates/deletes, which doesn't seem ludicrously out of line with what I'd expect.<br>
><br>
> It doesn't seem likely to me that the amount of memory that you have is terribly relevant to performance; the processing of a stream of 200K-ish I/U/Ds won't be RAM-hungry, it's mostly hungry in:<br>
> a) Chewing CPU for the parsing and planning of each statement;<br>
> b) Chewing disk I/O for the processing of the I/U/Ds and logging updates in WAL.<br>
><br>
<br>
</div>The disk hardware is pretty zippy and showing no signs of iowait, but it's definitely burning up the CPU and there's no way to thread that up to use more cores in current versions, right? Even if we split the tables across multiple sets, do they still process in serial?<br>
<div class=""><br></div></blockquote><div>Yep, they need to process in serial. Not much way to "improve" on that.</div><div><br></div><div>An attempt was made to parallelize processing in the eRServer replication system, a predecessor to Slony, and we had to shut that off in practice.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">
> I would expect Slony version 2.2 to be a fair bit quicker, as it uses COPY protocol to copy the data in, which dramatically reduces the amount of effort that the subscriber server needs to do parsing and planning the SQL for the INSERT/UPDATE/DELETE statements.<br>
<br>
</div>That was going to be my next question!<br>
<br>
Does that help just for the inserts or with the updates/deletes as well?<br>
</blockquote></div><br></div><div class="gmail_extra">It'll help for all three, yep. All going into one stream.</div></div>