On Wed, Apr 2, 2014 at 4:33 PM, Jeff Frost <jeff at pgexperts.com> wrote:

>
> On Apr 2, 2014, at 1:28 PM, Christopher Browne <cbbrowne at afilias.info>
> wrote:
>
> > It's taking a thousand-ish seconds to process ~200K
> inserts/updates/deletes, which doesn't seem ludicrously out of line with
> what I'd expect.
> >
> > It doesn't seem likely to me that the amount of memory that you have is
> terribly relevant to performance; the processing of a stream of 200K-ish
> I/U/Ds won't be RAM-hungry, it's mostly hungry in:
> > a) Chewing CPU for the parsing and planning of each statement;
> > b) Chewing disk I/O for the processing of the I/U/Ds and logging updates
> in WAL.
> >
>
> The disk hardware is pretty zippy and showing no signs of iowait, but it's
> definitely burning up the CPU and there's no way to thread that up to use
> more cores in current versions, right?  Even if we split the tables across
> multiple sets, do they still process in serial?
>
> Yep, they need to process in serial.  Not much way to "improve" on that.

An attempt was made to parallelize processing in the eRServer replication
system, a predecessor to Slony, and we had to shut that off in practice.


> > I would expect Slony version 2.2 to be a fair bit quicker, as it uses
> COPY protocol to copy the data in, which dramatically reduces the amount of
> effort that the subscriber server needs to do parsing and planning the SQL
> for the INSERT/UPDATE/DELETE statements.
>
> That was going to be my next question!
>
> Does that help just for the inserts or with the updates/deletes as well?
>

It'll help for all three, yep.  All going into one stream.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.slony.info/pipermail/slony1-general/attachments/20140402/7cf16c49/attachment.htm