On Apr 2, 2014, at 1:28 PM, Christopher Browne <cbbrowne at afilias.info> wrote:

> It's taking a thousand-ish seconds to process ~200K inserts/updates/deletes, which doesn't seem ludicrously out of line with what I'd expect.
> 
> It doesn't seem likely to me that the amount of memory that you have is terribly relevant to performance; the processing of a stream of 200K-ish I/U/Ds won't be RAM-hungry, it's mostly hungry in:
> a) Chewing CPU for the parsing and planning of each statement;
> b) Chewing disk I/O for the processing of the I/U/Ds and logging updates in WAL.
> 

The disk hardware is pretty zippy and showing no signs of iowait, but it's definitely burning up the CPU and there's no way to thread that up to use more cores in current versions, right?  Even if we split the tables across multiple sets, do they still process in serial?

> I would expect Slony version 2.2 to be a fair bit quicker, as it uses COPY protocol to copy the data in, which dramatically reduces the amount of effort that the subscriber server needs to do parsing and planning the SQL for the INSERT/UPDATE/DELETE statements.



That was going to be my next question!

Does that help just for the inserts or with the updates/deletes as well?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 208 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://lists.slony.info/pipermail/slony1-general/attachments/20140402/4b3102f6/attachment.pgp