On Fri, Sep 25, 2009 at 5:36 PM, roctaiwan <nettreeinc at gmail.com> wrote:

> I am bulkloading (copying)about 10million of records (total size about 1G)
> with replication runing. Takes me about 2h30m to load whole data into
> Master, but after that I give it like another an hour or so to load into my
> slave ( I only have one slave), but zero record got inserted. I then give it
> overnight to do its work, which gives almost 20 hours and still nothing
> happen. I examing the log on Slave and seem no obvious activity has sigh of

I find it amusing to "select datname,current_query from
pg_stat_activity where usename='slony'" on my slave to see what it is
up to when I've got a large backlog to get through. Also looking at
the size and a arbitrary records in _clustername.sl_log_1 and
_clustername.sl_log_2.

> data inserting. (I have attached part of my log)

no you didn't, or something swallowed it ;)

> I do see slony replication behave as records won't show up in slave table
> until all records got processed then it will load WHOLE records all at once
> into the table. I observe this by monitoring slave log.

If you add all the records in a single transaction, they are added on
the slave as a single transaction too.

> This makes me think, before records load on the table physically and while
> data is in processing, are they temporary storted in the memory and waiting
> to be load into the table once done with read records from copying?

No - pending changes are serialized in a PostgreSQL table. Have a look
at what is in _yourclustername.sl_log_1 and_youclustername.sl_log_2.

> Here is my concern. I copying both 10k and 1million of records with success,
> but why can't 10Million records? Is this has something to do with my memory
> size, which loading 10Million of records is too much for my server to
> handle?
>
> I have 512MB memory on both master and slave.





-- 
Stuart Bishop <stuart at stuartbishop.net>
http://www.stuartbishop.net/