Rod Taylor pg
Wed May 31 07:01:26 PDT 2006
I've been thinking of the initial COPY process.

The problem is that with a large amount of data you end up with a very
large transaction on the data provider. The transaction on the
subscriber isn't as important since it will normally be an otherwise
idle database.

COPY in is one part, but building indexes on the subscriber is the
painful part and during much of this process the data provider has an
idle connection.


I haven't looked at the log shipping support, but I suspect it holds
part of the answer. The data provider does not need to be active while
the new subscriber is catching up to the transaction log when log
shipping is enabled.

Is there any reason you could not convert between a shipped log and the
standard "online" mode?

If possible, that would reduce the initial transaction size on the data
provider to the time required to dump out the initial copy of the
tables. Still a significant amount of time, one transaction per table
would be better, but much shorter than today.


A first approximation of a transition between log shipping and standard
modes would be to write the data to a temporary area on the data
provider machine (avoid network overhead) and later on send it across
the wire without being connected to the data provider database.

-- 




More information about the Slony1-general mailing list