> On Monday November 8 2004 8:59, Jan Wieck wrote:
>> On 11/8/2004 9:58 PM, Ed L. wrote:
>> > I just had a 1.0.5 initial "copy set" fail after 2 hours of data
>> > copying. This was not slony's fault; the slon process got clobbered.
>> > Unfortunately, it rolled back all 2 hours of progress.  I don't have a
>> > patch to offer for this, but it would be nice to be able to commit
>> > partially thru the copy set and pick up where it left off ...
>>
>> Right, it would be nice. But I failed to figure out how to do this
>> without relying on the masters MVCC features while at the same time
>> ensuring that restarting against a possibly all the time heavily updated
>> master would still lead to a consistent initial copy. I don't expect a
>> patch ... I would settle for an idea.
>
> I wonder if you could somehow take note of a progress checkpoint after
> some
> amount of time on the nearest "truncateTable(...); copy table" boundary
> after the amount of time elapses?  Maybe sl_table.tab_initialized = false
> until each table is committed on the slave, then sl_table.tab_initialized
> := 'true'?  Then, if lights go out, you pick up with slave listing of
> tables where sl_table.tab_initialized = false before moving on to events.
> Not sure that qualifies as a usable idea.

Alas, no...  The problem is in determining which table updates since the
COPY SET started need to be applied to the subscriber.

If everything is done as one transaction, the answer is simple:  All of it.

Elsewhere lies madness, or at least something similar to madness.

What would be nice, if you could get away with it, would be to split a set
into (say) one for each table, so that while there would need to be a
whole bunch of subscriptions, they would be on much more "byte-sized"
components.    You'd merge the sets together in the end.

In 1.0.5, there is slonik support for splitting tables off an existing
set, so that you could go through, table by table, creating a new set for
each large table, and thereby subscribe a new node "in pieces."  That
would address your desire about as well as it gets.