Richard Yen wrote:
> On Nov 18, 2008, at 2:22 PM, Christopher Browne wrote:
>>
>> Haven't had the Round Tuits to get to it; if you could provide the
>> beginnings of it, that would make it easier to find the (hopefully
>> fewer, if effort is shared!) hours of implementation effort.
>
> Well, I've put some effort into getting finishTableAfterCopy and 
> ANALYZE into a separate thread.  The problem that we run into is that 
> the entire copy_set is under one transaction.  This means three things:
>
> 1.  Entries in sl_table are not visible to other connections to the 
> database, so tab_id doesn't exist when you try to run 
> finishTableAfterCopy
> 2.  If you provide full name of a table, reindex and analyze don't do 
> anything because you're doing this on stale data.
> 3.  If you put each iteration of setAddTable, prepareTableForCopy, and 
> COPY under remote_worker.c:2906 in its own transaction, if anything 
> fails, we can't roll back, so the old data is all gone.  This implies 
> that the reindex and analyze need either to happen *in* the 
> transaction or *after* everything is done.
>
> 1 and 2 are expected, but I wonder if you guys have anything to say 
> about 3.  I'm probably going to be speaking some serious DBA-heresy 
> here, but the way I see it, if you're going to turn a node into a 
> subscriber, you don't care about whether the data gets lost anyways.  
> Therefore, we *do* put setAddTable, prepareTableForCopy and COPY in 
> one transaction, with the reindex and analyze in a separate 
> transaction, even if anything fails, we'll still be okay.  Do you have 
> anything to say to this?
>
> Are there any other suggestions?
#3 comes as no great surprise.

If you try to "multithread" on the subscriber, then it comes as no 
surprise that this needs to represent several transactions on the 
subscriber, and that failures could potentially take place in *any* of 
those transactions.

It is a given that, by separating *any* of the subscriber work into 
another transaction, you have multiple transactions.  Once that is 
accepted, then, yes, some "heresy" becomes implicit :-).

Yes, indeed, if [SetAddTable, prepareTable, COPY] and [reindex, analyze] 
are in separate transactions, that has to be considered OK. 

Failure of *any* transaction breaks the subscription attempt; that is 
actually no worse than what we have today, because what we have, today, 
is one giant transaction for the subscription, where if the slightest 
bit of it falls over, the whole transaction rolls back.  So while the 
change means that there are more transactions (by count) that need to 
succeed in order for the whole to succeed, that isn't introducing any 
new real *work*.

Seems fine to me!

-- 
let name="cbbrowne" and tld="ca.afilias.info" in String.concat "@" [name;tld];;
<http://dba2.int.libertyrms.com/>
Christopher Browne
(416) 673-4124 (land)