On Fri, 2008-04-04 at 20:35 -0700, Craig James wrote:
> Jan Wieck wrote:
> > It appeared to me that Craig needed to get something off his chest after 
> > getting very frustrated. And getting frustrated with Slony is something 
> > that we all know is possible. The only thing that really worried me in 
> > the whole rant was that he obviously ran into all these problems while 
> > attempting to use Slony in a large scale production environment without 
> > bothering to test and learn in a lab setup prior to risking all that 
> > precious data.
> > 
> > Slony was not developed with that sort of irresponsible user in mind.
> 
> Your attitude illustrates the very problem I wrote about -- a lack of 
> concern for the real-life problems of your users, and outright disrepect 
> for criticism.  I rarely engage in these tit-for-tat insult exchanges, 
> as they're not productive.  But in this case, I will at least correct 
> the factual errors in your reply.
> 
> I have been using Slony for quite some time.  I started with small 
> databases, "learned the ropes" exactly as you suggest, read the 
> documentation carefully, asked a number of questions on this forum, 
> then replicated a medium-sized database, and finally worked up to a 
> full-sized database.  We have been using Slony successfully for about 
> 6 months, on this very same large production database that caused me 
> all the trouble this week.

This is very much like what my experience of using slony at times when I
was the only DBA at skype. 
Initially it was a heaven sent and worked marvelously for small/simple
setups but then deteriorated as it was used for more complicated tasks.

When databases, replication complexity and loads grew, it started to
cause more and more trouble and frustration, up to the point we needed
to abandon it.

Especially bad were schema changes at high loads which caused several
really bad experiences, both for db users and especially for the DBA.

Fortunately, by the time we finally had to abandon it, we had hired more
people with good postgresql knowledge and were able to do the
"abandoning" in stages.

First we took what is good in slony (snapshot based replication, logging
triggers), and extracted and rewrote it into simpler building blocks,
essentially separating out a generic queueing system pgQ. 

Marko Kreen, who did all the actual coding, will talk about it on pgCon
2008 ( http://www.pgcon.org/2008/schedule/events/79.en.html )

Then we built a replication system (actually two of them ;) ) on top of
it.

Like you, we had a lots of problems with slonys sets, so our replication
system was designed without sets, trading lack of sets for more work at
setup time. This enabled us to add/remove tables to replication sets
without disturbing already running replications.

> So, you are simply flat-out wrong in your assumption.  EVEN AFTER doing 
> exactly what you suggest is the correct way to use Slony, and EVEN AFTER
>  using it successfully for over a year, I have decided to abandon it, 
> based on my extensive experience with Slony itself, and on my expertise 
> as a computer scientist.

You might want to try out SkyTools, which includes pgQ and replication
tools https://developer.skype.com/SkypeGarage/DbProjects/SkyTools

It may have even less docs, but its failure modes are less dramatic and
it usually is much easier to debug if things go wrong.

It also has less protection (slave tables are not write-protected by
special triggers), so you have to take that into account when doing the
replication.

> ...

> Calling my carefully-worded critique a "rant" is hardly the way a computer 
> scientist should treat a fellow computer scientists who is offering his time
>  and experience to give your project his expert advice.
> 
> > I am sorry that Slony does not meet Craig's ease of use expectations. As 
> > has been said many times, there can not be a one size fits all approach 
> > to replication. Hopefully he will find that some other project, like 
> > Mammoth (which is being open sourced right now) can deliver the 
> > functionality, performance as well as simplicity that he expects.
> 
> My "rant" was about quality, not ease of use. 

The devilish thing about slony is that from initial tests and ones
postgresql experience you get the false sense of robustness similar to
what postgresql itself offers and only later you find out that it is not
so at all.

It gives you a false sense of a nice and simple to use product and then
fails in mysterious and convoluted ways when you least expect and can
afford it.

> If you've never read "Zen and 
> the Art of Motorcycle Maintenance," you may not know the difference.  It's a 
> book every real programmer should read ... twice.  It's not an easy book to 
> read, but if you get it, it will change your life.  Ease of use is a natural
> side effect of a Quality attitude, but quality comes from an appreciation
> for the beauty in a well-crafted system, not from school.

The sad thing about slony is that I still can't put my finger on one
thing that is wrong with slony and say "if you fix this then it is ok".

--------------------
Hannu Krosing