Hi people.<br><br>I have 52 nodes in a slony installation. I'll give you more information.<br><br>First, I have few writes on that database, about 1 TPS, on the other hand, I've done some 20K rows updates and I've got a lag of 20 minutes. The database size is less than 10GB.<br>
<br>This is a very simple database, only 3 tables and only 2 really importants. We implemented what Steve said, a cascade structure: one "forward-only" database on each DC, then it forwards to the real slaves or for others "forward-only" databases, something like this:<br>
<br>MASTER<br>|<br>` DC1<br>| ` slave<br>| ...<br>` DC2<br>
| ` slave<br>
| ...<br>` DC3<br>
| ` slave<br>
...<br><br>Other rule that we implemented: no more than 8 nodes on each forward-only node, so, we have 3 or 4 tree depth to real slaves.<br><br>We didn't measure too much how much bandwitch we use, because we know that we have light database traffic, but it's very important to check that.<br>
<br><div class="gmail_quote">On Tue, Jun 28, 2011 at 4:00 PM, <span dir="ltr"><<a href="mailto:slony1-general-request@lists.slony.info" target="_blank">slony1-general-request@lists.slony.info</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send Slony1-general mailing list submissions to<br>
<a href="mailto:slony1-general@lists.slony.info" target="_blank">slony1-general@lists.slony.info</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.slony.info/mailman/listinfo/slony1-general" target="_blank">http://lists.slony.info/mailman/listinfo/slony1-general</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:slony1-general-request@lists.slony.info" target="_blank">slony1-general-request@lists.slony.info</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:slony1-general-owner@lists.slony.info" target="_blank">slony1-general-owner@lists.slony.info</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Slony1-general digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: How many slaves can be supported by slony2.0? (rodney)<br>
2. Re: How many slaves can be supported by slony2.0? (rodney)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Tue, 28 Jun 2011 09:27:11 +0800<br>
From: rodney <<a href="mailto:yl.wang@invantest.com" target="_blank">yl.wang@invantest.com</a>><br>
Subject: Re: [Slony1-general] How many slaves can be supported by<br>
slony2.0?<br>
To: Steve Singer <<a href="mailto:ssinger@ca.afilias.info" target="_blank">ssinger@ca.afilias.info</a>><br>
Cc: slony1-support <<a href="mailto:slony1-general@lists.slony.info" target="_blank">slony1-general@lists.slony.info</a>><br>
Message-ID: <1309224431.2201.0.camel@scadmin-ThinkPad-T61><br>
Content-Type: text/plain; charset="UTF-8"<br>
<br>
Steve<br>
<br>
I will try the log shipping and let you know.<br>
<br>
Thanks<br>
<br>
<br>
On Mon, 2011-06-27 at 09:26 -0400, Steve Singer wrote:<br>
> On 11-06-26 10:48 PM, rodney wrote:<br>
> > Hi<br>
> ><br>
> > I'm trying to setup slony to replicate table from one master to around<br>
> > 200 slaves. Can slony support slave number at this scale? It is observed<br>
> > in pgadmin that when I have 1 slave, the num of listener is 1. However,<br>
> > when I have 2 slaves, the num of listener increase from 1 to 4 (not 2).<br>
> > Will this be a problem if the slave go to a large number, like 100?<br>
> ><br>
> ><br>
> > Best regards<br>
><br>
> Running 200 slaves isn't a common use case for slony.<br>
> That doesn't mean it won't work. I know of at least one person who is<br>
> doing about 50 slaves, but I don't have details on the setup.<br>
><br>
> If your going to try Slony with 200 slaves this would be my advice<br>
><br>
> * Expect to encounter issues along the way, you will need to figure out<br>
> the best way to deploy and tune things<br>
><br>
> * Slony supports the idea of cascaded subscribers. You can have node<br>
> something like<br>
><br>
> 1<br>
> |<br>
> V<br>
> 2<br>
> / \<br>
> 3 4<br>
><br>
> so you can have a number of direct subscribers then feeding a number of<br>
> slaves. Creating a layered network would probably work better than<br>
> having 50 or 100 subscribers off a single node.<br>
><br>
> * Slony also supports something called 'log shipping' where the slon on<br>
> a slave generates .sql files that can then be applied to other nodes<br>
> downstream. Since these other nodes are not technically part of the<br>
> slony cluster (they don't have paths, or slon daemons) you can<br>
> distribute the .sql files to as many machines as you want. This might<br>
> work for you<br>
><br>
> * In a slony cluster each node must confirm SYNC messages from EVERY<br>
> other node even when a direct path between them doesn't exist (this<br>
> explains for the number of listeners you see in pgadmin). At a 100<br>
> nodes there will be a lot of SYNC/confirm messages. You will probably<br>
> have to increase the SYNC interval so you have fewer syncs going on.<br>
><br>
><br>
> * If you try any of this let us know what works or doesn't<br>
><br>
> Steve<br>
><br>
<br>
--<br>
rodney <<a href="mailto:yl.wang@invantest.com" target="_blank">yl.wang@invantest.com</a>><br>
<br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Tue, 28 Jun 2011 09:35:25 +0800<br>
From: rodney <<a href="mailto:yl.wang@invantest.com" target="_blank">yl.wang@invantest.com</a>><br>
Subject: Re: [Slony1-general] How many slaves can be supported by<br>
slony2.0?<br>
To: Steve Singer <<a href="mailto:ssinger@ca.afilias.info" target="_blank">ssinger@ca.afilias.info</a>><br>
Cc: slony1-support <<a href="mailto:slony1-general@lists.slony.info" target="_blank">slony1-general@lists.slony.info</a>><br>
Message-ID: <1309224925.2201.3.camel@scadmin-ThinkPad-T61><br>
Content-Type: text/plain; charset="UTF-8"<br>
<br>
After reading the log shipping document from<br>
<a href="http://slony.info/documentation/logshipping.html" target="_blank">http://slony.info/documentation/logshipping.html</a>, find it not<br>
straightforward to configure. I'll just give up on this solution.<br>
<br>
Thanks.<br>
<br>
<br>
On Mon, 2011-06-27 at 09:26 -0400, Steve Singer wrote:<br>
> On 11-06-26 10:48 PM, rodney wrote:<br>
> > Hi<br>
> ><br>
> > I'm trying to setup slony to replicate table from one master to around<br>
> > 200 slaves. Can slony support slave number at this scale? It is observed<br>
> > in pgadmin that when I have 1 slave, the num of listener is 1. However,<br>
> > when I have 2 slaves, the num of listener increase from 1 to 4 (not 2).<br>
> > Will this be a problem if the slave go to a large number, like 100?<br>
> ><br>
> ><br>
> > Best regards<br>
><br>
> Running 200 slaves isn't a common use case for slony.<br>
> That doesn't mean it won't work. I know of at least one person who is<br>
> doing about 50 slaves, but I don't have details on the setup.<br>
><br>
> If your going to try Slony with 200 slaves this would be my advice<br>
><br>
> * Expect to encounter issues along the way, you will need to figure out<br>
> the best way to deploy and tune things<br>
><br>
> * Slony supports the idea of cascaded subscribers. You can have node<br>
> something like<br>
><br>
> 1<br>
> |<br>
> V<br>
> 2<br>
> / \<br>
> 3 4<br>
><br>
> so you can have a number of direct subscribers then feeding a number of<br>
> slaves. Creating a layered network would probably work better than<br>
> having 50 or 100 subscribers off a single node.<br>
><br>
> * Slony also supports something called 'log shipping' where the slon on<br>
> a slave generates .sql files that can then be applied to other nodes<br>
> downstream. Since these other nodes are not technically part of the<br>
> slony cluster (they don't have paths, or slon daemons) you can<br>
> distribute the .sql files to as many machines as you want. This might<br>
> work for you<br>
><br>
> * In a slony cluster each node must confirm SYNC messages from EVERY<br>
> other node even when a direct path between them doesn't exist (this<br>
> explains for the number of listeners you see in pgadmin). At a 100<br>
> nodes there will be a lot of SYNC/confirm messages. You will probably<br>
> have to increase the SYNC interval so you have fewer syncs going on.<br>
><br>
><br>
> * If you try any of this let us know what works or doesn't<br>
><br>
> Steve<br>
><br>
<br>
--<br>
rodney <<a href="mailto:yl.wang@invantest.com" target="_blank">yl.wang@invantest.com</a>><br>
<br>
<br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Slony1-general mailing list<br>
<a href="mailto:Slony1-general@lists.slony.info" target="_blank">Slony1-general@lists.slony.info</a><br>
<a href="http://lists.slony.info/mailman/listinfo/slony1-general" target="_blank">http://lists.slony.info/mailman/listinfo/slony1-general</a><br>
<br>
<br>
End of Slony1-general Digest, Vol 52, Issue 13<br>
**********************************************<br>
</blockquote></div><br><br clear="all"><br>-- <br>José Arthur Benetasso Villanova<br>