Hey,
I found a problem in misc.c (line 175) causing segfaults with slon.  Here's
the breakdown of what happened and what was found:

Started seeing this after a long import (200M row table subscribe)

2008-06-01 01:08:18 EDT DEBUG2 slon: child terminated status: 11; pid:
31047, current worker pid: 31047
2008-06-01 01:08:18 EDT DEBUG1 slon: restart of worker in 10 seconds

dmesg was reporting:

slon[30612]: segfault at 0000000000273139 rip 0000003b22876150 rsp
0000000041ded7a8 error 4

(not the same segfault pid, but they were all the same addresses)

I broke out gdb and found the following backtrace on the child:

[Switching to LWP 31128]
0x0000003b22876150 in strlen () from /lib64/libc.so.6
(gdb) bt
#0  0x0000003b22876150 in strlen () from /lib64/libc.so.6
#1  0x0000003b2284605b in vfprintf () from /lib64/libc.so.6
#2  0x0000003b2286728a in vsnprintf () from /lib64/libc.so.6
#3  0x0000000000417488 in slon_log (level=3D<value optimized out>,
fmt=3D0x41ec38 " ssy_action_list value: %s\n") at misc.c:175
#4  0x00000000004094a6 in sync_event (node=3D0x7cb3d70, local_conn=3D<value
optimized out>, wd=3D0x7cb3840, event=3D0x2aaaac0010f0) at remote_worker.c:=
4333
#5  0x000000000040bb03 in remoteWorkerThread_main (cdata=3D<value optimized
out>) at remote_worker.c:630
#6  0x0000003b234062f7 in start_thread () from /lib64/libpthread.so.0
#7  0x0000003b228ce85d in clone () from /lib64/libc.so.6
(gdb)

I went into main.c and had it printf some of the output before the segfault
and found:

2008-06-01 01:15:49 EDT DEBUG2 remoteListenThread_1: queue event 1,1342 SYNC
2008-06-01 01:15:49 EDT DEBUG4  ssy_action_list value:
'52977','52978','52979','52980','52981','52982','52983','52984','52985','52=
986','52987','52988','52989','52990','52991','52992','52993','52994','52995=
','52996','52997','52998','52999','53000','53001','53002','53003','53004','=
53005','53006','53007','53008','53009','53010','53011','53012','53013','530=
14','53015','53016','53017','53018','53019','53020','53021','53022','53023'=
,'53024','53025','53026','53027','53028','53029','53030','53031','53032','5=
3033','53034','53035','53036','53037','53038','53039','53040','53041','5304=
2','53043','53044','53045','53046','53047','53048','53049','53050','53051',=
'53052','53053','53054','53055','53056','53057','53058','53059','53060','53=
061','53062','53063','53064','53065','53066','53067','53068','53069','53070=
','53071','53072','53073','53074','53075','53076','53077','53078','53079','=
53080','53081','53082','53083','53084','53085','53086','53087','53088','530=
89','53090','53091','53092','53093','53094','53095','53096','53097','53098'=
,'53099','53100','53101','53102','53103','53104','53105','53106','53107','5=
3108','53109','53110','53111','53112','53113','53114','53115','53116','5311=
7','53118','53119','53120','53121','53122','53123','53124','53125','53126',=
'53127','53128','53129','53130','53131','53132','53133','53134','53135','53=
136','53137','53138','53139','53140','53141','53142','53143','53144','53145=
','53146','53147','53148','53149','53150','53151','53152','53153','53154','=
53155','53156','53157','53158','53159','53160','53161','53162','53163','531=
64','53165','53166','53167','53168','53169','53170','53171','53172','53173'=
,'53174','53175','53176','53177','53178','53179','53180','53181','53182','5=
3183','53184','53185','53186','53187','53188','53189','53190','53191','5319=
2','53193','53194','53195','53196','53197','53198','53199','53200','53201',=
'53202','53203','53204','53205','53206','53207','53208','53209','53210','53=
211','53212','53213','53214','53215','53216','53217','53218','53219','53220=
','53221','53222','53223','53224','53225','53226','53227','53228','53229','=
53230','53231','53232','53233','53234','53235','53236','53237','53238','532=
39','53240','53241','53242','53243','53244','53245','53246','53247','53248'=
,'53249','53250','53251','53252','53253','53254','53255','53256','53257','5=
3258','53259','53260','53261','53262','53263','53264','53265','53266','5326=
7','53268','53269','53270','53271','53272','53273','53274','53275','53276',=
'53277','53278','53279','53280','53281','53282','53283','53284','53285','53=
286','53287','53288','53289','53290','53291','53292','53293','53294','53295=
','53296','53297','53298','53299','53300','53301','53302','53303','53304','=
53305','53306','53307','53308','53309','53310','53311','53312','53313','533=
14','53315','53316','53317','53318','53319','53320','53321','53322','53323'=
,'53324','53325','53326','53327','53328','53329','53330','53331','53332','5=
3333','53334','53335','53336','53337','53338','53339','53340','53341','5334=
2','53343','53344','53345','53346','53347','53348','53349','53350','53351',=
'53352','53353','53354','53355','53356','53357','53358','53359','53360','53=
361','53362','53363','53364','53365','53366','53367','53368','53369','53370=
','53371','53372','53373','53374','53375','53376','53377','53378','53379','=
53380','53381','53382','53383','53384','53385','53386','53387','53388','533=
89','53390','53391','53392','53393','53394','53395','53396','53397','53398'=
,'53399','53400','53401','53402','53403','53404','53405','53406','53407','5=
3408','53409','53410','53411','53412','53413','53414','53415','53416','5341=
7','53418','53419','53420','53421','53422','53423','53424','53425','53426',=
'53427','53428','53429','53430','53431','53432','53433','53434','53435','53=
436','53437','53438','53439','53440','53441','53442','53443','53444','53445=
','53446','53447','53448','53449','53450','53451','53452','53453','53454','=
53455','53456','53457','53458','53459','53460','53461','53462','53463','534=
64','53465','53466','53467','53468','53469','53470','53471','53472','53473'=
,'53474','53475','53476','53477','53478','53479','53480','53481','53482','5=
3483','53484','53485','53486','53487','53488','53489','53490','53491','5349=
2','53493','53494','53495','53496','53497','53498','53499','53500','53501',=
'53502','53503','53504','53505','53506','53507','53508','53509','53510','53=
511','53512','53513','53514','53515','53516','53517','53518','53519','53520=
','53521','53522','53523','53524','53525','53526','53527','53528','53529','=
53530','53531','53532','53533','53534','53535','53536','53537','53538','535=
39','53540','53541','53542','53543','53544','53545','53546','53547','53548'=
,'53549','53550','53551','53552','53553','53554','53555','53556','53557','5=
3558','53559','53560','53561','53562','53563','53564','53565','53566','5356=
7','53568','53569','53570','53571','53572','53573','53574','53575','53576',=
'53577','53578','53579','53580','53581','53582','53583','53584','53585','53=
586','53587','53588','53589','53590','53591','53592','53593','53594','53595=
','53596','53597','53598','53599','53600','53601','53602','53603','53604','=
53605','53606','53607','53608','53609','53610','53611','53612','53613','536=
14','53615','53616','53617','53618','53619','53620','53621','53622','53623'=
,'53624','53625','53626','53627','53628','53629','53630','53631','53632','5=
3633','53634','53635','53636','53637','53638','53639','53640','53641','5364=
2','53643','53644','53645','53646','53647','53648','53649','53650','53651',=
'53652','53653','53654','53655','53656','53657','53658','53659','53660','53=
661','53662','53663','53664','53665','53666','53667','53668','53669','53670=
','53671','53672','53673','53674','53675','53676','53677','53678','53679','=
53680','53681','53682','53683','53684','53685','53686','53687','53688','536=
89','53690','53691','53692','53693','53694','53695','53696','53697','53698'=
,'53699','53700','53701','53702','53703','53704','53705','53706','53707','5=
3708','53709','53710','53711','53712','53713','53714','53715','53716','5371=
7','53718','53719','53720','53721','53722','53723','53724','53725','53726',=
'53727','53728','53729','53730','53731','53732','53733','53734','53735','53=
736','53737','53738','53739','53740','53741','53742','53743','53744','53745=
','53746','53747','53748','53749','53750','53751','53752','53753','53754','=
53755','53756','53757','53758','53759','53760','53761','53762','53763','537=
64','53765','53766','53767','53768','53769','53770','53771','53772','53773'=
,'53774','53775','53776','53777','53778','53779','53780','53781','53782','5=
3783','53784','53785','53786','53787','53788','53789','53790','53791','5379=
2','53793','53794','53795','53796','53797','53798','53799','53800','53801',=
'53802','53803','53804','53805','53806','53807','53808','53809','53810','53=
811','53812','53813','53814','53815','53816','53817','53818','53819','53820=
','53821','53822','53823','53824','53825','53826','53827','53828','53829','=
53830','53831','53832','53833','53834','53835','53836','53837','53838','538=
39','53840','53841','53842','53843','53844','53845','53846','53847','53848'=
,'53849','53850','53851','53852','53853','53854','53855','53856','53857','5=
3858','53859','53860','53861','53862','53863','53864','53865','53866','5386=
7','53868','53869','53870','53871','53872','53873','53874','53875','53876',=
'53877','53878','53879','53880','53881','53882','53883','53884','53885','53=
886','53887','53888','53889','53890','53891','53892','53893','53894','53895=
','53896','53897','53898','53899','53900','53901','53902','53903','53904','=
53905','53906','53907','53908','53909','53910','53911','53912','53913','539=
14','53915','53916','53917','53918','53919','53920','53921','53922','53923'=
,'53924','53925','53926','53927','53928','53929','53930','53931','53932','5=
3933','53934','53935','53936','53937','53938','53939','53940','53941','5394=
2','53943','53944','53945','53946','53947','53948','53949','53950','53951',=
'53952','53953','53954','53955','53956','53957','53958','53959','53960','53=
961','53962','53963','53964','53965','53966','53967','53968','53969','53970=
','53971','53972','53973','53974','53975','53976','53977','53978','53979','=
53980','53981','53982','53983','53984','53985','53986','53987','53988','539=
89','53990','53991','53992','53993',
2008-06-01 01:15:49 EDT DEBUG2 slon: child terminated status: 11; pid:
31548, current worker pid: 31548

The error looks like it was the DEBUG4 output of all the action list, as it
segfaults in building the string in the while loop on main.c:175.

Sorry I don't have a fix, I just commented out the loop for now, I could
have switched to debug1 or so though, it appears.  I was more concerned with
getting up and running than fixing.  If I have time tomorrow I might try and
figure out what can be done.  My off the cuff suggestion is when outbuf > a
max, split the message, send this one, and construct a new message.

Cheers,

Gavin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.slony.info/pipermail/slony1-general/attachments/20080601/=
1faad7d1/attachment-0001.htm