seg fault

[ignaden]

Hi,

I keep getting segfaults when running icegridnode and icegridadmin. Running it on solaris galaxy, uname -a is:

SunOS 5.10 Generic_127112-02 i86pc i386 i86pc

The exceptions occurs on shutdown, whether Ctrl-C or normally. Here's the gdb printout for icegridadmin,

(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /abc/platform/eq/sparc_SunOS_5.6/ace/3rdParty/solaris-x86-64/ice/3.2.1/bin/icegridadmin
[New LWP 1 ]
(no debugging symbols found)
warning: Temporarily disabling breakpoints for unloaded shared library "/usr/lib/amd64/ld.so.1"
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
warning: Lowest section in /lib/amd64/libpthread.so.1 is .dynamic at 00000000000000b0
warning: Lowest section in /lib/amd64/libthread.so.1 is .dynamic at 00000000000000b0
[New Thread 1 (LWP 1)]
[New LWP 2 ]
[New LWP 3 ]
[New LWP 4 ]
user id: boo
password:
[New LWP 5 ]
Ice 3.2.1 Copyright 2003-2007 ZeroC, Inc.
>>> quit
[LWP 5 exited]
[New Thread 5 ]
[LWP 3 exited]
[New Thread 3 ]
[LWP 4 exited]
[New Thread 4 ]
[New Thread 2 (LWP 2)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 2 (LWP 2)]
0xfffffd7fff3dd591 in dlamd64getunwind () from /lib/amd64/ld.so.1
(gdb)

[bernard]

Hi Denis,

The preferred debugger on Solaris 10 for Sun C++ binaries is dbx. I'd recommend to use a recent version, such as the dbx that comes with Sun Studio 11 or 12.

Are you using the Ice 3.2.1 binary distribution for Solaris 10 x86/x64? Or is it your own build?

Naturally, none of our programs should crash at exit. I suspect a problem with a system library version, such as libCrun.so.1 or maybe the dynamic linker. Getting a stack trace may give us more clues.

Here is my stack trace of an icegridadmin core on Solaris 10 amd64; I got this core by sending SIGQUIT to the icegridadmin process:

 dbx `which icegridadmin` core.icegridadmin.20125 
For information about new features see `help changes'
To remove this message, put `dbxenv suppress_startup_message 7.6' in your .dbxrc
Reading icegridadmin
core file header read successfully
Reading ld.so.1
Reading libGlacier2.so.3.3.0
Reading libIceXML.so.3.3.0
Reading libIceGrid.so.3.3.0
Reading libIcePatch2.so.3.3.0
Reading libIceBox.so.3.3.0
Reading libIce.so.3.3.0
Reading libIceUtil.so.3.3.0
Reading libpthread.so.1
Reading libCstd.so.1
Reading libCrun.so.1
Reading libm.so.2
Reading libthread.so.1
Reading libc.so.1
Reading libexpat.so.0.5.0
Reading libssl.so.0.9.7
Reading libcrypto.so.0.9.7
Reading libbz2.so.1
Reading libdl.so.1
Reading libsocket.so.1
Reading librt.so.1
Reading libnsl.so.1
Reading libaio.so.1
Reading libmd5.so.1
Reading libssl_extra.so.0.9.7
Reading libcrypto_extra.so.0.9.7
Reading nss_files.so.1
t@1 (l@1) terminated by signal QUIT (Quit)
0xfffffd7ffe7ce3aa: _read+0x000a:       jae      _read+0x16     [ 0xfffffd7ffe7ce3b6, .+0xc ]
(dbx) where
current thread: t@1
=>[1] _read(0x0, 0xfffffd7ffe7fb2e4, 0x400, 0x1, 0x0, 0x0), at 0xfffffd7ffe7ce3aa 
  [2] __filbuf(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ffe7adeb7 
  [3] getc(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ffe7b1ffa 
  [4] IceGrid::Parser::getInput(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x474feb 
  [5] yy_get_next_buffer(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x45c0c3 
  [6] yylex(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x45b541 
  [7] yyparse(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x4557a9 
  [8] IceGrid::Parser::parse(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x475f42 
  [9] Client::run(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x576a32 
  [10] Client::main(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x5717c4 
  [11] main(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0x570fa0 
(dbx) threads
o>    t@1  a  l@1   ?()   signal SIGQUIT in  _read() 
      t@2  a  l@2   sigwaitThread()   LWP suspended in  ___sigtimedwait() 
      t@3  a  l@3   startHook()   sleep on 0x6fe388  in  __lwp_park() 
      t@4  a  l@4   startHook()   sleep on 0x703528  in  __lwp_park() 
      t@5  a  l@5   startHook()   LWP suspended in  __pollsys() 
      t@6  a  l@6   startHook()   LWP suspended in  __pollsys() 
      t@7  a  l@7   startHook()   sleep on 0x70c5c8  in  __lwp_park()

Also make sure you have 'ulimit -c' set to unlimited to get core files.

Best regards,
Bernard