Ice 'hangs' in destroy when called through DLL

[DaVinci79]

Hi everyone,

I'm wrapping Ice 3.3.0 into a DLL (VS 2008) However I am not able to get it up and running properly. The problem is that Ice hangs when I try to unload the DLL. Even if I did not do anything but load and unload the DLL. I did not create any proxy instances, and not created any connections.

I have the following DLL main:

BOOL APIENTRY DllMain( HANDLE hModule, 
                       DWORD  ul_reason_for_call, 
                       LPVOID lpReserved) 
{
	if (ul_reason_for_call == DLL_PROCESS_ATTACH) 
	{
		try 
		{
			// initialize Ice when DLL is loaded.
			Ice::PropertiesPtr props = Ice::createProperties();
			props->setProperty("Ice.MessageSizeMax", "10240");

			Ice::InitializationData data;
			data.properties = props;
			ic = Ice::initialize(data);
		} catch (const Ice::Exception& ex) {
			return FALSE;
		}
	} else if (ul_reason_for_call == DLL_PROCESS_DETACH) {
		// destroy Ice when DLL is unloaded.
		try {
			if (ic) {
				/*ic->shutdown();
				ic->waitForShutdown();*/
				ic->destroy();
			}
		} catch (const Ice::Exception& ex) {
			// return true, even though things go wrong.
			return TRUE;
		}
	}
    return TRUE;
}

And i wrote the following test application:

int _tmain(int argc, _TCHAR* argv[])
{
		/* get handle to dll */
	HINSTANCE hGetProcIDDLL = LoadLibrary(_T("H:\\vc\\magnumDll\\Debug\\magnum.dll"));

   /* Release the Dll */
   FreeLibrary(hGetProcIDDLL);

	return 0;
}

When I run this it hangs at ic->destroy().

Here is the call-stack

 	ntdll.dll!7c90e4f4() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]	
 	ntdll.dll!7c90df3c() 	
 	kernel32.dll!7c8025db() 	
 	kernel32.dll!7c802542() 	
>	iceutil33d.dll!IceUtil::ThreadControl::join()  Line 61 + 0x10 bytes	C++
 	iceutil33d.dll!IceUtil::Timer::destroy()  Line 44	C++
 	ice33d.dll!IceInternal::Instance::destroy()  Line 1246 + 0x18 bytes	C++
 	ice33d.dll!Ice::CommunicatorI::destroy()  Line 74 + 0x12 bytes	C++
 	magnum.dll!DllMain(void * hModule=0x10000000, unsigned long ul_reason_for_call=0, void * lpReserved=0x00000000)  Line 72 + 0x30 bytes	C++
 	magnum.dll!__DllMainCRTStartup(void * hDllHandle=0x10000000, unsigned long dwReason=0, void * lpreserved=0x00000000)  Line 543 + 0x11 bytes	C
 	magnum.dll!_DllMainCRTStartup(void * hDllHandle=0x10000000, unsigned long dwReason=0, void * lpreserved=0x00000000)  Line 507 + 0x11 bytes	C
 	ntdll.dll!7c90118a() 	
 	ntdll.dll!7c91e024() 	
 	kernel32.dll!7c801bea() 	
 	kernel32.dll!7c80ac87() 	
 	callDll.exe!main(int argc=1, char * * argv=0x00330f80)  Line 31 + 0xc bytes	C++
 	callDll.exe!mainCRTStartup()  Line 259 + 0x19 bytes	C
 	kernel32.dll!7c817067() 	
 	ice33d.dll!IceInternal::Instance::Instance(const IceInternal::Handle<Ice::Communicator> & communicator={...}, const Ice::InitializationData & initData={...})  Line 980 + 0x59 bytes	C++
 	c483003e()

As far as I know it waits for the timer to die, but it never does. From the Autos window I can see that _destroyed has been set to true.

The _timer instance contains the following values:

-		this	0x00a2bec0 {_monitor={...} _destroyed=true _tokens=[0]() ...}	IceUtil::Timer * const
+		IceUtil::Shared	{_ref=1 _noDelete=false }	IceUtil::Shared
-		IceUtil::Thread	{_stateMutex={...} _started=true _running=false ...}	IceUtil::Thread
+		IceUtil::Shared	{_ref=1 _noDelete=false }	IceUtil::Shared
+		__vfptr	0x003d890c const IceUtil::Timer::`vftable'{for `IceUtil::Thread'}	*
+		_stateMutex	{_mutex={...} }	IceUtil::Mutex
		_started	true	bool
		_running	false	bool
		_handle	0x0000009c	void *
		_id	3300	unsigned long
-		_monitor	{_cond={...} _mutex={...} _nnotify=0 }	IceUtil::Monitor<IceUtil::Mutex>
+		_cond	{_internal={...} _gate={...} _queue={...} ...}	IceUtil::Cond
+		_mutex	{_mutex={...} }	IceUtil::Mutex
		_nnotify	0	int
		_destroyed	true	bool
		_tokens	[0]()	std::set<IceUtil::Timer::Token,std::less<IceUtil::Timer::Token>,std::allocator<IceUtil::Timer::Token> >
		_tasks	[0]()	std::map<IceUtil::Handle<IceUtil::TimerTask>,IceUtil::Time,IceUtil::Timer::TimerTaskCompare,std::allocator<std::pair<IceUtil::Handle<IceUtil::TimerTask> const ,IceUtil::Time> > >
+		_wakeUpTime	{_usec=26506316554 }	IceUtil::Time

I had a similar problem with Ice 3.2.1, at that point the ConnectionMonitor did not die (I didn't create any connections then either, connections set was empty). That is why I upgraded the whole thing to Ice 3.3.0.

Strangly enough if I do not build a DLL, but instantiate Ice from a Console application it works fine. Is there something I am missing?

Regards,
Vincent

[matthew]

Are you mixing the debug & release runtime DLLs in your application?

[DaVinci79]

Hi Metthew,

No, I created a DLL using /MDd, and linking the iced and iceutild libs.

This is the commandline for the compiler creating the DLL:

/Od /I "C:\Ice-3.3.0-VC90\include" /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /D "_USRDLL" /D "MAGNUMDLL_EXPORTS" /D "_WINDLL" /D "_UNICODE" /D "UNICODE" /Gm /EHsc /RTC1 /MDd /Fo"Debug\\" /Fd"Debug\vc90.pdb" /W3 /nologo /c /ZI /TP /errorReport:prompt

And this for the linker, linking the DLL:

/OUT:"\\Rijnh\shares\Users\beveren\vc\magnumDll\Debug\magnum.dll" /INCREMENTAL /NOLOGO /LIBPATH:"C:\Ice-3.3.0-VC90\lib" /DLL /MANIFEST /MANIFESTFILE:"Debug\magnum.dll.intermediate.manifest" /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"\\rijnh\shares\Users\beveren\vc\magnumDll\Debug\magnum.pdb" /SUBSYSTEM:WINDOWS /DYNAMICBASE /NXCOMPAT /MACHINE:X86 /ERRORREPORT:PROMPT iced.lib iceutild.lib  kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib

Regards,
Vincent

[DaVinci79]

As an addition I noticed it only goes wrong when invoking it from the DllMain function (invoked by the kernel when FreeLibrary is invoked).

If I would create a DLL function deinit, and invoke ic->destory() there it does go right. However, I wish to perform clean up outside of the control of the people using the library, since it will probably be used by non-programmers in LabView.

[bernard]

Hi Vincent,

I think this the problem:
The Old New Thing : Another reason not to do anything scary in your DllMain: Inadvertent deadlock

destroy() on the Ice communicator instructs the communicator to destroy its thread pools, and "join" these threads. Join is waiting on the thread's handle (Terminating a Thread (Windows)):

    int rc = WaitForSingleObject(_handle, INFINITE);

I suspect this handle is signaled only after the thread has completed the DLL_THREAD_DETACH, which is blocked on the loader lock. You should be able to confirm this in the debugger.

Cheers,
Bernard

[DaVinci79]

Hey Bernard,

Thank you for your reply. Yes, that is exactly where it hangs. From what I gather it is very likely that this is indeed what is going on.

If I understand correctly there is no solution for it inside the scope of the DllMain.

I could in theory start another thread to invoke the ic->destroy()... but that would be evil, since it will run partially after the DLL_PROCESS_DETACH has been completed.

Do you have a suggestion for a solution, (besides moving it outside the loader lock)?

Regards,
Vincent

[bernard]

Hi Vincent,

The best solution is to destroy your Ice communicator before this DLL unloading. I hope you can find a way to do that in your app.

Cheers,
Bernard

[DaVinci79]

Yes, I was afraid so... well, we'll just hope that they use the LabView components properly

Thank you for your quick reply.

Vincent

[adam3914]

I am having the same problem, I am using a c++ dll in matlab. If I unload the dll without destroying the communicator, matlab crashes after a few minutes. What I ended up doing was constantly connecting and disconnecting so that when the dll was unloaded it would already be disconnected. Their didn't appear to be much of a performance loss doing this way.