Why concurrent access to FreezeMap is so slow?

[kingbo]

When I test the concurrent access to FreezeMap,I found it is much slower than single-thread method.My code like following:

ObjectAdapterPtr	g_pAdapter;
vector<string> g_GetNamesFromMap(long beginTime,long endTime)
{
	vector<string> back;
	Freeze::ConnectionPtr pConn = Freeze::createConnection(g_pAdapter->getCommunicator(), "db");
	IHisDataMap hisDataMap(pConn, "map");
	
	IHisDataMap::const_iterator pEnd = hisDataMap.upperBoundForTime(endTime);
	IHisDataMap::const_iterator pos;
	for(pos = hisDataMap.lowerBoundForTime(beginTime);pos!=pEnd;++pos)
	{
		back.push_back(pos->first);
	}
	return back;
}
vector<string> g_SetNamesToMap(string name,long time)
{
	Freeze::ConnectionPtr pConn = Freeze::createConnection(g_pAdapter->getCommunicator(), "db");
	IHisDataMap hisDataMap(pConn, "map");

	IHisDataMap::iterator pos = hisDataMap.find(name);	
	if(pos != hisDataMap.end())
	{
		pos.set(time);
	}else
	{
		hisDataMap.insert(make_pair(name, time));	
	}
	
}


void JobThread::run()
{
	m_bRunning = true;
	IceUtil::Time sleepTime = IceUtil::Time::milliSeconds(1000);
	while(true)
	{
		if(!m_bRunning)
		{
			break;
		}else
		{
			vector<string> names = g_GetNamesFromMap(0,ULONG_MAX);
		}
		ThreadControl::sleep(sleepTime);
	}
}

void main()
{
	//Init communicator and g_pAdapter
	.............................
	
	int iThreadNum = 30;
	for(int i=0;i<iThreadNum;++i)
	{
		JobThread pThread = new JobThread();
		pThread->start();
	}
}

1. When iThreadNum = 1, the g_GetNamesFromMap will use 600ms to finish per time. When iThreadNum = 30, it will takes 33s to finish. Is there anything wrong?
2. When some threads are invoking g_GetNamesFromMap, I will call g_SetNamesToMap to update the map. In this case, do I have to serialize the two function or just call g_SetNamesToMap directly?

[bernard]

1. When iThreadNum = 1, the g_GetNamesFromMap will use 600ms to finish per time. When iThreadNum = 30, it will takes 33s to finish. Is there anything wrong?

Your code is not very clear to me: each thread is an infinite loop that gets the names and sleeps for one full second? Which time are you measuring?

In g_GetNamesFromMap, you can avoid creating transactions by using a const map:

const IHisDataMap hisDataMap(pConn, "map");

This could improve performance a little.

2. When some threads are invoking g_GetNamesFromMap, I will call g_SetNamesToMap to update the map. In this case, do I have to serialize the two function or just call g_SetNamesToMap directly?

You don't need to serialize, since you're using separate connections/maps in each function. However, you need to deal with deadlocks. See the Ice manual for details.

Cheers,
Bernard

[kingbo]

vow! So quick response in weekend. Thank you a lot!

Your code is not very clear to me: each thread is an infinite loop that gets the names and sleeps for one full second? Which time are you measuring?

In fact,the code is:

void JobThread::run()
{
	m_bRunning = true;
	IceUtil::Time sleepTime = IceUtil::Time::milliSeconds(1000);
	while(true)
	{
		if(!m_bRunning)
		{
			break;
		}else
		{
			long beginPos = GetTickCount();
			vector<string> names = g_GetNamesFromMap(0,ULONG_MAX);
			long endPos = GetTickCount();
			printf("g_GetNamesFromMap takes %d\n",endPos-beginPos);
		}
		ThreadControl::sleep(sleepTime);
	}
}

I just tested your suggestion of "const IHisDataMap hisDataMap(pConn, "map");" and found the performence is improved a little but not satisfied. It still takes 20s to 30s per time.

[bernard]

Another improvement would be to keep a Map opened in your main thread:

Freeze::ConnectionPtr pConn = Freeze::createConnection(g_pAdapter->getCommunicator(), "db");
	IHisDataMap hisDataMap(pConn, "map");

This way, you would avoid opening/closing the map all the time.

Cheers,
Bernard

[kingbo]

There is huge distance between 33s and 600ms. I guess I must miss some important configuration in DB_CONFIG. My DB_CONFIG is:

set_cachesize 0 1073741824 1
set_lk_max_lockers 40000
set_lk_max_locks 40000
set_lk_max_objects 40000
set_tx_max 20000
set_lg_bsize 2097152
set_lg_max 2097152
set_flags DB_TXN_NOSYNC
set_flags DB_TXN_WRITE_NOSYNC