Trouble when using Evictor!

[kent]

Hi,
I use Evictor to freeze the user's hint message, the number of users is about 100,000 and each user can contain max 100 messages. If one user gets a new message, the oldest message will be removed if the number of messages exceeds the limit. In general, each user can receive about 50-80 messages per day. The message will be mark read if the user has read it.
In this case, the Evictor often add/remove message, update message's stat, so its saving thread is busy in saving database, This causes the process takes about 99% cpu resource, I have set DB_CONFIG file:

set_cachesize 0 1073741824 1
set_lk_max_lockers 20000
set_lk_max_locks 20000
set_lk_max_objects 20000
set_tx_max 20000
set_lg_bsize 2097152
set_lg_max 2097152
set_flags DB_TXN_NOSYNC
set_flags DB_TXN_WRITE_NOSYNC

And set DbEnv's property:
<property name="Freeze.DbEnv.PnlDispatchNew.CheckpointPeriod " value="600"/>

I assure the max servant's size is less than 4k(the default Bdb's page size), so there are not dirty pages in Bdb's database file.

I trace the Evictor's log, and find that saving transaction is so slowly. There are 4 transaction log in Bdb's env directory.

I think the Evictor is a very good component and I am like to use it. I believe that the Evictor could deal with it perfect, but I don't know how to tune it. I cry for your kindness to help me resovle this problem!

Thanks.

[matthew]

You should look at tuning the size of the evictor queue. In addition, you should look at the tuning the values of the various save trigger parameters SaveSizeTrigger, SavePeriod and MaxTxSize.

Without knowing more about the architecture, design and usage patterns of your application its hard to give more specific advice. If you need more help it might be a good idea to get consulting services from us (I live in Shanghai). For more information contact us at sales@zeroc.com.

[kent]

I set SaveSizeTrigger=100, SavePeriod=300000, Size=100000. What's the trouble with it ?

[matthew]

If your SaveSizeTrigger is 100 and you are having, say 100 accesses that mutate the servant per second then you'll be saving every second which will definitely kill your performance.

[kent]

Ok, thanks.
If in that case that 100 accesses mutate the servant per second, what will I do? I have set SaveSizeTrigger=500, sometimes the process takes many cpu's resource too, but is better than before. Can I set SaveSizeTrigger as bigger as possible to resovle this problem?

[matthew]

The SaveSizeTrigger and SavePeriod are really an expression of how much data you are willing to risk losing in the event of a crash. If your application never crashes (and your hardware never fails) then its not necessary to set them at all

If you want to disable SaveSizeTrigger altogether then set the value to a negative number.

As I said I suspect it all really depends on your application, acceptable level of risk, what you need to do and how your users use the system. For example, if they log into the system and logout, you might want to pin logged in users into memory (so they are never evicted) and then save all changes every 10 minutes, and when the logout.

[kent]

Ok, I see. Thank you very much again. By the way, If I can accept the risk of all data lost and disable the 2 properties, does it cause to leave a lot of transaction logs in Bdb's home directory? And is it a bad result? And If I don't care about transaction saving, can I disable the transaction saving, just use directly saving?

[matthew]

There is no way to avoid saving the data within the scope of a transaction. Therefore, you will always have transaction logs. However, unused logs are automatically removed by Freeze by default (see the OldLogsAutoDelete property for details).