How to handle distributed transactions

[robert]

My following question is not specific to Ice, but concerns a general issue in distributed computing.

For our DMS we need transactions which write files to a fileserver and simultaneously the corresponding metadata to a database. The file upload is handled by an interface CFileUpload(). This interface first writes the file to a temporary location on the file server, so that it can easily be removed in case of a failure.

CFileUpload::commit() transfers the file to the persistent area on the file server and writes the metadata into the database. It can however not commit the database transaction as within that same transaction other database operations are executed via other interfaces to create other objects which may be associated with the file. For the client, the whole shebang must look like one transaction - either all the database objects and all uploaded files are committet or nothing of all. But how to hook this together?

I suggested to proceed like this:
---------------
class CDMSSessionI : virtual public CDMSSession {
...
CFileUploadSequence openFileUploads;
CDatabaseSession* dbSession;
void registerFileUpload(CFileUploadPtr pFileUPload) {
openFileUploads.push_back(pFileUpload);
}
void commit() {
foreach pFileUpload in openFileUploads { // Pseudocode
pFileUpload->commit();
}
dbSession->commit();
}
...
};
...
CFileUploadI::CFileUploadI(Ice::Current& current) {
CDMSSessionI* pSess = getDMSSessionFromCurrent(current);
pSess->registerFileUpload(this);
}
-----------------

So CFileUpload registers itself at the DMS Session which it uses to enable that DMS Session to commit all relevant file uploads within it's commit() method.

Our project team is not sure if this is the best approach as for simplicity, elegance and encapsulation. Can you comment on it? Should you have a better idea, it would be welcome, of course!

And by the way, can you give us an estimated target date for the release of Ice 1.6?

Thanks! Robert.

[marc]

Ice 1.6 (actually, we decided to give it the version number 2.0, because it has so many new features) will most likely be available within the next 4-8 weeks.

As for your question about distributed transactions, this is one of the most complicated topics, regardless of what middleware you use. I'm afraid any meaningful answer is way beyond the scope of the free support we can give here on this mailing list.

[stephan]

Hi Marc!

I'm afraid any meaningful answer is way beyond the scope of the free support we can give here on this mailing list.

Well, as far as I know about distributed transactions, this topic is quite complex, that's true. But there must be an opinion or recommendations on how to work with dt and Ice (or: how not to work with them ).

For example, is it likely that Ice 1.6 aka 2.0 will bring something like a transaction manager as seen in the CORBA specs or is there no customer demand apart from this question?

So many critical points of distributed applications where solved with Ice due to its simplicity, so I'd expect something similar for the holy grail of distributed applications (-> distributed transactions) as well.

I think, some additional comments from your side would be worth reading for the entire Ice community

Kind regards,

Stephan

[matthew]

Originally posted by stephan
Hi Marc!



Well, as far as I know about distributed transactions, this topic is quite complex, that's true. But there must be an opinion or recommendations on how to work with dt and Ice (or: how not to work with them ).

For example, is it likely that Ice 1.6 aka 2.0 will bring something like a transaction manager as seen in the CORBA specs or is there no customer demand apart from this question?

So many critical points of distributed applications where solved with Ice due to its simplicity, so I'd expect something similar for the holy grail of distributed applications (-> distributed transactions) as well.

I think, some additional comments from your side would be worth reading for the entire Ice community

Kind regards,

Stephan

With my former company I was the lead designer and engineer on an implementation of the CORBA OTS specification. Furthermore, I sat on the OTS OMG comittee as did Michi, and Bernard was the chair -- so I think it's safe to say we have lots of experience in this area However, so far we have no plans to implement a DTS as there is little customer demand.

With respect to your problem. the simplest approach would be to avoid a distributed transaction. Distributed transactions are only really needed if you actually have physically seperate objects cooperating on multiple operations that should together have one atomic result.

The simplest way to avoid this is to encapsulate the entire request in one operation call.

If this isn't reasonable then the next question is whether the cooperating objects exist over multiple processes and use seperate databases. If it is using a single database in one process then your job becomes much easier since you can arrange to share the transaction contexts internally in your server and you have alot less error conditions, and database integration code to worry about.

Regards, Matthew