ICE and exception restrictions

[yonik]

First I would like to congradulate you on ICE! Finally a spec that doesn't make me cringe or make my eyes bleed ;-)

I have only spent an hour browsing the docs so far, just looking at the protocol and slice sections. There are a couple of things I don't really agree with at first blush, but I think I'll try to split them up into separate threads as I have time instead of one long email.

There are a few places where "can't easily be supported in all languages" is used as rational for a restriction. My own design philosophy is to make the general case nice, and the outliers possible. The first that jumps out at me are the restrictions on exceptions.

"""
Exceptions are not first-class data types and first-class data types are not exceptions:
•You cannot pass an exception as a parameter value.
•You cannot use an exception as the type of a data member.
•You cannot use an exception as the element type of a sequence.
•You cannot use an exception as the key or value type of a dictionary.
•You cannot throw a value of non-exception type (such as a value of type int or string).

The reason for these restrictions is that some implementation languages use a specific and separate type for exceptions (in the same way as Slice does).
"""

All of the object oriented languages in wide use support exceptions as first class types. C++, Java, and C# must make up at least 90% of the OO languages in wide use, and exceptions are objects and can be copied in all of them AFAIK. Lesser known languages such as Python also support this fine.

The first 3 restrictions you impose would seem to prevent the following scenarios:
- nesting exceptions (throwing a new type of exception but including the info about the old)
- forwarding exceptions, passing them to a logging server as part of a larger amount of info.
- serializing exceptions

For the rare languages that are OO, but don't support "exception is also an object" semantics, then a simple specific mapping for that language could be developed. This appears to be the exception rather than the rule, however ;-)

-Yonik

[marc]

Actually, Python was one of the languages for which we put this restriction into place. Have a look at the following thread:

Classes vs. Exceptions

Python might support classes as exception at the language level, but we couldn't figure out how to support this with the C API.

[yonik]

Thanks for the thread reference - I had missed it before.

I agree with the difficulties of throwing an arbitrary type (Java class has to implement Throwable, etc), and that's perfectly reasonable. Having a separate exception type in slice is fine, I think it actually makes a spec more readable.

However I do think that the exception type should have all the abilities of a struct. It should be passable as a parameter, and be able to be a member of a class or struct. This shouldn't present a mapping problem since exceptions are normal objects in C#, Java, C++, and Python.

[yonik]

> exception type should have all the abilities of a struct

Looking at it again, exceptions need many of the abilities of a class (since structs don't inherit, and don't have a base class with which to treat them polymorphicly.

[michi]

Originally posted by yonik
> exception type should have all the abilities of a struct

Looking at it again, exceptions need many of the abilities of a class (since structs don't inherit, and don't have a base class with which to treat them polymorphicly.

Hmmm... Making such a change would force the Ice protocol to be versioned because the on-the-wire format would have to change. This is something that I don't want to do lightly. I suspect that we'd wait until we'd accumulate a number of changes and then put them all into a single release, in order to avoid versioning the protocol as few times as possible.

Cheers,

Michi.

[marc]

Note that nesting exceptions wouldn't be so easy, except if you would nest the exact exception type. However, normally you would want to use an exception base class as nested data member.

Since exceptions don't have pointer but value semantics, this wouldn't work in C++. The "interesting" part of the exception would be sliced off as soon as you copy it into the data member.

Giving exceptions pointer semantics in C++ would lead to an unnatural programming style IMO. Besides, you would have to generate a lot more code, for all the smart pointers. And the smart pointers would have to reflect the exception hierarchy. This can only be done by using up-casts in the smart pointer base class, or by duplicating the exception pointer in every class in the hierarchy.

The same problem would apply if you put an exception into a sequence, or if you pass an exception as a return value. In both cases, the exceptions would be truncated, if the sequence or return type is a base exception type.

If you really want to pass an exception as parameter, it seems easier to me to use the exception as wrapper only, and to put the real exception type as data member into the exception.

[yonik]

Originally posted by marc
Note that nesting exceptions wouldn't be so easy, except if you would nest the exact exception type. However, normally you would want to use an exception base class as nested data member.

Right... the value is being able to pass around an exception without knowing the exact details of it.

Giving exceptions pointer semantics in C++ would lead to an unnatural programming style IMO. Besides, you would have to generate a lot more code, for all the smart pointers.

I haven't read the C++ mapping, but it appears all the necessary implementation code and details have already been done since you already have classes.

And the smart pointers would have to reflect the exception hierarchy.

They don't already? That would appear to be an undesirable limitation in general - not just for exceptions. Is there a technical limitation that prevents this?

If smart pointers can't be made to mirror the inheritance hierarchy, there would still appear to be another way out...
Give exceptions all the properties of classes, but still throw them by value, as you do now. If you want to save it, or send it somewhere else, then use the clone() method (and assign to a smart pointer if you want).

The same problem would apply if you put an exception into a sequence, or if you pass an exception as a return value. In both cases, the exceptions would be truncated, if the sequence or return type is a base exception type.

Right, but all this holds true for classes as well.

If you really want to pass an exception as parameter, it seems easier to me to use the exception as wrapper only, and to put the real exception type as data member into the exception.

Yeah, but that doesn't buy you much. The whole idea of nesting exceptions is that you don't want to loose information about root causes of exceptions. I guess the best workaround absent being able to store an exception in another class is to store the representation of it (reason string) and hope that has all the info you need. That means it would have to contain full stack dumps in languages that support it.

[marc]

Originally posted by yonik
They don't already? That would appear to be an undesirable limitation in general - not just for exceptions. Is there a technical limitation that prevents this?

No, they don't. Doing so has several disadvantages. First, you would have to add virtual functions to smart pointers, and of course also a virtual destructor. Then you would have to choose from either doing dynamic casts all the time from base to derived, or you would have to cache the derived pointers in the derived classes. The first means runtime overhead, the second means size overhead.

Instead of reflecting the hierarchy it is common to allow implicit upcasts using template member functions, like:

template<typename T>
class Handle : public HandleBase<T>
{
public:
// ...
template<typename Y>
Handle(const Handle<Y>& r)
{
// ...
}
// ...
};

Originally posted by yonik
If smart pointers can't be made to mirror the inheritance hierarchy, there would still appear to be another way out...
Give exceptions all the properties of classes, but still throw them by value, as you do now. If you want to save it, or send it somewhere else, then use the clone() method (and assign to a smart pointer if you want).

I don't think this is a good idea, because without smart pointers, you would need explicit delete's. This is very CORBA-style, and something we better avoid.

Of course we could add smart-pointers just for passing exceptions as parameters, or when they are used as data members, but still throw them directly. But this would add more complexity and rules to the mapping, and more code would have to be generated. We like to keep things simple

Originally posted by yonik
Yeah, but that doesn't buy you much. The whole idea of nesting exceptions is that you don't want to loose information about root causes of exceptions. I guess the best workaround absent being able to store an exception in another class is to store the representation of it (reason string) and hope that has all the info you need. That means it would have to contain full stack dumps in languages that support it.

This is what we usually do. This way, we can also get information about local exceptions and language specific exception information (such as stack dumps in Java) "across the wire".

[yonik]

Originally posted by marc
First, you would have to add virtual functions to smart pointers

I'm not sure why this is... at first glance it seems like the virtual machinery of the object pointed to will do the job. Your non-virtual smart pointer function calls the real virtual function on the contained ptr.

However I'll defer to you for now since I don't know the details of your implementation, haven't tried to implement it myself, and I don't know what other problems you have to solve at the same time (and I've also been out of the C++ world for almost 2 years) ;-)

> Give exceptions all the properties of classes, but still throw them by value
I don't think this is a good idea, because without smart pointers, you would need explicit delete's. This is very CORBA-style, and something we better avoid.

Of course we could add smart-pointers just for passing exceptions as parameters, or when they are used as data members, but still throw them directly. But this would add more complexity and rules to the mapping, and more code would have to be generated. We like to keep things simple

The only difference will be in the C++ mapping.

// examples of code with current exception restrictions
catch(MyException& e) {
cout << e.reason << endl;
}
catch(MyException& e) {
MyCls.theErrInfo.reason = e.reason;
MyCls.theErrInfo.errCode = e.errCode;
}

// examples of code with exception as classes (thrown by value though)
catch(MyException& e) {
cout << e.reason << endl;
}
catch(MyException& e) {
MyCls.theErrInfo.reason = e.reason;
MyCls.theErrInfo.errCode = e.errCode;
}
catch(MyException& e) {
MyCls.theErr = e.clone();
}

Hence, for the C++ mapping, no complexity is added for the user as existing code will continue to work.

OK so here is my biased pro/con list.
Advantages:
- enables nesting exceptions
- enables forwarding exceptions
- more natural semantics for most OO languages (exception is just another object, except that it may need to implement a specific interface).
- no changes necessary to most language mappings
- C++ mapping would change to be a superset of the existing API and functionality - no user code should break.

Disadvantages:
- slightly more code produced per exception for C++ (though most should be inline)
- possibly slower exception creation/throwing for C++??? (not sure about this one)

Note I didn't add "greater implementation complexity for C++" because all the code generation for exceptions as members, etc, should be shared between classes and exceptions.

[marc]

Originally posted by yonik
I'm not sure why this is... at first glance it seems like the virtual machinery of the object pointed to will do the job. Your non-virtual smart pointer function calls the real virtual function on the contained ptr.

Whenever you can treat a derived class as base, and can delete the class through the base, you must at the very minimum have a virtual destructor.

Originally posted by yonik
However I'll defer to you for now since I don't know the details of your implementation, haven't tried to implement it myself, and I don't know what other problems you have to solve at the same time (and I've also been out of the C++ world for almost 2 years) ;-)

Trust me, I had an implementation for this :-)

Originally posted by yonik

The only difference will be in the C++ mapping.

// examples of code with current exception restrictions
catch(MyException& e) {
cout << e.reason << endl;
}
catch(MyException& e) {
MyCls.theErrInfo.reason = e.reason;
MyCls.theErrInfo.errCode = e.errCode;
}

// examples of code with exception as classes (thrown by value though)
catch(MyException& e) {
cout << e.reason << endl;
}
catch(MyException& e) {
MyCls.theErrInfo.reason = e.reason;
MyCls.theErrInfo.errCode = e.errCode;
}
catch(MyException& e) {
MyCls.theErr = e.clone();
}

Hence, for the C++ mapping, no complexity is added for the user as existing code will continue to work.

That's correct, but this assumes that theErr is a smart pointer. I just wanted to point out that you couldn't just clone w/o a smart pointer, except if you want to go back to CORBA-style memory management

Originally posted by yonik

OK so here is my biased pro/con list.
Advantages:
- enables nesting exceptions
- enables forwarding exceptions
- more natural semantics for most OO languages (exception is just another object, except that it may need to implement a specific interface).
- no changes necessary to most language mappings
- C++ mapping would change to be a superset of the existing API and functionality - no user code should break.

Disadvantages:
- slightly more code produced per exception for C++ (though most should be inline)
- possibly slower exception creation/throwing for C++??? (not sure about this one)

Note I didn't add "greater implementation complexity for C++" because all the code generation for exceptions as members, etc, should be shared between classes and exceptions.

Well, it depends: If we don't catch exceptions as smart-pointer, then the smart pointer for exceptions could be "dumber", because it doesn't have to follow the exception hierarchy.

I guess this would indeed be possible: In C++, exceptions are handles as they are now for throw/catch, but they are used with a dumb smart-pointer (now, that's an oximoron), that doesn't reflect the exception hierarchy, if used as parameter or data member.

This is doable. However, quite frankly, it's not on top of our priority list right now But I agree with you, it would be cleaner.

[yonik]

Originally posted by marc
Whenever you can treat a derived class as base, and can delete the class through the base, you must at the very minimum have a virtual destructor.

Well, most of the time at least ;-) It's not a language requirement, but a please-don't-crash requirement. If the memory footprint for the whole hierarchy is the same (no new members, introduction of the first virtual function, no introduction of multiple inheritance) then you can get away with the base class destructor as long as the bits (memory footprint) have the same semantics.

But don't you have a virtual destructor already? All classes derive from Ice::Object, and I figure that must have a virtual destructor. As far as the Handle itself, that should always be pass-by-value, so you don't have to worry. Even if someone wants to heap allocate a Handle, I imagine a Handle object would fit the requirements above for having the same memory footprint and thus you could get away with having no virtual destructor. Anything that needs to be virtual can be delegated to the contained pointer, which should havea virtual destructor along with other virtual methods. Am I making sense, or just missing your point?

Trust me, I had an implementation for this :-)

Actually, now that I think about it, I sort of did too (andmore than once).
The last was a class hierarchy for data exchange consisting of primitive and constructed (dict, recordset, sequence, etc) types. It started off with more-or-less manual memory management for the constructed types like sequences (which just stored bare pointers) and I changed it to use smart-pointers. Not quite the same situation, but analagous.

That's correct, but this assumes that theErr is a smart pointer. I just wanted to point out that you couldn't just clone w/o a smart pointer, except if you want to go back to CORBA-style memory management

Yes, I assumed that it was a smart pointer since that is what you always use for a class that is a member of a class (and my proposal was to treat exceptions as classes).

As far as the CORBA C++ mapping goes, OMG! (get it ;-) What happened?
As I remember, the C++ standard (with STL, etc) hadn't been issued yet, but there had been ongoing work forever, and people knew where it was going. Even so, an update (or a complete alternate mapping) should have been done at a later date. Things like namespaces, basic STL things (container rules, std::string, std::vector) far predate the actual ISO C++ standard ratification.

Still, I guess I could see how it could happen (the not-using-STL part at least). Back in 1993, I was involved in the NMF/XOpen standardization of a C++ API for the TMN stack (ASN.1, CMIP/CMISE, ROSE, GDMO). It was a tough battle to use new C++ features that weren't in any standard yet, and weren't implemented in all of the vendor C++ compilers. I remember my arguments being:
a) by the time our standard gets standardized, most of the C++ compiler vendors will have implemented the features
b) there were some features (basic templates, basic string and vector usage) that everyone knew was not going to change in the standard. Besides, compiler vendors were implementing ahead of the standard.
c) even if a C++ vendor didn't implement a particular feature, that doesn't
stop the TMN toolkit vendor from implementing just the needed subset their toolkit relied on.

Luckily, cooler heads prevailed, and we didn't go with the lowest common denominator approach. Also luckily, TMN (and OSI protocols) didn't spread much beyond the telecom industry - I wouldn't wish that on anyone ;-)
[END RANT]

But I agree with you, it would be cleaner.

Oh, now y'r just trying to quiet me down ;-) ;-)

[marc]

Originally posted by yonik

But don't you have a virtual destructor already? All classes derive from Ice::Object, and I figure that must have a virtual destructor. As far as the Handle itself, that should always be pass-by-value, so you don't have to worry. Even if someone wants to heap allocate a Handle, I imagine a Handle object would fit the requirements above for having the same memory footprint and thus you could get away with having no virtual destructor. Anything that needs to be virtual can be delegated to the contained pointer, which should havea virtual destructor along with other virtual methods. Am I making sense, or just missing your point?

I was referring to the handles, not to the objects. I.e., what would it take to have the handles reflect the hierarchy of the objects they point to. And yes, you are right, as long as the handles would always be passed by value no virtual destructor would be necessary But you would still need the dynamic cast from base to derived, or a derived pointer in each derived handle.

In any case, since handles would not be thrown, but the exceptions directly, this would not be necessary. This means your idea is implementable without overhead.

Originally posted by yonik

Oh, now y'r just trying to quiet me down ;-) ;-)

Well, I'm always sceptical at first, but after thinking about this now for some time, I indeed like the idea It's backwards compatible, makes the type system more complete, and has very little overhead (only slightly more code). For the protocol, it would mean that exceptions are handled exactly as classes. So, yes, you convinced me

[michi]

I've just been through the exercise of looking in detail at what adding this feature would require. It turns out that the changes are a lot more intrusive than first meets the eye. The changes go through all levels of Ice, from the parser and code generator right down to the protocol. My estimate is that it will take a minimum of two weeks solid work to implement this.

You are right in that C++ and Java permit me to use arbitrary types (in Java, almost arbitary types) as exceptions. But then again, how often does an exception actually get passed as data or stored in a struct? Not very often, I think.

So, for the time being, we've decided to put this feature on ice (pun intended). If you want to pass exception info, or make causality chains of exceptions, you can do so using the following:

class ExceptionBase {
    ExceptionBase previous; // Link to lower-level exception
};

class ErrorCondition1 extends ExceptionBase {
    // Data members here...
};

class ErrorCondition2 extends ExceptionBase {
    // Data members here...
};

// etc...

exception Error1 {
    ErrorCondition1 details;
};

exception Error2 {
    ErrorCondition2 details;
};

// etc...

With that, you can pass exception data as a parameter, store exceptions in structure members or similar, and you can chain exceptions together to create a causality chain as a low-level exception is thrown and handled at various levels of the exception handling hierarchy.

That provides pretty much the same functionality as making exceptions first-class data types and should be sufficient for the few applications that need it, I would hope.

Cheers,

Michi.