We can use a Freeze map to add persistence to the file system server, and we present C++ and Java implementations in this section. However, as you will see in
Section 40.5, a Freeze evictor is often a better choice for applications (such as the file system server) in which the persistent value is an Ice object.
2.
If no suitable key or value types are found, define new (possibly derived) types that capture your persistent state requirements. Consider placing these definitions in a separate file: these types are only used by the server for persistence, and therefore do not need to appear in the “public” definitions required by clients. Also consider placing your persistent types in a separate module to avoid name clashes.
Our goal is to implement the file system using a Freeze map for all persistent storage, including files and their contents. Our first step is to select the Slice types we will use for the key and value types of our map. We will keep the same basic design as in
Chapter 35, therefore we need a suitable representation for persistent files and directories, as well as a unique identifier for use as a key.
Unfortunately, the selection of a value type is more complicated. Looking over the
Filesystem module in
Chapter 35, we do not find any types that capture all of our persistent state, so we need to extend the module with some new types:
module Filesystem {
class PersistentNode {
string name;
};
class PersistentFile extends PersistentNode {
Lines text;
};
class PersistentDirectory extends PersistentNode {
NodeDict nodes;
};
};
Our Freeze map will therefore map from Ice::Identity to
PersistentNode, where the values are actually instances of the derived classes
PersistentFile or
PersistentDirectory. If we had followed the advice at the beginning of
Section 40.4, we would have defined
File and
Directory classes in a separate
PersistentFilesystem module, but in this example we use the existing
Filesystem module for the sake of simplicity.
In this section we present a C++ file system implementation that utilizes a Freeze map for persistent storage. The implementation is based on the one discussed in
Chapter 35, and, in this section, we only discuss code that illustrates use of the Freeze map.
$ slice2freeze ‑I$(ICE_HOME)/slice ‑‑dict \
IdentityNodeMap,Ice::Identity,Filesystem::PersistentNode\
IdentityNodeMap Filesystem.ice \
$(ICE_HOME)/slice/Ice/Identity.ice
The server’s main program is responsible for initializing the root directory node. Many of the administrative duties, such as creating and destroying a communicator, are handled by the class
Ice::Application, as described in
Section 8.3.1. Our server
main program has now become the following:
#include <FilesystemI.h>
#include <Ice/Application.h>
#include <Freeze/Freeze.h>
using namespace std;
using namespace Filesystem;
class FilesystemApp : public virtual Ice::Application {
public:
FilesystemApp(const string& envName) :
_envName(envName) { }
virtual int run(int, char*[]) {
// Terminate cleanly on receipt of a signal
//
shutdownOnInterrupt();
// Install object factories
//
communicator()‑>addObjectFactory(
PersistentFile::ice_factory(),
PersistentFile::ice_staticId());
communicator()‑>addObjectFactory(
PersistentDirectory::ice_factory(),
PersistentDirectory::ice_staticId());
// Create an object adapter (stored in the NodeI::_adapter
// static member)
//
NodeI::_adapter = communicator()‑>
createObjectAdapterWithEndpoints(
"FreezeFilesystem", "default ‑p 10000");
//
// Set static members used to create connections and maps
//
NodeI::_communicator = communicator();
NodeI::_envName = _envName;
NodeI::_dbName = "mapfs";
// Find the persistent node for the root directory, or
// create it if not found
//
Freeze::ConnectionPtr connection =
Freeze::createConnection(communicator(), _envName);
IdentityNodeMap persistentMap(connection, NodeI::_dbName);
Ice::Identity rootId = communicator()‑>stringToIdentity("RootDir");
PersistentDirectoryPtr pRoot;
{
IdentityNodeMap::iterator p =
persistentMap.find(rootId);
if (p != persistentMap.end()) {
pRoot =
PersistentDirectoryPtr::dynamicCast(
p‑>second);
assert(pRoot);
} else {
pRoot = new PersistentDirectory;
pRoot‑>name = "/";
persistentMap.insert(
IdentityNodeMap::value_type(rootId, pRoot));
}
}
// Create the root directory (with name "/" and no parent)
//
DirectoryIPtr root = new DirectoryI(rootId, pRoot, 0);
// Ready to accept requests now
//
NodeI::_adapter‑>activate();
// Wait until we are done
//
communicator()‑>waitForShutdown();
if (interrupted()) {
cerr << appName()
<< ": received signal, shutting down" << endl;
}
return 0;
}
private:
string _envName;
};
int
main(int argc, char* argv[])
{
FilesystemApp app("db");
return app.main(argc, argv);
}
Let us examine the changes in detail. First, we are now including Freeze/Freeze.h instead of
Ice/Ice.h. This Freeze header file includes all of the other Freeze (and Ice) header files this source file requires.
Next, we define the class FilesystemApp as a subclass of
Ice::Application, and provide a constructor taking a string argument:
FilesystemApp(const string& envName) :
_envName(envName) { }
One of the first tasks run performs is installing the Ice object factories for
PersistentFile and
PersistentDirectory. Although these classes are not exchanged via Slice operations, they are marshalled and unmarshalled in exactly the same way when saved to and loaded from the database, therefore factories are required. Since these Slice classes have no operations, we can use their built‑in factories.
communicator()‑>addObjectFactory(
PersistentFile::ice_factory(),
PersistentFile::ice_staticId());
communicator()‑>addObjectFactory(
PersistentDirectory::ice_factory(),
PersistentDirectory::ice_staticId());Next, we set all the NodeI static members.
NodeI::_adapter = communicator()‑>
createObjectAdapterWithEndpoints(
"FreezeFilesystem", "default ‑p 10000");
NodeI::_communicator = communicator();
NodeI::_envName = _envName;
NodeI::_dbName = "mapfs";
Then we create a Freeze connection and a Freeze map. When the last connection to a Berkeley DB environment is closed, Freeze automatically closes this environment, so keeping a connection in the main function ensures the underlying Berkeley DB environment remains open. Likewise, we keep a map in the
main function to keep the underlying Berkeley DB database open.
Freeze::ConnectionPtr connection =
Freeze::createConnection(communicator(), _envName);
IdentityNodeMap persistentMap(connection, NodeI::_dbName);
Now we need to initialize the root directory node. We first query the map for the identity of the root directory node; if no match is found, we create a new
PersistentDirectory instance and insert it into the map. We use a scope to close the iterator after use; otherwise, this iterator could keep locks and prevent subsequent access to the map through another connection.
Ice::Identity rootId = communicator()‑>
stringToIdentity("RootDir");
PersistentDirectoryPtr pRoot;
{
IdentityNodeMap::iterator p =
persistentMap.find(rootId);
if (p != persistentMap.end()) {
pRoot =
PersistentDirectoryPtr::dynamicCast(
p‑>second);
assert(pRoot);
} else {
pRoot = new PersistentDirectory;
pRoot‑>name = "/";
persistentMap.insert(
IdentityNodeMap::value_type(rootId, pRoot));
}
}
Finally, the main function instantiates the
FilesystemApp, passing
db as the name of the database environment.
int
main(int argc, char* argv[])
{
FilesystemApp app("db");
return app.main(argc, argv);
}
We also must change the servant classes to incorporate the Freeze map. We are maintaining the multiple-inheritance design from
Chapter 35, but we have added some methods and changed the constructor arguments and state members.
Let us examine the definition of NodeI first. You will notice the addition of the
getPersistentNode method, which allows
NodeI to gain access to the persistent node in order to implement the
Node operations. Another alternative would have been to add a
PersistentNodePtr member to
NodeI, but that would have forced the
FileI and
DirectoryI classes to downcast this member to the appropriate subclass.
namespace Filesystem {
class NodeI : virtual public Node {
public:
// ... Ice operations ...
static Ice::ObjectAdapterPtr _adapter;
static Ice::CommunicatorPtr _communicator;
static std::string _envName;
static std::string _dbName;
protected:
NodeI(const Ice::Identity&, const DirectoryIPtr&);
virtual PersistentNodePtr getPersistentNode() const = 0;
PersistentNodePtr find(const Ice::Identity&) const;
IdentityNodeMap _map;
DirectoryIPtr _parent;
IceUtil::RecMutex _nodeMutex;
bool _destroyed;
public:
const Ice::Identity _ID;
};
}
Other changes of interest in NodeI are the addition of the members
_map and
_destroyed, and we have changed the
NodeI constructor to accept an
Ice::Identity argument.
The FileI class now has a single state member of type
PersistentFilePtr, representing the persistent state of this file. Its constructor has also changed to accept
Ice::Identity and
PersistentFilePtr.
namespace Filesystem {
class FileI : virtual public File,
virtual public NodeI {
public:
// ... Ice operations ...
FileI(const Ice::Identity&, const PersistentFilePtr&,
const DirectoryIPtr&);
protected:
virtual PersistentNodePtr getPersistentNode() const;
private:
PersistentFilePtr _file;
};
}
The DirectoryI class has undergone a similar transformation.
namespace Filesystem {
class DirectoryI : virtual public Directory,
virtual public NodeI {
public:
// ... Ice operations ...
DirectoryI(const Ice::Identity&,
const PersistentDirectoryPtr&,
const DirectoryIPtr&);
// ...
protected:
virtual PersistentNodePtr getPersistentNode() const;
// ...
private:
// ...
PersistentDirectoryPtr _dir;
};
}
Let us examine how the implementations have changed. The FileI methods are still fairly trivial, but there are a few aspects that need discussion.
First, each operation now checks the _destroyed member and raises
Ice::ObjectNotExistException if the member is
true. This is necessary in order to ensure that the Freeze map is kept in a consistent state. For example, if we allowed the
write operation to proceed after the file node had been destroyed, then we would have mistakenly added an entry back into the Freeze map for that file. Previous file system implementations ignored this issue because it was relatively harmless, but that is no longer true in this version.
Next, notice that the write operation calls
put on the map after changing the
text member of its
PersistentFile object. Although the file’s
PersistentFilePtr member points to a value in the Freeze map, changing that value has no effect on the persistent state of the map until the value is reinserted into the map, thus overwriting the previous value.
Finally, the constructor now accepts an Ice::Identity. This differs from previous implementations in that the identity used to be created by the
NodeI constructor. However, as we will see later, the caller needs to determine the identity prior to invoking the subclass constructors. Similarly, the constructor is not responsible for creating a
PersistentFile object, but rather is given one. This accommodates our two use cases: creating a new file, and restoring an existing file from the map.
Filesystem::Lines
Filesystem::FileI::read(const Ice::Current&)
{
IceUtil::RWRecMutex::RLock lock(_nodeMutex);
if (_destroyed)
throw Ice::ObjectNotExistException(__FILE__, __LINE__);
return _file‑>text;
}
void
Filesystem::FileI::write(const Filesystem::Lines& text,
const Ice::Current&)
{
IceUtil::RWRecMutex::WLock lock(_nodeMutex);
if (_destroyed)
throw Ice::ObjectNotExistException(__FILE__, __LINE__);
_file‑>text = text;
_map.put(IdentityNodeMap::value_type(_ID, _file));
}
Filesystem::FileI::FileI(const Ice::Identity& id,
const PersistentFilePtr& file,
const DirectoryIPtr& parent) :
NodeI(id, parent), _file(file)
{
}
Filesystem::PersistentNodePtr
Filesystem::FileI::getPersistentNode()
{
return _file;
}
The DirectoryI implementation requires more substantial changes. We begin our discussion with the
createDirectory operation.
Filesystem::DirectoryI::createDirectory(
const std::string& name,
const Ice::Current& current)
{
IceUtil::RWRecMutex::WLock lock(_nodeMutex);
if (_destroyed)
throw Ice::ObjectNotExistException(__FILE__, __LINE__);
checkName(name);
PersistentDirectoryPtr persistentDir
= new PersistentDirectory;
persistentDir‑>name = name;
DirectoryIPtr dir = new DirectoryI(
communicator()‑>stringToIdentity(IceUtil::generateUUID()),
persistentDir, this);
assert(find(dir‑>_ID) == 0);
_map.put(make_pair(dir‑>_ID, persistentDir));
DirectoryPrx proxy = DirectoryPrx::uncheckedCast(
current.adapter‑>createProxy(dir‑>_ID));
NodeDesc nd;
nd.name = name;
nd.type = DirType;
nd.proxy = proxy;
_dir‑>nodes[name] = nd;
_map.put(IdentityNodeMap::value_type(_ID, _dir));
return proxy;
}
After validating the node name, the operation creates a PersistentDirectory1 object for the child directory, which is passed to the
DirectoryI constructor along with a unique identity. Next, we store the child’s
PersistentDirectory object in the Freeze map. Finally, we initialize a new
NodeDesc value and insert it into the parent’s node table and then reinsert the parent’s
PersistentDirectory object into the Freeze map.
The implementation of the createFile operation has the same structure as
createDirectory.
Filesystem::FilePrx
Filesystem::DirectoryI::createFile(const std::string& name,
const Ice::Current& current)
{
IceUtil::RWRecMutex::WLock lock(_nodeMutex);
if (_destroyed)
throw Ice::ObjectNotExistException(__FILE__, __LINE__);
checkName(name);
PersistentFilePtr persistentFile = new PersistentFile;
persistentFile‑>name = name;
FileIPtr file = new FileI(
communicator()‑>stringToIdentity(IceUtil::generateUUID()),
persistentFile, this);
assert(find(file‑>_ID) == 0);
_map.put(make_pair(file‑>_ID, persistentFile));
FilePrx proxy = FilePrx::uncheckedCast(
current.adapter‑>createProxy(file‑>_ID));
NodeDesc nd;
nd.name = name;
nd.type = FileType;
nd.proxy = proxy;
_dir‑>nodes[name] = nd;
_map.put(IdentityNodeMap::value_type(_ID, _dir));
return proxy;
}
The next significant change is in the DirectoryI constructor. The body of the constructor now instantiates all of its immediate children, which effectively causes all nodes to be instantiated recursively.
For each entry in the directory’s node table, the constructor locates the matching entry in the Freeze map. The key type of the Freeze map is
Ice::Identity, so the constructor obtains the key by invoking
ice_getIdentity on the child’s proxy.
Filesystem::DirectoryI::DirectoryI(
const Ice::Identity& id,
const PersistentDirectoryPtr& dir,
const DirectoryIPtr& parent) :
NodeI(id, parent), _dir(dir)
{
// Instantiate the child nodes
//
for (NodeDict::iterator p = dir‑>nodes.begin();
p != dir‑>nodes.end(); ++p) {
Ice::Identity id = p‑>second.proxy‑>ice_getIdentity();
PersistentNodePtr node = find(id);
