This is a proof of concept implementation of Object Prevalence for Common Lisp. PLEASE NOTE: This now a project at CL.net as http://common-lisp.net/project/cl-prevalence.
Object Prevalence is a simple but interesting concept first proposed by Klaus Wuestefeld in 2001. IBM developerWorks has a reasonable Introduction to Object Prevalence article. The main Java implementation is called Prevayler, with a (chaotic) wiki site with lots of information and discussions. Basically, the idea is this:
That is all there is to the concept of object prevalence. Here are some more details as well as some advantages and limitations:
This code was written by Sven Van Caekenberghe using OpenMCL, an open source Common Lisp implementation for Mac OS X (Darwin) and LinuxPPC. You can learn more about OpenMCL on http://openmcl.clozure.com/. This code is known to run on LispWorks 4.3. CMU CL used to run this code successfully in the past.
You can download this software from my homepage at http://homepage.mac.com/svc/prevalence/prevalence.tgz. This code is also meant to be included as examples in the OpenMCL the distribution.
All software and documentation is Copyright (C) 2003 Sven Van Caekenberghe. You are granted the rights to distribute and use this software as governed by the terms of the Lisp Lesser GNU Public License (see http://opensource.franz.com/preamble.html), also known as the LLGPL.
There are currently two main files, three optional files and two examples in this package:
Basically, just use the provided ASDF file. You will have to download my Simple XML Parser for Common Lisp (or read about it), from which you'll need the file xml.lisp.
Using a prevalence system is quite simple. The main thing to keep in mind is that every destructive operation on the system must be made explicit. When using the standard transaction object, this means that you have to use an explicit named function to do the work (an anonymous lambda won't do). Here is a simple example:
? (in-package :clp)
#<Package "CLP">
? (defvar *test-system*)
*TEST-SYSTEM*
? (setf *test-system* (make-prevalence-system #p"/tmp/test-prevalence-system/"))
#<PREVALENCE-SYSTEM #x56F1156>
? (defun tx-create-counter (system)
(setf (get-root-object system :counter) 0))
TX-CREATE-COUNTER
? (execute *test-system* (make-transaction 'tx-create-counter))
0
? (defun tx-increment-counter (system &optional (increment 1))
(incf (get-root-object system :counter) increment))
TX-INCREMENT-COUNTER
? (execute *test-system* (make-transaction 'tx-increment-counter))
1
? (execute *test-system* (make-transaction 'tx-increment-counter 10))
11
? (get-root-object *test-system* :counter)
11
T
? (close *test-system*)
NIL
At this point the persistent system state consists of a transaction log with three transactions (the creation of the counter, the incrementing of the counter by 1 and the incrementing of the counter by 10). When you re-open the same system (using the same directory), an automatic restore operation will re-execute these three transaction in order to re-create the exact same in-memory state. You can have a look at the file transaction-log.xml in the directory /tmp/test-prevalence-system/. Closing a system is optional (at the expense of one open file descriptor). A closed system will be re-opened automatically when used (the object model itself is untouched by the close operation).
Creating a snapshot can be done at any time, like this:
? (snapshot *test-system*) T
The effect of this is that the whole system (all objects transitively connected to the named root objects) is written out and the transaction log is truncated. You can have a look at the file snapshot.xml in the directory /tmp/test-prevalence-system/. When you re-open the same system (using the same directory), the automatic restore operation will directly reload all named root objects (and their subobjects) from the snapshot to re-create the exact same in-memory state.
There are two examples included in the package that are equivalent to the two main examples in the Java Prevalence ('Prevalyer') implementation:
Managed prevalence implements a scheme to deal with prevalence. It assumes that objects have an id that is set by the system upon creation. For each class, two data structures are maintained: the extent, or the list of all instances, and the id index, or an hashtable mapping an id to the actual instance. Two accessor functions are provided: find-all-objects and find-object-with-id. Three predefined transactions maintain the internal datastructures: tx-create-object, tx-delete-object and tx-change-object-slots. This is all the machinery you need to store, retrieve and change simple objects. Note that for inter-object relations you still need to write explicit transaction functions (not doing so is a well-known prevalence bug: upon restore you will end up with multiple copies of the same objects!).
Blobs are a mechanism to store larger binary data objects outside RAM in an ordinary file. Blobs are real prevalence objects that automatically manage their contents. As far as the API is concerned, you cannot see that the actual bytes are stored in a file.
The Prevalence API is can be found here.
This serialization/deserialization functionality uses a simple XML representation. A more native, possibly binary represenation should be faster. The protocol knows about most primitive Common Lisp types like numbers, strings and symbols, datastructures like structures, sequences and hashtables, as well as CLOS classes. For maximum portability across implementations, we require a generic function called serializable-slots to be defined for each class or struct. For OpenMCL, CMUCL and Lispworks, an implementation using the MOP is provided, returning a list of all slots.
You can re-implement serializable-slots for custom behavior (for example, when you want to make some slots transient).
? (in-package :serialization)
#<Package "SERIALIZATION">
? (defclass person ()
((id :initarg :id :accessor get-id)
(firstname :initarg :firstname :accessor get-firstname)
(lastname :initarg :lastname :accessor get-lastname)))
#<STANDARD-CLASS PERSON>
? (defvar *jlp*)
*JLP*
? (setf *jlp* (make-instance 'person :id 100 :firstname "Jean-Luc" :lastname "Picard"))
#<PERSON #x5648636>
? (with-output-to-string (out) (serialize-xml *jlp* out))
; artificial XML intendation added for readability ;-)
"<OBJECT ID=\"1\" CLASS=\"SERIALIZATION::PERSON\">
<SLOT NAME=\"SERIALIZATION::ID\">
<INT>100</INT>
</SLOT>
<SLOT NAME=\"SERIALIZATION::FIRSTNAME\">
<STRING>Jean-Luc</STRING>
</SLOT>
<SLOT NAME=\"SERIALIZATION::LASTNAME\">
<STRING>Picard</STRING>
</SLOT>
</OBJECT>"
? (with-input-from-string (in *) (deserialize-xml in))
#<PERSON #x565721E>
? (describe *)
#<PERSON #x565721E>
Class: #<STANDARD-CLASS PERSON>
Wrapper: #<CCL::CLASS-WRAPPER PERSON #x564860E>
Instance slots
ID: 100
FIRSTNAME: "Jean-Luc"
LASTNAME: "Picard"
The two main functions accept an optional serialization-state parameter. This is an object that you create using (make-serialization-state). By reusing some datastructures, you make things just a bit more efficiency (and quite a bit more efficient if you do many serializations and deserializations).
The Serialization API is can be found here.
$Id: readme.html,v 1.7 2004/01/13 14:21:34 sven Exp $