XML-RPC for OpenMCL

Introduction

This is an implementation of XML-RPC for OpenMCL. PLEASE NOTE: This now a project at CL.net as http://common-lisp.net/project/s-xml-rpc.

release 6, Januari 21, 2004: ported to LispWorks, added *xml-rpc-package*, *xml-rpc-call-hook* and execute-xml-rpc-call
release 5, Januari 20, 2004: added ASDF support, included some contributed patches
release 4, June 10, 2003: tracking a minor, but incompatible change in the XML parser
release 3, May 25, 2003: the XML parser has been put in a separate package, we are using the more efficient SAX interface of the parser, improved documentation
release 2, Januari 6, 2003: we now pass the XML-RPC validator tests, various important bug fixes, improved documentation
release 1, December 15, 2002: first public release of working code, not optimized, limited testing

XML-RPC is a de facto standard for making remote procedure calls between software running on disparate operating systems, running in different environments and implemented using different languages. XML-RPC is using HTTP as the transport and XML as the encoding. XML-RPC is designed to be as simple as possible, while allowing complex data structures to be transmitted, processed and returned. The protocol is described in detail on http://www.xmlrpc.com/. Some key features (both positive and negative) of the XML-RPC protocol are:

It is a published protocol implemented in a variety of languages and operating systems.
It allows communication between applications implemented in different languages.
It allows communication over the internet and through firewalls.
It is not intended for time-critical communications, or for the transmission of large volumes of data.
It is an asymmetrical protocol allowing calls from the client to the server but not callbacks from the server to the client.
It is a very simple protocol that it not defined, controlled or endorsed by a standards body.
It works.

OpenMCL is a open source Common Lisp implementation for Mac OS X (Darwin) and LinuxPPC. You can learn more about OpenMCL on http://openmcl.clozure.com/

This implementation was written by Sven Van Caekenberghe in the (northern hemisphere) winter of 2002. It contains a base64 encoder/decoder, an independent module that could be useful in other applications. Both client as well as server side XML-RPC is implemented. There is a standalone networking implementation (using some of OpenMCL's multi-processing and socket API), as well a networking implementation using (Portable) Allegro Serve (see http://opensource.franz.com/aserve/ or http://sourceforge.net/projects/portableaserve/), which you have to install first. An implementation of both the client and server part of the XML-RPC validator tests is also included.

Distribution

You can download this software from my homepage at http://homepage.mac.com/svc where you will find the archive http://homepage.mac.com/svc/xml-rpc.tgz. This code is also meant to be included as an example in the OpenMCL the distribution.

All software and documentation is Copyright (C) 2002, 2003 Sven Van Caekenberghe. You are granted the rights to distribute and use this software as governed by the terms of the Lisp Lesser GNU Public License (see http://opensource.franz.com/preamble.html), also known as the LLGPL.

The implementation consists of 2 (optionally 4) files:

xml-rpc.lisp - the main file that require-s base64 and xml contains encoding and decoding support as well as a standalone networking implementation of both the client and the server side.
base64.lisp - a base64 encoder/decoder.
aserve.lisp - optional supplement to use (portable) allegro serve to do the networking.
validator1.lisp - optional supplement implementing the validator1 test suite.

You will have to download my Simple XML Parser for Common Lisp (or read about it), from which you'll need at least the file xml.lisp.

Installation

Basically, you load the files respecting their dependency order: xml-rpc depends on base64 and xml. To make things load faster, compile the files respecting their dependency and load order. The optional aserve code can only be loaded or compiled once you install the (portable) aserve package. The optional validator1 test suite obviously depends on the whole implementation. It is also possible to use the provided ASDF file.

Usage

Client

Making an XML-RPC call is a two step process: first you encode the call, then you make the call.

? (xml-rpc-call (encode-xml-rpc-call "currentTime.getCurrentTime") :host "time.xmlrpc.com")
#<XML-RPC-TIME 20021216T06:36:33>

? (xml-rpc-call (encode-xml-rpc-call "examples.getStateName" 41) :host "betty.userland.com")
"South Dakota"

If you are behind an HTTP proxy, you have to pass that information along using keyword arguments. For all keyword arguments, there are default global variables (because most of the time you talk to the same host).

? (xml-rpc-call (encode-xml-rpc-call "currentTime.getCurrentTime") :host "time.xmlrpc.com" 
                                                                   :proxy-host "myproxy" 
                                                                   :proxy-port 8080)

When all goes well, an XML-RPC call returns a result that is decoded into a Common Lisp value. A number of things can go wrong:

There could be a problem encoding the call.
There could be a problem in the networking, up to the HTTP server responding with a code different from 200 OK.
The result of the call could be an XML-RPC fault: this is how the server responds to an error that occured on the server while executing the call there.
There could be a problem decoding the call's result.

At the moment, all these cases are reported by the standard error mechanism. Later we could use different condition types. There are two structs, xml-rpc-time and xml-rpc-struct, that represent two XML-RPC types, iso8601.dateTime and struct respectively. Two convenience functions with the same name come in handy when creating these structs:

? (xml-rpc-struct :foo 1 :bar -1)
#<#XML-RPC-STRUCT (:BAR . -1) (:FOO . 1)>
	
? (xml-rpc-time 3000000000)
#<XML-RPC-TIME 19950125T06:20:00>

The function xml-rpc-aserve:xml-rpc-call-aserve does the same thing, but uses the (portable) aserve HTTP client API for the networking.

Unit tests throughout the source code (marked by #+unit-test and usually wrapped inside an assert) can serve as (executable) examples. A more complicated example is the server and client implementation of some tests in validator1.lisp. Remember that XML-RPC method (function) names are case-sensitive, as are the names of XML-RPC structure members.

Server

Only a single function call is needed to get the server up and running:

? (start-xml-rpc-server :port 8080)

From now on, your lisp image becomes an XML-RPC server, listening for HTTP requests on port 8080. By default each function in your lisp image becomes available remotely, provided its signature is compatible. The is not very secure, but it is easy and powerful. You should customize this behavior for production usage. In more detail, this is what happens:

The XML-RPC call arrives as XML encoded characters in the body of an HTTP POST request
The characters received are parsed by the XML parser and decoded on the fly (using a SAX-like interface) following XML-RPC semantics into a a string method name and a possibly empty list of Lisp objects that are the arguments
The value of *xml-rpc-call-hook* is applied on the string method name and optional argument list
The default value of *xml-rpc-call-hook* is execute-xml-rpc-call which interns the method name in the package *xml-rpc-package* (whose default value is the XML-RPC package itself) and applies the function bound to that name to the argument list (if any)
The result is encoded as an XML-RPC result and returned to the client
If anything goes wrong in any of these steps, a Lisp condition is thrown which is catched and then encoded as an XML-RPC fault and returned to the client

Customization points are *xml-rpc-package* or *xml-rpc-call-hook*. Setting the variable xml-rpc::*xml-rpc-debug* to t makes the server more verbose. Note that XML-RPC method names are case sensitive: for example, clients have specify "LISP-IMPLEMENTATION-TYPE" for the corresponding Lisp function; a server has to define a function named |login| if his clients look for an implementation of "login".

AppleScript can make client-side XML-RPC calls. So provided you have your lisp XML-RPC server running, you can have lisp to the math like this:

tell application "http://localhost:8080/RPC2"
  set call_result to call xmlrpc {method name:"+", parameters:{10, 20, 30}}
end tell
display dialog the call_result buttons {"OK"}

Calling the functions xml-rpc-aserve:start-xml-rpc and xml-rpc-aserve:publish-aserve-xml-rpc-handler does the same thing but uses the (portable) aserve server framework to handle incoming HTTP requests.

XML-RPC Client and Server API

The XMl-RPC Client and Server API can be found here.

Type Mapping

This XML-RPC implementation for Common Lisp maps types as in the following table. There is a small difference between what types are accepted by the encoder and what types are returned by the decoder.

XML-RPC Type Accepted Comon Lisp Type Returned Common Lisp Type

string string string

int, i4 integer integer

boolean t or nil t or nil

double float float

base64 any array of 1 dimension with at least (unsigned-byte 8) as element type an array of 1 dimension with (unsigned-byte 8) as element type

is08601.dateTime struct xml-rpc-time struct xml-rpc-time

array list or vector list

struct struct xml-rpc-struct struct xml-rpc-struct

Later, generic functions to encode and decode arbitrary CLOS instances could be added.

Base64

The code in the package "BASE64" is an implementation of Base64 encoding and decoding (part of RFC 1521). Encoding takes bytes (a binary stream or a byte array) and produces characters (a character stream or a string). Decoding goes the other way.

Base64 API

The Base64 API can be found here.

$Id: readme.html,v 1.14 2004/01/21 11:48:30 sven Exp $

XML-RPC Type	Accepted Comon Lisp Type	Returned Common Lisp Type
string	string	string
int, i4	integer	integer
boolean	t or nil	t or nil
double	float	float
base64	any array of 1 dimension with at least (unsigned-byte 8) as element type	an array of 1 dimension with (unsigned-byte 8) as element type
is08601.dateTime	struct xml-rpc-time	struct xml-rpc-time
array	list or vector	list
struct	struct xml-rpc-struct	struct xml-rpc-struct