Watershed's Communications staff use an in-house desktop application, named Communicate, to edit and control "what's on" information for the organisation. Communicate uses a Cocoa persistence framework, I wrote some years ago in Objective-C, to fetch and save objects into an Openbase relational database. The framework uses plug-ins to directly connect to and communicate with databases from different vendors.

An Objective-C, Cocoa, business objects framework is written on top of the persistence classes which describes and implements all the business logic. This framework is a copy of a server-side, WebObjects, Java framework.

Network performance is good, especially as most users use the applications over a LAN or local wireless connection. As the size of the database has grown, unarchiving large streams of data into the object graph is beginning to use a significant amount of time.

The persistence framework was created, over five and a half years ago, in 2003. New technologies used to develop desktop applications have become available. The persistence framework uses its own classes for sorting and qualifiing object fetch requests. For instance, we could reduce our maintenance footprint by adopting NSSortDescriptor and NSPredicate.

A new persistence framework must give Watershed database independence. We would like to move from Openbase to MySQL some point in the medium term. The flexability is currently provided by database specific plug-ins and we would like to change this to use a three tier architecture for desktop client applications.

Using a set of desktop frameworks, written in Objective-C, mirroring a set of server frameworks, written in Java, means defining and implementing any object relationships, validation rules and convenience routines twice. It's difficult to create comparable classes using Java and Objective-C and it's obviously twice the work and twice the maintenance. A new framework should help alleviate this duplication.

Core Data is Apple's object life-cycle and persistence framework. Using Core Data has advantages, such as automatic undo, and moving forward will gain new features such as possible automatic AppleScript access to the model. Core Data has a natural synergy with bindings, KVC and KVO. NSManagedObject implements many techniques, including uniquing, identification and change tracking, used in a persistence framework. It would be sensible for a new persistence framework to take advantage of existing technology in Core Data.

Web applications, such as Basecamp, Lighthouse and soon Watershed, frequently provide a REST API to the services. Often the API sends and receives XML over an HTTP connection. This API can be used to build a wide range of software but is especially useful for desktop applications, widgets and iPhone applications communicating with the web service. A persistence framework would be all the more valuable if it helped us to communicate with these types of services.

The existing persistence framework can still be used. There is nothing intrinsicaly wrong and we can write adapters for almost all networked, relational databases. The framework connects to the database and fetches the data it needs. This approach where the framework connects and authenticates directly to the database just feels wrong viewed alongside mordern techniques for over the internet delivery.

Some performance issues could be resolved by adding the ability to partially load business objects, similar to the way raw rows can be used in EOF. We suspect performance improvements could also be found in the routines that deal with the model lookups and the creation of Cocoa objects from huge streams of database data.

We have an object persistence framework that we have been using for the past five years. We could leverage some of the existing code by moving the framework onto Core Data and modernising the interfaces. Using an in-memory store and the notifications we could replicate the functionality we currently have. Moving our base business object to a subclass of NSManagedObject we can reuse many of the change tracking functionality built into NSManagedObject to store our objects into a network RDBMS.

FBTFPersistentObjects uses built-in database adapters. For each database vendor, there will be an adapter. The database adapter subclasses manage SQL generation and network communication. The database adapter is not dynamically loaded but is compiled and linked into the framework therefore changing databases could require substantial work meaning no real world database independence.

Clients connecting directly to the database requires negotiating network security and authenticating users' requests on the database, loosing our independent user management. Business logic and validation written into the client can't be easily reused in other applications.

The third option is to essentially use Core Data as a local cache for REST web services. Using a local SQLite store, each client would keep its own copy of the used portions of the server data. The managed object model would be interrogated to generate network request URLs and formulate request data. A mapping delegate or configuration file would also be available to replace the defaults on a project-by-project basis.

Using the delegate notifications from the managed object context, changes would be propagated through RESTful HTTP POST and PUT methods to URLs on the server. At this point objects can be updated, inserted and removed from the server store. Fetches are likely to be performed against the server if a network connection is available and the local cache updated. Data schema formats could include XML, JSON or Binary Property Lists.

By using REST stores we're complementing the work we have been doing to add RESTful request handing to WebObjects applications and would be useful when writing desktop clients for the upcoming changes in dShed.net as well as the ever increasing number of web applications with REST APIs.

To be a truly effecient companion to REST web services, NSManagedObjects should be capable of sub-atomic operations. We would need to find out if managed objects could be made to support property values that can be trickled in as they are needed, in a similar fashion to the way relationships are faulted in.

iPhone

Working closely with Core Data prohibits using the persistence framework with iPhone client applications. With the limited memory and performance available on the iPhone, I suspect it would be advantageous to create a separate, simpler, resource saving mapping routines that deserialze XML or binary Property LIsts.

WebObjects REST Handler

We've been using WebObjects' frameworks to create web applications for some time now and have recently been working on a method to quickly and effortlessly add RESTful interfaces to applictions.

The REST interface over HTTP was designed, not only to provide a publicy available API, but also as low configurtion data provider to a desktop persistence framework.

Ruby on Rails

Whilst, not heavy user of Ruby on Rails, it's impressive how simply and amazingly quickly a web application can be developed to vend XML, or JSON, data for a desktop client. These applications use type hints in the attribrutes of the XML and expect XML elements to be named to a convention. Any Cocoa persistence framework developed should be able to work closely, and cleanly, with a generated scaffolded Rails application with little configuration.

It appears to be with increasing frequency that one can hear experienced Cocoa developers reminiscing about EOF for desktop applications and new developers wondering why Core Data doesn’t use stores in network RDBMS’s from a range of vendors.

As Watershed’s persistence framework will shortly be replaced, I’ve been working on some possible replacements. One of the best things about being an in-house developer is, I get to to see people using my applications every day. I walk past people’s desks and see which windows are open and what tasks users perform. An incredibly frequent task runs as follows:

1. The user needs to make a quick edit to an item stored in a relational database.
2. The user opens their Cocoa client and a window opens, normally containing a NSTableView listing all the entries.
3. The persistence routines generate the required SQL, sends it to the database and receives a stream of data.
4. The persistence routines turn the data into a object graph generating thousands and thousands of objects. Each business object, even with relationships represented by proxy objects, contains many attributes each of which is represented by an object.
5. Once the persistence routines have finished creating the thousands and thousands of objects, making copies & caches and linking them together by setting object instance variables: the user opens the first item in the list, corrects a typo and quits the application.

The problems evident in the example could be seen as a UI issue. Possibly the way forward is to follow iPhone applications’ methods of display a small number of entries then prompting the user to load then next batch. iTunes displays thousands of tracks in a table on launch so why can’t my applications do the same? The business object implemented using NSManagedObject or an EOGenericRecord is an atomic unit. Why does the application need to populate an entire business object just to display the title in one column?

Caching large chunks of data on the client made efficiency enhancements that were just not needed. Launch times are incredibly slow, damaging the user experince. Worried about excess network traffic, we started off designing clients with aggressive caching. Our controllers in the Cocoa applications now save changes more frequently, and with less modality in the edits. This results in a constant trickle of data across the network but a much smoother user experience.

Public applications with a large number of users render bulky local caches ineffective. As the number of editors increases, the probability that a local cache is no longer synchronised increases.

Frameworks such as EOF or ActiveRecord work very well over high speed, high quality networks but many network connections for Macintosh, and iPhone, clients are mobile and made wirelessly or are intermittent. Many clients connect over fragile internet connections where the performance will destroy the user experience.

So, no I don’t want Apple to re-release EOF for Objective-C or allow Core Data stores to reside in a network RDBMS—I want something better.