Application Interoperability

(I've edited this page from the discussion page for a rhizome called "Mobile Productivity Challenges. Thanks alaric!*)

Alaric Snell-Pym is a computer programmer and blogger from the U.K. who works on database systems among other cool projects, you can find more about him on his home page at snell-pym.org.uk/alaric/.

Part One

I think that application interoperability for mobile computing discussed in the original post is quite important, and often under-estimated.

It's always struck me that in Star Trek, interfacing anything to anything just seems to take a few keypresses. While in the real world, it's often weeks of skilled work.

I can only presume that, in the future, they have better standards for data interchange.

But making better standards isn't just about making more of them; most standards for interconnecting things these days are hobbled by assumptions that mean they are only useful in a limited domain, and so, eventually, become outdated. Rather than making more standards in more domains, we really need to think about how to make standards that assume less to begin with (being closed to the fundamental essence of the problem, as it where, rather than to the currently in-vogue instance of the problem), and standards that are more able to evolve and grow with time.

So in the personal information management arena in particular, I'm taking a serious look at knowledge representation based on propositional calculus. That means storing information as a series of objects (identified by some unique key, such as a URL or an inherent identity - the number five might as well be represented as '5' by universal convention), and relationships relating them.

One way of doing this is embodied by RDF, in which relationships always link exactly two objects (and objects identified by URLs are handled slightly different from objects represented by their value, which get called properties, but the basic ideas the same). This is simple, but can be awkward at times. Also, RDF suffers from confusion about the meaning of objects. In the RDF model, a URL, what the URL refers to (which could be a concept, or a real-world object, or anything), and what the URL resolves to (eg, a web page or something) are all different concepts, but the latter two are usually hopelessly confused. When one makes statements about http://www.snell-pym.org.uk/alaric/, are you talking about me or my homepage?

Another way is that embodied by Prolog, in which relationships can be between any number of objects - we might say "created(foo, 2011, 08, 14, bar)" to say that some object Foo was created on the fourteenth of August, 2011, by somebody called bar; it is purely a matter of convention that the first part of a "created" relationship is the created object, the second part the year, and so on. This is more compact than doing the same thing in RDF, where you'd probably create a separate "creation event" object linked by relationships to the created (foo), and to the date, and to the creator (bar), but is also more complex to work with.

A third way is that embodied by Lojban, where one makes statements that involve any number of objects (like Prolog), but where the objects involved in the relationship are identified by tags and can all be omitted. Sparing you the syntax and vocabulary of Lojban, this is something like "created(object=foo, year=2011, month=8, day=14, creator=bar)". This becomes somewhat equivalent to the RDF concept of having a creation-event object linked to things with simple relationships, while retaining the compactness of Prolog's model and not creating a two-tier system of simple and complex relationships (the latter being those that need an extra kind of object to embody them), especially if you have a mechanism to give statements names so they can be objects to make statements about themselves.

So what's so neat about this? Well, it's extensible. Consider the traditional approach for representing data - as a table, like in an SQL database, with one row per 'object'. If you have some new kind of information to store about your objects, you need to create a new column to put it in. Every object has to be assigned a unique type by being put in the correct table. It's all very rigid. In a predicate calculus model, you can make multiple statements about the same object. That "foo" we spoke about the creation of above might be a person, and therefore also be involved in "friend-of" relationships, while other things that were created might be computers, and therefore be involved in "plugged-into" relationships. You can record the "type" of an object with a single-object relationship, such as "is-person(foo)", but of course, an object can have many types - "is-manager(foo)" might also be true, as well as "is-physical-object(foo)". In effect, rather than reserving a bit of storage somewhere in the database for each object, we instead reserve storage for each statement about objects, and find out the "truth" about an object by assembling all the statements about it. That means your "schema" of what you can say about objects isn't limited and can easily grow. And it also means that you can merge data from multiple sources by just learning more statements about the same objects. Merging data in SQL databases and other more traditional types of storage is a lot trickier.

So imagine if all your personal information apps shared a predicate calculus store full of objects. They might agree on a core set of relationship types - name(thing, "The name of the thing") would assign names to people, titles to TODO items and appointments, and so on; so any object that has a name relationship can be represented to the user by that name. The appointments app might have a vague understanding of people (so they can be invited to appointments), just looking for things in your database that match is-person(X) to give you a list of people to invite, while the contact list manager might have a much wider vocabulary of relationships involving people.

However, when you installed new applications that shared this store, they would register themselves. The contact list manager would record that it can display and edit objects that match is-person(X). So when you select the sender of an email in the messages application, it would know that the contact list manager can be called upon to display your stored information about that person. If more than one application is registered for a type of object - then they can share the screen real estate together... So if one wrote an app to handle PGP key management, then it could also respond to people and handle relationships binding them to PGP keys, and then whenever you viewed a person, their PGP keys would appear alongside the information displayed by the core contact list manager; and when you edited them, a button to try and fetch a key for them from the keyservers would appear (if they didn't already have a key), and so on.

It would also be possible to attach "I am something that needs doing!" relationships to anything and have the task list manager represent them. If foo is an email that needs replying to, we can record that with:

 requires-attention(foo) 
 priority(foo, "urgent") 
 deadline(foo, 2011, 08, 25)

Or an email with multiple tasks in it could be handled by creating a heap of fresh new dedicated task objects, and simply linking them back to the email text with an appropriate relationship. The task manager need not know anything about emails, or other objects that might be tasks; it would just focus on displaying their names, in priority or deadline order, and handling their "done" relationship, and all that - and delegating the rest to other applications that have registered to deal with that type of object.

Part Two

There are plenty of other awesome things about knowledge bases I didn't think to include the first time and should go back and add... For instance you can write inference rules in them (fully Turing-complete, in fact; that's what Prolog is) to work stuff out, as well as storing static facts.

Say you have a "part of" relationship you use to, say, organise TODO items into projects, and things like that. These relationships might be used by the user interface to provide useful navigation between objects. But then say you want to attach multiple TODOs to an email message somebody's sent you asking you to do things, so you can easily navigate from the TODOs back to the email - "part of" isn't quite right for that, and besides, you'd like to point to the email and point to the position in the email where they ask you to do the task, so it's clear and you don't have to scroll. So you make a new "derived from" relationship that links something to an email message, and specifies a position in the text as well.

And so you end up extending your UI to look for "part of" and "derived from" relationships in order to provide useful links. And this process continues with other kinds of navigational relationships, and your UI starts to become more complicated because of all of this and, worse, the UI ends up with lots of "domain specific knowledge"; and you need to keep extending the UI to understand new kinds of navigational link.

At this point, you should introduce a new relationship, called something nice and generic like "is related to", that links an object to an object it's related to, with other attributes: a string describing the relationship, for a start.

You might actually use that relationship in your knowledge base, but most important, you put in inference rules, saying "X is related to Y with relationship 'Is a part of' if X is a part of Y", "X is related to position Y within Z with relationship 'Originates from' if X is derived from position Y within Z", and so on. So when the UI, displaying an object X, queries the knowledge base for "Anything that is related to X" and "Anything that X is related to" in order to draw the links, the KB notices the inference rules and knows to check all these other things as well.

And then you can get funky. Write an inference rule that marks two people as related with relationship "Shares a birthday" if they have the same birthday. In fact, write an inference rule for the calendar system that says there's a note on the date YYYY-MM-DD saying "It's X's birthday!" if there's a person with name X whose birthmonth is MM and whose birthdate is DD. And pow! Without extra logic in your UI, people's birthdays appear as notes in the calendar. All this cool stuff can happen inside the knowledge base, as libraries of "useful knowledge" you can load in from a public repository and put alongside all your personal knowledge about your own life.

I'd really like to have all my email and other forms of messaging feed into my knowledge base, so I can put relationships between people and messages I've received from them or sent to them. This could be done by modularising the "backend storage" of the knowledge base, so that I could write an IMAP interface that queries my mailboxes and returns what it finds as series of knowledge-base statements: "X is an email. X has message ID . X is from address A. X has subject . X has body ..." But why stop there?

For many years I was a freelancer, with a big home directory full of files relating to past projects that I might need to delve back into if the project came back to life. How about having a filesystem interface that produces statements like "X is a directory. X has name . X contains file Y. Y has name . Y has type .", which I can combine with some static facts about what subdirectory relates to what project, and who the client was, and what people are associated with that company. Then I'd easily be able to find all the files and emails pertaining to a given project. And if I put my timesheets in the knowledge base, that information would all be available, too.

And a generic knowledge-base browser might not know what files and directories and projects were, but the KB would tell it that files and directories and projects have names, and are related to each other, and sometimes are related to objects like people that it might have a dedicated handler for, so it'd be able to navigate the objects of my electronic world to some degree, even without having any custom plugins written.