Omniscient Inference, The Promise of SOA, REST, and ROA

I've been doing a lot of reading lately on SOA and REST. I've even spent a little time looking at ROA. There is a common theme that permutes all of these approaches to architecture. It's something I can best sum up as omniscient inference. The idea that software will be able to infer the usefulness of the service or resource through some omniscient capability.  What do I mean? Well, let me tackle each architectural pattern individually.

SOA proponents argue that you can't have a true SOA without UDDI. The promise of UDDI is that software that needs a service can make SOAP calls and discover what services are available. Once discovering these services, your software is now able to infer the mechanisms by which the service can be invoked. It's quite impressive really. Your application will be able to infer from the WSDL the entities that are required to interact with the service, understand the boundary conditions of the service, and therefore be able to immediately leverage the service to extend its functionality.

Okay, this isn't completely accurate.  The intent of UDDI is to standardize the mechanism for organizations to discover what services are available. The problem is that it really tries to serve two masters. Software obviously can't leverage a service a priori. Developers need to design their applications based on the semantics of the service. The problem is that WSDL is a mediocre way to specify the full semantics of an interface for developers. It's difficult to read and even more difficult to document. Proponents would argue that developers don't use WSDL, they use code generators to translate into native languages familiar to the developer. But a Java method doesn't define semantics. Ultimately, the developer needs associated documentation, which challenges the need for UDDI as a service library.

For deployment and routing discovery, UDDI is useful. It's also incredibly complex as most of the information in UDDI is not required simply to manage the rendezvous between a client and service. In fact, most coordination of services does not require the WSDL but rather system qualities about the service provider.  The result is that UDDI has too few semantics defined for developers and too much semantics defined for client/service binding.

REST tries to go in a different direction than SOA.  The goal is simplicity and it does achieve a simpler interaction style than SOAP for simple tasks.  This is a good thing.  The problem with SOAP (and CORBA before it) is that even simple things require a significant amount of overhead. You have to design WSDL, after reading several hundred pages of specifications. You have to translate the WSDL with compilers, and install services frameworks. The performance of your application is hampered by the SOAP marshaling and the framework layers that exist to protect you from SOAP's complexities.  This is reasonable for complex problems but when I want to retrieve is a list of news items, a simple object in itself, then this is unnecessarily complex.  The REST camp is justified in crying foul.

But REST also proposes magic.  Micro-formats will save developers from complex entities.  They will standardize the way information is exchanged and allow applications to just work across the Internet.  In fact, micro-formats are simple enough that applications may be able to extract information from formats they have never seen before, or so the claim. And now we're back to this style of programming that allows our applications to possess omniscient inference.

Micro-formats face two major challenges. First, standardizing across organizations will prove to be challenging. Especially where the contents represents business entities that are offered by competing companies.  What incentive exists for company A to cooperate with company B on a standard definition of their common entity? Businesses thrive in a competitive market through differentiation. It's conceivable that some least common denominator format can be agreed upon, but each will still want to extend it to attract customers to their business.  Even relatively common data will face the temptation for extensions that give one company an edge in the market.

More problematic though is micro-formats are appropriate for simple entities.  As entities become more complex though, the format will no longer be micro.  For example, ATOM and MIME are both formats for representing a message.  ATOM is perfectly sufficient for a broad range of messages but it is not complete.  That's okay because the goal is to make simple messages simple and it does that admirably. But as the message semantics expand,  you have to resort to MIME to adequately model them. Messages are not the only entities that will either outgrow or potentially never fit within a micro-format.

ROA is an intriguing proposition.  Applications are freed from worrying about services at all.  A resource, which is effectively an entity, exposes the state transitions it is willing to accept.  Applications don't even care what the resource is, they simply decide whether they are interested in the state transitions available.  The level of omniscient inference with this approach is difficult to even explain.  So, if the resource is a tennis court and the state transition is reserve (is that a state change or operation?) an application can decide, without caring about the resource per se, that it wants to reserve it.  Of course if this is a seat at a concert you may be committing to a financial transaction. At the very least, the state may change back without the applications knowledge (except it's omniscient so that isn't possible).

The reader may take away from this article that I'm anti-SOA, anti-REST, and anti-ROA. That would be unfortunate as it doesn't characterize my position at all. SOA is necessary for proper decoupling of large architectures. REST has tackled the problem of making simple services, simple.  ROA is truly intriguing, although I'd like to see more attention given to the concept.  No, my objection is to bold claims that can't realistically be achieved. It isn't necessary and detracts from the real value that these approaches bring to the world of architecture.  There are meritorious aspects of each approach so let's focus on how to best leverage those in appropriate ways.

Technorati Tags: architecture, atom, engineering, programming, protocol, uddi, rest, scalability, services, soa, software, to_read, toread, web

Comments

[Pete Lacey]

When I was working at Systinet (now part of HP), we use to call this kind of thinking "science fiction." I'm curious, though, where you encountered this notion. In my experience, reasonable people (SOA or ROA, SOAP or REST) do not promote anything like this. This typically comes from the overzealous sales and marketing guys or the clueless technology reporter. It's along the same lines as being told that someday your fridge will order more milk for you when you're low.

BTW, I think you mean "omniscient" not "omnipotent."

Pete

[Dan Pritchett]

First, thinks for correcting my grammar. All powerful, all knowing, both seem unlikely but yes, you are correct, I meant omniscient and have updated the article accordingly.

Overzealous proponents or maybe I'm just reading too much into the postings but the material is there. And I'm happy to see you agree it's science fiction.

[Bill de hOra]

When I was working at Systinet (now part of HP), we use to call this kind of thinking "science fiction."

It's now called the semantic web.

[Alex Bunardzic]

I've commented in more details here:

http://jooto.com/blog/index.php/2006/12/02/resource-oriented-architecture-growing-pains/

[Chris Stiles]

IMHO .. every generation of programmers is doomed to relive this attempt to define some kind of 'services architecture' - an IT version of the groundhog day where we attempt to find an environment which fosters 'ultimate' reuse.

The previous iteration of such attempts ground to a halt because of problems with tooling - and the fact that then exeunt processors meant that everything was so damn slow.

Now we have tools, but we are approaching two areas that are restatements of other problems; that of complex type inference and that of defining complete and practical formal languages.

We are chasing after things that won't be possible in practice for many computer generations - unless you have a implementation of large scale quantum computing - and may not even be practically possible for real systems.

Chimeras all. The only thing that matters with SOA is roughly what can be done with it now. Most of the rest is just persiflage.

[Vadim Geshel]

My simple theory for why discussions of REST seem to imply omniscient inference is this. Many proponents of REST say, basically: just use what we've used since 95, why add complexity, things already work. Which tends to leave out, or at least not make explicit, the fact that what we've used since 95 is not just resources and statelessness and four verbs, but also a single data format with fixed semantics (HTML), client apps with those semantics hard-coded (browsers), and, most importantly, humans, aka (sufficiently) omniscient inference engines.

[orcmid]

I want to disagree with Vadim about whether there are any semantics to speak of in HTML.

I think part of our problem is that we tend to believe that markup conveys semantics.

I don't think semantics is coded into formats and utilities much, it is more that semantics tends to be preserved by the tool, not that it is in any sense incorporated in the tool.

We are the ones who make all the meaning, and we are mostly unaware of our doing that, which is why there is so much reliance on tacit knowledge in systems.

In all other respects, I am completely aligned, and I think this is a great post and discussion.

[Roy T. Fielding]

Sounds like a paper tiger to me. There are certainly a lot of software sales companies and marketing brochures for tools that describe various buzzwords as a means of enabling omniscient applications, but lumping REST proponents into that category is bizarre. I have yet to see a REST sales pitch. I wouldn't even know where to start one.

Most REST-based applications have a human in the loop directing the selection of transitions based on the content of data presented to them -- they don't need to be omniscient, just aware of the current state and trusting that the resource owner is not misrepresenting the transitions (links, buttons, etc.). We know that works well enough to use it as an example of REST in action and see the advantages of that architecture over other architectures that also have a human in the loop.

Some REST-based applications do not have a human in the loop. I wrote one of the very first -- MOMspider -- which performs a simple task through a preconfigured set of interactions. It only needed to know enough about HTML to recognize safe links and follow them, but the principle is the same in general. The more standardized a media type is for a given purpose, the more automation can be applied to the transitions presented by that media type.

If you can preconfigure an application with enough semantics to find and select the appropriate transitions based on the current state presented to them by the server, then a REST-based application can be performed without a human in the loop.

The distinction here between SOA/ROA/Py and REST is not the presence or absence of omniscient behavior, but rather how much knowledge is necessary to comprehend the application alternatives at each step of the application such that the desired end state is reached (without accidentally purchasing a pink motorhome along the way).

I argue that REST-based architecture is better for that purpose because the states are simpler to understand. Obviously, if you had omniscience then the degree of simplicity in understanding each state would be irrelevant. That is why it makes no sense to say that REST advocates are not aware of the need.

In other words, I agree that this question of omniscience is critical to understanding claims about architecture, but I disagree that anyone who actually understands REST would make such claims. The purpose of REST constraints is to construct an artificial context in which the shared understanding and opportunities for partial failures are as small as possible. It would be absurd to go to all that trouble if we thought such understanding is unnecessary.

I also disagree with your characterization that companies will not define standard media types because they feel a need to compete. The only companies that compete based on data formats are closed-product vendors. The people who buy software, or make it in-house with open source tools, clamor for open standards in data formats and frequently require them in RFPs. They do that for vendor independence and the serendipitous reuse gained from using data formats that can be analyzed independently from the applications used to create them.

[Dan Pritchett]

I agree with your characterizations of what is required to write applications. And I thank you for clearly stating what context a REST client must have.

I think the lack of standardized types is already prevalent. Yes, it is by closed product companies, but ultimately it still occurs. Video offers a concrete example. MPEG is a perfectly reasonable format yet any robust player needs to support Quicktime and Windows Media as well. Google and Yahoo both offer map interfaces but these are not compatible. What we see from consumer driven demand is the requirement for applications to support all of the formats, not a demand for a single format.

[Danno]

RISE FROM YOUR GRAVE!

I always thought the REST omniscience arguments went a little like this:

It'll be pretty easy to jigger with the RESTful API by just making GET, POST, PUT, and DELETE calls against the server and looking at the XML output that gets handed back to you.

So instead of looking through WS-Whatever, you just get your code working with the subset of application state you care about.

As a bonus, given a sufficiently smart REST client, you get some intelligence about the HTTP codes, a set of standard headers, existing HTTP authentication, and one or two other things I forget about.

If you get lucky, a couple of people *have* standardized on some XML data representation and your code now works, no fuss no muss with all of them (or well, optimistically all of them, someone probably misread something).

See, I thought the whole SemWeb thing was about getting the applications to figure out what the data meant in an omniscient manner.