« Build vs Buy - One Perspective | Main | In Support of Non-Stop Software »

Thursday, May 24, 2007


TrackBack URL for this entry:

Listed below are links to weblogs that reference Chaotic Perspectives:

» Thankgod for chaos! from EwanSilver.com
Its always nice to get an Ebay perspective on things - especially when you totally agree with what is being said. For a long time I have felt that if we really want to embrace the power of large scale distributed systems we have to accept that w... [Read More]

» Unappreciated Chaos from Pragmatic Dictator
Dan Pritchett presents some good insight in respect of large scale systems. Im left to ponder the fact that whilst Dan is deliberately creating these systems. I find myself looking at nondeterministic systems a lot lately. Many solutions ... [Read More]



I totally agree with what you are saying. Dealing with non determinstic systems is a major mind shift for most people. The systems they work on, whilst often "big", do not actually require them to think in anything other than a linear 2PC style fashion. Thats where the orthodoxy lies. Unfortunately internet scale requires dumping those assumptions (or at least questioning them!)

Andres Kütt

This is a very revealing perspective, actually. And to comfort you, not only engineers find it usually hard to deal with the un-deterministic, non-linear systems but so also do mathematicians. One of the most challenging courses I saw through my studies were "non-linear dynamics and chaos" and "differential geometry" which both deal with what happens if we add complex input to complex systems. The mathematical tooling we have developed over the centuries just does not suit well for analysis of these kinds of things. Which is probably related to the fact that a human being is not built to handle non-linear or parallel processes. Hence, the challenge of coping with systems of increasing complexity is fundamentally not of engineering but of psychology, learning and mental models.

Ben Kruger

Bringing chaos to the organization is difficult since there isn't a well defined way. We need a list in order of precedence of things that should be considered along with their pro's and con's. We need more reference points.

I think abandoning the traditional thinking is relatively easy once you have that "ah-ha" moment. Unfortunately, he difficulty lies with not just he implementation but the convincing of peers and those higher up in the food chain. What compounds that, is most often those higher up are already set in their mindset.


This from the Inktomi guys in 1998:



What would it take for you to add the one.org banner to your blog to support charity? I have added it to my own and would love to see other bloggers amplify the need to stomp out poverty.

If the activism irritates you then I understand...

James McGovern

Rohit Gandhe

This post was special to me because it very much applies to a system I am working on.

We have a parking system which allows a car to enter a parking lot after collecting Id(s), which can be a Ticket, License plate, Credit card or an RF Id. The same id(s) can be used to exit the parking lot. The patron enters through an entry lane and exits through an exit lane (If that wasn't obvious enough).

Mostly the entry lanes are unmanned and sometimes the exit lanes are too. This adds a lot of complexity to the system, as the software has to take corrective actions in case a device(Ticket Issuing machine) fails. Also, each lane can have multiple cars (as much as three) interacting with the system at one time. There are about 11 devices in each lane and depending on the car sensor location, a car can be in 20 different states (blocking sensors). This creates 220 ways a car can interact with a device and the fact that there can three cars just adds to the complexity.

We created an event driven state machine to handle these states, but as we soon found out, the event driven system works perfectly in the test lab, but regularly breaks down in real world. It turns out that we did not consider temporal factors and continues degradation of various equipment. Our initial reaction was exactly as you predicted; one to add more constraints to the system and consider "time" as well as "events". The more temporal constraints we added, the more event constraints we broke.

We eventually gave up and ended up making assumptions for now (It's a version 1.0 system). For example, in ambiguous situations, we assume that the car is moving forward and the RF id can only be read when the car is at a certain location in the lane... etc.

I am sure there are ways to solve these problems without adding unnecessary constraints, but I just haven't had time to think about them.

The comments to this entry are closed.