« Compute Power, Is Your Architecture Green? | Main | Build vs Buy - One Perspective »

Friday, February 09, 2007


Noah Campbell

Can you provide some references on data architecture for performance?

Geva Perry

Excellent post, Dan. These are very similar principles to the ones we refer to with our Space-Based Architecture. Other ideas for good design for low latency include:

1. Co-location of the tiers (logic, data, messaging, presentation) on the same physical machine (but with a shared-nothing architecture so that there is minimal communication between machines)

2. Co-location of services on the same machine

3. Maintaining data in memory (caching)

4. Asynch communication to a persistent store and across geographical locations

There's more about this here: http://www.gigaspacesblog.com/2006/12/09/sbas-general-principles/

Dossy Shiobara

As I explain it to colleagues, "latency is 'death by a thousand papercuts' to your poor design."

I always warn folks away from premature micro-optimization (shaving microseconds off an operation, etc.) but an effective design needs to treat overall latency as a first-order constraint, as you point out. Because, you know, "the speed of light isn't going to change"), and latency translates into effective throughput.

Guy Nirpaz

I find the part on data decomposition as most insightful. One of the killers for this state of mind is the ORM trend on top of relational databases.

Bob Carpenter

It's a problem even if you don't have global networking issues.

My first real-world programming assignment was to integrate the SpiderMonkey implementation of JavaScript into a cross-platform, multi-threaded, highly memory constrained speech recognition engine. The Javascript was being used to design grammars that defined what people could say in response to prompts. Any latency was a customer waiting for the system to say something.

To make a long story short, the requirements were something like 50ms average response time and a maximum 200ms latency. The first integration hit 20ms average response time over the test collection (over 48 threads sharing the Javascript code). But the spikes in latency were terrible -- up to 2000ms.

Of course, the culprit was garbage collection. The solution (hat tip to Sol Lerner) was to build a completely new Javascript engine every call on every thread (that is, every dialogue turn). The setup and teardown cost was about 30ms, which was just fast enough to squeak by under the average requirements and sail under the maximum latency requirements.

jony mill

thanks for the article
all engineering topics team

The comments to this entry are closed.