Build vs Buy - One Perspective

I'm an engineer. It's in my blood. As an engineer, I want to build things. So anytime the build vs buy discussion arises, I have to fight the urge to say build and get right to it. I am definitely happier building my own stuff than configuring and deploying that built by somebody else. But I'm also a pragmatic engineer that accepts that I should leverage existing products wherever possible. But why should we ever build in problem areas where products exist?

A build vs buy decision is primarily about determining if a vendor product can be sufficiently customized to solve the problem that your organization faces. Products are designed to solve a wide range of problems. They have to be or the product vendor limits their potential customer base. The need to solve a broad set of problems does limit the optimizations a vendor can make. This follows the adage that anything that does several things, does none of them well.

How is it that custom built solutions can be better than commercial products though? You will have less engineering resources and less testing than a commercial product. Shouldn't it be obvious that commercial solutions will have an edge in capabilities and quality? Commercial products lack one crucial element though. A specific problem.

Knowing your problem domain allows you to careful choose your compromises. Every problem has unique constraints that can be leveraged to reduce complexity or improve performance. This is a reality that is inescapable and becomes the crux of the build vs buy decision process. For what you are balancing is the optimizations you can achieve against the resources the vendor can offer in terms of support and on going R&D.

The largest challenge that I've seen is adequately defining the requirements for your current problem and being realistic about your future needs. Comparing feature sets from a vendor with your internally developed solutions is a pointless exercise. The goal is to find a good fit, not to have the broadest set of features. The requirements can be difficult to clearly derive though. There are often second order requirements that are less than obvious but in fact lead to the largest opportunities.

As an example, when looking at messaging systems, we knew we needed reliable delivery. The major revelation however was that we did not need exactly once delivery or ordered delivery. The majority of the information we were propagating possessed inherent keys for managing idempotent delivery and there was no inter-event dependencies. There is a great deal of complexity that can be eliminated and tremendous performance gains that can be achieved when you eliminate ordered, exactly once delivery from the messaging infrastructure. Of course, this is not a good idea in general, but this example illustrates how a careful analysis and understand of the problem can lead to more clarity on the requirements.

Okay, so I'm obviously advocating build over buy. No, not really. There are several factors to consider before building a custom solution that has clear overlap with a vendor product. Some of the key factors are:

• Business Impact - How much does the problem you are trying to solve impact your bottom line? The less impact, the less critical a highly optimized solution becomes. I know this is obvious, but it is worth stating. Optimizing marginal problems is largely pointless.
• Incremental Benefits - How big are the gains you are likely to achieve by building your own solution vs using a commercial product. In the example above, we discovered we could reduce the number of SQL statements per message by 9X which made it very compelling. Had the improvements been 30% or less, we probably would not have embarked on building a solution.
• Holistic Costs - It's very easy to focus on the cost associated with the primary solution and ignore the overall life cycle costs. Not only does the NRE for the component need to be considered, but also all ongoing support costs as well as the second order infrastructure components that are required to support the solution.

If you consider these (and others which I would appreciate hearing about) and still find that the benefits out weigh the costs for a high impact problem, then build makes sense. One thing that I find organizations also fall victim to, is assuming that product companies have smarter engineers. Ultimately, you should understand your problem better than anybody else. Therefore, you should be able to deliver the best solution to your problems. There are lots of reasons to rely upon vendors to deliver your solution, but presumed superior engineering talent should not be one of them.

As always, I welcome your feedback.

Technorati Tags: architecture, engineering, performance, programming, scalability, software, to_read, toread

Comments

[wpbarr]

The decision isn't build vs. buy, it's overcoming the institutional inertia of not invented here. :)

[Arun Jacob]

I maintain a blog with a similar "simplicity" theme among other things. Got pointed to this blog by a colleague who reads both :-)

Great blog. I am hooked.

http://simplicity-works-everytime.blogspot.com/2007/03/simplicity-serendipity.html

[Gunnar]

To add another rail to this discussion, buy and build are not mutually exclusive. Pat Christiansen and I wrote an essay on this callled the Buy/Build Balance

http://arctecgroup.net/040304.htm

If you buy SAP you ae likely to still write tons and tons of code. Was that a buy or a build?

If you build your own J2EE stuff you are still constrained and pushed in certain directions by BEA or JBoss. Was that a build or a buy?

I like to use Clayton Christensen's work to get at the issues you describe as Business Impact, Incremental Benefits, and Holistic Costs. Specifically the law of conservation of modularity is at the heart of this

(summarized by Phil Windley)
http://www.windley.com/archives/2004/03/osbc2004_clayto.shtml

"Clayton introduces the “law of conservation of modularity.” The idea is that either the integrated system or the subsystems need to be modular and comfortable in order to optimize performance for the other. He uses Microsoft OS and Linux as examples. Applications in Microsoft are suboptimized in order for the OS to be optimized (this is a result of closed source) whereas in Linux, the OS is suboptimized via an open and modular architecture in order to optimize the application (i.e. being able to change code in the OS to optimize application performance).
In Clayton’s world, “modular” implies “suboptimal” or “inefficient.” You can’t make money on the modular layers in the value-added chain (they’re the integators). You make money at the borders to the modular layers. Another way of looking at this is that no one is going to make money on SOAP, you’re going to make money on the parts that talk SOAP, the services."

I think this is also at the center of a lot of the Rest/WS-* debates. It is interesting that Redmond invests heavily in opening up while Google shuts down its SOAP pipe...

I don't think there is a one size fits all. So may be you buy more where you need modularity (so you get economies of scale) and build more where you have your secret sauce.

If you look at Ping Identity, they wrap JBoss and you deploy a Ping binary that gives you federation. They have relatively simple needs from a JBoss standpoint, HTTP i/o.

Then you have Cape Clear which deploys inside a JBoss container, you deploy the JBos bits and then load Cape Clear. Cape Clear uses many, many JBoss services - HTTP, messaging, on and on.

[Stan]

"A build vs buy decision is primarily about determining if a vendor product can be sufficiently customized to solve the problem that your organization faces." But don't forget to factor in whether your users can be sufficiently customized to use the product. The more flexible they are, the less work you have to do. My sad experience has been that the customers want what they want, and we spend so much to customize the product into what they had in mind all along that the economics of buy vs build go all wrong.

Or maybe I just really enjoy building. :)

[Rajgo]

Hi Dan,

You have stated some of the important points that must be kept in mind for a build vs buy. However,I think you have missed one important point. The build vs buy decision is also hugely determined by the current execution capability of the team to deliver.

Also, the build or buy decisions are different for Software Product Companies, and for in house IT departments

Large companies probably have more resources and this point might or might not come up depending on the team. But in a small team like the one that I run, this is a major factor for me.

I too am tempted always to go and write some capability that I think I need, but sometimes it is not just worth it.

Here are some horror stories from Hyderbad about some over enthusiastic Developers

One company in Hyderabad chose to build their own bug tracking system, instead of using a third party tool (free or commercial). What a waste of people's time and company money.

In another company, a developer who was a purist OO Guy, wrote an XMI to C# code generator and spent 8 months in generating an object model that should have been built in a month.

The moral of these stories is that the build or buy decision must be taken by mature brains and must not be given just to hardcore developers who have no other point of view.

[Andres]

Alas, would the build vs. buy decision be a rational one! More often than not, however, we encounter emotional resistance. Microsoft (and Oracle and so forth) is a swearword for a large number of people and getting their stuff into the system even if it would make perfect sense is often fought with vengeance. And there's the problem for decision-makers who inevitably can not be familiar with all the technical details of a huge million-dollar system: how to distinguish between real and subjective arguments? Even more: if there is considerable resistance to the system among the engineers for whatever reason, can the system still be bought and implemented successfully?

Good example of this is MS Exchange that I have seen working perfectly fine and fitting nicely in in a reasonably big organization. In another organization of approximately the same size, however, the thing never seems to work properly. It can't be for the skill of people and the environment is fairly similar too. So I think it all comes down to whether you can look at a bought system neutrally without any prejudice.

[Ben Kruger]

Time to market (TTM) and resources should also be major contributing factors. Additionally in smaller or medium sized companies, often times uninformed people are making decisions to buy rather than to build. What compounds these decisions is the product chosen to buy rather than build is not the right fit or necessarily the best product out there. I have been bitten in the ass time and time again with bought solutions that were then replaced by built solutions. However those bought solutions did enable TTM to happen in a quick manner.

[Geva Perry]

Dan - from a different perspective (one that may not interest you as much), I wrote about the Build Vs. Buy issue and why it makes it so hard to market software platforms to software developers.

See here: http://gevaperry.typepad.com/main/2007/02/marketing_to_de.html

[Ian Rae]

Build/Buy decisions are interesting in terms of adding simplicity. Using 3rd-party stuff generally involves getting lots of 'features' you don't need in order to get a few things you do.

I like the adding simplicity idea. My blog talks about an Ira Glass video on simplicity in storytelling.

[Paul Anderson]

Another major consideration with "buy" is that you seldom are buying just that product/API - you are usually buying hidden or opportunistic thin-end-of-wedge vendor lock-in potential.

[Custom Essays]

Thanks for your information. Most of the posts in the blog is really valuable. Regards

[danial]

good blog

[yoyo]

It's not so simply to bring a good enough custom written essay, especially if you are booked. I give advice you to find buy essays online and to be spare from doubt that your work will be done by essays writers

[www.google.com/accounts/o8/id?id=AItOawlDl5cfvrCBkEsG5oD4MRZbdExq8mRtvu4]

Excellent blog

[www.google.com/accounts/o8/id?id=AItOawlDl5cfvrCBkEsG5oD4MRZbdExq8mRtvu4]

nice one

[Narayan]

Your informative article is really helpful for the people who're seeking a good articles on the internet online. It would have helped.....

http://www.webartsense.com