The Web Services Technology Pipeline

A Preliminary Essay

[This is one of a series of essays I'm publishing for comment and discussion. Each essay is based on a concept I'm addressing in my forthcoming book, Strategies for Web Services. ...doug]

 

If you were to embark on a simple networking project today, from what source would you obtain your TCP/IP protocol-stack software? The answer is: You wouldn't obtain it at all—TCP/IP is built into virtually every computing platform that supports a networking interface. But this hasn't always been the case. Thirty years ago, anyone wanting to use TCP/IP had to first implement the protocols—that means write the code, not merely install a software package. If you were evaluating or planning a TCP/IP-based project in 1975, your recommendations would likely have been very different from what they would be today.

In 1985, TCP/IP software entered its technology pipeline as an expensive do-it-yourself solution, and came out the other end ten years later as a technology we now expect to be bundled with every operating system, even those that are free. Over that ten-year period, the interoperability, quality, and performance of TCP/IP continued to improve as the cost declined. If you were trying to plan a TCP/IP project during the decade from 1985 to 1995, you were aiming at a moving target, and the successful (financial and otherwise) outcome of your project depended a great deal on its timing relative to where TCP/IP was in the technology pipeline at the moment you decided how you would implement that part of your system.

Sound familiar?

Today, web services present similar challenges. From what source should you obtain your solutions? The choices include, for example, application servers, web-services networks (WSNs), XML proxies, and gateways. We could post yet another matrix and compare the strengths and weaknesses of each of these deployment solutions, but there's one problem: The relative strengths and weaknesses of these options are changing, because the underlying technologies are working their way through the technology pipeline. Furthermore, as observers, we're inside the pipeline as well. It's a relativity issue. We can't tell how much the technologies and deployment options are changing because we're changing right alongside them.

We need to step back, and figure out where each of the technologies that are critical to our projects are in the pipeline, relative to one another and to the pipeline itself. For we know one thing for certain: Technology comes out the end of the pipeline in a form that's more standardized, more reliable and less expensive than when it went in.

Technology Pipeline for Web-Services Software

From the above figure we see that new technologies enter at the top of the pipeline, where the only way to deploy them is through home-brew or do-it-yourself code. As the need for a particluar solution becomes more widespread—among early adopters—the challenge is first taken on by web-services networks (WSNs) who specialize in solving cutting-edge problems, particularly for which the standards have yet to be adopted or widely implemented. As soon as the standards appear, specialty ISVs begin to deliver solutions in the form of relatively high-priced software. Initially, the software is deployed as part of a gateway or proxy, then later packaged with hardware as a complete appliance. Soon thereafter, the solutions are available from a new breed of service provider: web-service delivery networks (WSDNs).

Eventually, the problems become so common, that the major hardware and software vendors release their own implementations. They, too, begin by delivering separate software packages, but eventually the solutions are bundled with application servers or even operating systems. Finally, as the solutions themselves become common, and clear favorites appear from among the crowd, web-service providers—similar to managed service providers (MSPs) on the Web—provide outsourced hosting of the most popular solutions.

The history of software technology is filled with examples of similar pipelines. Consider what happened with TCP/IP. It started out as a DIY-only technology, and was then sold as a fairly expensive add-on by specialty ISVs, most of whom are now long since gone. Eventually the major software vendors such as Microsoft released their own versions of the TCP/IP protocol stack, but as a package priced separately from their operating systems. Finally, with the launch of Windows 95, Microsoft bundled TCP/IP with the operating system, and all other vendors were forced to follow suit. The result was the explosion of the Internet.

Similar descriptions can be written for many of the popular programming languages and for packet-filtering firewalls that began as very expensive systems and are now included with many operating systems. Along the way—through the pipeline—each of these technologies was delivered by a different type of vendor and at a progressively decreasing cost.

In summary, as you consider the many options for the deployment of your web-services projects, be aware that all of the technologies are working their way through the pipeline. If you need bleeding-edge (pre-standard) technology, you should be looking at the earlier stages, whereas those whose requirements are less demanding can avail themselves of later-stage (and less expensive) options.

Doug Kaye, 27 October 2002