The Who’s and What’s of ESBs and eSBs

The software integration business is notorious for its acronyms, and any new integration technology or technique worth its salt must have a compelling TLA (three letter acronym) to go with it. The latest TLA making the rounds is ESB, which stands for enterprise service bus. We asked Jim O’Leary, vice president of product management at integration software vendor EXTOL International to give us a rundown on ESB and its little brother, eSB.

Alex Woodie: ESB–that refers to Red Hook Brewery’s award-winning Extra Special Bitter, right?

Jim O’Leary: Correct, although eSB (lower-case “e”) tends to be less filling and taste better.

AW: How is an ESB different from other types of integration technologies, such as EDI and MQ Series?

JO: Although not identical, ESBs and message queuing products like WebSphere MQ (MQ Series) are closely related. They provide a robust mediation infrastructure for connecting different kinds of systems to each other, and in many cases include services and tools that facilitate interoperability between systems that were designed and built independently. ESBs also have become parts of larger product suites, so they are both software products and components of other software products. Larger ESB suites may include EDI and message queuing support. Where that is the case, the role of an ESB is to integrate EDI and other integration services into the enterprise infrastructure (whether SOA or otherwise). In the past five years or so, ESBs have evolved in parallel with other integration middleware. As so often happens in our business, good ideas from ESB implementations have been incorporated in integration broker suites, and vice versa.

AW: I’ve heard ESB mentioned in relation to SOA. What’s the connection between the two?

JO: ESBs are good choices for implementing large-scale, multi-domain integration. SOA, on the other hand, is an architecture, one that is particularly well suited to implementation using ESBs. The focus of most SOAs is on enterprise-level business processes, and the integration of those processes using services. ESBs provide message binding, routing, persistence, and other core services, plus connections to process automation, data transformation, enveloping, resource adapters, and other services needed for SOA integration. SOA doesn’t go anywhere without significant success in integration.

Whether all of these services are included as part of the ESB Suite or instead must be separately acquired and integrated depends on the ESB product in question. The lines between ESBs and other categories of integration middleware have now blurred so much that a prominent industry analyst recently had this to say: “Whether you buy an integration suite, an ESB, or an ESB suite doesn’t matter. Ultimately, organizations with SOA and integration projects will need the combined set of features, and these product categories overlap so much that they are increasingly indistinguishable.”

AW: How do I know if I need an ESB if I work at a typical mid size manufacturer or distributor that relies on the AS/400?

JO: Good question. It’s a fact of life that most companies using the AS/400 have very limited IT staffs, with little or no experience with ESBs or other integration middleware. Sometimes just a handful of people manage all the IT function. So (relative) simplicity and ease of use are critical factors.

A manufacturer or distributor–and other SMBs, for that matter–is more likely to be driven by external requirements such as customer mandates or compliance reporting. It is less often that an internal initiative will be adopted that will justify the acquisition and use of a technology like an Enterprise Service Bus. But the nature of ESB (or “little e” eSB) is not always something that external requirements will demand. So, we have mixed signals. Requirements for and success with eSBs will come where the technology can be cost-justified for the current project, and then can be extended for the future. Because “little eSBs” have a focus on results, productivity, and usability, which are critically important aspects because of the tight timeframes, they’re more likely to satisfy a requirement in the small-to-medium-sized manufacturer or distributor.

AW: What ESB products from what vendors should be on my “look at” list if I was a typical AS/400 shop?

JO: Of course, we’d like to see our Business Integrator product on everyone’s list, but that’s a hard question to answer without some qualification. So here are some factors to think about.

First, make sure that your project requirements really call for an ESB, rather than something less capable but better suited to your business needs. Vendors like EXTOL offer ESB subsets that are easier to cost-justify initially, but enable you to grow into a full ESB suite later.

Second, look beyond the immediate needs of your current projects and identify requirements you see coming down the road. If you can justify an ESB acquisition for multiple projects, you’ll be able to consider a broader range of options.

Third, look at the specific capabilities you need for current and planned projects: Are you implementing a SOA? Integrating internal applications? Connecting external partners using traditional or non-traditional EDI? Supporting data syndication and analysis? This is where the possibilities become too numerous to make a single recommendation possible.

Fourth, take a hard look at your IT staff and skill sets. Even a crackerjack IT staff can become overwhelmed by some of the larger and more complex ESB suites. So consider carefully where the right trade-off lies between power and ease of use.

Finally, if you’re a 400/System i shop, platform support, portability, and interoperability are obvious concerns. While EXTOL and a few other vendors can deploy on IBM i OS, running your ESB on i OS will limit your choice of offerings. And if your System i is heavily utilized, adding the overhead of an ESB could be counter-productive. Most businesses that use System i also use Windows, Linux, and other platforms, and i OS offers excellent interoperability with these platforms, so it’s worth considering outboard deployment of your ESB on a Windows or Linux server, as an integration “appliance.”

And remember that ESB is a middleware category, not a product name. As the analyst quoted earlier observed, ESB Suites, integration broker suites, and other middleware products have a great deal in common. Many middleware products that include ESB functionality (like ours) are not marketed as ESBs or ESB suites. So look at your current and future project requirements and match them to the best solution for your business, regardless of the product’s name or marketing spin.

AW: Why are ESBs not more broadly adopted outside of larger enterprises?

JO: Early adopters of any technology are typically in large enterprises; smaller organizations will adopt technologies after they have been proven-out in larger enterprises. Based on that experience, one would expect to see a strong foothold for ESBs among the more progressive small-to-medium businesses at this point, because they have, indeed, been proven in larger enterprises. But that isn’t the case, that foothold doesn’t exist. So, we have to conclude that there are inherent differences between larger and smaller companies, thus accounting for the resistance in the smaller companies, at least in the case of ESB adoption.

So how do SMBs differ from large enterprises in ways that might explain this? As suggested earlier, small to medium-sized businesses (SMBs) have highly-constrained IT staffs and skill sets, and they often find it difficult to consume and deploy technology that may require a significant integration effort before the benefits emerge. Because they tend to invest in new technology at a project level, SMBs are somewhat more price sensitive. They usually don’t invest with the consideration of multiple-year enterprise requirements. They generally look at benefits and projects one at a time. SMBs are more likely to be driven by external requirements such as customer mandates or compliance reporting. It is less often that an internal initiative will be adopted that will justify the acquisition and use of a technology like an Enterprise Service Bus. But the nature of ESB is not typically something that external requirements will demand. Finally, technology acquisition in midsize and smaller companies is usually part of a project with a specific implementation in mind, so the timeframe is highly compressed. While vendors are working on this, the integration and deployment effort is still an issue with many ESB products.

What does this profile mean? What are the consequences for the adoption of technologies like ESBs by mainstream companies?

There is generally less interest in acquiring large-scale infrastructure because, as mentioned, most investments occur at a project level. Also, companies are willing to buy technology that can be cost-justified for the current project and can be extended for the future, and less willing to buy something that needs to be cost-justified over multiple projects. SOAs generally focus on enterprise business processes and often go beyond the scope of individual projects. The resultant lower SOA adoption rate in smaller companies has had an effect on the adoption rate of ESBs in these companies. Lastly, these companies usually focus on results, productivity, and usability, which are critically important aspects because of the tight timeframes.

There is little dispute over the value of ESBs in early-adopter companies. But there is an open question as to whether the reason that these companies adopted ESBs also applies to many smaller companies. ESBs have the ability to address issues in companies of all sizes, but as of now, they have not been brought in to mainstream companies of small to medium size. That’s where that concept of the “less filling” eSB, the “little-e” eSB, came from, a technology approach that is more suitable to the technology and business realities of the SMB.

AW: What is an eSB and how does it differ from an ESB?

JO: The “little-e” eSB is a kind of ESB that better suits the requirements of smaller or medium-size enterprises, or departments or divisions of larger enterprises. eSBs specifically address the need for integration of heterogeneous systems. That’s really the type of business problem that SMBs or divisions and departments are focusing on, more than, for example, SOA. From a functional, black-box point of view, the pure-play ESB and ESB suites are really the same thing as the eSB, in that they each consume various kinds of service requests, data, and event signals, and produce the same kinds of service requests, data, and event signals out the other end.

So, what’s really different? How you get there is one of the big distinctions: more pre-built functionality versus customization requirements. Generally the eSB uses a message bus internally; the message bus may not necessarily be externally exposed, except indirectly, through adapters.

But the primary difference is that the eSB provides pre-built services that are designed to be used together, as building blocks. For example: event monitoring, scheduling, data analysis, enveloping/de-enveloping, routing, process automation, data transformation, data aggregation, etc. The “little e” eSB exposes its built-in services through a matched graphical modeling environment, enabling users to more easily create and combine complex, business-level services using the built-in core services supplied by the eSB.

Because “big-E” ESBs are designed to support a very wide range of integration requirements, they have more moving parts, require more effort to integrate and configure, and focus less on providing application-level building block services. As a result, many ESBs fall back on low-level services or even hand-coding for some common application requirements. For example, inbound envelope analysis, acknowledgement generation, outbound data grouping and enveloping for B2B integration are generally absent. Compensating for this lack of application-level services requires a rather sophisticated grasp of both implementation details and the set of available basic services that must be combined to create B2B applications.

“Little-e” eSBs make this easier by providing pre-defined packages of application-level services. For example, a package for EDI might include the services just mentioned–envelope analysis, acknowledgements, data grouping, enveloping–plus validation against standards, log-based reprocessing, hierarchical auditing, and other services and tools. When combined with user-specified routine and transformation rules, such packaged services make implementation easier, faster, and less dependent on specialist expertise. Little eSBs may provide similar service packages to facilitate database integration, global data synchronization, or integration with commercial application software.

So in general, the primary difference is really one of packaging–how much comes with the product. It’s one of a pre-built, pre-defined set of services, as opposed to more of a developer tool kit, which is what the traditional ESB really looks like, where it comes with a set of primitive functions that let you build your own tools. Little-e eSB is less filling (less demanding), but still tastes great (gets the job done).

AW: What are the different applications for eSBs and ESBs?

JO: Project-level requirements generally govern most of the requirements of companies below a certain size, which we can arbitrarily set at $1 billion. SMBs normally look for such features as fast ramp-up, high productivity, and ease of use; all three are asserted as important.

ESBs focus on large-scale integration, multi-domain coverage, SOA and the like. The focus is on enterprise-level business processes, and the integration of those processes using services. On the other hand, eSBs, because they are project-focused, tend to be better suited to specific target domains, like B2B integration, or simpler forms of data integration. Service-level integration and business process integration are still important at this scale, but even more important is the ability to support heterogeneous system integration and service mediation. There’s much more emphasis on making systems work together to support the specific integration requirements of the service, and less emphasis on things like service composition, where you are building larger and larger services out of smaller ones. Services and integration are much more complex across an enterprise.

“Big” ESBs generally have a wider range of process or service granularity. A higher degree of service composition leads to larger services and longer, more complex integration and synchronization across an enterprise. In the smaller case, the requirements are more specific for integrating a few systems, smaller number of services. Service composition is less of a requirement, although it still does exist, to a certain extent, in some “little” eSB implementations.

Finally, governance. Because “Big-E” ESBs deal with multiple domains, governance is critical in terms of service levels, performance, reuse, and controlling access. In “small-e” eSBs, governance is more constrained and focuses primarily on access control. Guaranteeing service levels and performance to different domain and users isn’t as much of a concern for “little” eSBs.

Bottom line: “Little” eSBs are better suited than “Big” ESBs when the need is for application-level built-in services and general-purpose heterogeneous system integration and service mediation, and where less need and interest exists in custom business-specific services, service composition, and enterprise-level business processes.

AW: Can there be peace between ESBs and eSBs, or are they doomed to a life of competition and conflict?

JO: Large-scale ESBs focus on SOA, and you could make an argument that using multiple ESBs in support of a single SOA could become an impediment to integration. The black-box characteristic of ESBs enables interoperability, but there is a large degree of “information hiding” at work there. So, if there is a requirement to compose services in a tightly coupled fashion, having multiple ESBs could get in the way. Using multiple ESBs from multiple providers also can make configuration and management more costly and difficult.

But using multiple ESBs to manage separate SOAs really isn’t a problem, in general. In the case of eSBs, we’re talking essentially about middleware platforms that are applied to specific integration requirements. Having multiple eSBs of this nature should not pose a problem, either, because the scope of application for each eSB is usually a discrete set of application domains, sometimes just a single project. This is not to say that you couldn’t have an eSB implement an SOA that goes beyond the project level. But as to whether project-level implementation of eSBs is a problem, we would assert that the answer is no.

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot