In-Depth

Think Integration

IT must move beyond mere maintenance and upgrades. Rather than being viewed by the suits as a money pit, IT needs to assume leadership and propose new business processes models.

• By Peter Varhol
• 08/01/2004

Most IT departments face the prospect of no significant new investment in systems or software for the foreseeable future, except for perhaps replacement and incremental version updates. By itself, this situation isn't necessarily bad. Many of us like experimenting with new things and making them work, but certainly most of those charged with keeping IT doing what it supposed to do are just as happy with what they have today.

But we're asked to do more than maintenance. With the same budget and likely less staff, we're told to improve IT performance. Often this kind of mandate comes in the form of a business imperative such as the need to cut costs or improve the efficiency of a given process. At its worst, the suits view IT as a cost center, an expensive affectation that can't be afforded to do any more than precisely what it is told. After all, when IT reaches beyond that into ambitious new projects of dubious value, those projects often fail altogether, or at least fail to deliver on their promises.

It's up to us to prove otherwise and to come down squarely on the side of value and innovation. We do this through leadership of thought and action in the use of IT to enable the business to run faster, more efficiently, or in new ways.

What does this state of affairs have to do with integration? And what is integration, anyway? These days we hear far too much of that simple word, used in ways we would never have thought possible. Let's examine it for a minute.

At the most obvious level, integration is the act of getting applications to work together, rather than separately. Integration has many layers; the most commonly used approach to integration is copy and paste. Users copy data from one application through the Clipboard and paste it into another. And it works as it's intended to be used-to get data from one location into another.

Automating Integration
However, this type of integration is inefficient because it requires individuals to judge what data is valuable and to perform the act of Select, Ctrl-C, Ctrl-V. It would be far better if integration could happen in a more automated fashion, without user intervention. And what about the data that can't be manually moved easily between applications? For many applications, cut and paste is either impractical or impossible. IT can help here by making that action occur automatically.

There are several ways of doing automatic cut and paste. Perhaps the simplest is to write off the data from one application to a file and have that file read by the other application. But this approach has problems because you can't count on the first application writing to the file when the second one needs it, and there's no verification that the correct data is being passed. After all, if the first application changes the format of the file, and the second doesn't get the word, what seemed like a good idea quickly becomes a bad idea once the wrong data becomes ingrained in the process.

Automatic cut and paste is one example of what might be called tight integration. While it doesn't seem very tight, each application has to know that the other exists, and any change in the data format or even the timing of the reading and writing of the file could disrupt the integration. Such integration is even tighter if the applications talk directly with one another. The primary advantage of tight integration is performance, in that applications tend to run more quickly if they have greater knowledge about what is going on around them.

Instead, a more complex but robust approach is to enable each application to perform its job without needing to know where the data comes from. A popular way of doing this is through Web services, which can be built in several ways. For example, a service can request the data from the application, perform any required manipulations, and send it to the receiving application in response to a query. Or the processing end of one or both of the applications can be turned entirely into a Web service having the primary goal to serve and exchange data for both.

A second way of accomplishing such integration is through publish-and-subscribe technology. This approach enables an application to publish data-in other words, put it into a known location-and make it available for other applications to subscribe to. Neither the publisher nor the subscribers need to know about each other. This approach is often implemented through the use of a message-passing system such as the Java Message Service (JMS).

Both Web services and publish-and-subscribe methodologies are loose integrations. Little or no change occurs to any of the applications involved; instead, they are joined by another piece of software. Integration between existing applications can be enabled in many other ways, but these are representative of the most popular approaches.

Hard Work
Such is the technology of integration. But, to be frank, that's the easiest part. A more difficult part is finding data that is worthwhile to be integrated. In some cases, the users themselves have already done this work manually. In the rest, however, painstaking hard work talking to users is required to determine what data they use in their business processes and how that data might be combined. You would do this research through a systems analysis of each of those processes. You're not done there, however, because you then have to decide which integrations add the most value to the process. You can make a judgment call here, based on experience, or perform a formal cost-benefit analysis.

And that's just the start of it. This kind of effort helps the business improve existing processes through the integration of data from separate but usually similar sources. Users no longer have to move data manually. That's important, but it's just the tip of the iceberg. There are still more efficiencies to be had in streamlining the overall process. If a particular business process takes two or three days to accomplish, there may exist application data that when combined can drastically shorten that process to perhaps two or three hours.

This type of integration requires a broader understanding of the purpose of the process, each of the steps involved, and the data inputs and outputs to those steps. Through a careful analysis of the overall process and the data needed to complete the process, IT can combine applications in ways that bring the necessary data to bear much earlier in the process. Integration supporting the redesign of business processes to shorten them significantly is where most of the value of application integration will be found.

But there's one more level beyond process improvement that requires IT involvement. The Holy Grail of application integration is to enable the creation of entirely new processes that redefine how work is accomplished. Such breakthroughs can launch new lines of business, and even new industries. We saw such cases happen over the last few years with several Web businesses, such as eBay, where the Web was able to bring large numbers of people together to participate in auctions, and data was aggregated in a way that made bidding possible by all.

Making this outcome happen might be more luck than skill. Unless you are a visionary, imagining that new and as-yet-untried business process and the business that can be derived from it seems difficult. I doubt my own ability to create this kind of vision. Instead, it's likely that the most we mere mortals can achieve is to recognize a new combination of data that might have implications beyond the business processes they support.

Making it happen is more difficult still, and once again not a technical challenge. Instead, it is first a challenge of the imagination to be able to envision something that doesn't yet exist anywhere. It is second a challenge of leadership to sell others on that vision and to make the organizational and business shifts that can make that outcome happen. Both of these are so rare that they are accomplished only a few times a year, in only a handful of enterprises.

Business Partnership
Now back to the original question: how can IT demonstrate business value in stabilizing or even increasing investment? It has to be more than just the technical side of application integration. We can be the best technicians available, but if we only keep the systems running and existing processes working as they were originally implemented, then we are little more than a cost center.

IT is at its best when it functions as a full partner with the business, understanding the business processes and the data and processing that go into them, and takes the lead in proposing new business process models. If this sounds too difficult, or out of the scope of enterprise computing, take note that no one else is better qualified to assume this role. IT professionals have the unique combination of knowing the data and processing that make existing processes work, along with the technical ability to understand how it might be possible to combine existing applications to compress inefficient business processes and create new processes.

How can IT make this happen? Take several experienced professionals and assign each a business process. Their role is to understand each of the steps in the process and the applications and data used along the way. They should also participate in the process, so that they can see how workers get their tasks accomplished. Using their existing knowledge of IT systems, they can then determine if those systems are used in the best way to reach the end result.

Achieving these goals will make IT indispensable in the enterprise. Possibly you already do the analysis portion today; that analysis has to be supplemented with actionable advice and leadership to the lines of business. Rather than being viewed by executive management as a bottomless money pit, IT becomes a force for creating new businesses and optimizing old ones. If we need or want greater resources or more people, we have to take on the leadership role for providing the business environment for application integration.