Watching an SOA strategy take shape

Before I get on with the technical SOA stuff I promised, here is a diversion you might find interesting. I have come across a mid-sized company that packages and sells materials through supermarkets - Its a well known brand name with a turnover in the £60m range. Just the range that is going to have to deal with the issues I have discussed thus far in this blog.

They have already set up a strategy group to look at the feasibility of Service Oriented Architecture so I have the perfect opportunity to watch the process as it develops and write about it in this blog.

As soon as they give me the information, I will tell you about both the politics of the venture and the technical decisions they take with their reasoning. First decision will be how to start - Agility first i.e a quick kill by identifying a process or related set of processes or the purist, technology led approach through the creation of an ESB and an entirely new process built with all new services.

I can tell you the toolset they have chosen is IBM's Webspere and Weblogic. Watch this space!

SOA - What should I expect in return for my investment

The more people I talk to about Service Oriented Architecture, the harder I find it to present SOA as a one stop suits all solution. I can split the conversations into two groupings. The first set I would describe as managerial in that the questions centre around what the measurable benefits of implementing the technology should be. The second set are technical and centre around how the technology should be implemented. In this blog I am going to try and address the first group. The second group I will address in the next.

I guess the place to start is to re-state the driving objective behind the move to implement SOA's and that is to reduce the cost of delivering solutions to users. Of course, this needs further definition to be useful, since some managers might think that the most effective way to reduce such costs is not to attempt to deliver any. There are two sides to the cost equation. There is the cost incurred while waiting for a solution and there is the cost of delivering that solution. The balance between the two will form the core of any measurable ROI.

Service Oriented Technology is aimed squarely at both sides of this equation - the cost incurred while waiting will be less if the time taken to deliver the solution is less and the cost of delivering the solution will be less if it really is simpler and quicker to create than with any other technology as it is supposed to be.

Unfortunately, adopting this technology is not without an initial set up cost. How large this cost is and therefore how long it will take to reap a rewards for its introduction depends on the interpretation of the technology used and, that in turn, depends on the needs of the organisation proposing to adopt it.

Lets look at the options in turn. At the top of the pile we have the SOA purists. The approach proposed here will eschew any attempt to make a start by attempting to expose services from within traditional existing systems as "Sticking lipstick on a pig". This immediately requires the organisation to invest in the creation of the entire architecture from scratch and therefore the time taken to begin seeing any reward will be long - very long. For a technology that is presented as making IT more agile, this can be a dubious start. The same purist approach will demand that an Enterprise Service Bus (ESB) is implemented and that individual services have to contract with the bus. This is deemed necessary to ensure that no process requires any knowledge of the working of any of the services it needs. Instead it just needs to know what services exist in the registry and how to ask the bus for information from them at the right time. The bus knows content in what format the requesting service needs the information and it also knows what content and in what format it can get it from its register of services. In theory, I am now free to extend the capability of existing services or exchange them for new services at any time and only the ESB needs to understand the changes. Remember, any one service may be a constituent of many different processes.

If by now you are wondering what happens if a service is changed such that it no longer meets the need of every process that uses it this too has been addressed. Governance is the key to ensuring that services are managed so that this does not happen.

The problem I have with this is that there is a real danger that the very agility that is being sought can be negated by the need for tight governance restricting the development of services in line with the needs of the business. This in turn will lead to the development of ever more new services instead of encouraging the re-use of existing ones, thus undermining the whole theory. Whether this concern is well founded is still to be determined - there are not enough experiences around yet.

In terms of getting an acceptable return on the investment, the likelihood is that such a project will be unlikely to give a return on investment in a period of less than three to five years if current results remain the norm. Projecting the future that far is notoriously difficult and is the preserve of the very wealthy or foolhardy

The alternative to the purists are the incrementalists. Here the objective is make quick kills to convince the user community of the value of the approach. By definition, to make a quick kill corners have to be cut. Lipstick will be put on the pig so that existing investments in systems are preserved, An ESB may not be used and the overhead incurred when a service is altered swallowed, possibly inadvisedly, as an acceptable price to pay to make swift gains now.

This approach does, however, have something to be said for it, particularly because recent technology developments are riding to its assistance. The first development that helps concerns detecting and talking to services. The big argument for an ESB is that when a service is changed, only its contract with the ESB needs updating. This avoids the need to update every process with the change. However, there is now technology available that can discover and use a service without any need for re-programming. This makes this task much easier and undermines some of the advantages of an ESB and as ESB's are expensive swings the ROI balance in favour of the incrementalists.

The second development is one that makes the creation of services a code free exercise. This has several impacts on the argument. Its quick and easy to put lipstick on the pig, its cheap to create new services on demand and, most importantly of all, its very simple to re-engineer existing services. Taken together, these technologies when used incrementally put SOA technology within reach of businesses right down the the SME level.

Overview of Example of practical approach to SOA

Imagine a typical medium sized company. It has an ERP system it implemented two years ago, a CAD system and a PDM system plus an separate CRM package. The company has identified a requirement to upgrade its quotation and order entry system which currently is largely manual supported by multiple spreadsheets. This process is labour intensive, prone to error and seen by their customers as unresponsive.

In Service Oriented technology terms, the new process will be made up of the following identifiable compound services:

1. Handling customer data both existing and new
2. Creating a quotation/order header
3. Capturing order line details including Configuring the complex products
4. Creating bills of material as required
5. Printing Quotations/order confirmations

All of these services will be made up of several elemental services that can be re-used over and over again.

Each of the compound services needs to interact with the ERP system. If we start with the customer data, the ERP system needs to deliver customer name and addresses and delivery address information on request and also accept and process new customer data. Unfortunately, although the ERP system was purchased only two years ago, it is of traditional construction and does not offer any services. The first task, therefore, is to add service functionality to the ERP system. In fact, several services have to be created and exposed to get this job done.

1. Customer details existing and new
2. Quotation/Order header and line details input and recall
3. Parts data including costs and new bills of material input.

To accomplish this, I use a technique I call creating Application Service Managers. (ASM's) which I create with a code less modeling tool called EdenAgileIT. Essentially, ASM's are an extension to the Host ERP system that is integrated into it and exposes functionality as, in most cases, a web service. To build an ASM takes me between a day and three days, depending on the complexity of the service being created and the architecture of the host ERP system.

Once I have these ASM's in place, I can go on and create the new compound services required before orchestrating them into the new quotation process. These compound services can be very complex, especially those concerned with configuring products or services. It is not unusual for a configuration service to be made up of more than fifty elemental services. I use the same tool set to build all the services. We have accomplished this particular task for a number of customers now, and the whole process from start to roll out requires between 30 and 50 man days.

SOA - where do I start?

I have just read a paper from IBM about predicting and measuring SOA projects ROI's. I'm not impressed. Urging people to be satisfied with instinct and hope is not going to fly with any CFO's I know. IBM's approach is flawed from the outset in that it fails to capitalise on, or at least preserve, the investment companies have made in their existing IT infrastructure

Very few of us are going to be able to begin with a greenfield site. The reality is that we start with a host of legacy systems, all of which were conceived long before Service Oriented Architectures were invented. Very wealthy companies with a history of being early adopters can afford to set up large teams and set them the task of re-creating their primary systems as collections of re-usable services. Most of us cannot. Nor do I believe that you should even if you could.

A better way is to identify quick kills and make a difference early. For example, re-designing your interface to your principal customer or supplier might be a good place to start. The questions is how? You still have that legacy ERP system to deal with.

The answer is to add a software component to your IT infrastructure I call an ASM - Application Service manager. It's function is to integrate closely with your ERP system and loosely with everything else. Each ASM should be designed to expose a specific function within your existing ERP system as a service. To get going with a project to allow major customers to place orders directly via a web service, you only need one ASM. Building an ASM, if you use the right tools, will take less than a man week, maybe much less, depending on your ERP system.

Once you have built one, move on to another - maybe to upgrade your supplier relationships this time. Pretty soon, you will have exposed a significant number of functions from within your existing ERP system as services. Along the way, you can introduce new functionality made up entirely of elemental services and create improved processes made up of services exposed via ASM's and new services composed from new elemental services.

Each step of the way you will have made a real, measurable difference. Users will see the benefits quickly and support the process. management will see a return and the ROI will be tangible.

SOA – Hooray, got web services, now what?

Just been talking to a friend of mine who was pleased to announce his company has just implemented a new version of their ERP environment. He was quite chuffed with it because now he had lots of new functionality inside his system. Turning this functionality into something that his business could use left him with somewhat disappointing consultancy costs, but never mind, he was happy.

“But”, he said, “this environment comes with Web Services. What do I do with them?”

Fair question. So I asked if it came with the ability to orchestrate these new web services. Looking at me somewhat puzzled, he asked: “ability to orchestrate ? Not sure what you mean.”

“A web service in itself is a nice thing to have but it doesn’t do anything. What you need is a way to use web services from different systems in order to make your way of doing business more agile.” – I replied. And I continued with an example.

Suppose your new environment can expose the outstanding balances of invoices. What you could do is define Key Performance Indicators (KPI’s) that keep track of the fact that the outstanding balances together with their due dates are enough to keep the cash flow position of the company stable. In order to do this, you need something that can calculate these KPI’s and use the outstanding balances from the web service to come to measurements.

If you have the ability to present these KPI’s as a service (possibly a web service) then it would be quite simple to present these figures though any web environment or even to a PDA.

Think about it, wouldn’t it be great to be sitting on the beach of Allicante and being able to check your company’s cash position without leaving your cocktail?

And all this doesn’t have to cost the earth. There are a few solutions available that actually make use of your existing web services and can turn other systems into a SOA-compliant system including web services.

One of my personal favorites is the Google Homepage. I customized it by simply adding a few links and now it shows the local weather, my calendar, my activities, some KPI’s for my company and the latest technology news. (And some other gimmicks that are not very relevant here)

An upgraded system containing web services is like a HD ready TV. It has got the HD socket but without the HD Box (orchestrator) it is still the same as a normal TV.

SOA - Loosing sight of agility

Its getting harder and harder to remember that the idea behind the SOA technology is agility - agility in the sense that the business can get the business processes it wants when it wants them. That timescale is increasingly measured in hours or days, not months or years.

The aspect of systems that is driven by change is the knowledge encapsulated within them. When you create a service to, say, configure a service or product within an order entry system, the service or product is changing to meet market demand. Insurance policy constraints change regularly due to legislation as well as market forces - in short, agility can only be achieved if it is possible to adapt and maintain elemental or replace composite services easily - within hours.

Its the technology used to create the services that will ultimately determine the level of agility achieved, and therefore the ROI gained. Coding services the old fashioned way - functional specification, program spec, programming in a language etc - is, therefore, not a realistic option. At the end of many man years effort, all a company will have is another legacy system, albeit made up of composite services carefully orchestrated.

Look for a technology that enables users to create composite services without any coding - that's where agility lies.