Watching an SOA strategy take shape

Before I get on with the technical SOA stuff I promised, here is a diversion you might find interesting. I have come across a mid-sized company that packages and sells materials through supermarkets - Its a well known brand name with a turnover in the £60m range. Just the range that is going to have to deal with the issues I have discussed thus far in this blog.

They have already set up a strategy group to look at the feasibility of Service Oriented Architecture so I have the perfect opportunity to watch the process as it develops and write about it in this blog.

As soon as they give me the information, I will tell you about both the politics of the venture and the technical decisions they take with their reasoning. First decision will be how to start - Agility first i.e a quick kill by identifying a process or related set of processes or the purist, technology led approach through the creation of an ESB and an entirely new process built with all new services.

I can tell you the toolset they have chosen is IBM's Webspere and Weblogic. Watch this space!

SOA - What should I expect in return for my investment

The more people I talk to about Service Oriented Architecture, the harder I find it to present SOA as a one stop suits all solution. I can split the conversations into two groupings. The first set I would describe as managerial in that the questions centre around what the measurable benefits of implementing the technology should be. The second set are technical and centre around how the technology should be implemented. In this blog I am going to try and address the first group. The second group I will address in the next.

I guess the place to start is to re-state the driving objective behind the move to implement SOA's and that is to reduce the cost of delivering solutions to users. Of course, this needs further definition to be useful, since some managers might think that the most effective way to reduce such costs is not to attempt to deliver any. There are two sides to the cost equation. There is the cost incurred while waiting for a solution and there is the cost of delivering that solution. The balance between the two will form the core of any measurable ROI.

Service Oriented Technology is aimed squarely at both sides of this equation - the cost incurred while waiting will be less if the time taken to deliver the solution is less and the cost of delivering the solution will be less if it really is simpler and quicker to create than with any other technology as it is supposed to be.

Unfortunately, adopting this technology is not without an initial set up cost. How large this cost is and therefore how long it will take to reap a rewards for its introduction depends on the interpretation of the technology used and, that in turn, depends on the needs of the organisation proposing to adopt it.

Lets look at the options in turn. At the top of the pile we have the SOA purists. The approach proposed here will eschew any attempt to make a start by attempting to expose services from within traditional existing systems as "Sticking lipstick on a pig". This immediately requires the organisation to invest in the creation of the entire architecture from scratch and therefore the time taken to begin seeing any reward will be long - very long. For a technology that is presented as making IT more agile, this can be a dubious start. The same purist approach will demand that an Enterprise Service Bus (ESB) is implemented and that individual services have to contract with the bus. This is deemed necessary to ensure that no process requires any knowledge of the working of any of the services it needs. Instead it just needs to know what services exist in the registry and how to ask the bus for information from them at the right time. The bus knows content in what format the requesting service needs the information and it also knows what content and in what format it can get it from its register of services. In theory, I am now free to extend the capability of existing services or exchange them for new services at any time and only the ESB needs to understand the changes. Remember, any one service may be a constituent of many different processes.

If by now you are wondering what happens if a service is changed such that it no longer meets the need of every process that uses it this too has been addressed. Governance is the key to ensuring that services are managed so that this does not happen.

The problem I have with this is that there is a real danger that the very agility that is being sought can be negated by the need for tight governance restricting the development of services in line with the needs of the business. This in turn will lead to the development of ever more new services instead of encouraging the re-use of existing ones, thus undermining the whole theory. Whether this concern is well founded is still to be determined - there are not enough experiences around yet.

In terms of getting an acceptable return on the investment, the likelihood is that such a project will be unlikely to give a return on investment in a period of less than three to five years if current results remain the norm. Projecting the future that far is notoriously difficult and is the preserve of the very wealthy or foolhardy

The alternative to the purists are the incrementalists. Here the objective is make quick kills to convince the user community of the value of the approach. By definition, to make a quick kill corners have to be cut. Lipstick will be put on the pig so that existing investments in systems are preserved, An ESB may not be used and the overhead incurred when a service is altered swallowed, possibly inadvisedly, as an acceptable price to pay to make swift gains now.

This approach does, however, have something to be said for it, particularly because recent technology developments are riding to its assistance. The first development that helps concerns detecting and talking to services. The big argument for an ESB is that when a service is changed, only its contract with the ESB needs updating. This avoids the need to update every process with the change. However, there is now technology available that can discover and use a service without any need for re-programming. This makes this task much easier and undermines some of the advantages of an ESB and as ESB's are expensive swings the ROI balance in favour of the incrementalists.

The second development is one that makes the creation of services a code free exercise. This has several impacts on the argument. Its quick and easy to put lipstick on the pig, its cheap to create new services on demand and, most importantly of all, its very simple to re-engineer existing services. Taken together, these technologies when used incrementally put SOA technology within reach of businesses right down the the SME level.

SOA – Hooray, got web services, now what?

Just been talking to a friend of mine who was pleased to announce his company has just implemented a new version of their ERP environment. He was quite chuffed with it because now he had lots of new functionality inside his system. Turning this functionality into something that his business could use left him with somewhat disappointing consultancy costs, but never mind, he was happy.

“But”, he said, “this environment comes with Web Services. What do I do with them?”

Fair question. So I asked if it came with the ability to orchestrate these new web services. Looking at me somewhat puzzled, he asked: “ability to orchestrate ? Not sure what you mean.”

“A web service in itself is a nice thing to have but it doesn’t do anything. What you need is a way to use web services from different systems in order to make your way of doing business more agile.” – I replied. And I continued with an example.

Suppose your new environment can expose the outstanding balances of invoices. What you could do is define Key Performance Indicators (KPI’s) that keep track of the fact that the outstanding balances together with their due dates are enough to keep the cash flow position of the company stable. In order to do this, you need something that can calculate these KPI’s and use the outstanding balances from the web service to come to measurements.

If you have the ability to present these KPI’s as a service (possibly a web service) then it would be quite simple to present these figures though any web environment or even to a PDA.

Think about it, wouldn’t it be great to be sitting on the beach of Allicante and being able to check your company’s cash position without leaving your cocktail?

And all this doesn’t have to cost the earth. There are a few solutions available that actually make use of your existing web services and can turn other systems into a SOA-compliant system including web services.

One of my personal favorites is the Google Homepage. I customized it by simply adding a few links and now it shows the local weather, my calendar, my activities, some KPI’s for my company and the latest technology news. (And some other gimmicks that are not very relevant here)

An upgraded system containing web services is like a HD ready TV. It has got the HD socket but without the HD Box (orchestrator) it is still the same as a normal TV.

SOA, Web Services, Ajax and Mash-up – an overview

Nowadays you can hear buzz words like Service Oriented Architecture, Web Services, Ajax and Mash-up. But for the average business oriented person they can be hard to understand. So what do they mean and what have they got to do with each other?

Let’s try to shed some light on this.

Service Oriented Architecture (SOA) is a concept of making functionality, complex or simple, that is handled by a specific system or server, available to the outside world. This outside world can be inside your own company or exposed over the internet. The service can be called upon as and when it is needed by any other component as part of various business processes.

Web Services is a bit of a misleading name. It implies it is a service exposed over the Web but that doesn’t need to be the case. A Web Service is the technology in which a service can be approached. This service can expose itself over any network connection, including the internet. Hence the misleading name.

Active Javascript And Xml (Ajax) is not really something new. The concept of Ajax has been around for years but of late, more and more development environments have supplied the software industry with tools to make it easier to build browser based applications that interact and feel like more traditional (windows-) based applications. So now you can see more and more application developments creating rich user interfaces for their web applications.

A mash-up can be compared to a portal where various functionalities from different sources and applications are placed on one screen and these various (usually small) functionalities can interact. A common example is the screen that shows the latest news, has an overview of the outstanding tasks and shows, once a task has been selected, the location for that task in a map.

So how does it all link together?

This question is probably best answered by using an example.

Suppose your company has a server that holds all the accounts receivable information. As part of an SOA, a service is created that can expose outstanding balances based on a customer code. This piece of software on the accounts receivable server uses a web service to expose itself.

On the actual webpage that you see, an area of the screen can be used to query these outstanding balances using Ajax. The customer code comes from another area of your screen where all your tasks for today are. Once a task has been selected (clicked on) it takes the customer code from that task (in this case a task could be a phone call) and queries the web service for all outstanding balances. The result is shown next to your calendar. And it does this without contacting the server for an entire new page. It only gets the little bit of info that is needed to display the balances. So now, before you make the call, you can see if there is anything in particular going on with this customer that you may be able to mention during your phone call.

So now the browser contains a mash-up page and the accounts receivable server fits into a Service Oriented Architecture.

Of course this is but a simple presentation of the reality and in real-life there are many issues to consider (security, availability and protocols) but it does give an overview of how it is tied together.

SOA - loosing sight of the objective

I visited an insurance company today to talk about their investment in creating an SOA. They had appointed a manager to lead the project and had forty five programmers working. They have been working for over a year. So far, there has been no tangible result. Yes, they are creating services but they need a lot before they can create a worthwhile business process.

Forty five Programmers for a year? They have obviously lost sight of the principal justification for such an investment - Agility! How can a technology that's touted as the answer to everyone's need for agility need more than forty five man years of effort just to get started.

Then there's another problem - Once the new services are created, say with an average of two man years of programming a piece in them, how easy is it to modify them? They desperately need a technology that can create services in a hundredth of the time. A technology that does away with the need for specification and programming. One that allows its users to create services based applications iteratively through dialogue with the users. They do exist.

Until you've got that, you have not got anything.

SOA Explained

Service Oriented Architectures hold the key to the creation of truly agile IT environments – Agile in the true sense in that user needs for improved or new business processes are met within hours or days instead of weeks or months.

The concept is a simple one – monolithic legacy programs that cannot share their functionality are gradually replaced with small software components that each perform a task. The components are constructed so that they expose what they can do as a service – a service to be called upon as and when needed by any other component without the need for any specific integration. This “Loosely Coupled” arrangement means that every component can be re-used endless times in any number of business processes.

To become a realistic solution, the technology to be used has to be agreed so that companies can call each others services. The chosen technology for this is web services. This technology uses a standardised approach to the problem by specifying the messaging protocols to be used and the way each service is described. The message technology is SOAP and the service description is an electronic document called WSDL.

What is Agile IT?

Agile IT systems are based on new technologies that allow existing business processes to be modified and new business processes to be developed at the same pace as the user can articulate them. To achieve this, the traditional development process of Analysis, specification, programming, testing and release has to be ditched and replaced with a single step, iterative process.Because the processes most likely to be developed this way are the key, customer facing knowledge based business processes such as quotation, configuration, order processing, pricing and margin management, the key enabling technology at the core of such systems is business rules server capable of capturing knowledge coupled to an advanced graphical forms designer. Using this technology, business processes can be expressed entirely in terms of inputs, rules and outputs.Unfortunately, having this technology alone is not enough. To be really effective, Agile IT solutions have to be an integral part of the overall existing IT infrastructure. Ideally, this should be accomplished by first exposing the services available in the participating legacy systems and then calling these services as and when required within the new and improved business processes as they are developed. This type of architecture is termed an SOA (Service Orientated Architecture) and the resulting integration is termed Loose. It is not essential that this type of integration technology should be used, and agile IT solutions can and are being developed without it by relying on more traditional, hard, integration techniques.When selecting an Agile IT toolset, it is important that it should be able to participate in an SOA environment, even if its first uses are based on more traditional technologies.