SOA Frameworks and Vendor Support

SOA seems to be getting a lot of press lately. It is the next defining (or disruptive) technology that will change enterprise computing for ever. Being a field consultant for an integration software vendor, I have had my share of exposure to organizations and their quest to enable SOA.

One of the first realizations you have when you start on such a venture is that nobody really has any idea about what it takes to enable SOA in the enterprise, not even the vendors. The one thing most people agree upon is to establish and enterprise-wide SOA framework.

Why create an enterprise SOA framework? (Few reasons that come to mind...)

• Ease of adoption - an enterprise framework will facilitate the adoption of SOA by centralizing the critical architecture and strategic planning functions and empowering application teams to concentrate on solving specific business problems
• Leveraging Core Infrastructure Services - Core infrastructure services include services such as location, security and rule-based routing. These can be federated services that are consumed by all business level services. Again (at the expense of sounding like an old record) aligning IT closer to business, where business developers don't need to worry about coding for security etc.
• Vendor Neutrality - Yes, as nice as Mr. Salesguy is, we want to be vendor neutral. Building our own framework will be one of the best ways to ensure that we can move from vendor to vendor without affecting "business applications"

"SOA" enabled integration products seem to try to do a lot of things for you. Such "ease-of-use" is helpful when adopting SOA especially if the implementation is quite straightforward.

You will be almost entirely satisfied with the product if you are willing to use the vendor provided location, security and routing services. Also if your requirements do not dictate the use of a transport other than those supported "out-of-the-box".

What most of these products don't address are the needs of a complex enterprise that needs an additional SOA framework layer.

What the heck do I mean? (Some scenarios that might drive you to override default out-of-the-box functionality)

• How would I augment the security functions of a product so that I can leverage an existing legacy authorization provider?
• I have a UDDI tree, I want the routing feature to override the endpoint specified in the incoming WS-Addressing header with the one returned by this UDDI query
  

I believe there are two hurdles SOA vendors need to overcome.

1. The relative lack of maturity of WS-* standards
2. The relative novelty of SOA products

The ultimate SOA platform will be one that will allow for flexibility. A pluggable framework that will allow users to provide custom extensions as necessary.

Such a product will not only allow both vendors and users to "fill-in-the-blanks" left by incomplete or immature WS-* standards, it will also allow vendors to provide "out-of-the-box type solutions while enabling more demanding users to override the defaults.

To me, what is most ingenious about the US Constitution is that it allows for the flexibility to change and adopt as times change. The founding fathers have acknowledged that there is no possible way for one group (even as brilliant as themselves) to possibly conjure the "perfect" solution, therefore they allowed for flexibility to ensure the longevity of their creation.

Now if only we can get our integration vendors to understand this....

Part 2 : Common component approach to software best practices

In Part 1...

The fundamental flaw in the audit technique to enforcing software best practices is the unfavorable relation of the number of concurrent development tasks to the number of available enterprise architects.

Reusable Software Components

Thousands of years ago, man discovered fire. Later that day, man discovered barbecue. Similarly, the concept of common components in software development has been around ever since the very early OO languages were released.

In addition to the tremendous productivity enhancements that accompany such an exercise, there are few, rather lucrative, fringe benefits.

Components as Best Practices

I was consulting with a large telco company that had identified that the root cause of most of their availability issues were due to bad communications logic. The communications logic was used to access legacy systems via proprietary protocols that were known to very few persons in the company.

Like most large organizations, this telco leveraged partners for software development. Since the protocols being used for communications were proprietary, the partner organizations had to write communications logic.

Unfortunately, no two designers thought the same way, and most of them did not really understand the run-time characteristics of these home-grown communication protocols.

Faced with this scenario, they decided to create a set of core components that all partners and internal development teams should use to access its proprietary middleware systems. The interface was extremely simple; any developer could use it by looking at a couple of samples.

The components were configured via a centralized configuration file which further reduced complexity for the developer.

Instead of requiring an architect to verify to 80 different "anti-patterns" for using the middleware, the design review and code review now only needs to verify that the published component has been used.

All the best practices, including the avoidance of the 80 "anti-patterns" were encapsulated in the component. Partner organizations that are oblivious to internal communications protocols now have the ability to deliver high quality software.

Summary

This is just one example, of how component best practices can be a winning strategy in IT Governance.

Rather than writing a 500 page document that defines the threading model to use and how session handshakes occur, this company decided to deliver a set of components. Instead of referring to this document and combing through the code with a fine toothed comb, a code review and design review will have one checkmark to tick that reads "Uses common component framework for all communications".

Part 1 : Common component approach to software best practices

Software best practices are used in IT organizations to promote uniformity in the use of technology. Uniform standards are a catalyst for higher software quality. What higher software quality means to us is fewer "middle of the night" support calls and far less (often non-compensated) overtime.

For many such organizations, best practices constitute a series of documents and other artifacts that may be published in a central repository, portal or otherwise be made available to the IT staff.

While lots of resources are consumed in the preparation of such artifacts, not many organizations have been successful in translating these efforts into higher software quality.

This may be partly due to the fact that best practices usually arise from a central architecture team. While they are experts in both the technology and business domains the core competency of such a team is not IT governance nor is it software delivery.

Best practices, when followed, leads to very high quality systems. Meanwhile, little or no attention is paid to enforcement.

One might be tempted to draw parallels between this and one of the most polarizing political issues in America, illegal immigration. While there are calls for the reform of immigration laws from many corners, opponents of change argue that the single biggest flaw of the immigration system in the US is that most of the existing immigration laws are not properly enforced. They contend that reforming or adding more immigration legislation is strictly an academic exercise unless the issue of enforcement is addressed.

Clearly, all the best practices in the world would not make any sense unless the organization is able to align its best practices with its IT governance.

There is more than one way of achieving such alignment. The traditional approach is to use an audit model. Most of us are familiar with "code reviews" and "design reviews". These are exercises commonly used by IT organizations that use the "audit" model.

The audit model for ensuring quality has been around for centuries. The best example of this approach at work is the financial markets. The SEC, Securities and Exchange Commission, requires publicly traded corporations to submit a standard set of documents each quarter (income statement, balance sheet etc.). These documents have to be prepared using certain standards or best practices (FASB standards). An independent auditor, a CPA firm, will audit these documents to ensure that they have been prepared according to the best practices (FASB standards) as directed by the SEC. They will only be accepted by the SEC when this auditor has signed offf on the statements.

There are strengths and weaknesses of this model. The strength is that it is a pretty effective model to ensure uniformity and adherence to standards. Despite a few blemishes such as the Enron-Anderson debacle the audit system works.

One major drawback of this approach is that the model is not leverageable. In techie terms, the audit model does not scale up.

A typical organization will include a placeholder in its methodology to perform "formal design reviews" and "formal code reviews" to get signoff from a representative from a central team of experts. This means that there has to be at least one resource from this central team assigned to each application development effort as they would need to sign off on both design and implementation of the app.

For an audit to be successful, the auditors need to understand the system that is being audited. This means the auditor needs not only strong technical knowledge but business domain expertise as well. Given this scenario, if we were to estimate and one architecture expert is assigned to each application team, all of a sudden we are faced with a 1 to 1 relationship between the number of available architects and the number of ongoing concurrent development efforts.

Since I have explained the context and laid a foundation of the issue I am trying to address (also I am getting very sleepy) I will explore a newer and rather radical approach to achieving higher software quality through best practices in my next post which I plan to title "Part 2 : Common component approach to software best practices".