Monday, December 29, 2008
Process Template
I hope this document will be helpful tool to lot of us to sail through EPM/BPM universe.
The document can be downloaded from Scribd.com.
Anatomy of Service
In simplistic terms service is logically grouped set of operations capable of performing related unit of work and operates over a message.
So service is units of processing logic, which are closely related.
Therefore service has two major components:
1. Message: Data required to complete partial or all parts of a unit of work.
2. Operation: The logic required to process message in order to complete a unit of work.
A well-designed service should have following attributes:
1. Service should be reusable
2. Service should exposes a formal interface contract
3. Service should abstract underlying logic
4. Service should be composable
5. Service should be autonomous
6. Service should be stateless
7. Service should be discoverable
8. Service should be available over network
In addition of these, if we consider SOA as confederation of services on more attribute is desired:
9. Services must be loosely coupled.
So how these attributes interact!!
Services may also be viewed from application perspective over which they are written or which they make in some composable fashion. The second perspective is process perspective, which view services from enterprise process perspective.
Application perspective of Service
Process perspective of Service
Hybrid perspective of services
Monday, December 22, 2008
SOA and Business
IT assets in general may be viewed from three perspectives.
1. Enterprise
2. Application
3. Product
Enterprise perspective include IT assets at inter and intra enterprise level. It considers Business value of IT assets on enterprise level; so it’s effects is also enterprise level. This perspective is generally elaborated in terms of inter and intra enterprise integration. This perspective is generally taken care by CIO and CTO level while deciding enterprise wide IT strategy. Enterprise Architects also play major role in shaping this perspective.
Application perspective limits IT assets within the boundaries of application – off the shelf or custom-made. This perspective considers Application or its significant functionality as service. This perspective makes sense to VP Engineering and CTO. This perspective gains significance in case of purchasing new enterprise application.
Product perspective becomes very important for product developing companies. This perspective focuses on the services offered within a product. Proper usage of this perspective helps in construction of robust and flexible product/product family. This perspective assumes significance for Technical Architects.
All three perspectives help business to save on cost and decreasing the recurring expenses in terms of maintenance & support. They also helps in reducing time to market.
SOA also brings technological breakthroughs which helps in cost saving and fasten the introduction of products in market.
1. Increased reusability of components/services.
2. Ease in integration – inter and intra enterprise.
3. Increased pace of development and adjusting IT assets as business evolve.
4. Facilitation of incremental construction (architecture, design & development) of IT assets.
5. Facilitation of geographically dispersed development in varying time zones.
6. Facilitation in outsourcing/off shoring the IT assets – construction and maintenance & support.
7. Facelift of legacy application and integration of them in modern IT landscapes.
8. Increased utilization of human resources.
9. Reduced and managed risk with respect to legacy IT assets and resources.
Sunday, December 21, 2008
Service Interface / Contract Definition
I hope this document will be helpful tool to lot of us to sail through SOA universe.
The document can be downloaded from Scribd.com.
Saturday, December 20, 2008
Book Review: SOA Governance by Todd Biske (ISBN-13: 978-1847195869)
SOA Governance is one of the most vigorously debated and misunderstood subject in SOA world. Because of inherent complexity of governance and SOA, SOA Governance becomes more complex subject.
Book is written is style of novel coupled with some hard technical talk in end of each chapter. This makes reading of book very relaxing though covering complex subject. Book covers governance issue in such a fluid manner that while reading it seems that one is just browsing a story book. Book starts with basic concept of governance and start peeling layers. It covers architectural, design and project management challenges to have good governance model from project as well as enterprise perspective.
Book also take advantage of case study approach and assume very realistic insurance company with all nuisance of IT Management and Architecture noise.
Book is certainly a good read for its target audiences – Architects, IT & Business management folks and certainly those who are interested in SOA Governance.
Go and grab the book at packtpub.com
Thursday, December 18, 2008
Players in SOA universe
In SOA universe there are two types of players. One those who are selling different products under SOA umbrella and second one are pure intellectual efforts who are defining SOA.
There is also third breed which consists of service providers who are mainly focusing on implementation strategy.
Breed one and two has lot of cross pollination to promote their respective product suite. This cross pollination has resulted in plethora of specification and some time at the expense of each other.
So the question arises is this leading to some where or it will turn SOA into one more kind of Lock-in strategy by product companies?
Wednesday, December 10, 2008
Enterprise Process Management
Keeping EPM into sight, what is process?
“A process is a coordinated thread of sequential and parallel activities needed to deliver value to stakeholders.”
So what is EPM?
“Enterprise Process Management is assuming controlling position on processes from inception/idea to its delivery, from very first activity to last, and analysis of each and every activity for continuous and step improvement.”
From definition it is very clear that EPM can be visualized from three view points:
- Inventive viewpoint
- Execution viewpoint
- Improvement viewpoint
1. Inventive viewpoint: This viewpoint is affected by strategic and tactical decisions of any enterprise. Strategic and tactical success of any enterprise demands effective and efficient processes with continuous improvement build in. As environment is dynamic and most of the time beyond the control of enterprise, processes must be adaptive in nature and if possible self-aware.
2. Execution viewpoint: This viewpoint focuses on execution part of processes’ lifecycle. Once process is released in any enterprise for execution purposes this viewpoint takes over. In any successful process this viewpoint has longest duration with numerous small changes but insignificant number of step changes.
3. Improvement viewpoint: In this viewpoint focus is on improvement related aspects of process. This viewpoint gets feed from execution and gives to inventive viewpoint. The improvements may be minor chunks (part of continuous improvement) or quantum leap.
On the basis of complexity, enterprise processes can be divided into three categories:
1. Material Processes
2. Information Processes
3. Human Interaction Processes
- Material Processes: MPs are primarily concerned to “THINGS”. The purpose of these processes is transforming the material and consume resources. These processes rely heavily on Industrial engineering concepts. These processes are found in primarily in manufacturing, assembly, process industrial environment. Generally speaking these processes are so matured over time period across the industries that they more or less resemble procedure or long running procedures. This means the input and output of these processes can be defined objectively.
- Information Processes: IPs are primarily concerned to “DATA”. These processes transform data into information or information into more abstract information. These processes rely on Computer science, Software engineering and Information science concepts. These processes are prevalent in service and non-service industrial environment. Again due to tremendous progress in computer science, information science and software engineering these processes resemble to procedures.
- Human Interaction Processes: HIPs are primarily focus on human cognitive aspects. HIPs are articulation of subjective issues and getting agreement among various stakeholders. HIPs are based on structure of human communication and coordination, which are prevalent in natural languages and cultures. HIPs are highly affected by cognitive abilities of participating actors and surrounding environment. Most of the time HIPs are highly subjective so their automation is a big challenge.
In real life settings, most of the enterprise processes are combination of MP, IP and HIP.
For simplicity let us consider a simple process: Opening a checking bank account by a new customer.
So to open checking bank account, customer has to submit a form to branch where he wishes to have account. The account opening form also need some supporting document like identity card of the prospect along with letter from an existing account holder. Upon submission of completed form and supporting document, a clerk at branch verifies the documents at preliminary level and sends to regional office for detailed verification. If verification result is OK, account is opened, base branch & customer are informed and checkbook, ATM card are dispatched to customer. If verification result is NOT OK, base branch & customer are informed.
In the first glance the process seems to be well structured and simple enough for automation. This process seems to fall in Material and Information Process category. So automation looks easy and within reach. Now let Human aspect enter into the process.
Mr. Clerk who accepts documents from prospective account holders has friendship with Mr. Tony. One fine day Mr. Tony reaches to branch where Mr. Clerk is stationed and submit Account opening form. Mr. Tony does not was not carrying supporting documents, so he talks to Mr. Clerk and makes promise to submit supporting documents on next Monday and asks Mr. Clerk to hold his docs till then. So human relations start tearing a well-structured process.
Sunday, December 7, 2008
Enterprise SOA Reference Architecture
Reference Architecture is independent of platforms offered by various vendors. It is also possible that none of the vendors in market provide complete solution to realize this reference architecture.
Data Persistent Layer is primarily consists of Enterprise Data - Master, Transaction and Meta. It may also include Datamart and Data driven reporting infrastructure.
In most of the large enterprises Data Persistent Layer consists of relational (RDBMS), hierarchical (LDAP, IMS), unstructured (Emails, chat sessions, file systems, etc) and any other type. These variations in data characteristics warrant unique data handling and treatment technologies.
This Layer consists of different business applications. These applications may be packaged (e.g. SAP, Reteck, etc) or may be custom developed in various technologies (e.g. Java, .Net, etc). Again application in this layer may be service enabled or legacy once, which do not expose their functionalities as service.
At this layer EAI exist. Some may argue that SOA is one form of integration then how EAI can exist at this layer. I will discuss this argument in one of my future posting but not now.
Services Layer
This layer is consists of multiple sub layers which encapsulate different class of services. It encompasses Connectivity Services, Data Services, Business Activity Services and Business Process Services. Each layer in turn consists of other sub-types of services.
This layer exposes different aspects of business - functional and infrastructural aspects. Connectivity and Data services are consisting of Infrastructural services while Business Activity and Process services expose functional Services.
These services though in layers but still breach layered architecture to expose these services independent of layers and for better performance. These services can also be exposed to multiple ESBs if situation warrants.
1. Connectivity services primarily cater to accessing system level functionalities, any adapter & connector services and partner (B2B) integration services.
2. Data services comprises of services, which caters to Business, Technical and Meta data related services.
3. Business Activity Services consist of Atomic business/functionalities and any custom business logic developed by combing one or more Data & Connectivity services.
4. Business Process Services primarily consist of Process integration, Machine and Human centric services. These services primarily comprise of Business Activity services.
Orchestration
Routing
Mediation
Enrichment
Some ESBs may provide one more functionality - Choreography. This functionality is associated with BPM systems and currently none of Commercially available ESB provides the same.
Enrichment may be of two types: Data Transformation and Channel Transformation. Most of the analysts & architects keep Data Transformation & Channel Transformation separate functionality.
Human
- Thin Client
Browser clients: May be based on traditional HTML based or AJAX based
Portable devices: May be based on open standards (WAP) or propitiatory (device or carrier specific)
- Thick Client
Alway Connected: These clients connect to service providers and consume the services in real time. Traditional think client falls in this category
Offline Clients: These clients do not connect to service providers in Always On fashion but an actor can work in offline mode and when ever connectivity available, services consumed. Java WebStart and Adobe AIR are two most widely used technologies in this field.
Voice Based Client
Voice based clients are gaining popularly to reduce load on contact center. To facilitate this type of user interface, IVR plays great role.
Machine
These types of interfaces are not very obvious but very prevent at enterprise level. Application level interaction (Software to software) and interaction with various devices like RFID tags, Bar code readers, etc (Hardware to Software or vice versa) are part of this type of interfaces.
Most of the large enterprises also get integrated with external world. This integration is both ways - information flow is in both directions. To further complicate the scenario, enterprises increasingly using SAAS. This situation requires special attention and tools. In case of integration with business partners B2B Engine plays major role in conjunction with Partner services. SAAS services may be consumed directly or via B2B engine. In case of Human interaction (CRM - Salesforce.com) B2B Engine should be bypassed but for m/c interaction or substantial data transfer (Document management for insurance companies) B2B engine should be engaged.
Security in SOA is one of the biggest concern due to distributed & heterogeneous systems interacting with each other. The security related aspects should be handled at each level of enterprise. In the proposed model security is considered from physical and virtual perspective. It also takes into account security from network and application view.
At monitoring front it take care of Technical - For support staff, Process & Managerial - For business process owners & managers and Executive - for top rung of business leadership requirements.
Governance is one of the burning issues in SOA environment. To discuss SOA Governance's Technical architecture, it is necessary to understand SOA Governance model.
Following figure depicts the SOA Governance Model.
The picture below intends to depict SOA Governance Technical Architecture.
Service Life Cycle Management essentially helps in managing service life cycle from its conception to decommission. This is one of the vital components, which determine the success of SOA initiative in any enterprise and also its long-term viability.
Rule/Policy Repository contains various Business Rules and Governance & Security Policies. The engine part of the same will help in execution of Rules and Policies as and when demanded.
Service directory essentially consists of UDDI, which facilitates consolidated listing of services.
To exploit SOA to its full extent event handling plays crucial role. In SOA environment Events can be classified into three categories
1. Generic Event
Event fired/raised in response of event or interaction with other service/s.
2. Time Based Event
Event fired/raised at particular moment of time to perform certain task
3. Heart Beat Event
Event fired/raised by services at regular interval to announce its existence and may be level of throughput it is working.
Generic Event Handling
These are events which fire/raise other events or results due to interaction of services in ecosystem.
1. Event Generator generates events with minimal data set.
2. These events should be formatted as per the event protocol defined in ecosystem, so that theses event are available to entire ecosystem for further processing.
Ø Event Pre-processing: Check that mandatory attributes are present in the incoming events
Ø Event Refinement: Introduce the other system-generated attributes like start time etc.
Ø Situation Refinement: Check the target status on which action would be performed
Ø Impact assessment: Required result assessment after that impact applies on the target
3. Event factory determine the specific type of action, which the given event have.
4. Event listener executes the specific action, which stated at the specific action code. There we have some general action apply to all type of system is present in generic action code and specific to specific action is present in the specific action listener
5. Event listener executes the action and calls the required system to complete it.
6. Some event, which completed their actions, and no further support needed from any other services are terminated
7. Some events which need further processing from some other services or may be generate other event in response of event is send back to the event engine
8. Event in response of event is again run as a new event and again it initiates its processing cycle in the event engine
Time Based Event Handling
Scheduled event delivered to a service at specific time in the future. Event deliveries can be deferred for short periods of time (such as seconds or minutes) or for long stretches of time (for example, hours later for batch processing). Until that delivery time, the Event is essentially invisible until it is delivered, allowing you to schedule work at a particular time in the future.
Heart Beat Event Handling
Heartbeats, a common software construct, verify the continual operation of a specific service. With heartbeats, a targeted service continually broadcasts a signal across its environment. System works normally when client services can detect a targeted service's heartbeat signals
There is number of interdependent services in SOA ecosystem such that, if one failed, others will also fail.
Heartbeat generally propagated across SOA ecosystem using publish/subscribe paradigm.
Transaction Management is one of the most difficult aspects of any enterprise architecture due to varying level of understanding among stakeholders, various technological platforms involvement and spread of transactions – intra & inter enterprise.
In general any enterprise grade transaction framework follows ACID but due to presence of long running business processes and involvement of asynchronous communication traditional ACID becomes irrelevant. To accommodate long running business processes and asynchronous communication relaxed ACID is used which allows violation of ACID but with capability of correcting the violations in reasonable time period and with little business impact.
SOA capable enterprise grade Transaction Framework should have following features:
· Multi level undo/redo
· Coalescing of transactions
· Automatic aggregation of nested transactions
· Supports batching of transactions (Aggregation)
· Rollback of aggregated transactions for error recovery
· Listener support
· Synchronous and Asynchronous Transaction
· Compensatory Service
· Transaction Propagation
Exception & Error Management is consist of
Exception & Error Handling
Exception & Error Logging
Exception & Error Notification
In any enterprise under SOA ecosystem Exception & Error Management poses following challenges:
Exception & Error Handling
No predefined format for passing exception information among participating services
Web services do not have capability of maintaining stack trace
Lack of common exception vocabulary across enterprise
Exception & Error Logging
Scattered logs across services/applications
Varying formats of logs across services/applications
Toggling logs at run time is service/application dependent
Exception & Error Notification
Every enterprise/LOB has its own unique notification requirement
Toggling Notification at run time is necessity
SLA and exceptions are coupled
To counter these challenges there must be a comprehensive Exception and Error Management framework which can manage at enterprise level and form the foundation for Remediation framework.
On the high level, Exception and Error Management framework’s architecture can be depicted as follows:
Remediation framework plays very important role in any SOA ecosystem. It can be though of extension of Error & Exception Management and Transaction Management frameworks. Remediation framework leverages Error & Exception Management Framework by utilizing logged error & exceptions. It also supplement Transaction Management Framework via helping to implement relaxed ACID where human intervention might be needed. Remediation framework also makes whole of SOA ecosystem robust via utilizing compensatory services.
In SOA ecosystem Security, Governance, Operations, Quality are closely related. Each face of SOA ecosystem affects other.
Saturday, December 6, 2008
Book Review: SOA Approach to Integration XML, Web services, ESB, and BPEL in real-world SOA projects
Book Review: SOA Approach to Integration XML, Web services, ESB, and BPEL in real-world SOA projects by Matjaz, Ramesh, Pooranchandra and Frank (ISBN: 978-1-904811-17-6)
At first glance this would appear to be another "me too" title, and while in some ways that may be true. However there are a couple of things that set this book apart and it can be of value to those experienced in Integration & SOA as well as those new to the subjects.
Those who are new to the book I will recommend read as much as possible and this book must be one those. This book covers basic essentials of Integration, SOA, WebServices and BPEL in very simple and lucid language, which can help any greenhorn.
The most important part of this book is its language. The most glaring lapse in this book is assuming SOA means web services.
It has explained concepts of EAI in very concise & to the point in first chapter. While explaining Integration Infrastructure authors have drawn nice picture, which covers almost all aspects of integration tool stack. But in the section of Integration Technologies section, it fails to mention that MoM, TPM, ORB, App Server and ESB are various generations of the same and no word about “Several hybrid and proprietary products”.
The second chapter covers SOA in technology agnostic as well as Java &. Net specific way. This chapter nicely covers basic concepts of SOA, which include not only ESB but also web services and its Java and .Net implementation. This chapter also tries to cover process-based architecture in the end, which makes things little bit clumsy.
The third chapter covers XML, Schema, XML document security, and XML parsing. This chapter covers a lot but also leaves behind a lot. It totally neglects non Java XML parsers.
The forth chapter seems to be inspired from IBM’s eBusiness patterns. This chapter also touches Web Services Interoperability (WS –I) stuff, which is pretty elaborated.
The fifth chapter covers BPEL and integration with emphasis on process orientation. This chapter is good reference for BPEL and especially Oracle way of BPEL in its Fusion Middleware.
The last chapter covers ESB in detail. The name of the chapter and content of it does not match.
The most wanted but missing part in this book is references for each chapter in particular and for book in general.
In summary I would suggest that this is a good book for anyone involved in SOA and integration project. It will provide a good introduction for those new to the discipline of Integration and SOA. While acting as a useful refresher for those with more experience
go and grab the book at packtpub.com
SOA and Business
SOA is for business enablement. It is for business to help manage IT assets more effectively and efficiently.
IT assets in general may be viewed from three perspectives.
- Enterprise
- Application
- Product
Enterprise perspective include IT assets at inter and intra enterprise level. It considers Business value of IT assets on enterprise level; so it’s effects is also enterprise level. This perspective is generally elaborated in terms of inter and intra enterprise integration. This perspective is generally taken care by CIO and CTO level while deciding enterprise wide IT strategy. Enterprise Architects also play major role in shaping this perspective.
Application perspective limits IT assets within the boundaries of application – off the shelf or custom-made. This perspective considers Application or its significant functionality as service. This perspective makes sense to VP Engineering and CTO. This perspective gains significance in case of purchasing new enterprise application.
Product perspective becomes very important for product developing companies. This perspective focuses on the services offered within a product. Proper usage of this perspective helps in construction of robust and flexible product/product family. This perspective assumes significance for Technical Architects.
All three perspectives help business to save on cost and decreasing the recurring expenses in terms of maintenance & support. They also helps in reducing time to market.
SOA also brings technological breakthroughs which helps in cost saving and fasten the introduction of products in market.
- Increased reusability of components/services.
- Ease in integration – inter and intra enterprise.
- Increased pace of development and adjusting IT assets as business evolve.
- Facilitation of incremental construction (architecture, design & development) of IT assets.
- Facilitation of geographically dispersed development in varying time zones.
- Facilitation in outsourcing/off shoring the IT assets – construction and maintenance & support.
- Facelift of legacy application and integration of them in modern IT landscapes.
- Increased utilization of human resources.
- Reduced and managed risk with respect to legacy IT assets and resources.
Sunday, November 30, 2008
SOA Components
- Application Front end: Generally it is web browser based but thick client is also preferred in some cases.
- Service: In most of the real time implementations it is Web service but other types of services are based on CORBA, RMI, etc.
- Service Repository: If service is implemented using Web Service, it is UDDI (Universal Description, Discovery and Integration)
- Service Hosting Environment: In most of the implementations it is ESB ( Enterprise Service Bus)
- Contract: If service is Web Service then contract is WSDL (Web Services Description Language)
- Implementation: It is web service in most of the cases.
- Interface: In most of the cases it is Web based ( SOAP over HTTP)
- Business Logic: It can reside in any language, platform.
- Data: It can reside in any language, platform.
SOA Principles
- Follow the standards
- Optimize the granularity of service
- Make conscious decision that when not to use SOA
- SOA is gradual process, do not emulate Big Bang.
- SOA effectiveness is not in number of services but in reusability and compos-ability of services
- SOA is for application and also for Enterprise, make conscious choice
Challenges in adopting SOA
- SOA is a strategic decision not the tactical
- Vendor centric SOA view
- Resistance to standardization
- Fear of losing control
Technical Challenges
- Lack of technical understanding
- Re-usability of services
- Isolated implementation of SOA
- High demand on network and hardware
What & Why SOA ??
Advantages of SOA
There are only three advantages of SOA:
- Re usability
- Frugality of IT Assets
- Agility
- Business
- Technical
All other advantages can be derived from these three.
Business & Technical Case for SOA
Business
- Fast pace business environment
- Customers taste change
- Merger, acquisition & split
- Change in business processes & procedures
- Ever changing legal environment
- Ever changing IT environment
- Never ending race for cost cutting
Technical
- Spaghetti of technologies
- Independence from Technology and/or Vendor
- Shrinks SDLC efforts
- Reusable, compos-able services available over network
SOA offer the following advantages over traditional approaches to distributed computing:
- Business services across the platforms
- Provide location independence
- Services need not be at a particular system or particular network
- Completely loosely coupled approach
- Authentication and authorization support at every level
- The search and connectivity to other services is dynamic
Short-term benefits of implementation:
- Enhances reliability
- Reduces hardware acquisition costs
- Leverages existing development skills
- Accelerates movement to standards-based server and application consolidation
- Provides a data bridge between incompatible technologies
Long-term benefits of implementation:
- Provides the ability to build composite applications
- Creates a self-healing infrastructure that reduces management costs
- Provides truly real-time decision-making applications
- Benefits from the perspective of Business Value Ability to more quickly meet customer demands
- Lower costs associated with the acquisition and maintenance of technology
- Management of business functionality closer to the business units
- Leverages existing investments in technology
- Reduces reliance on expensive custom development
When to say NO to SOA
Every one is advocating for SOA. But there must be some scenarios when not to use SOA.
- In Homogeneous IT environment.
- When real time performance is critical.
- When Business and IT environment is static.
- When tight coupling is advantageous not a disadvantageous.
- When huge amount of data transfer is happening.
- When data sharing is in batch mode.
Friday, November 21, 2008
Just Started
In this blog I will iterate my thoughts on:
1. Enterprise Architecture
2. Enterprise SOA
3. Enterprise BPM
4. EAI - A2A and B2B
5. Application Architecture
6. Application level SOA
7. Application level BPM