Showing posts with label BPM. Show all posts
Showing posts with label BPM. Show all posts
Wednesday, March 9, 2011
Sunday, May 9, 2010
Monday, February 8, 2010
Thursday, February 4, 2010
Modelling Processes
The Process Modeling (PM) is the one of the primary requirement of any BPM project. The objective of PM is to record definition of as-is process. This recording results in analysis of existing process to understand strengths and weakness of the same. Any PM exercise needs to answer following questions:
1. What are the inputs (3Ms and One I – Man/Human Resources/Roles, Material/Document, Machine/Processing capability and Data/Information and Knowledge) to the Process?
2. What are the outcomes of the Process (tangible and non-tangible as well as direct and side effects)?
3. What are the different activities are performed in the course of execution of the Process?
4. What is the order of activities?
5. Who perform the activities?
6. What are the different events occur during the course of process execution?
7. What is the order of events?
8. What the pre-requisites of the Process to initiate?
9. What do/does trigger/s the Process?
10. What are incomplete-nesses or strengths of the Process?
11. Any historical information related to the Process.
The Process Modeling is carried out to achieve following objectives:
1. To specify the exact result of the process and to understand the value of this result.
2. To enable evolution of the Process.
3. To optimize the process (3Ms, One I and One T – Man/Human Resources, Material/Document, Machine/Processing capability, Data/Information and Knowledge and Time to execute)
4. To understand correlation among different processes.
Process Modeling results is variety of artifacts:
1. Process Map: This document – textually and/or graphically depicts various processes in enterprise and their interrelationship at very high level.
2. Roles and Relations Structure: This document - and/or graphically depicts various roles/groups and their relation in graph structure ( not organizational hierarchy) with respect to a Process.
3. As is Process Model: This model consists of following:
a. Process Environment Diagram: Detailed relationship of the Process with other interacting/intersecting processes.
b. Detailed Process Map: Detailed process map of each activity which getting executed in Process Execution all probable conditions.
c. Business rule: List of Business Rules and how they affect execution of process.
d. Exception Handling Diagram: Here detailing is carried out for error and exceptional cases for the process.
4. Publishing and Communication Process Modeling artifacts.
To carry out Process Modeling, simulation plays important role. Any simulation platform/tool/facility should exhibit following traits:
1. Simulate processes with respect to load distribution with time and non time factors
2. Simulate processes with respect to resource consumption ( 3Ms, One I and One T)
3. Statistical analysis of simulation result.
4. What if analysis.
5. Textual and graphical representation of simulation results
6. Publication of simulation results.
While carrying out Process Modeling following principles should be followed:
1. Stick to standards such as BPMN, ARIS methodology, etc.
2. Models must be semantically correct, means all activities, events and 3Ms, One I & One T are taken care of.
3. Beware of Paralysis by Analysis phenomenon.
4. Do Cost Benefit Analysis
5. Minimize technical jargon
While doing Process Modeling following challenges are common:
1. Absence of Process owner
2. Persons aware about process details are not cooperative enough
3. Modeler's point of view gets incorporated in Process Model.
4. Usage of jargon ( Business as well as IT specific)
5. As is Model start depicting To Be.
1. What are the inputs (3Ms and One I – Man/Human Resources/Roles, Material/Document, Machine/Processing capability and Data/Information and Knowledge) to the Process?
2. What are the outcomes of the Process (tangible and non-tangible as well as direct and side effects)?
3. What are the different activities are performed in the course of execution of the Process?
4. What is the order of activities?
5. Who perform the activities?
6. What are the different events occur during the course of process execution?
7. What is the order of events?
8. What the pre-requisites of the Process to initiate?
9. What do/does trigger/s the Process?
10. What are incomplete-nesses or strengths of the Process?
11. Any historical information related to the Process.
The Process Modeling is carried out to achieve following objectives:
1. To specify the exact result of the process and to understand the value of this result.
2. To enable evolution of the Process.
3. To optimize the process (3Ms, One I and One T – Man/Human Resources, Material/Document, Machine/Processing capability, Data/Information and Knowledge and Time to execute)
4. To understand correlation among different processes.
Process Modeling results is variety of artifacts:
1. Process Map: This document – textually and/or graphically depicts various processes in enterprise and their interrelationship at very high level.
2. Roles and Relations Structure: This document - and/or graphically depicts various roles/groups and their relation in graph structure ( not organizational hierarchy) with respect to a Process.
3. As is Process Model: This model consists of following:
a. Process Environment Diagram: Detailed relationship of the Process with other interacting/intersecting processes.
b. Detailed Process Map: Detailed process map of each activity which getting executed in Process Execution all probable conditions.
c. Business rule: List of Business Rules and how they affect execution of process.
d. Exception Handling Diagram: Here detailing is carried out for error and exceptional cases for the process.
4. Publishing and Communication Process Modeling artifacts.
To carry out Process Modeling, simulation plays important role. Any simulation platform/tool/facility should exhibit following traits:
1. Simulate processes with respect to load distribution with time and non time factors
2. Simulate processes with respect to resource consumption ( 3Ms, One I and One T)
3. Statistical analysis of simulation result.
4. What if analysis.
5. Textual and graphical representation of simulation results
6. Publication of simulation results.
While carrying out Process Modeling following principles should be followed:
1. Stick to standards such as BPMN, ARIS methodology, etc.
2. Models must be semantically correct, means all activities, events and 3Ms, One I & One T are taken care of.
3. Beware of Paralysis by Analysis phenomenon.
4. Do Cost Benefit Analysis
5. Minimize technical jargon
While doing Process Modeling following challenges are common:
1. Absence of Process owner
2. Persons aware about process details are not cooperative enough
3. Modeler's point of view gets incorporated in Process Model.
4. Usage of jargon ( Business as well as IT specific)
5. As is Model start depicting To Be.
Friday, August 14, 2009
Top 16 lessons from Real world BPM Projects
DO
• Start with an low hanging fruits – easy to implement, least politicized
• Document the process in detail
• Establish a simple ROI metric for initiative that is meaning for business
• Do engineering not over engineering
• Insist on multiple iterations
• Use business process needs to compare offerings
• Hire help
• Plan for enterprise convergence
• Change is inevitable
• Work in cooperation with process improvement (Six Sigma, Reengineering, Lean etc) initiatives.
DON’T
• Revolution in process engineering
• Go overboard with process performance metrics
• Wait for complete consensus on the new process before getting something up and running
• Allow Scope creep
• Focus on technology instead of Business functionality of BPMS
• Try implement complex process in first go
• Start with an low hanging fruits – easy to implement, least politicized
• Document the process in detail
• Establish a simple ROI metric for initiative that is meaning for business
• Do engineering not over engineering
• Insist on multiple iterations
• Use business process needs to compare offerings
• Hire help
• Plan for enterprise convergence
• Change is inevitable
• Work in cooperation with process improvement (Six Sigma, Reengineering, Lean etc) initiatives.
DON’T
• Revolution in process engineering
• Go overboard with process performance metrics
• Wait for complete consensus on the new process before getting something up and running
• Allow Scope creep
• Focus on technology instead of Business functionality of BPMS
• Try implement complex process in first go
Sunday, August 2, 2009
Enterprise Process Assets
Enterprise Process Assets are artifacts which facilitate execution, maintenance, retirement and improvement of a process. Enterprise Process assets can be categories into:
· Processes and Procedures
· Corporate Knowledge Base
Processes and procedures are part of any Enterprise Assets because a process or procedure facilitates lifecycle of another process. For example retirement process manages retirement of a process.
Corporate Knowledge Base forms large chunk of Enterprise Process Assets. It get accumulated a various projects get executed. Few of the examples of Enterprise Process Assets which are of Corporate Knowledge Base are:
· Formal and informal plans
· Policies
· Lesson learned
· Risk data
· Historical information
· Earned Value data
· Process measurement data
· Project files from previous projects
· Issue and defect management data
Enterprise Process Assets management is of very important to manage processes and projects in efficient manner.
· Processes and Procedures
· Corporate Knowledge Base
Processes and procedures are part of any Enterprise Assets because a process or procedure facilitates lifecycle of another process. For example retirement process manages retirement of a process.
Corporate Knowledge Base forms large chunk of Enterprise Process Assets. It get accumulated a various projects get executed. Few of the examples of Enterprise Process Assets which are of Corporate Knowledge Base are:
· Formal and informal plans
· Policies
· Lesson learned
· Risk data
· Historical information
· Earned Value data
· Process measurement data
· Project files from previous projects
· Issue and defect management data
Enterprise Process Assets management is of very important to manage processes and projects in efficient manner.
Sunday, January 18, 2009
Process Design
In most of the cases BPM deals with existing process where process designer picks up the existing process, optimize it as per the given mandate and finish off his/her work. Does this expose him/her to best practices of process design?
So the steps of process design are:
1. Concept: The Idea what do you want to do?
2. Form: The Process The process and the result of process(product)
3. Content: The Meaning Does form satisfies the concept?
In a well designed process cater to following class of changes:
1. Change in State: Transition to one state to another during the course of execution of process. For example in Quote to Cash process change in state of Quote to Order.
2. Internal Changes: Changes in other processes which are within the realm of enterprise/s to whom process is catering. For example change in price of an item while a Quote to Cash process is in middle of execution. Price change is another process.
3. Environmental Changes: These changes are most of time beyond control of participating enterprises. For example change in tax structure while a Quote to Cash process is in middle of execution.
In any process Class 1 changes are accounted for because they form the normal exertion path of a process. Class 2 changes are also accounted in most of well designed process either as Class 1 changes or exceptions or via versioning path. Class 3 changes are not taken care in most of the process design due to resultant complexity and thin chances of occurrence of these changes. Class 3 changes can be accounted either as Exception, using versioning or catastrophe. If Class 3 changes are not accounted in process design this may result in chaos.
To design a process following models can be used by various players of game.
1. Value Stream Mapping: This mapping technique is widely used by Six Sigma and Lean professionals.
2. Event Modeling: This technique is widely used by Process Reengineering professionals.
3. Object Modeling: This technique is widely used by software professionals to model software as real life objects. For details you can refer to OOPS and OOAD literature.
4. Data Modeling: This is one more technique which is widely used by software professionals to design data intensive software.
5. Process Interaction Model: This is the models which caters to intra and inter process interaction. This model is borrowing concepts from Control engineering and ARIS methodology. This model is under development and soon I will release more details about it.
Business and System Analysts use Value Stream Mapping, Event and Process Interaction Models to define, refine and model a process. Technical Architects and Designers use Object Model, Data and Process Interaction Models to come out with technical specifications and actual design.
Process Design Good Practices
1. Design for present but do not loose sight from future.
2. Take account of Class 1 events and Class 2 events.
3. Manage Class 3 events keeping complexity in full view.
4. Balance between automation and manual tasks to keep process simple.
5. Strive for perfection but watch out for complexity, so balance.
Reference
1. http://en.wikipedia.org/wiki/Value_Stream_Mapping
2. http://www.isixsigma.com/dictionary/Value_Stream_Mapping-413.htm
3. http://en.wikipedia.org/wiki/Object-oriented_analysis_and_design
4. http://www.ooad.org/
5. http://en.wikipedia.org/wiki/Data_flow_diagram
6. http://www.agilemodeling.com/artifacts/dataFlowDiagram.htm
7. http://en.wikipedia.org/wiki/Dataflow
8. http://en.wikipedia.org/wiki/Event_model
9. http://msdn.microsoft.com/en-us/library/ms533023(VS.85).aspx
10. http://www.actiontech.com/BPM/BIM.cfm
11. http://process-interaction-models.info/
So the steps of process design are:
1. Concept: The Idea what do you want to do?
2. Form: The Process The process and the result of process(product)
3. Content: The Meaning Does form satisfies the concept?
In a well designed process cater to following class of changes:
1. Change in State: Transition to one state to another during the course of execution of process. For example in Quote to Cash process change in state of Quote to Order.
2. Internal Changes: Changes in other processes which are within the realm of enterprise/s to whom process is catering. For example change in price of an item while a Quote to Cash process is in middle of execution. Price change is another process.
3. Environmental Changes: These changes are most of time beyond control of participating enterprises. For example change in tax structure while a Quote to Cash process is in middle of execution.
In any process Class 1 changes are accounted for because they form the normal exertion path of a process. Class 2 changes are also accounted in most of well designed process either as Class 1 changes or exceptions or via versioning path. Class 3 changes are not taken care in most of the process design due to resultant complexity and thin chances of occurrence of these changes. Class 3 changes can be accounted either as Exception, using versioning or catastrophe. If Class 3 changes are not accounted in process design this may result in chaos.
To design a process following models can be used by various players of game.
1. Value Stream Mapping: This mapping technique is widely used by Six Sigma and Lean professionals.
2. Event Modeling: This technique is widely used by Process Reengineering professionals.
3. Object Modeling: This technique is widely used by software professionals to model software as real life objects. For details you can refer to OOPS and OOAD literature.
4. Data Modeling: This is one more technique which is widely used by software professionals to design data intensive software.
5. Process Interaction Model: This is the models which caters to intra and inter process interaction. This model is borrowing concepts from Control engineering and ARIS methodology. This model is under development and soon I will release more details about it.
Business and System Analysts use Value Stream Mapping, Event and Process Interaction Models to define, refine and model a process. Technical Architects and Designers use Object Model, Data and Process Interaction Models to come out with technical specifications and actual design.
Process Design Good Practices
1. Design for present but do not loose sight from future.
2. Take account of Class 1 events and Class 2 events.
3. Manage Class 3 events keeping complexity in full view.
4. Balance between automation and manual tasks to keep process simple.
5. Strive for perfection but watch out for complexity, so balance.
Reference
1. http://en.wikipedia.org/wiki/Value_Stream_Mapping
2. http://www.isixsigma.com/dictionary/Value_Stream_Mapping-413.htm
3. http://en.wikipedia.org/wiki/Object-oriented_analysis_and_design
4. http://www.ooad.org/
5. http://en.wikipedia.org/wiki/Data_flow_diagram
6. http://www.agilemodeling.com/artifacts/dataFlowDiagram.htm
7. http://en.wikipedia.org/wiki/Dataflow
8. http://en.wikipedia.org/wiki/Event_model
9. http://msdn.microsoft.com/en-us/library/ms533023(VS.85).aspx
10. http://www.actiontech.com/BPM/BIM.cfm
11. http://process-interaction-models.info/
Monday, December 29, 2008
Process Template
Recently I have developed a document which will be helpful in documenting enterprise process. This document is in MS WORD format. The document is structured in technology independent way. I will certainly welcome any suggestion and changes with respect to technology and platform.
I hope this document will be helpful tool to lot of us to sail through EPM/BPM universe.
The document can be downloaded from Scribd.com.
I hope this document will be helpful tool to lot of us to sail through EPM/BPM universe.
The document can be downloaded from Scribd.com.
Wednesday, December 10, 2008
Enterprise Process Management
Enterprise Process Management should be considered as superset of Business Process Management. In current scenario whenever we talk about BPM we always think of business organization. But in real world every organization is not business organization. There are various organizations which have interests beyond business e.g. government organizations, non-government organizations (NGO), philanthropic organizations, etc. So the term BPM is not sufficient to cater their needs. I propose Enterprise Process Management (EPM) because Enterprise covers not only business organization but also lot of other non-business organizations.
1. Inventive viewpoint: This viewpoint is affected by strategic and tactical decisions of any enterprise. Strategic and tactical success of any enterprise demands effective and efficient processes with continuous improvement build in. As environment is dynamic and most of the time beyond the control of enterprise, processes must be adaptive in nature and if possible self-aware.
2. Execution viewpoint: This viewpoint focuses on execution part of processes’ lifecycle. Once process is released in any enterprise for execution purposes this viewpoint takes over. In any successful process this viewpoint has longest duration with numerous small changes but insignificant number of step changes.
3. Improvement viewpoint: In this viewpoint focus is on improvement related aspects of process. This viewpoint gets feed from execution and gives to inventive viewpoint. The improvements may be minor chunks (part of continuous improvement) or quantum leap.
On the basis of complexity, enterprise processes can be divided into three categories:
1. Material Processes
2. Information Processes
3. Human Interaction Processes
Keeping EPM into sight, what is process?
“A process is a coordinated thread of sequential and parallel activities needed to deliver value to stakeholders.”
So what is EPM?
“Enterprise Process Management is assuming controlling position on processes from inception/idea to its delivery, from very first activity to last, and analysis of each and every activity for continuous and step improvement.”
From definition it is very clear that EPM can be visualized from three view points:
- Inventive viewpoint
- Execution viewpoint
- Improvement viewpoint
1. Inventive viewpoint: This viewpoint is affected by strategic and tactical decisions of any enterprise. Strategic and tactical success of any enterprise demands effective and efficient processes with continuous improvement build in. As environment is dynamic and most of the time beyond the control of enterprise, processes must be adaptive in nature and if possible self-aware.
2. Execution viewpoint: This viewpoint focuses on execution part of processes’ lifecycle. Once process is released in any enterprise for execution purposes this viewpoint takes over. In any successful process this viewpoint has longest duration with numerous small changes but insignificant number of step changes.
3. Improvement viewpoint: In this viewpoint focus is on improvement related aspects of process. This viewpoint gets feed from execution and gives to inventive viewpoint. The improvements may be minor chunks (part of continuous improvement) or quantum leap.
Classification of Enterprises Processes
On the basis of complexity, enterprise processes can be divided into three categories:
1. Material Processes
2. Information Processes
3. Human Interaction Processes
- Material Processes: MPs are primarily concerned to “THINGS”. The purpose of these processes is transforming the material and consume resources. These processes rely heavily on Industrial engineering concepts. These processes are found in primarily in manufacturing, assembly, process industrial environment. Generally speaking these processes are so matured over time period across the industries that they more or less resemble procedure or long running procedures. This means the input and output of these processes can be defined objectively.
- Information Processes: IPs are primarily concerned to “DATA”. These processes transform data into information or information into more abstract information. These processes rely on Computer science, Software engineering and Information science concepts. These processes are prevalent in service and non-service industrial environment. Again due to tremendous progress in computer science, information science and software engineering these processes resemble to procedures.
- Human Interaction Processes: HIPs are primarily focus on human cognitive aspects. HIPs are articulation of subjective issues and getting agreement among various stakeholders. HIPs are based on structure of human communication and coordination, which are prevalent in natural languages and cultures. HIPs are highly affected by cognitive abilities of participating actors and surrounding environment. Most of the time HIPs are highly subjective so their automation is a big challenge.
In real life settings, most of the enterprise processes are combination of MP, IP and HIP.
For simplicity let us consider a simple process: Opening a checking bank account by a new customer.
So to open checking bank account, customer has to submit a form to branch where he wishes to have account. The account opening form also need some supporting document like identity card of the prospect along with letter from an existing account holder. Upon submission of completed form and supporting document, a clerk at branch verifies the documents at preliminary level and sends to regional office for detailed verification. If verification result is OK, account is opened, base branch & customer are informed and checkbook, ATM card are dispatched to customer. If verification result is NOT OK, base branch & customer are informed.
In the first glance the process seems to be well structured and simple enough for automation. This process seems to fall in Material and Information Process category. So automation looks easy and within reach. Now let Human aspect enter into the process.
Mr. Clerk who accepts documents from prospective account holders has friendship with Mr. Tony. One fine day Mr. Tony reaches to branch where Mr. Clerk is stationed and submit Account opening form. Mr. Tony does not was not carrying supporting documents, so he talks to Mr. Clerk and makes promise to submit supporting documents on next Monday and asks Mr. Clerk to hold his docs till then. So human relations start tearing a well-structured process.
Sunday, December 7, 2008
Enterprise SOA Reference Architecture
Here I propose SOA Enterprise Reference Architecture. This is for enterprise and certainly for an application it should be inspiration.
Reference Architecture is independent of platforms offered by various vendors. It is also possible that none of the vendors in market provide complete solution to realize this reference architecture.
Data Persistent Layer is primarily consists of Enterprise Data - Master, Transaction and Meta. It may also include Datamart and Data driven reporting infrastructure.
In most of the large enterprises Data Persistent Layer consists of relational (RDBMS), hierarchical (LDAP, IMS), unstructured (Emails, chat sessions, file systems, etc) and any other type. These variations in data characteristics warrant unique data handling and treatment technologies.
This Layer consists of different business applications. These applications may be packaged (e.g. SAP, Reteck, etc) or may be custom developed in various technologies (e.g. Java, .Net, etc). Again application in this layer may be service enabled or legacy once, which do not expose their functionalities as service.
At this layer EAI exist. Some may argue that SOA is one form of integration then how EAI can exist at this layer. I will discuss this argument in one of my future posting but not now.
Services Layer
This layer is consists of multiple sub layers which encapsulate different class of services. It encompasses Connectivity Services, Data Services, Business Activity Services and Business Process Services. Each layer in turn consists of other sub-types of services.
This layer exposes different aspects of business - functional and infrastructural aspects. Connectivity and Data services are consisting of Infrastructural services while Business Activity and Process services expose functional Services.
These services though in layers but still breach layered architecture to expose these services independent of layers and for better performance. These services can also be exposed to multiple ESBs if situation warrants.
1. Connectivity services primarily cater to accessing system level functionalities, any adapter & connector services and partner (B2B) integration services.
2. Data services comprises of services, which caters to Business, Technical and Meta data related services.
3. Business Activity Services consist of Atomic business/functionalities and any custom business logic developed by combing one or more Data & Connectivity services.
4. Business Process Services primarily consist of Process integration, Machine and Human centric services. These services primarily comprise of Business Activity services.
Orchestration
Routing
Mediation
Enrichment
Some ESBs may provide one more functionality - Choreography. This functionality is associated with BPM systems and currently none of Commercially available ESB provides the same.
Enrichment may be of two types: Data Transformation and Channel Transformation. Most of the analysts & architects keep Data Transformation & Channel Transformation separate functionality.
Human
Browser clients: May be based on traditional HTML based or AJAX based
Portable devices: May be based on open standards (WAP) or propitiatory (device or carrier specific)
Alway Connected: These clients connect to service providers and consume the services in real time. Traditional think client falls in this category
Offline Clients: These clients do not connect to service providers in Always On fashion but an actor can work in offline mode and when ever connectivity available, services consumed. Java WebStart and Adobe AIR are two most widely used technologies in this field.
Voice Based Client
Voice based clients are gaining popularly to reduce load on contact center. To facilitate this type of user interface, IVR plays great role.
Machine
These types of interfaces are not very obvious but very prevent at enterprise level. Application level interaction (Software to software) and interaction with various devices like RFID tags, Bar code readers, etc (Hardware to Software or vice versa) are part of this type of interfaces.
Most of the large enterprises also get integrated with external world. This integration is both ways - information flow is in both directions. To further complicate the scenario, enterprises increasingly using SAAS. This situation requires special attention and tools. In case of integration with business partners B2B Engine plays major role in conjunction with Partner services. SAAS services may be consumed directly or via B2B engine. In case of Human interaction (CRM - Salesforce.com) B2B Engine should be bypassed but for m/c interaction or substantial data transfer (Document management for insurance companies) B2B engine should be engaged.
Security in SOA is one of the biggest concern due to distributed & heterogeneous systems interacting with each other. The security related aspects should be handled at each level of enterprise. In the proposed model security is considered from physical and virtual perspective. It also takes into account security from network and application view.
At monitoring front it take care of Technical - For support staff, Process & Managerial - For business process owners & managers and Executive - for top rung of business leadership requirements.
Governance is one of the burning issues in SOA environment. To discuss SOA Governance's Technical architecture, it is necessary to understand SOA Governance model.
Following figure depicts the SOA Governance Model.
The picture below intends to depict SOA Governance Technical Architecture.
Service Life Cycle Management essentially helps in managing service life cycle from its conception to decommission. This is one of the vital components, which determine the success of SOA initiative in any enterprise and also its long-term viability.
Rule/Policy Repository contains various Business Rules and Governance & Security Policies. The engine part of the same will help in execution of Rules and Policies as and when demanded.
Service directory essentially consists of UDDI, which facilitates consolidated listing of services.
To exploit SOA to its full extent event handling plays crucial role. In SOA environment Events can be classified into three categories
1. Generic Event
Event fired/raised in response of event or interaction with other service/s.
2. Time Based Event
Event fired/raised at particular moment of time to perform certain task
3. Heart Beat Event
Event fired/raised by services at regular interval to announce its existence and may be level of throughput it is working.
Generic Event Handling
These are events which fire/raise other events or results due to interaction of services in ecosystem.
1. Event Generator generates events with minimal data set.
2. These events should be formatted as per the event protocol defined in ecosystem, so that theses event are available to entire ecosystem for further processing.
Ø Event Pre-processing: Check that mandatory attributes are present in the incoming events
Ø Event Refinement: Introduce the other system-generated attributes like start time etc.
Ø Situation Refinement: Check the target status on which action would be performed
Ø Impact assessment: Required result assessment after that impact applies on the target
3. Event factory determine the specific type of action, which the given event have.
4. Event listener executes the specific action, which stated at the specific action code. There we have some general action apply to all type of system is present in generic action code and specific to specific action is present in the specific action listener
5. Event listener executes the action and calls the required system to complete it.
6. Some event, which completed their actions, and no further support needed from any other services are terminated
7. Some events which need further processing from some other services or may be generate other event in response of event is send back to the event engine
8. Event in response of event is again run as a new event and again it initiates its processing cycle in the event engine
Time Based Event Handling
Scheduled event delivered to a service at specific time in the future. Event deliveries can be deferred for short periods of time (such as seconds or minutes) or for long stretches of time (for example, hours later for batch processing). Until that delivery time, the Event is essentially invisible until it is delivered, allowing you to schedule work at a particular time in the future.
Heart Beat Event Handling
Heartbeats, a common software construct, verify the continual operation of a specific service. With heartbeats, a targeted service continually broadcasts a signal across its environment. System works normally when client services can detect a targeted service's heartbeat signals
There is number of interdependent services in SOA ecosystem such that, if one failed, others will also fail.
Heartbeat generally propagated across SOA ecosystem using publish/subscribe paradigm.
Transaction Management is one of the most difficult aspects of any enterprise architecture due to varying level of understanding among stakeholders, various technological platforms involvement and spread of transactions – intra & inter enterprise.
In general any enterprise grade transaction framework follows ACID but due to presence of long running business processes and involvement of asynchronous communication traditional ACID becomes irrelevant. To accommodate long running business processes and asynchronous communication relaxed ACID is used which allows violation of ACID but with capability of correcting the violations in reasonable time period and with little business impact.
SOA capable enterprise grade Transaction Framework should have following features:
· Multi level undo/redo
· Coalescing of transactions
· Automatic aggregation of nested transactions
· Supports batching of transactions (Aggregation)
· Rollback of aggregated transactions for error recovery
· Listener support
· Synchronous and Asynchronous Transaction
· Compensatory Service
· Transaction Propagation
Exception & Error Management is consist of
Exception & Error Handling
Exception & Error Logging
Exception & Error Notification
In any enterprise under SOA ecosystem Exception & Error Management poses following challenges:
Exception & Error Handling
No predefined format for passing exception information among participating services
Web services do not have capability of maintaining stack trace
Lack of common exception vocabulary across enterprise
Exception & Error Logging
Scattered logs across services/applications
Varying formats of logs across services/applications
Toggling logs at run time is service/application dependent
Exception & Error Notification
Every enterprise/LOB has its own unique notification requirement
Toggling Notification at run time is necessity
SLA and exceptions are coupled
To counter these challenges there must be a comprehensive Exception and Error Management framework which can manage at enterprise level and form the foundation for Remediation framework.
On the high level, Exception and Error Management framework’s architecture can be depicted as follows:
Remediation framework plays very important role in any SOA ecosystem. It can be though of extension of Error & Exception Management and Transaction Management frameworks. Remediation framework leverages Error & Exception Management Framework by utilizing logged error & exceptions. It also supplement Transaction Management Framework via helping to implement relaxed ACID where human intervention might be needed. Remediation framework also makes whole of SOA ecosystem robust via utilizing compensatory services.
In SOA ecosystem Security, Governance, Operations, Quality are closely related. Each face of SOA ecosystem affects other.
Reference Architecture is independent of platforms offered by various vendors. It is also possible that none of the vendors in market provide complete solution to realize this reference architecture.
Data Persistent Layer
Data Persistent Layer is primarily consists of Enterprise Data - Master, Transaction and Meta. It may also include Datamart and Data driven reporting infrastructure.
In most of the large enterprises Data Persistent Layer consists of relational (RDBMS), hierarchical (LDAP, IMS), unstructured (Emails, chat sessions, file systems, etc) and any other type. These variations in data characteristics warrant unique data handling and treatment technologies.
Service Providers
This Layer consists of different business applications. These applications may be packaged (e.g. SAP, Reteck, etc) or may be custom developed in various technologies (e.g. Java, .Net, etc). Again application in this layer may be service enabled or legacy once, which do not expose their functionalities as service.
At this layer EAI exist. Some may argue that SOA is one form of integration then how EAI can exist at this layer. I will discuss this argument in one of my future posting but not now.
Services Layer
This layer is consists of multiple sub layers which encapsulate different class of services. It encompasses Connectivity Services, Data Services, Business Activity Services and Business Process Services. Each layer in turn consists of other sub-types of services.
This layer exposes different aspects of business - functional and infrastructural aspects. Connectivity and Data services are consisting of Infrastructural services while Business Activity and Process services expose functional Services.
These services though in layers but still breach layered architecture to expose these services independent of layers and for better performance. These services can also be exposed to multiple ESBs if situation warrants.
1. Connectivity services primarily cater to accessing system level functionalities, any adapter & connector services and partner (B2B) integration services.
2. Data services comprises of services, which caters to Business, Technical and Meta data related services.
3. Business Activity Services consist of Atomic business/functionalities and any custom business logic developed by combing one or more Data & Connectivity services.
4. Business Process Services primarily consist of Process integration, Machine and Human centric services. These services primarily comprise of Business Activity services.
Enterprise Service Bus
ESB provides four functionalities:Orchestration
Routing
Mediation
Enrichment
Some ESBs may provide one more functionality - Choreography. This functionality is associated with BPM systems and currently none of Commercially available ESB provides the same.
Enrichment may be of two types: Data Transformation and Channel Transformation. Most of the analysts & architects keep Data Transformation & Channel Transformation separate functionality.
Presentation Services
These services support various user interfaces (human or machine centric) and let outer world (intra & inter enterprise) interact with enterprises IT assets. The consumers of presentation services areHuman
- Thin Client
Browser clients: May be based on traditional HTML based or AJAX based
Portable devices: May be based on open standards (WAP) or propitiatory (device or carrier specific)
- Thick Client
Alway Connected: These clients connect to service providers and consume the services in real time. Traditional think client falls in this category
Offline Clients: These clients do not connect to service providers in Always On fashion but an actor can work in offline mode and when ever connectivity available, services consumed. Java WebStart and Adobe AIR are two most widely used technologies in this field.
Voice Based Client
Voice based clients are gaining popularly to reduce load on contact center. To facilitate this type of user interface, IVR plays great role.
Machine
These types of interfaces are not very obvious but very prevent at enterprise level. Application level interaction (Software to software) and interaction with various devices like RFID tags, Bar code readers, etc (Hardware to Software or vice versa) are part of this type of interfaces.
Integration with Outer World
Most of the large enterprises also get integrated with external world. This integration is both ways - information flow is in both directions. To further complicate the scenario, enterprises increasingly using SAAS. This situation requires special attention and tools. In case of integration with business partners B2B Engine plays major role in conjunction with Partner services. SAAS services may be consumed directly or via B2B engine. In case of Human interaction (CRM - Salesforce.com) B2B Engine should be bypassed but for m/c interaction or substantial data transfer (Document management for insurance companies) B2B engine should be engaged.
Security
Security in SOA is one of the biggest concern due to distributed & heterogeneous systems interacting with each other. The security related aspects should be handled at each level of enterprise. In the proposed model security is considered from physical and virtual perspective. It also takes into account security from network and application view.
At monitoring front it take care of Technical - For support staff, Process & Managerial - For business process owners & managers and Executive - for top rung of business leadership requirements.
Governance
Governance is one of the burning issues in SOA environment. To discuss SOA Governance's Technical architecture, it is necessary to understand SOA Governance model.
Following figure depicts the SOA Governance Model.
The picture below intends to depict SOA Governance Technical Architecture.
Service Life Cycle Management
Service Life Cycle Management essentially helps in managing service life cycle from its conception to decommission. This is one of the vital components, which determine the success of SOA initiative in any enterprise and also its long-term viability.
Rule/Policy Repository & Engine
Rule/Policy Repository contains various Business Rules and Governance & Security Policies. The engine part of the same will help in execution of Rules and Policies as and when demanded.
Service Directory
Service directory essentially consists of UDDI, which facilitates consolidated listing of services.
Event Management
To exploit SOA to its full extent event handling plays crucial role. In SOA environment Events can be classified into three categories
1. Generic Event
Event fired/raised in response of event or interaction with other service/s.
2. Time Based Event
Event fired/raised at particular moment of time to perform certain task
3. Heart Beat Event
Event fired/raised by services at regular interval to announce its existence and may be level of throughput it is working.
Generic Event Handling
These are events which fire/raise other events or results due to interaction of services in ecosystem.
1. Event Generator generates events with minimal data set.
2. These events should be formatted as per the event protocol defined in ecosystem, so that theses event are available to entire ecosystem for further processing.
Ø Event Pre-processing: Check that mandatory attributes are present in the incoming events
Ø Event Refinement: Introduce the other system-generated attributes like start time etc.
Ø Situation Refinement: Check the target status on which action would be performed
Ø Impact assessment: Required result assessment after that impact applies on the target
3. Event factory determine the specific type of action, which the given event have.
4. Event listener executes the specific action, which stated at the specific action code. There we have some general action apply to all type of system is present in generic action code and specific to specific action is present in the specific action listener
5. Event listener executes the action and calls the required system to complete it.
6. Some event, which completed their actions, and no further support needed from any other services are terminated
7. Some events which need further processing from some other services or may be generate other event in response of event is send back to the event engine
8. Event in response of event is again run as a new event and again it initiates its processing cycle in the event engine
Time Based Event Handling
Scheduled event delivered to a service at specific time in the future. Event deliveries can be deferred for short periods of time (such as seconds or minutes) or for long stretches of time (for example, hours later for batch processing). Until that delivery time, the Event is essentially invisible until it is delivered, allowing you to schedule work at a particular time in the future.
Heart Beat Event Handling
Heartbeats, a common software construct, verify the continual operation of a specific service. With heartbeats, a targeted service continually broadcasts a signal across its environment. System works normally when client services can detect a targeted service's heartbeat signals
There is number of interdependent services in SOA ecosystem such that, if one failed, others will also fail.
Heartbeat generally propagated across SOA ecosystem using publish/subscribe paradigm.
Transaction Management
Transaction Management is one of the most difficult aspects of any enterprise architecture due to varying level of understanding among stakeholders, various technological platforms involvement and spread of transactions – intra & inter enterprise.
In general any enterprise grade transaction framework follows ACID but due to presence of long running business processes and involvement of asynchronous communication traditional ACID becomes irrelevant. To accommodate long running business processes and asynchronous communication relaxed ACID is used which allows violation of ACID but with capability of correcting the violations in reasonable time period and with little business impact.
SOA capable enterprise grade Transaction Framework should have following features:
· Multi level undo/redo
· Coalescing of transactions
· Automatic aggregation of nested transactions
· Supports batching of transactions (Aggregation)
· Rollback of aggregated transactions for error recovery
· Listener support
· Synchronous and Asynchronous Transaction
· Compensatory Service
· Transaction Propagation
Exception & Error Management
Exception & Error Management is consist of
Exception & Error Handling
Exception & Error Logging
Exception & Error Notification
In any enterprise under SOA ecosystem Exception & Error Management poses following challenges:
Exception & Error Handling
No predefined format for passing exception information among participating services
Web services do not have capability of maintaining stack trace
Lack of common exception vocabulary across enterprise
Exception & Error Logging
Scattered logs across services/applications
Varying formats of logs across services/applications
Toggling logs at run time is service/application dependent
Exception & Error Notification
Every enterprise/LOB has its own unique notification requirement
Toggling Notification at run time is necessity
SLA and exceptions are coupled
To counter these challenges there must be a comprehensive Exception and Error Management framework which can manage at enterprise level and form the foundation for Remediation framework.
On the high level, Exception and Error Management framework’s architecture can be depicted as follows:
Remediation Framework
Remediation framework plays very important role in any SOA ecosystem. It can be though of extension of Error & Exception Management and Transaction Management frameworks. Remediation framework leverages Error & Exception Management Framework by utilizing logged error & exceptions. It also supplement Transaction Management Framework via helping to implement relaxed ACID where human intervention might be needed. Remediation framework also makes whole of SOA ecosystem robust via utilizing compensatory services.
Relationship
In SOA ecosystem Security, Governance, Operations, Quality are closely related. Each face of SOA ecosystem affects other.
Friday, November 21, 2008
Just Started
In this blog I will try to jouney through software architecture mostly independent of tools and technology.
In this blog I will iterate my thoughts on:
1. Enterprise Architecture
2. Enterprise SOA
3. Enterprise BPM
4. EAI - A2A and B2B
5. Application Architecture
6. Application level SOA
7. Application level BPM
In this blog I will iterate my thoughts on:
1. Enterprise Architecture
2. Enterprise SOA
3. Enterprise BPM
4. EAI - A2A and B2B
5. Application Architecture
6. Application level SOA
7. Application level BPM
Subscribe to:
Posts (Atom)
Posts
