Reference Architecture is independent of platforms offered by various vendors. It is also possible that none of the vendors in market provide complete solution to realize this reference architecture.
Data Persistent Layer
Data Persistent Layer is primarily consists of Enterprise Data - Master, Transaction and Meta. It may also include Datamart and Data driven reporting infrastructure.
In most of the large enterprises Data Persistent Layer consists of relational (RDBMS), hierarchical (LDAP, IMS), unstructured (Emails, chat sessions, file systems, etc) and any other type. These variations in data characteristics warrant unique data handling and treatment technologies.
Service Providers
This Layer consists of different business applications. These applications may be packaged (e.g. SAP, Reteck, etc) or may be custom developed in various technologies (e.g. Java, .Net, etc). Again application in this layer may be service enabled or legacy once, which do not expose their functionalities as service.
At this layer EAI exist. Some may argue that SOA is one form of integration then how EAI can exist at this layer. I will discuss this argument in one of my future posting but not now.
Services Layer
This layer is consists of multiple sub layers which encapsulate different class of services. It encompasses Connectivity Services, Data Services, Business Activity Services and Business Process Services. Each layer in turn consists of other sub-types of services.
This layer exposes different aspects of business - functional and infrastructural aspects. Connectivity and Data services are consisting of Infrastructural services while Business Activity and Process services expose functional Services.
These services though in layers but still breach layered architecture to expose these services independent of layers and for better performance. These services can also be exposed to multiple ESBs if situation warrants.
1. Connectivity services primarily cater to accessing system level functionalities, any adapter & connector services and partner (B2B) integration services.
2. Data services comprises of services, which caters to Business, Technical and Meta data related services.
3. Business Activity Services consist of Atomic business/functionalities and any custom business logic developed by combing one or more Data & Connectivity services.
4. Business Process Services primarily consist of Process integration, Machine and Human centric services. These services primarily comprise of Business Activity services.
Enterprise Service Bus
ESB provides four functionalities:Orchestration
Routing
Mediation
Enrichment
Some ESBs may provide one more functionality - Choreography. This functionality is associated with BPM systems and currently none of Commercially available ESB provides the same.
Enrichment may be of two types: Data Transformation and Channel Transformation. Most of the analysts & architects keep Data Transformation & Channel Transformation separate functionality.
Presentation Services
These services support various user interfaces (human or machine centric) and let outer world (intra & inter enterprise) interact with enterprises IT assets. The consumers of presentation services areHuman
- Thin Client
Browser clients: May be based on traditional HTML based or AJAX based
Portable devices: May be based on open standards (WAP) or propitiatory (device or carrier specific)
- Thick Client
Alway Connected: These clients connect to service providers and consume the services in real time. Traditional think client falls in this category
Offline Clients: These clients do not connect to service providers in Always On fashion but an actor can work in offline mode and when ever connectivity available, services consumed. Java WebStart and Adobe AIR are two most widely used technologies in this field.
Voice Based Client
Voice based clients are gaining popularly to reduce load on contact center. To facilitate this type of user interface, IVR plays great role.
Machine
These types of interfaces are not very obvious but very prevent at enterprise level. Application level interaction (Software to software) and interaction with various devices like RFID tags, Bar code readers, etc (Hardware to Software or vice versa) are part of this type of interfaces.
Integration with Outer World
Most of the large enterprises also get integrated with external world. This integration is both ways - information flow is in both directions. To further complicate the scenario, enterprises increasingly using SAAS. This situation requires special attention and tools. In case of integration with business partners B2B Engine plays major role in conjunction with Partner services. SAAS services may be consumed directly or via B2B engine. In case of Human interaction (CRM - Salesforce.com) B2B Engine should be bypassed but for m/c interaction or substantial data transfer (Document management for insurance companies) B2B engine should be engaged.
Security
Security in SOA is one of the biggest concern due to distributed & heterogeneous systems interacting with each other. The security related aspects should be handled at each level of enterprise. In the proposed model security is considered from physical and virtual perspective. It also takes into account security from network and application view.
At monitoring front it take care of Technical - For support staff, Process & Managerial - For business process owners & managers and Executive - for top rung of business leadership requirements.
Governance
Governance is one of the burning issues in SOA environment. To discuss SOA Governance's Technical architecture, it is necessary to understand SOA Governance model.
Following figure depicts the SOA Governance Model.
The picture below intends to depict SOA Governance Technical Architecture.
Service Life Cycle Management
Service Life Cycle Management essentially helps in managing service life cycle from its conception to decommission. This is one of the vital components, which determine the success of SOA initiative in any enterprise and also its long-term viability.
Rule/Policy Repository & Engine
Rule/Policy Repository contains various Business Rules and Governance & Security Policies. The engine part of the same will help in execution of Rules and Policies as and when demanded.
Service Directory
Service directory essentially consists of UDDI, which facilitates consolidated listing of services.
Event Management
To exploit SOA to its full extent event handling plays crucial role. In SOA environment Events can be classified into three categories
1. Generic Event
Event fired/raised in response of event or interaction with other service/s.
2. Time Based Event
Event fired/raised at particular moment of time to perform certain task
3. Heart Beat Event
Event fired/raised by services at regular interval to announce its existence and may be level of throughput it is working.
Generic Event Handling
These are events which fire/raise other events or results due to interaction of services in ecosystem.
1. Event Generator generates events with minimal data set.
2. These events should be formatted as per the event protocol defined in ecosystem, so that theses event are available to entire ecosystem for further processing.
Ø Event Pre-processing: Check that mandatory attributes are present in the incoming events
Ø Event Refinement: Introduce the other system-generated attributes like start time etc.
Ø Situation Refinement: Check the target status on which action would be performed
Ø Impact assessment: Required result assessment after that impact applies on the target
3. Event factory determine the specific type of action, which the given event have.
4. Event listener executes the specific action, which stated at the specific action code. There we have some general action apply to all type of system is present in generic action code and specific to specific action is present in the specific action listener
5. Event listener executes the action and calls the required system to complete it.
6. Some event, which completed their actions, and no further support needed from any other services are terminated
7. Some events which need further processing from some other services or may be generate other event in response of event is send back to the event engine
8. Event in response of event is again run as a new event and again it initiates its processing cycle in the event engine
Time Based Event Handling
Scheduled event delivered to a service at specific time in the future. Event deliveries can be deferred for short periods of time (such as seconds or minutes) or for long stretches of time (for example, hours later for batch processing). Until that delivery time, the Event is essentially invisible until it is delivered, allowing you to schedule work at a particular time in the future.
Heart Beat Event Handling
Heartbeats, a common software construct, verify the continual operation of a specific service. With heartbeats, a targeted service continually broadcasts a signal across its environment. System works normally when client services can detect a targeted service's heartbeat signals
There is number of interdependent services in SOA ecosystem such that, if one failed, others will also fail.
Heartbeat generally propagated across SOA ecosystem using publish/subscribe paradigm.
Transaction Management
Transaction Management is one of the most difficult aspects of any enterprise architecture due to varying level of understanding among stakeholders, various technological platforms involvement and spread of transactions – intra & inter enterprise.
In general any enterprise grade transaction framework follows ACID but due to presence of long running business processes and involvement of asynchronous communication traditional ACID becomes irrelevant. To accommodate long running business processes and asynchronous communication relaxed ACID is used which allows violation of ACID but with capability of correcting the violations in reasonable time period and with little business impact.
SOA capable enterprise grade Transaction Framework should have following features:
· Multi level undo/redo
· Coalescing of transactions
· Automatic aggregation of nested transactions
· Supports batching of transactions (Aggregation)
· Rollback of aggregated transactions for error recovery
· Listener support
· Synchronous and Asynchronous Transaction
· Compensatory Service
· Transaction Propagation
Exception & Error Management
Exception & Error Management is consist of
Exception & Error Handling
Exception & Error Logging
Exception & Error Notification
In any enterprise under SOA ecosystem Exception & Error Management poses following challenges:
Exception & Error Handling
No predefined format for passing exception information among participating services
Web services do not have capability of maintaining stack trace
Lack of common exception vocabulary across enterprise
Exception & Error Logging
Scattered logs across services/applications
Varying formats of logs across services/applications
Toggling logs at run time is service/application dependent
Exception & Error Notification
Every enterprise/LOB has its own unique notification requirement
Toggling Notification at run time is necessity
SLA and exceptions are coupled
To counter these challenges there must be a comprehensive Exception and Error Management framework which can manage at enterprise level and form the foundation for Remediation framework.
On the high level, Exception and Error Management framework’s architecture can be depicted as follows:
Remediation Framework
Remediation framework plays very important role in any SOA ecosystem. It can be though of extension of Error & Exception Management and Transaction Management frameworks. Remediation framework leverages Error & Exception Management Framework by utilizing logged error & exceptions. It also supplement Transaction Management Framework via helping to implement relaxed ACID where human intervention might be needed. Remediation framework also makes whole of SOA ecosystem robust via utilizing compensatory services.
Relationship
In SOA ecosystem Security, Governance, Operations, Quality are closely related. Each face of SOA ecosystem affects other.
Posts
This reference architecture looks solid. I particularly like to see the services’ “tiers”. If SOA will continue to achieve it’s mantra of integrating the “Business” into information technology, then we as architects must continue to put an emphasis on service composition and consolidation.
ReplyDeleteWith this reference diagram, you propose four layers of service organization and that is definitely on the right track. If more companies saw SOA as a way to truly expose data, interfaces, systems, functionality, etc as Business Services or Business “Processes” rather, than maybe more companies would start modeling their businesses. All too often we see SOA being reduced to connectivity and data centricity where many companies bind themselves to another IT project of overpriced and under delivering (not to mention complex) software. Shouldn’t the software model the business and not the other way around? Is this not where EAI went fell short in the first place?
Where do you think SOA purports to diverge from EAI? Do you really believe EAI was only relegated to the service providers? Did not EAI birth the genesis of transaction management as it relates to event handling?
Again, good job highlighting operational efficiency in your reference here. Rollbacks, compensating transactions, exception / error management and of course a true remediation framework are the best ways to achieve the enterprise strength, capable of the availability nines desperately being sought by IT today. Some may argue the future of SOA is now two-fold – distancing itself from connectivity and aligning with practical IT value like high-availability and governance solutions.
My hope – that SOA can finally transcend IT all-together and be seen as the architect’s holy grail to the business need. Representing not another software package or initiative, but rather the missing link between all the things IT likes to talk about (personalization, virtualization, CEP, BPM, BAM, Rapid Development, Agility, etc…) and it really happening. When the business stops asking is this even possible and starts requesting functionality without fear of complexity, dependencies, timelines and cost – the answer will be SOA.
Shawn P. Simon – Enterprise Architect - Oshyn, Inc. - Powering Innovation ™
www.Oshyn.com
Follow Oshyn on... Twitter & LinkedIn
Thanks for such detailed comments.
ReplyDeleteI see EAI as sub set of SOA. Cetainly SOA can achive what ever EAI can do - integrating business application but SOA goes beyond it. SOA also proposes composable services/processes which can realize applications really fast.
In one of another post I have tried to visulaize evolution of SOA which clearly indicate that SOA must be business driven rather than IT driven. To achive this paradigm enterprises has to focus on BPM/EPM which will bring concept of process ownership in enterprise structure/hierarchy.
As business grow complex and bigger enterprises naturally bring in Event driven architecture, CEP, EPM/BPM, autonomous services etc.
In general this article describe a layered architectural style in each layer exist components the interact in a request-reply fashion with components in other layers, e.g., process services invoke composite services, composite services invoke atomic services, and so on.
ReplyDeleteif loosely coupling is the key characteristic of SOA, This article only considers decoupling in design time (consumers decoupled from the service implementations), but what about runtime coupling?
In runtime the services are tightly couple, so if a service is unavailable the dependent services stop to work, in Service orientation a service must be work although other service are down, e.g., the sales people can continue processing order regardless billing people or shipping people.
Additionally, this article considers sequential business process, when in the real life the process in a organization are more complex and cannot be realized with with sequential and procedural paradigms. take in account of this...
Labels: Enterprise SOA Reference Architecture Barcode Reader