AI is a challenge of leadership instead of innovation

 

AI doesn’t fail primarily due to lack of ideas or technology. It fails because leaders don’t make the hard decisions AI forces into the open.

Innovation problems are about can we build it?
AI problems are about should we, where, and under what constraints?

That’s a leadership problem.

1. AI collapses the gap between decision and consequence

Traditional innovation lets leaders delegate:

• Engineers build
• Product experiments
• Leaders review outcomes later

AI doesn’t allow that comfort.

• AI executes decisions at scale
• Errors propagate instantly
• “Small” choices become policy

Leadership challenge

• You must decide in advance what decisions are allowed to scale.
• You own failures you didn’t personally approve line-by-line.

 

2. AI exposes organizational contradictions

AI systems force answers to questions leaders often avoid:

• Do we value speed or safety?
• Growth or trust?
• Consistency or discretion?
• Efficiency or employment?

Humans can navigate contradictions informally.
AI cannot.

Result

• Leadership indecision becomes model ambiguity.
• Political compromises turn into technical debt.

 

3. Innovation tolerates ambiguity. AI amplifies it.

Innovation thrives on exploration.
AI systems:

• Act even when uncertain
• Sound confident when wrong
• Hide edge cases until damage occurs

Leadership failure mode

• Treating AI like a prototype instead of an operational actor.
• Confusing model accuracy with decision readiness.

 

4. AI shifts accountability upward, not downward

In classic innovation:

• Failure belongs to the team.
• Leaders sponsor and shield.

In AI:

• Failures trigger legal, ethical, and reputational consequences.
• “The model did it” is not a defense.

Hard truth

You cannot delegate moral agency to software.

That accountability sits with leadership whether acknowledged or not.

 

5. The real bottleneck is not data or models; it’s permission

Most AI programs stall because leaders won’t decide:

• Which workflows can be automated
• Which roles change
• Which risks are acceptable
• When humans must override the system

Teams can build models faster than leaders can grant authority.

 

6. AI forces explicit value tradeoffs

 

Innovation asks: What’s possible?
AI asks: What is acceptable?

Examples:

• Fairness vs profitability
• Transparency vs performance
• Personalization vs privacy

These are normative decisions, not technical ones.

Only leaders can make them and be accountable.

 

7. AI success looks boring, not innovative

Well-led AI:

• Quietly prevents bad decisions
• Stops scaling the wrong things
• Reduces variance, not creativity

Poorly led AI:

• Demos well
• Fails publicly
• Surprises leadership

Innovation celebrates novelty.
Leadership values reliability.

AI rewards the second.

 

The core insight

AI is a mirror. It reflects leadership clarity or the lack of it - at machine speed.

If values, ownership, escalation paths, and risk tolerance are unclear, AI will surface that confusion faster than any other technology.

That’s why organizations with strong leadership but mediocre tech outperform those with brilliant models and weak governance.

 

A simple litmus test for leaders

If a leader cannot clearly answer:

1. What decisions this AI is allowed to make
2. What data it is allowed to use
3. What failure looks like
4. Who shuts it off
5. Who apologizes publicly

They are not leading AI.
They are experimenting with it.

A Small Step for Service Governance

While talking about SOA Governance, one visualizes big fat software and tools which costs millions of dollars and a platoon of support staff to “govern” SOA Governance platform.
In my experience, I noticed that small baby steps always more helpful and governance should be embedded in architecture and design. Instead of SOA Governance, I like it to be service governance first.
Recently, talking to one of my counterpart at my client place, I encountered a classic case of mis governance in services space. Once service is created and deployed, its contract ( wsdl in case of web service) is freely available across enterprise which makes unknowns its customer (sic) without any controlling authority. This uncontrolled distribution and usage of contract leads to nightmares and fights when service performance decreases or new version of service need to be releases and older version to be retired.

How to avoid such dogfight!

Simply create a registry (not UDDI) of service and make sure that this registry contains the information that who is calling whom and authentication has to pass through this registry. I understand this suggestion violets purist form of SOA but in this world nothing is perfect.

Service in eGovernment

Definition:

Service can be defined as system or organization that provides stakeholders with some thing that they need/ask for or and consequence of execution of it.

Stakeholders involved may be people, organizations or systems.

Stakeholders:

1. Client
a. Citizen
b. Business
c. Public Administration Organization
2. Service Organization
a. Service Provider
b. Evidence Verifier
c. Evidence Provider
i. Primary
ii. Auxiliary
d. Consequence (Auxiliary Outputs) Receiver
e. Primary Output Receiver
3. Service Broker
a. Service Repository
b. Interoperability Agent

Service Interaction:

The Client contacts a Service Provider (Public Administration Organization) and asks for a service that requires for its execution input from organizations located in national/states/municipalities across India. The Service Provider, after gathering from these countries all evidences needed for Service execution, executes the Service based on predefined Rules and provides Client with an output (Primary Output) and all organizations with the consequences (Auxiliary Outputs) of the Service.

Preconditions:

A client identifies service organization and ensures that potential service exists. Client provides minimum set of evidences (primary inputs) needed for initiating the service identification. Client may also supply Auxiliary inputs ( in form of evidences and references to evidences) as required by service contract.

Post conditions:

Client receives the Output (primary output) of service execution and participatory organizations receive consequences (Auxiliary Outputs).


Interaction Model:



Description:

Participating Roles

Client
Primary Output Receiver
Service Provider
Broker (Service Repository, Interoperability Agent)
Evidence Provider
Evidence Verifier
Consequence Receiver

Description of Steps

Step 1. A client requests for service. He/she supplies some evidences and some references to evidences. Evidence supplied by client may be categorized as:
1. Evidence required for identification purpose
2. Evidences needed to fetch more evidences from evidence providers
3. Evidences needed to execute service

Less evidences supplied by client to Service Provider, transparent is the service. Of course number and type of evidences required are service specific.

Step 2. Service provider supplies the evidences/credentials of client Broker that in turn passes the same to Evidence Verifier. Evidence Verifier verifies credentials of Client and accordingly supplies the results to Service provider via Broker. Now Client has an Identity.

Step 3. Service Provider looks up for more evidences on the based of identity established for client and references provided by client with the help of Evidence Providers via Broker.

Step 4. Evidences collected from Client, Evidence Suppliers are passed to Evidence Verifiers via Broker.

Step 5. On positive out come of Evidence Verifiers, Service is executed. The outcomes of service are passed to Primary and Auxiliary Output Receivers.

Class Model:

The interaction model can be depicted in class model in two parts

1. The Operational (Transactional) Part
2. The Knowledge (Planning) Part

The Operational Part



The Knowledge Part



Reference:

1. Fowler, M., Analysis Patterns. Object Technology Series, 1997: Addison-Wesley

Book Review: SOA Governance by Todd Biske (ISBN-13: 978-1847195869)

SOA Governance is one of the most vigorously debated and misunderstood subject in SOA world. Because of inherent complexity of governance and SOA, SOA Governance becomes more complex subject.

Book is written is style of novel coupled with some hard technical talk in end of each chapter. This makes reading of book very relaxing though covering complex subject. Book covers governance issue in such a fluid manner that while reading it seems that one is just browsing a story book. Book starts with basic concept of governance and start peeling layers. It covers architectural, design and project management challenges to have good governance model from project as well as enterprise perspective.

Book also take advantage of case study approach and assume very realistic insurance company with all nuisance of IT Management and Architecture noise.

Book is certainly a good read for its target audiences – Architects, IT & Business management folks and certainly those who are interested in SOA Governance.

Go and grab the book at packtpub.com

Enterprise SOA Reference Architecture

Here I propose SOA Enterprise Reference Architecture. This is for enterprise and certainly for an application it should be inspiration.



Reference Architecture is independent of platforms offered by various vendors. It is also possible that none of the vendors in market provide complete solution to realize this reference architecture.

Data Persistent Layer

Data Persistent Layer is primarily consists of Enterprise Data - Master, Transaction and Meta. It may also include Datamart and Data driven reporting infrastructure.

In most of the large enterprises Data Persistent Layer consists of relational (RDBMS), hierarchical (LDAP, IMS), unstructured (Emails, chat sessions, file systems, etc) and any other type. These variations in data characteristics warrant unique data handling and treatment technologies.

Service Providers

This Layer consists of different business applications. These applications may be packaged (e.g. SAP, Reteck, etc) or may be custom developed in various technologies (e.g. Java, .Net, etc). Again application in this layer may be service enabled or legacy once, which do not expose their functionalities as service.

At this layer EAI exist. Some may argue that SOA is one form of integration then how EAI can exist at this layer. I will discuss this argument in one of my future posting but not now.
Services Layer

This layer is consists of multiple sub layers which encapsulate different class of services. It encompasses Connectivity Services, Data Services, Business Activity Services and Business Process Services. Each layer in turn consists of other sub-types of services.

This layer exposes different aspects of business - functional and infrastructural aspects. Connectivity and Data services are consisting of Infrastructural services while Business Activity and Process services expose functional Services.

These services though in layers but still breach layered architecture to expose these services independent of layers and for better performance. These services can also be exposed to multiple ESBs if situation warrants.

1. Connectivity services primarily cater to accessing system level functionalities, any adapter & connector services and partner (B2B) integration services.

2. Data services comprises of services, which caters to Business, Technical and Meta data related services.

3. Business Activity Services consist of Atomic business/functionalities and any custom business logic developed by combing one or more Data & Connectivity services.

4. Business Process Services primarily consist of Process integration, Machine and Human centric services. These services primarily comprise of Business Activity services.

Enterprise Service Bus

ESB provides four functionalities:

Orchestration
Routing
Mediation
Enrichment

Some ESBs may provide one more functionality - Choreography. This functionality is associated with BPM systems and currently none of Commercially available ESB provides the same.

Enrichment may be of two types: Data Transformation and Channel Transformation. Most of the analysts & architects keep Data Transformation & Channel Transformation separate functionality.

Presentation Services

These services support various user interfaces (human or machine centric) and let outer world (intra & inter enterprise) interact with enterprises IT assets. The consumers of presentation services are

Human

• Thin Client

Browser clients: May be based on traditional HTML based or AJAX based

Portable devices: May be based on open standards (WAP) or propitiatory (device or carrier specific)

• Thick Client

Alway Connected: These clients connect to service providers and consume the services in real time. Traditional think client falls in this category

Offline Clients: These clients do not connect to service providers in Always On fashion but an actor can work in offline mode and when ever connectivity available, services consumed. Java WebStart and Adobe AIR are two most widely used technologies in this field.

Voice Based Client

Voice based clients are gaining popularly to reduce load on contact center. To facilitate this type of user interface, IVR plays great role.

Machine

These types of interfaces are not very obvious but very prevent at enterprise level. Application level interaction (Software to software) and interaction with various devices like RFID tags, Bar code readers, etc (Hardware to Software or vice versa) are part of this type of interfaces.

Integration with Outer World

Most of the large enterprises also get integrated with external world. This integration is both ways - information flow is in both directions. To further complicate the scenario, enterprises increasingly using SAAS. This situation requires special attention and tools. In case of integration with business partners B2B Engine plays major role in conjunction with Partner services. SAAS services may be consumed directly or via B2B engine. In case of Human interaction (CRM - Salesforce.com) B2B Engine should be bypassed but for m/c interaction or substantial data transfer (Document management for insurance companies) B2B engine should be engaged.

Security

Security in SOA is one of the biggest concern due to distributed & heterogeneous systems interacting with each other. The security related aspects should be handled at each level of enterprise. In the proposed model security is considered from physical and virtual perspective. It also takes into account security from network and application view.

At monitoring front it take care of Technical - For support staff, Process & Managerial - For business process owners & managers and Executive - for top rung of business leadership requirements.

Governance

Governance is one of the burning issues in SOA environment. To discuss SOA Governance's Technical architecture, it is necessary to understand SOA Governance model.

Following figure depicts the SOA Governance Model.



The picture below intends to depict SOA Governance Technical Architecture.

Service Life Cycle Management

Service Life Cycle Management essentially helps in managing service life cycle from its conception to decommission. This is one of the vital components, which determine the success of SOA initiative in any enterprise and also its long-term viability.

Rule/Policy Repository & Engine

Rule/Policy Repository contains various Business Rules and Governance & Security Policies. The engine part of the same will help in execution of Rules and Policies as and when demanded.

Service Directory

Service directory essentially consists of UDDI, which facilitates consolidated listing of services.

Event Management

To exploit SOA to its full extent event handling plays crucial role. In SOA environment Events can be classified into three categories


1. Generic Event
Event fired/raised in response of event or interaction with other service/s.

2. Time Based Event
Event fired/raised at particular moment of time to perform certain task

3. Heart Beat Event
Event fired/raised by services at regular interval to announce its existence and may be level of throughput it is working.

Generic Event Handling

These are events which fire/raise other events or results due to interaction of services in ecosystem.



1. Event Generator generates events with minimal data set.

2. These events should be formatted as per the event protocol defined in ecosystem, so that theses event are available to entire ecosystem for further processing.

Ø Event Pre-processing: Check that mandatory attributes are present in the incoming events
Ø Event Refinement: Introduce the other system-generated attributes like start time etc.
Ø Situation Refinement: Check the target status on which action would be performed
Ø Impact assessment: Required result assessment after that impact applies on the target

3. Event factory determine the specific type of action, which the given event have.

4. Event listener executes the specific action, which stated at the specific action code. There we have some general action apply to all type of system is present in generic action code and specific to specific action is present in the specific action listener

5. Event listener executes the action and calls the required system to complete it.

6. Some event, which completed their actions, and no further support needed from any other services are terminated

7. Some events which need further processing from some other services or may be generate other event in response of event is send back to the event engine

8. Event in response of event is again run as a new event and again it initiates its processing cycle in the event engine

Time Based Event Handling

Scheduled event delivered to a service at specific time in the future. Event deliveries can be deferred for short periods of time (such as seconds or minutes) or for long stretches of time (for example, hours later for batch processing). Until that delivery time, the Event is essentially invisible until it is delivered, allowing you to schedule work at a particular time in the future.



Heart Beat Event Handling

Heartbeats, a common software construct, verify the continual operation of a specific service. With heartbeats, a targeted service continually broadcasts a signal across its environment. System works normally when client services can detect a targeted service's heartbeat signals

There is number of interdependent services in SOA ecosystem such that, if one failed, others will also fail.

Heartbeat generally propagated across SOA ecosystem using publish/subscribe paradigm.

Transaction Management

Transaction Management is one of the most difficult aspects of any enterprise architecture due to varying level of understanding among stakeholders, various technological platforms involvement and spread of transactions – intra & inter enterprise.

In general any enterprise grade transaction framework follows ACID but due to presence of long running business processes and involvement of asynchronous communication traditional ACID becomes irrelevant. To accommodate long running business processes and asynchronous communication relaxed ACID is used which allows violation of ACID but with capability of correcting the violations in reasonable time period and with little business impact.



SOA capable enterprise grade Transaction Framework should have following features:

· Multi level undo/redo
· Coalescing of transactions
· Automatic aggregation of nested transactions
· Supports batching of transactions (Aggregation)
· Rollback of aggregated transactions for error recovery
· Listener support
· Synchronous and Asynchronous Transaction
· Compensatory Service
· Transaction Propagation

Exception & Error Management

Exception & Error Management is consist of
Exception & Error Handling
Exception & Error Logging
Exception & Error Notification
In any enterprise under SOA ecosystem Exception & Error Management poses following challenges:

Exception & Error Handling

No predefined format for passing exception information among participating services
Web services do not have capability of maintaining stack trace
Lack of common exception vocabulary across enterprise

Exception & Error Logging

Scattered logs across services/applications
Varying formats of logs across services/applications
Toggling logs at run time is service/application dependent

Exception & Error Notification

Every enterprise/LOB has its own unique notification requirement
Toggling Notification at run time is necessity
SLA and exceptions are coupled
To counter these challenges there must be a comprehensive Exception and Error Management framework which can manage at enterprise level and form the foundation for Remediation framework.

On the high level, Exception and Error Management framework’s architecture can be depicted as follows:

Remediation Framework

Remediation framework plays very important role in any SOA ecosystem. It can be though of extension of Error & Exception Management and Transaction Management frameworks. Remediation framework leverages Error & Exception Management Framework by utilizing logged error & exceptions. It also supplement Transaction Management Framework via helping to implement relaxed ACID where human intervention might be needed. Remediation framework also makes whole of SOA ecosystem robust via utilizing compensatory services.

Relationship

In SOA ecosystem Security, Governance, Operations, Quality are closely related. Each face of SOA ecosystem affects other.