Large Language Models have rapidly become ubiquitous tools for problem-solving, but not all problems are created equal when it comes to LLM performance. Understanding where these systems shine and where they stumble requires thinking beyond simple categorizations of "easy" versus "hard." The reality is far more nuanced.
When analyzing a problem to decide use of LLM or traditional Software architecture approach, I test it with two distinct frameworks.
First Three-Dimensional framework and second one is DIKW framework.
The current post is focusing on Three-Dimensional framework.
The X-Axis: Open-Endedness
This axis measures whether a problem has a definitive answer or requires creative exploration. On one end, we have closed problems with objectively verifiable solutions—legal precedents, mathematical proofs, documented historical facts, or established scientific findings. On the other end lie open-ended challenges: brainstorming marketing campaigns, exploring philosophical questions, generating creative narratives, or proposing novel research directions.
The Y-Axis: Context-Specificity
Context-specificity reflects how much specialized domain knowledge a problem demands. Low context-specificity problems draw on general knowledge and common reasoning—explaining everyday concepts, writing casual correspondence, or summarizing publicly accessible information. High context-specificity problems require deep expertise—interpreting medical imaging, applying niche legal frameworks, debugging domain-specific codebases, or navigating proprietary organizational processes.
The Z-Axis: Complexity
Complexity encompasses the volume and interconnectedness of information required. Simple problems involve few variables and straightforward relationships. Complex problems require synthesizing vast amounts of information, tracking multiple dependencies, managing intricate cause-and-effect chains, or integrating knowledge across disparate domains.
Where LLMs Excel
The Sweet Spot: Low Context-Specificity + Variable Open-Endedness + Low to Moderate Complexity
LLMs perform remarkably well when working with problems that draw on broad, publicly available knowledge. They excel at:
- Synthesis and explanation: Taking complex general-knowledge topics and making them accessible
- Creative exploration within known patterns: Generating marketing copy, writing stories, or brainstorming ideas based on established conventions
- Low-stakes closed problems: Answering factual questions, providing definitions, or explaining well-documented concepts
- Pattern-based tasks: Code generation for common patterns, translating languages, or reformatting content
The key advantage is their training on massive diverse datasets. When a problem falls within the distribution of their training data—even if complex—they can leverage statistical patterns to produce impressive results.
Where LLMs Struggle
The Danger Zones
High Context-Specificity + Closed Problems = Hidden Failure Mode
This is perhaps the most dangerous territory. When problems require specialized expertise AND have objectively correct answers, LLMs often fail silently. They'll confidently generate plausible-sounding responses that may be subtly or catastrophically wrong. Examples include:
- Niche legal interpretations
- Specialized medical diagnoses
- Domain-specific technical debugging
- Proprietary system configurations
The problem isn't just that they get things wrong—it's that the output sounds authoritative enough to be trusted.
High Complexity + High Context-Specificity
Even when problems are open-ended, combining deep expertise requirements with vast interconnected information creates significant challenges. LLMs may:
- Miss critical domain-specific nuances
- Fail to recognize when specialized knowledge contradicts general patterns
- Struggle with long-chain reasoning across specialized domains
- Generate solutions that ignore important constraints only visible to experts
Pure Factual Accuracy on Closed Problems
Paradoxically, even seemingly simple closed problems can trip up LLMs when they require:
- Information published after training cutoffs
- Precise numerical data or statistics
- Current event details
- Specific citations or attributions
LLMs are not databases—they're pattern predictors. They can hallucinate facts with alarming confidence.
Real-Time or Hyper-Specific Data Needs
Though this class of problems do not fit into the framework but still it is worthwhile to mention that LLMs falter with real-time or hyper-localized data, like predicting today’s stock market or analyzing a company’s internal data, unless paired with external tools.
Complexity as the Hidden Multiplier
As complexity (Z-axis) increases — with more entangled variables, dependencies, or cross-domain relationships — LLMs’ effectiveness declines sharply unless supplemented with structured reasoning tools, external databases, or human orchestration.
High complexity problems often require models of the world, not just patterns of language.
The Practical Implications
When to Trust LLMs
- First-draft generation for well-understood problem types
- Brainstorming and exploration where multiple perspectives are valuable
- Explaining established concepts to different audiences
- Transforming or reformatting existing content
- Accelerating routine tasks with clear patterns
When to Exercise Caution
- Any high-stakes decision requiring specialized expertise
- Problems with verifiable correct answers that you can't easily check
- Situations where subtle errors have serious consequences
- Domains where training data may be sparse or biased
The Verification Imperative
The most effective use of LLMs involves treating them as powerful assistants rather than autonomous experts. The human role becomes:
- Framing problems in ways that play to LLM strengths
- Providing domain context that may not exist in training data
- Verifying outputs against ground truth, especially for closed problems
- Iterating and refining based on expert judgment
Looking Forward
As LLMs evolve, we're seeing movement along all three axes:
- Retrieval-augmented generation addresses context-specificity by grounding responses in specific documents
- Improved reasoning capabilities help with complexity management
- Longer context windows enable handling more intricate problems
- Multimodal models expand the types of problems LLMs can address
Yet the fundamental framework remains valuable. Understanding where a problem sits in this three-dimensional space helps us predict where LLMs will excel and where human expertise remains irreplaceable.
Conclusion
LLMs are transformative tools, but they're not uniformly capable across all problem types. By thinking critically about open-endedness, context-specificity, and complexity, we can deploy these systems where they add the most value while maintaining healthy skepticism in domains where their limitations pose risks.
The future isn't about replacing human expertise—it's about understanding precisely where human-AI collaboration creates the most powerful outcomes.
Posts
