Showing posts with label AWS. Show all posts
Showing posts with label AWS. Show all posts

Wednesday, September 16, 2026

The 5 Levels of Running Coding Agents

 

I have classified coding using AI agents into five levels, shifting the focus from simple prompting to the logistical challenges of infrastructure and environment management.

The 5 Levels of Agent Management:

  1. Level 1: The IDE: Starting with a single agent inside editors like Cursor or Copilot. While easy to use, it can lock you into specific proprietary workflows.
  2. Level 2: The Terminal: Running agents as direct processes in the terminal. This offers more control and flexibility by decoupling the agent from a specific IDE.
  3. Level 3: Multi-Agent Management: Scaling up to multiple agents simultaneously. Tools like T-Mux or, more specifically, Herder to manage background sessions and track agent status.
  4. Level 4: Remote Machine: Moving agents off your local machine to a desktop or server using Tailscale and SSH. This allows for GPU-intensive tasks and ensures agents continue running when you close your laptop.
  5. Level 5: The Cloud: Running agents in disposable cloud sandboxes, such as Control Plane. This provides total environmental control, persistence, and the ability to connect agents to internal infrastructure securely.

Level 1

Level 1 represents the starting point for most users, which is running a single AI coding agent directly inside your IDE. Common examples include tools like Cursor or Copilot or Claude Code or Codex integrated into your code editor.

Key characteristics of this level include:

  • Workflow simplicity: You provide a task, the agent modifies files, and you review the resulting "diff" before proceeding.
  • Ease of use: It is a highly effective way to "vibe code" and complete a significant amount of work without needing additional configuration.
  • Constraints: While convenient, the IDE often dictates the agent's workflow, how you interact with it, and what tools are exposed. Additionally, it frequently involves extra subscription costs on top of the base model fees

Level 2

Level 2 shifts the workflow from the IDE to the terminal. By treating the AI agent as a standalone process rather than a plugin, you gain greater control over the environment and avoid proprietary limitations.

Key advantages of this approach include:

  • Flexibility: You can choose your preferred model, specific machine, and custom environment tools without being locked into a single editor's ecosystem.
  • Automation: It becomes significantly easier to script interactions, chain multiple tools together, or move your entire setup via SSH to different machines.
  • Open Ecosystem: This level often relies on open-source tools such as OpenCode or Claude via command-line interfaces which can be more cost-effective than proprietary IDE subscriptions.

Level 3

Level 3 focuses on the challenges of managing multiple AI agents simultaneously. As you scale your projects, you will likely find yourself needing to run several agents at once, which can quickly lead to terminal clutter and difficulty tracking the status of each task.

Key tools for this level include:

  • T-Mux: A traditional terminal multiplexer that allows you to keep multiple sessions alive in the background. It is effective for managing numerous processes through different panes, enabling you to detach and reattach to your workflows easily.
  • Herder: A modern alternative to T-Mux specifically designed for agent management. Herder solves the "mental overhead" of tracking which agents are active, waiting for input, or have completed their tasks, making it a more purpose-built solution for this workflow.

Ultimately, this level is about moving from manually juggling terminal windows to utilizing specialized software that organizes your agent fleet, preventing the chaos of having dozens of processes running without clear oversight.

Level 4

Level 4 focuses on decoupling your coding agents from your local hardware. As you scale, you may run into limitations like needing a more powerful GPU for machine learning or simply wanting your agents to continue working without being tied to your laptop.

Key strategies for this level include:

  • Remote Access via SSH: By setting up your secondary machine (e.g., a desktop) and your laptop on the same Tailscale network, you can remotely control your powerful machine from anywhere.
  • Background Persistence: Using T-Mux or Herder on a remote machine ensures that your agents keep running even if you close your laptop.
  • Simplified Workflows: Using the remote flag in Herder to handle the SSH connection automatically, making it seamless to manage your agent fleet across different physical devices

Level 5

Level 5 moves the entire agent infrastructure away from physical hardware and into disposable cloud environments. This stage focuses on gaining maximum control while eliminating dependencies on your local machine.

Key features of this level include:

  • Environmental Control: Unlike vendor-managed solutions (like Cursor or Claude's built-in cloud agents), using tools like Control Plane allows you to define custom images containing your specific dot files, tools, and preferred agent workflows.
  • Persistence and Flexibility: You can spin up a sandbox in under a minute, hand it tasks to run uninterrupted, and then suspend the sandbox when idle to reduce compute costs to zero, resuming exactly where you left off later.
  • Secure Infrastructure Access: A major advantage over IDE-based cloud agents is the ability to securely connect your sandbox to internal infrastructure (e.g., AWS, GCP, or Azure) using identity management, enabling the agents to interact with your private services without compromising security.


Friday, April 3, 2026

From Infrastructure to Apps: A Deep Dive into the AWS AI Stack

 

Amazon’s AI stack, centered around Amazon Web Services (AWS), is vertically integrated. It spans from raw infrastructure all the way to end-user AI applications. The key idea: AWS doesn’t just give you models; it gives you every layer needed to build, train, deploy, and scale AI systems in production.


 

Let’s break it down cleanly, layer by layer.

Layer 1: Data Layer

Purpose: Enables data collection, storage, processing, and preparation for ML

  • Amazon S3: Scalable object storage for datasets and model artifacts
  • Amazon Redshift: Data warehousing for analytics and ML training
  • AWS Glue: Serverless data integration and ETL (Extract, Transform, Load) service
  • Amazon Kinesis: Real-time data streaming for ML applications
  • AWS Lake Formation: Build, secure, and manage data lakes for ML

Layer 2: Machine Learning Platform Layer (Bedrock)

Purpose: Hosts Amazon’s in-house Foundation models as well as APIs to access partners’ models

·       Amazon Foundation Models: Amazon has Nova family of Foundation models to cater to a variety of needs

o   Nova Pro: Designed for complex, multi-step tasks, offering top-tier performance for reasoning and understanding

o   Nova Lite: An efficient, fast, and cost-effective model suitable for text, image, and video tasks

o   Nova Micro: An extremely fast and lightweight model focused on high-throughput, low-latency text tasks

o   Nova 2 Omni (Preview): A multimodal reasoning model capable of processing text, images, video, and speech, while natively generating text and images

o   Nova Reel: Dedicated to generating short video content, including the ability to take reference images to guide video creation

o   Nova Canvas: Generates images and offers editing capabilities

o   Nova Act: Foundation model that interacts with UIs: clicks buttons, fills forms, navigates apps - ideal for legacy system automation

    • AWS IoT Greengrass: Run ML models locally on edge devices
    • AWS Panorama: Computer vision at the edge
    • AWS Outposts: Run AWS infrastructure on-premises for low-latency ML

Layer 3a: Tools for Builders’ Layer

Purpose: Tools for building custom ML models and workflows

  • SageMaker Studio: Integrated development environment (IDE) for ML
  • SageMaker Autopilot: Automated model building
  • SageMaker JumpStart: Pre-built models and solutions
  • SageMaker Pipelines: Orchestrate ML workflows
  • SageMaker Feature Store: Central repository for ML features
  • SageMaker Clarify: Detect bias and explain model predictions
  • SageMaker Edge Manager: Deploy models to edge devices
  • AWS Deep Learning AMIs: Pre-configured environments for deep learning frameworks (TensorFlow, PyTorch, etc.)
  • AWS Deep Learning Containers: Docker images for deep learning

Layer 3b: AI Application Layer

Purpose: Provide pre-build plug-n-play AI APIs for ML workloads

  • Amazon Rekognition: Image and video analysis (e.g., object detection, facial recognition)
  • Amazon Polly: Text-to-speech service
  • Amazon Lex: Build conversational interfaces (chatbots, voice assistants)
  • Amazon Comprehend: Natural language processing (NLP) for text analysis
  • Amazon Forecast: Time-series forecasting
  • Amazon Personalize: Real-time personalized recommendations
  • Amazon Textract: Extract text and data from documents
  • Amazon Transcribe: Automatic speech recognition
  • Amazon Translate: Language translation
  • Amazon Forecast: Time-series forecasting for demand planning, inventory optimization, etc.
  • Amazon Fraud Detector: Fraud prevention for transaction risk scoring, account takeover detection
  • Q Developer: Assistant for developers, acts as a pair programmer, helping write, debug, and upgrade code (including complex tasks like migrating legacy Java code). Amongst its capabilities include coding suggestions and security scanning.
  • Q Business: Ingests data from 40+ systems like Amazon S3 and Salesforce to help build “Q Apps” for sales, lawyers etc. to perform Q&A to help users get answers to their questions, provide summaries, generate content, and securely complete tasks based on data and information in their enterprise systems.

Users can also use Amazon Q Apps to generate apps in a single step from their conversation with or by describing their requirements.

  • Q in QuickSight: Natural language BI for asking questions about data

 

Which Layer Should You Use?

 

Requirement

Layer

Key Service

I want to build a custom LLM from scratch

Tools for Builders’ Layer

SageMaker

I want to build an AI agent for my app

Machine Learning Platform Layer

Bedrock

I need an AI to help my employees work faster

AI Applications Layer

Amazon Q