The Next Phase of AI is Rewiring the Enterprise Software Stack
The Next Phase of AI is Rewiring the Enterprise Software Stack
The Next Phase of AI is Rewiring the Enterprise Software Stack
In just two years, large language models (LLMs) have evolved from conversational tools into sophisticated agents. This has given rise to autonomous systems capable of reasoning, taking action, and managing complex workflows to boost efficiency. Much like the early days of cloud computing, this shift is now driving a fundamental transformation in business software architecture.
From a Growth Equity perspective, we believe building the trusted guardrails—including development environments, evaluation suites, orchestration frameworks, governance systems, and memory layers—will be key to ensure AI’s autonomous evolution is reliable, safe, and economically compelling.
The Agent Inflection Point
Businesses are seriously engaging with the next phase of AI due to technical advances and organizational readiness:
- Evolved Models: Models are now context-rich decision engines, leveraging longer context windows, reliable tool-use APIs, and multimodal capabilities for reasoning and software interaction.
- LLM-Ready Platforms: Enterprise platforms are 'LLM-ready,' with well-defined APIs across data, event streams, and SaaS, enabling agents to act beyond isolated UIs.
- Encouraging ROI signs: Early signals show faster triage, reduced ticket volume, and cycle-time compression in customer support, IT, and security, driving increased adoption and investment.
A New AI Software Stack for Enterprises
Deploying agents at scale requires new software capabilities not well served by today's cloud, data, security, or developer tooling stacks. As a result, new architecture is taking shape.
Agent Security
- This focuses on controls and safeguards that limit what agents can access or do. This includes permissioning, policy enforcement, abuse prevention, and protection against unsafe or unintended actions.
- Companies include Astrix, Obsidian, RelyanceAi, Strata Identity, SGNL, Invariant Labs, Operant, Clerk, EQTY Lab, Lakera, Promptfoo, Zenity, Semgrep, Virtue AI, Straiker, Descope.
Reinforcement Learning:
- This involves techniques that allow agents or models to improve through feedback and rewards. This typically occurs by learning from repeated interactions with environments rather than fixed examples.
- Companies include Halluminate, Poetiq, Mechanize, Augento, Osmosis, Theta, Judgement Labs, General Reasoning, Adaptive, OpenPipe, Pokee AI, Habitat.
Agent Development Environments & Testing
- This refers to sandboxed environments and testing platforms where agents can be safely developed, simulated, and stress-tested before deployment, using realistic tools, data, and workflows.
- Companies include Daytona, Coder, Warp, Fly.io, Ona, RunLoop, Novita, E2B, Okteto.
Evaluation & Observability
- Systems that monitor, evaluate, and explain agent behavior in production. This includes tracking decisions, tool usage, failures, and overall performance across multi-step workflows.
- Companies involved in this layer include: LangSmith, Weights & Biases, Galileo, AgentOps.ai, COVAL, Langfuse, Patronus AI, braintrust, arize, fiddler, comet, LMArena.
Agent Frameworks & Orchestration
- Tools and frameworks developers use to build, chain, and coordinate agents. This includes defining how agents plan tasks, call tools, interact with other agents, and execute multi-step workflows.
- Companies include: LangChain, Microsoft AutoGen, Crewai, Stack AI, LIamaIndex, n8n, AG2, make, inngest, Hatchet, Temporal, Pydantic AI.
Memory & State
- This includes digital architecture that allows agents to remember past actions, context, and outcomes across steps and sessions. This enables them to maintain continuity and improve decisions over time.
- Companies include: Sid AI, Zep, Cognee, ontotext, Prometheux, ZeroEntropy, Letta, Mem0, MemGPT.
Model Serving & Routing (Inference Software Efficiency)
- Software that runs AI models in production, manages inference requests, routes traffic across multiple models or providers, and optimizes for cost, latency, and reliability.
- Companies include: Together.ai, FireworksAI, baseten, replicate, OpenRouter, vLLMv, Portkey, Ollama, LM Studio, Fal, Hugging Face, LiteLLM. Vellum, prodia.
Core Architecture
Core Architecture encompasses the foundational elements that facilitate AI systems and functions, including compute and chips (GPUs/NPUs), enterprise data platforms, vector databases for retrieval, and the underlying foundation models. Here are some examples within each category:
- Foundation Models : OpenAI, ANTHROP\C, Gemini, Mistral AI, LLaMA by Meta, Qwen, Cohere.
- Embeddings & Retrieval (vector databases): Pinecone, Weaviate, Chroma, turbopuffer, HelixDB, Qdrant.
- Enterprise Data Plane: Snowflake, Databricks, MongoDB, Oracle, Google Big Query, supabase, PostgreSQL, redis.
- Compute : NVIDIA, Intel, AMD, Google, Apple, Samsung, Coral, Qualcomm, hp, BROADCOM, HAILO, Atmel, Imagination.
Where Will Value Accrue?
In our view, "trusted rails" will be key, encompassing development environments, evaluation, orchestration, governance, and memory, to enable reliable, safe, and economically compelling autonomy.
- Orchestration, Memory, and Agent-Native Tools: These are key innovation areas. The complexity of coordinating multi-step workflows drives demand for frameworks offering structured planning, error handling, tool graphs, and multi-agent coordination. Memory systems that maintain context across sessions are also gaining importance as enterprises move beyond stateless LLM interactions.
- Evaluation, Observability, and Safety Layers: These represent new and underserved categories. As agents take actions, there's a critical need for systems to trace reasoning, test behaviors, detect drift, and enforce policies. Traditional Application Performance Monitoring (APM) and Machine Learning Operations (MLOps) tools are not designed for non-deterministic, tool-using agents, creating opportunities for new vendors.
- Model Serving and Routing: This area is becoming crowded and increasingly commoditized. Hyperscalers are rapidly integrating routing, caching, and optimization features. This makes it challenging for independent hosting platforms to differentiate unless they develop strong proprietary telemetry or control-plane intelligence.
- Agent Development Environments: These could become a foundational category. High-fidelity sandboxes that allow agents to safely practice tool-use, run end-to-end simulations, and undergo stress-testing are analogous to the role continuous integration / continuous delivery played in human software development. Though still early, these environments may become essential for enterprise autonomy.
The Future Path Forward
The evolution of generative AI to AI with agentic capabilities is experiencing significant momentum driven by several macro trends. Expanding AI budgets and the natural fit of autonomous AI workloads into existing spending categories contribute to a large and growing Total Addressable Market (TAM). A capital expenditure super-cycle from hyperscalers and model labs is simultaneously lowering inference costs and enhancing model performance. This environment fosters shorter adoption cycles, with some enterprises moving from pilot to production in under a year, and offers clear operational benefits in bounded, high-volume enterprise workflows.
Despite this momentum, the pace of AI's autonomous shift is uneven due to several challenges. High integration complexity, particularly with fragmented data and legacy systems, remains a significant hurdle. Security concerns, such as prompt injection and unintended actions by non-deterministic agents, necessitate robust, and potentially difficult to implement, guardrails. Furthermore, organizational readiness varies widely, with many teams still developing the necessary internal expertise. The risk of rapid commoditization, especially as hyperscalers embed autonomous features into their platforms and / or partner with frontier model providers to deliver an end-to-end software suite, also poses a challenge, potentially leading to uneven deployment cycles even amidst strong underlying investment.
While it's premature to predict which layers will consolidate or which companies will lead, the direction is clear: enterprises are moving towards increasingly autonomous AI workflows. This is supported by an emerging technology stack that looks significantly different from the SaaS and cloud architectures of the last decade. We believe trusted guardrails will be key, including development environments, evaluation suites, orchestration frameworks, governance systems, and memory layers, that ensure autonomy is reliable, safe, and economically compelling, a process that demands close monitoring, adaptation, and vigilance from investors.
If you are building in this environment or exploring related opportunities, the Growth Equity team at Goldman Sachs Alternatives is ready to explore them with you.
