Ask ten teams what their AI stack looks like and you will get ten different tool lists. Look at what those stacks actually do, and the same six layers show up nearly every time.
This is a map of those layers, with an honest note on which ones teams skip — because the skipped layers are remarkably consistent, and so are the consequences.
Layer 1 — Language and fundamentals
Python, REST APIs, Git, cloud basics.
The unglamorous foundation. Almost nobody stalls on a production AI project because they misunderstood transformer architecture; they stall on environment mismatches, leaked keys, and code that exists on one laptop.
The AI-specific wrinkle is that prompts are code — they change behaviour and therefore need version control, review and rollback like anything else.
Layer 2 — Model providers
Anthropic Claude, OpenAI, Google Gemini, open-weight models.
The reasoning engine. The decision that matters is not "which model is best" — it is which model per task, at what cost, under what data-residency constraints. Classification and routing rarely need your most capable model; reserve that for reasoning that justifies the spend.
Treat the provider as a dependency that can change beneath you: pin versions explicitly where possible, and expect that a silent provider update can shift your quality baseline without a single line of your code changing.
Compare the frontier models → · Reasoning models explained →
Layer 3 — Orchestration
LangGraph, CrewAI, LlamaIndex Workflows, Google ADK — and MCP underneath.
The layer that turns a model call into a system: multi-step workflows, tool use, state, retries and human approval gates.
The main trap here is adopting a framework before you need one. A large share of production "agents" are one model call, a tool and a retry — roughly eighty lines of plain Python. MCP matters more than framework choice in the long run, because it makes tool integrations portable across whatever framework you end up on.
Layer 4 — Vector databases and retrieval
pgvector, Pinecone, Qdrant, Weaviate.
How your system reaches knowledge it was not trained on. The usual advice holds: if your team already runs Postgres, pgvector handles the vast majority of real workloads under about ten million vectors, and one fewer specialised system is worth a great deal operationally.
The database is rarely the problem. Chunking strategy and retrieval quality are — which is why roughly 80% of RAG failures trace to ingestion rather than the model.
Embeddings and vector DBs → · Production RAG patterns →
Layer 5 — Evaluation and observability
RAGAS, DeepEval, LangSmith, Braintrust, Langfuse, Arize Phoenix, OpenTelemetry.
This is the layer teams skip, and it is the reason most pilots stall. Without it you cannot prove a change improved anything, cannot explain a regression, and cannot see failures until customers report them.
The two halves answer different questions. Evaluation tells you whether quality moved. Observability tells you why. Keep them connected — a low groundedness score should link straight to the trace that produced it.
Instrument on OpenTelemetry GenAI conventions so your instrumentation is not welded to one vendor. And treat content capture as a deliberate decision, not a default — see what to log and what never to log.
Evaluation → · LLM-as-a-judge → · Observability →
Layer 6 — Deployment and LLMOps
Docker, FastAPI, CI/CD, cost controls, monitoring.
Getting it running reliably, affordably, with a rollback path. The distinctive parts are non-determinism (evals replace exact-match tests), a dependency that changes underneath you, usage-based cost that scales with success, and prompts as deployable artefacts.
The seventh layer nobody draws
Most versions of this diagram stop at six. In any regulated Malaysian context — finance under BNM RMiT, healthcare, anything touching PDPA-covered personal data — there is a seventh that runs alongside all of them: guardrails and security.
Input, dialog, retrieval and output rails constrain what the system can do regardless of what a user types. Prompt injection sits at LLM01 in the OWASP Top 10 for LLM Applications, and OWASP is explicit that fool-proof prevention may not exist — which shifts the work from prevention to blast-radius reduction.
Guardrails → · Prompt injection defence →
Where to start if you are building your first system
- Fundamentals — Python async, Pydantic, Git discipline with prompts as versioned files.
- One model, one task — resist multi-model complexity until something forces it.
- Retrieval if you need it — pgvector first; invest in chunking, not in the database.
- Evaluation before orchestration — fifty real test cases beats an elegant agent graph you cannot measure.
- Observability from day one — retrofitting tracing is far more painful than starting with it.
- Guardrails proportionate to blast radius — an internal tool and a customer-facing agent with write access are different risk objects.
- Deployment discipline — containers, pinned versions, CI gates, cost caps, a tested kill switch.
Nothing on that list is exotic. The teams that get AI into production are usually not the ones with the most sophisticated architecture — they are the ones who did the boring layers properly.
Our AI Engineering and AI Orchestration programmes cover this stack end to end, hands-on — HRD Corp SBL-KHAS claimable for eligible Malaysian employers.
