calmrocks/ai-engineer-notebooks: Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API. · GitHub

Learn the applied-LLM stack the way you’ll actually be interviewed on it —
framework-free, on a free API, from prompting all the way to serving, fine-tuning,
and a red-team benchmark.

Runnable Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE)
skill set: building working systems on top of foundation models — model APIs, RAG,
evals, agents, adaptation, serving — using raw APIs, not frameworks.

• Framework-free, on purpose. You write the agent loop, RAG, and evals from
  raw API calls first — so you understand what LangChain/LlamaIndex actually do
  before you reach for them (and can judge when not to). Patterns are durable;
  wrappers churn.
• Evals are the spine. “Measure before you tune” is installed early and
  returns in every section — the habit that separates an engineer who shipped a
  system from one who built a demo.
• Free to run, end to end. Everything runs on the free
  Groq API (no credit card). The two topics Groq
  can’t host — LoRA fine-tuning (06) and self-hosted serving (09) — are
  concept-first with optional, fenced Colab-GPU appendices, verified on a real
  Colab T4.
• Real case studies, not toy demos. Three end-to-end
  case studies show the skills combined under
  real constraints — a support assistant debugged in production, a
  pipeline-vs-agent cost showdown, and a red-team robustness benchmark.
• OpenAI-compatible throughout, so every pattern transfers directly to OpenAI
  and (with small changes) Anthropic — the seam is swappable, the skills aren’t.

Built as the hands-on companion to
Plan: Transitioning to Forward Deployed Engineer / AI Engineer.
The plan explains what to learn and why; these notebooks are where you run it.

Backend or full-stack engineers moving into AI Engineer, FDE, Applied AI,
or Solutions Engineer (AI) roles — different titles, largely the same job.
You can ship production code; you want the applied-model layer on top.

Work top to bottom. Each notebook is self-contained (installs its own
dependencies, reads API keys from Colab secrets) and ends with exercises.

Notebook	What you’ll learn
Measuring outputs	Golden sets and metrics on the section-01 task — install the “measure before you tune” habit before building anything you’d need to tune. Evals is the spine; it returns in every section after this

Notebook	What you’ll learn
Golden sets	Building a golden set for the RAG system from section 03
LLM as judge	Judge prompts, agreement with humans, and the judge’s own failure modes
Regression evals	Evals as CI: catching quality regressions when you change a prompt or model

Notebook	What you’ll learn
Fine-tune vs RAG vs prompt	When to change the model’s weights vs its inputs; what LoRA/QLoRA are and cost; the argument you’ll have in the room — plus an optional real LoRA fine-tune on a free GPU

Notebook	What you’ll learn
Prompt injection & the trust boundary	Direct & indirect prompt injection, output handling, PII, excessive agency — the OWASP LLM Top 10 risks, failing live then defended

Notebook	What you’ll learn
Observability & LLMOps	Tracing every call, safe prompt logging, cost/latency/error metrics, drift detection, and the observe→eval feedback loop
Reliability & fallbacks	Retries with backoff, timeouts, fallback models, output validation, circuit breakers, graceful degradation
Experiment tracking & registry	MLflow end to end: log runs/params/metrics from the section-04 eval harness, register and version a model, and promote by stage — the tooling that turns “I ran an eval” into a tracked, reproducible workflow

Where the free Groq API can’t run the topic (these frameworks need a GPU),
the notebook teaches it concept-first and fences an optional Colab-GPU
appendix — the same pattern as the section-06 LoRA appendix.

Notebook	What you’ll learn
Serving frameworks	The serving stack an AI engineer actually picks between — vLLM, TGI, Triton, TensorRT-LLM — what each optimizes, how they map onto the raw API you’ve been calling, and when to reach for which
Inference performance	The levers behind throughput and latency: continuous batching, the KV cache, quantization, and the throughput-vs-latency trade — with the napkin math to size a deployment

Notebook	What you’ll learn
Designing an inference service	Concept: the ML system design interview, worked end to end — QPS/VRAM/latency/cost estimation, replica scaling, queueing, caching, and the SLA trade-offs, on a realistic LLM-serving prompt

Notebook	What you’ll learn
Scoping & discovery	Turn a vague customer ask into a scoped, evaluable system: discovery questions, a one-page scoping doc, the demo discipline — the customer-scenario interview round most engineers can’t evidence

Where the skills come together into projects. First a case study — one
realistic scenario worked end to end, runnable — then the capstone, the
deployed repo you build yourself. (Section overview.)

Notebook	What you’ll learn
Case study A — Customer-support assistant	One scenario scoped → built → served → debugged in production: a vague ask becomes a deployed, evaluated RAG+agent assistant, then a live quality regression (a stale index after a corpus migration) that you diagnose and fix. A build-to-debug arc threading sections 02–11
Case study B — Contract extraction: pipeline vs agent	The judgment call interviewers love: build the same extraction task as both an agent and a pipeline, then prove with accuracy + token cost that the pipeline wins when the steps are known
Case study C — Red-team robustness benchmark	A different kind of system — a harness that evaluates a model instead of serving one: an attacker→target→judge (PAIR) loop that measures attack success rate, composing the agent loop, LLM-judge, security, and evals

Capstone: the brief for the deployed
project that goes on your resume — a real repo with a serving component and an
eval report. Case studies are for learning; the capstone is for hiring.

• Raw model APIs, no frameworks. Patterns are durable; wrappers churn.
• One shared corpus (data/) across RAG and eval sections, so evals
  measure the retrieval you actually built.
• Self-contained notebooks. First cell installs, second cell calls
  from aien import setup; client, MODEL = setup() to load your key from
  Colab secrets (or a local env var). No hidden state between notebooks.
  aien is the tiny shared-setup package in this repo — one place to change
  credential loading — installed automatically by the first cell.
• Every notebook ends with exercises — do them before moving on.

1. Get a free API key at console.groq.com — no
   credit card required.
2. In Colab: the key icon in the left sidebar → add GROQ_API_KEY as a secret,
   and toggle notebook access on.
3. Open any notebook via its badge and run top to bottom.

Running locally instead: pip install -r requirements.txt && pip install -e .
(the second installs the aien setup helper), export GROQ_API_KEY=...,
open with Jupyter.