AI Infrastructure I: Deployment

Here is the settling fact before anything else: you are not going to train a model, and nobody is asking you to. The models are trained by a handful of labs at a cost measured in datacentres; what companies actually need — in numbers that dwarf the demand for machine-learning researchers — is someone who can stand a model up inside their estate, behind their identity system, on their network, under their cost controls. That is not a new discipline. It is the last thirty-two classes pointed at a new resource type. The model arrives as a finished artifact, like the database engine in Class Twelve or the container runtime in Class Twenty-Six; nobody expected you to write PostgreSQL either.

Draw the Class One line through an AI workload and it lands in a familiar place. Microsoft owns the model weights, the GPUs, the serving infrastructure, and the uptime of the endpoint. You own everything that makes calling it safe and accountable: which identities may reach it, which network paths exist to it, how much it may spend, what gets logged, and which version answers when your application asks a question. The provider will never take that half — and the provider's half is not what your employer is worried about. Their half of the model is, as ever, the half you are paid to hold.

The model is theirs. The meter is yours.

Strip the branding away and Azure's AI offering is an ordinary Azure resource with an unusually famous tenant.¹ You create an Azure OpenAI resource (the portal now walks you through it under the Microsoft Foundry banner, the latest name for what was Azure AI Foundry) the way you have created everything since Class Seven: it lands in a resource group, in a region, with tags, RBAC, diagnostic settings, and a line on the invoice. Everything you know applies. What you get for it is small and specific:

The resource

The billable, governable container. Region matters twice: data is processed where you put it — the compliance conversation — and model availability differs by region, so the model you want may dictate the region you choose.

An endpoint

One HTTPS URL, shaped like https://<name>.openai.azure.com. A REST API, Class Sixteen's kind — everything that calls the model calls this, and everything you know about securing an endpoint applies to it.

Two API keys

Born with the resource, granting full data-plane access to anyone holding them. §4 is about why the pair exists — and why your goal is to need neither.

Nothing else

No model answers yet. An Azure OpenAI resource with no deployments is a locked, empty room with a famous address — the model only exists for you once you deploy one into it, which is §3.

The distinction in that last row is the one interviews probe, so take it slowly: the resource is not the model. The resource is yours — a governance boundary. The model becomes reachable only through a deployment you create inside it, and the two are separated for the same reason Class Eleven separated the VM from its disks: they have different lifecycles, different limits, and different failure modes.

A model deployment is a named instance of a specific model, at a specific version, with a specific slice of capacity, living inside your resource. Your application never asks for "GPT-4o"; it asks for campux-chat — the deployment name you chose — and you decide, on the infrastructure side, what answers to that name. This indirection is the whole trick, and it is one you have seen twice: it is Class Thirteen's load-balancer frontend and Class Nineteen's branch pointer, reapplied. When the model version retires — and they retire on published schedules, roughly annually — you stand up a new deployment, test it, and repoint. The application never changes.

Deployment

A named, versioned instance of a model inside your resource, with capacity you assign — the level of indirection that lets models change underneath an application that only ever knows a name.

Capacity is where AI infrastructure stops resembling web infrastructure, because the unit is new: the token, roughly three-quarters of a word, counted in both directions — the question and the answer both meter. Quota arrives as TPM, tokens per minute, granted per model, per region, per subscription, and you assign slices of it to deployments; a request-per-minute allowance rides along in proportion.² When a deployment exhausts its TPM in a given minute, callers get HTTP 429 — throttled, Class Sixteen's status codes still doing the work — until the minute rolls over. Two consequences follow. First, TPM is a ceiling, not a purchase: standard deployments bill per token consumed, so an idle deployment with generous TPM costs nearly nothing, while Class Thirty-Two's use-it-or-lose-it warning belongs to the provisioned tier, where you buy dedicated throughput by the hour whether you use it or not. Second, quota is a finite pool: TPM parked on a forgotten dev deployment is TPM production cannot have — the §2 orphan sweep gains a new species.

The resource was born with two API keys, and here is the full measure of what they are: bearer tokens with no name attached. A request presenting the key is served — whoever sent it, from wherever, for whatever. The audit log records that the key was used, never who used it; the Class Thirty-One question "who accessed this in May" is unanswerable by design. Two exist so you can rotate one while the other serves, which tells you Microsoft's own expectation: keys are a thing you rotate, which means a thing that leaks. And an AI key that leaks is a special kind of leak — the finder does not deface your website; they quietly run their chatbot on your invoice, at token prices, until the Class Thirty-Two budget alert is the thing that catches it.

You have climbed this ladder before — it is Class Nine's hygiene ladder, and this is its final rung. The top looks the same as it did there: the calling app's managed identity holds the data-plane role Cognitive Services OpenAI User on the resource, requests carry an Entra token instead of a key, and every call is attributable to a named identity with a revocable role assignment.³ No secret exists to store, leak, or rotate. Then take the step that makes the posture real: the resource accepts a disable local auth setting that turns key access off entirely — the same move as Class Twelve's storage keys, and the difference between "we prefer identity" and "keys do not work here." Flip it, and the §2 keys become inert strings; flip it by policy, per Class Eight, and nobody re-enables them quietly.

"AI experience" appears in job postings the way "cloud experience" appeared in 2015 — demanded everywhere, defined nowhere, and filtered for by people who cannot themselves define it. That vagueness is an opportunity, because it means you get to supply the definition in the interview, and the definition you can now supply is concrete: "I deploy and operate AI infrastructure on Azure" — a resource in the right region for the model, deployments pinned to versions with a tested repointing path, TPM budgeted like the finite pool it is, key auth disabled in favour of managed identity, and the spend visible in cost analysis by tag. Every noun in that sentence is something you can screen-share; most candidates claiming AI experience can screen-share a chat window.

Be equally precise about the boundary, because naming it is what makes the rest credible — the Class Thirty-One move, reapplied. You do not train models, tune models, or evaluate model quality; when asked about prompt engineering, the honest answer is that you build the infrastructure under the people who do. Interviewers are drowning in candidates who blur that line upward. The one who draws it crisply — infrastructure, not models; the meter, not the math — reads as the only person in the stack of resumes who has actually run this in production.