Building an AI Ingestion Pipeline on Azure: From Prototype to Production
A 130-product catalog silently truncated GPT-4o's output every time — not because the model couldn't handle it, but because of a deployment-level completion-token cap that had nothing to do with model capability. The real fix, and the two-pass schema-discovery design that's next
Vendor product catalogs are one of the most boring documents in existence, and that's exactly why they're hard to automate. Every vendor exports their catalog differently — a PDF with three merged header rows, a spreadsheet where "SKU" is sometimes "Item #" and sometimes "Product Code," a table that silently switches units of measure halfway down the page. ProfitForge (internally, Vendor Vault OCR) exists to turn that mess into one clean, standardized product table, automatically. The pipeline is simple to describe — OCR extracts the raw text and tables, GPT-4o maps the columns and pulls out structured products, a human approves before anything hits production — and, like most "simple" AI pipelines, the interesting engineering was entirely in the part that description skips over.
The pitch and the first wall
The three-stage shape — Document Intelligence for OCR, GPT-4o for mapping, a manual review gate before export — worked immediately on small catalogs. Upload a ten-page PDF, get back clean JSON, done. Then a real 130-product catalog came through and the AI mapping step failed, silently, every single time: the model would truncate mid-JSON and hand back a response that didn't parse.
The instinct is to blame the model's intelligence — "GPT-4o can't handle a big catalog." That's not what was happening, and figuring out what actually was happening is the part worth writing down.
The actual constraint: two different limits wearing one name
Azure OpenAI deployments have two separate token limits, and it's easy to conflate them because both get called "the token limit" in casual conversation:
- Context window — the combined size of the prompt plus the response. GPT-4o's is 128K tokens. Even the largest catalogs here, at roughly 30K input tokens once OCR output is serialized to JSON, were nowhere near that ceiling.
- Completion tokens — how much the model is allowed to generate in one response. This is a deployment-level setting configured in Azure AI Foundry, independent of the model's own architecture.
The gpt-4o-2024-05-13 deployment had a hard completion cap of 4,096 tokens. A 130-product catalog needs roughly 9,000 output tokens to describe. The model wasn't failing to understand the catalog — it was being cut off mid-sentence by a config value that had nothing to do with model capability, on every single call past a fairly small product count.
The fix for the immediate problem was almost anticlimactic once the real constraint was visible: upgrade the Pulumi-managed deployment to gpt-4o-2024-08-06, which supports a 16,384-token completion cap. That alone raised the safe ceiling to roughly 200 products per call — enough for the catalogs actually coming through at the time.