AI Spending Visibility Is Broken: How Engineering Leaders Should Build the Financial Infrastructure to Fix It

Most engineering organisations running AI in production cannot tell you, with any precision, what a given workflow actually costs to operate. They can produce an API invoice and a cloud bill, but the connection between those numbers and specific products, teams, or business outcomes is missing. That gap is not a finance problem. It is an instrumentation problem, and the data pipelines required to close it need to be designed at the architecture stage, not retrofitted after the budget review.

Companion piece to our broader work on AI cost measurement. See Token ROI for AI: Breaking the Measurement Loop for why token-based ROI frameworks fail without the underlying attribution infrastructure this article addresses.

Why AI Cost Fragmentation Is an Instrumentation Problem

AI spend arrives through at least three distinct channels: model provider APIs, cloud compute for self-hosted inference and fine-tuning, and the application-layer infrastructure that coordinates them. Each channel produces its own billing artefact, on its own schedule, in its own schema. No single team typically owns all three, which means no single team has the full picture.

The consequence is that cost accountability gets assigned to whoever holds the invoice, rather than to the team generating the usage. A platform team pays the OpenAI bill, an ML team owns the GPU cluster, and a product team ships the feature. When spend increases, the conversation degrades into a negotiation between cost centres rather than an analysis of which workload drove the change.

Solving this requires treating cost data as a first-class telemetry signal, with the same instrumentation discipline applied to latency or error rates. That means deciding, at the architecture level, how usage data will be captured, tagged, attributed, and retained before any model goes into production.

Token-Level Attribution: The Minimum Viable Unit of Measurement

The token is the correct unit of measurement for model API costs because it is the unit the provider bills against. Aggregating to request-level or session-level data loses the granularity needed to distinguish between a well-engineered prompt and a verbose one that costs three times as much.

Every request to a model API should be instrumented to capture prompt token count, completion token count, model identifier, timestamp, and a set of attribution tags that map the request to a product, feature, team, and environment. This is not complex engineering, but it requires a decision to do it consistently before the system is built, because retrofitting attribution tags onto an existing codebase is expensive and error-prone.

The attribution taxonomy matters as much as the data capture. Tags need to be standardised across teams so that aggregations are meaningful. A tag schema that allows each team to define its own conventions produces data that cannot be joined at the platform level, which defeats the purpose of centralised observability.

Building the Log Retention Architecture

Raw usage logs are the audit trail that makes cost attribution defensible. Without them, any cost figure is an estimate derived from billing summaries, and estimates cannot support the kind of root-cause analysis needed when spend spikes unexpectedly.

Retention Tiers and Cost Trade-offs

Log retention should be tiered by query frequency and age. Hot storage, covering the past thirty days, supports real-time dashboards and incident investigation. Warm storage, covering the past twelve months, supports quarterly reviews and trend analysis. Cold storage, covering everything older, satisfies audit requirements without incurring the cost of keeping infrequently accessed data in high-performance systems.

The schema for retained logs should be designed for analytical queries from the start. Storing logs as unstructured JSON in object storage is cheap but slow to query at scale. Columnar formats with partitioning by date, team, and product reduce query costs and latency significantly when the data volume reaches hundreds of millions of records per month.

Cloud Compute Attribution

Cloud compute costs require a different instrumentation approach because the billing unit is the instance-hour or accelerator-hour, not the token. Jobs need to be tagged at submission time with the same attribution schema used for API calls, and those tags need to propagate through the job scheduler into the cloud provider's cost allocation system.

This is where most organisations have the largest gap. GPU cluster usage for training, fine-tuning, and batch inference is frequently untagged or tagged inconsistently, which means it appears in the cloud bill as an undifferentiated block that cannot be attributed to any specific workload or team.

Total Cost of Ownership: What the Invoice Misses

API invoices and cloud bills capture direct spend, but they exclude the engineering time spent on prompt engineering, evaluation, and the failed runs that precede a production deployment. These costs are real, they are substantial, and they are invisible to any system that only reads financial ledgers.

A complete TCO model for an AI workload needs to account for model API costs, cloud compute for inference and training, engineering hours for development and maintenance, and the cost of the evaluation infrastructure used to validate outputs. Organisations that measure only the first two systematically understate the cost of AI, which produces investment decisions based on incomplete data.

The practical implication is that TCO measurement requires integrating cost data from the financial systems with effort data from engineering tooling. That integration is not trivial, but it is the only way to produce a cost figure that can be compared meaningfully against the business outcome the workload is intended to generate.

The Governance Layer: Connecting Spend to Business Outcomes

Cost observability without outcome data is accounting, not governance. The goal is to produce a view that connects a dollar of AI spend to a measurable change in a business metric, whether that is time saved, revenue influenced, or error rate reduced. Without that connection, cost data can identify where money is going but cannot inform whether it should continue going there.

Building that connection requires two things. First, the business outcome metrics need to be instrumented alongside the cost metrics, so that both time series can be joined in analysis. Second, there needs to be an organisational process that reviews the relationship between spend and outcomes on a regular cadence, with the authority to redirect or reduce investment where the relationship is weak.

The governance layer is where engineering leadership and finance need to operate from the same data. That requires a shared reporting layer, typically a cost analytics platform or a purpose-built internal dashboard, that presents cost and outcome data in a format both audiences can interrogate. The engineering team should own the instrumentation that populates it. The business review process should be a joint function.