The tempting comparison
Five requests becoming two is a clear improvement in request count. But a workflow does not always follow its successful path. Invalid structured output or a failed grouped response can trigger extra work.
The Paperless-GPT example
The grouped path keeps the title request, tries one combined metadata request, and returns to the four individual requests if grouping fails. A failed group followed by successful fallback therefore totals six calls, compared with five in the original path.
The useful calculation
If grouping succeeds with probability p, the simplified expected call count is 2p + 6(1 − p), or 6 − 4p. It falls below five when success exceeds 25%. That is only a request-count threshold: requests differ in length and price, and the calculation assumes the fallback succeeds without further retries.
What to track
Record first-attempt success, fallback frequency, total usage, final acceptance, and time to an accepted result. A reliable fallback is valuable. Its cost belongs in the comparison, alongside the user disruption it prevents.
About the evidence
Fallback behavior checked locally. Formula is illustrative, not a billed-cost result.
Project detail and source links ↗