During the World Cup this year, I built Rune Score, a sports analytics chatbot. It appears simple on the surface, but internally it runs a distributed system that must manage up‑to‑date sports data, user rephrasing, extraction failures, LLM inference costs, streaming, memory management, fallbacks, and security. The lessons from this project are applicable to virtually any AI product that must remain fast, cost‑effective, and accurate.
Before proceeding, here are concise definitions, as both tools are not widely known. Redis is an open‑source, in‑memory data structure store that functions as a database, cache, message broker, and streaming engine. In AI applications it serves as a fast cache, storing a precise answer associated with an exact key. Retrieval‑augmented generation (RAG) is an AWS‑described technique that augments a large language model with external data sources, such as internal documents. Rather than matching an exact label, RAG queries a broader knowledge base using semantic meaning instead of exact wording. Redis offers precise, inexpensive storage but is inflexible, whereas RAG provides greater flexibility and durability at higher operational cost.
Key Takeaways
- It’s often wiser to begin with a simple cache, which is inexpensive and fast, and reveals user needs before adding complexity.
- A basic cache’s limitation is that it matches only exact phrasing; any rewording can cause it to miss pre‑existing answers.
- A more sophisticated memory system incurs higher runtime costs and slight latency, but it comprehends diverse phrasing, retains information longer, and can be reused across projects.
While the World Cup was underway this year, I developed Rune Score, a sports analytics chatbot. It appears simple on the surface, but internally it runs a distributed system that must manage up‑to‑date sports data, user rephrasing, extraction failures, LLM inference costs, streaming, memory management, fallbacks, and security. The lessons from this project are applicable to virtually any AI product that must remain fast, cost‑effective, and accurate.
Two quick definitions before proceeding, since neither tool is widely known. Redis is an open‑source, in‑memory data structure store that functions as a database, cache, message broker, and streaming engine. In AI applications it serves as a fast cache, storing a precise answer associated with an exact key. Retrieval‑augmented generation (RAG) is an AWS‑described technique that augments a large language model with external data sources, such as internal documents. Rather than matching an exact label, RAG queries a broader knowledge base using semantic meaning instead of exact wording. Redis offers precise, inexpensive storage but is inflexible, whereas RAG provides greater flexibility and durability at higher operational cost.
Why I didn’t start with RAG
I began not with a vector database but with a Redis cache that used a straightforward labeling rule—identical requests received identical keys, such as “Argentina vs. Spain.” The impact was immediate. In my LinkedIn article on Rune Score, I noted that without the cache, a fresh extraction followed by analysis required up to 23,000 input tokens (including scraped content, context, and history). When the cache hit the same key, token usage fell to roughly 5,000, representing a 4.5‑fold reduction in the most costly pipeline component.
AI costs typically stem from unnecessary repetitions. If ten users ask the same question within a short period, executing the full, expensive pipeline ten times wastes money with no benefit.
Where Redis breaks down
Users rarely phrase queries identically. A request such as “FC Barcelona vs Real Madrid” or “Barça – Madrid” would be treated as distinct keys by a strict cache, causing duplicate processing, missed opportunities to reuse existing answers, and unnecessary reruns of the costly pipeline simply due to wording differences.
The part that actually breaks quality
A key lesson applies to any system: the primary quality bottleneck in a RAG pipeline is not the retrieval technology itself, but the input it receives. As TechTarget’s definition of “garbage in, garbage out” (GIGO) illustrates, output quality is dictated by input quality. Whenever I reduced the amount of content extracted from a source, the downstream analysis suffered visibly.
Redis’s engineering blog notes that “the consistent takeaway across these studies is that combining retrievers tends to surface more relevant results at the top positions than either method alone.” Merging exact keyword matches, semantic similarity scores, and recency signals into a unified ranking yields better top‑position relevance than depending on any single signal.
RAG costs more, and that’s fine
In my Rune Score experiment, RAG reduced token usage from 20,000 to 8,000—a 60% drop. While this is less cost‑effective than the 80% reduction achieved by an exact Redis hit and runs slightly slower, the difference is not a drawback. Instead, RAG offers tolerance for varied phrasing, persistent memory beyond cache expiration, and a reusable architecture that extends beyond a single product.
The bottom line
Redis and RAG are not opposing options; they are sequential. The optimal refactor is not the theoretically perfect one but the one that enables rapid rollback if real‑world metrics fall short. Start with a cheap, fast cache to learn user queries, then add a full retrieval system only where the cache alone cannot suffice.

