When graphs lose

Somewhere on this site there is a scoreboard. It grades both paradigms on the same generated questions. No language model does the judging. It runs deterministic (plain, repeatable) checks against what is actually in the data. I built it. I like it.

The docs describe it in four words. The "when graphs win" scoreboard.

Stop and think about that. A scoreboard that can only print one result is not a real evaluation. It is a demo pretending to be one.

So here is a very plain question. What is Acme's contract end date?

A team had put their whole retrieval stack behind GraphRAG. They chose it for reasons that sound a lot like the last twenty pages you just read. The date came back correct. But it also came back slower, at roughly forty times the tokens, and no more correct than a plain vector lookup would have managed in one shot. The fact sat in one clause of one document. There was nothing to traverse. They had built a tool for walking through links, then handed it a question that needs no walking at all.

Retrieval has regimes

A 2026 paper accepted at ICLR asked exactly this. Then it did the slow, useful work. It ran both kinds of retrieval across several kinds of question and published all of it, even the results that did not fit a neat story.

The authors' own finding is not that GraphRAG is bad. It is that basic retrieval is as good as or better than GraphRAG on simple fact retrieval that needs no complex reasoning. And GraphRAG shows a clear advantage on complex reasoning, contextual summarization, and creative generation.

Now the numbers. Fact retrieval: basic RAG 60.92%, with the graph methods landing anywhere between 36.92% and 60.14%. Read the top of that range, not the bottom. The top is the fair comparison, and there the best graph method only draws level. Now change the kind of question and watch. Complex reasoning: 42.93% against 53.38% for HippoRAG2. Contextual summarization: 51.30% against 64.40%. Creative generation: 38.26% against 70.85% for RAPTOR. Same systems, same document sets, four different questions. The winner flips because the question changed and nothing else did.

Now the correction, before this turns into its own slogan.

You will see "GraphRAG frequently underperforms" quoted around. It is a real sentence in real papers. But in those papers it is the opening summary of earlier work, not their own result. Their own result is the flip.

And "graphs only win on complex reasoning" is too neat as well. On one medical benchmark a graph method beat basic retrieval at plain fact retrieval, 66.28% against 64.73%. Not by much. But enough to break the rule you were about to write down. A separate systematic evaluation, revised in March 2026, said it as plainly as a paper can: the two exhibit complementary behaviors rather than a consistent winner. On single-hop questions plain retrieval won, 64.78% to 60.04%. No one gets a simple slogan out of this research. What you get instead is a rule that depends on the question and on the document set.

Building the structure up front is a bet

A graph does not show up on its own. Somebody builds it. That build is the second bill.

That same systematic evaluation timed both. Indexing the documents for plain retrieval: 135 seconds. Building the knowledge graph over the same documents: 5,560 to 7,702 seconds. Call it about an hour and a half against about two minutes. And you pay it before the first question even shows up.

Then you pay again, on every query, forever. Microsoft's own GraphRAG running in global mode has been measured at up to 331,375 tokens for a single query, against roughly 879 for a plain lookup. That is not a small extra cost. It is a whole different scale, every time a person types.

So building the structure up front is a bet, and the bet is simple to state. You are betting that enough of the questions coming at you will need the structure to make it worth building. Win that bet and the graph pays for itself: you paid the hour and a half once, and questions that could not be answered before are now routine. Lose it and you have built an expensive map of links nobody walks, and you are paying forty times over to answer questions a plain vector lookup already had covered.

Finding more also means finding more junk

There is a quieter finding in the ICLR work, and it is the one that explains the Acme contract. Graph retrieval scored higher on recall and lower on relevance than plain retrieval.

Both, at the same time. It found more of the right material and it pulled in more of the wrong material with it. This makes sense once you say it out loud. A walk does not stop at the answer. It follows edges. Edges lead to neighbors, and those neighbors have neighbors of their own. Ask about a contract and the traversal happily brings you the customer, their orders, their shipments, and their other three contracts, because all of it is connected. "Everything is connected" is the graph's whole promise. Here it is the graph's whole problem.

That is where the forty times comes from, by the way. Not from the walk being slow. From the walk bringing back too much. It fills the prompt with everything next to the answer, and then the model has to find the answer inside the pile you handed it. More context is not more grounding. Past a point it is just a bigger stack to search, and you paid to build it.

The scoreboard has two axes

So put both axes up properly. Accuracy on one. Cost on the other, and cost means all of it: build time, query tokens, and the ongoing work of keeping the graph true.

Here is the part that took me longest to accept. The cost axis has a consistent winner, and it is not the graph. It never is. The accuracy axis has no consistent winner at all. It flips with the kind of question, and it shifts with the document set. That gap is the honest summary of every number on this page. You always know what the graph costs. You do not always know what it buys.

Which of these criticisms actually applies here

Before the mapping, the honest part. It works both ways, and roughly half of this page does not apply to us.

Look at how those benchmarks were built. They test GraphRAG over unstructured text, with the knowledge graph extracted by a language model reading that text. That is where the worst number in the systematic evaluation comes from: only about 65.5% to 65.8% of the answer entities were present in the built graphs at all. A third of the answers were not retrieved badly. They were simply not there. The extractor read the documents and missed them, and no traversal can recover what was never built.

That failure does not reach us. The lab you are working through, and Fabric IQ underneath it, do not ask a model to guess a graph out of text. The graph is built from the tables. A foreign key match is a fact about the data, not a guess about it. Run it twice, get the same graph twice. A key match is not a guess. So when someone waves that 65% at an ontology built on relational data, they are quoting a measure of an LLM extractor's recall and calling it a fact about graphs. It is not.

Now the part that does reach us, and I would rather say it than have you find it later. The question-dependence carries over completely: a fixed, rule-based graph is still the wrong tool for "what is this customer's contract end date." Prompt inflation carries over too: a traversal over real foreign keys is just as happy to hand the model the whole neighborhood as one over extracted text. And the build-and-carry cost carries over, in a different form but really. You are not paying two hours of LLM extraction. You are paying modeling time, binding, and the ongoing work of keeping an ontology true while your data keeps changing. Cheaper. Not free.

Being clear about which criticism actually applies beats either side's marketing. A third set of 2026 work makes the same point from the other direction. It finds that existing GraphRAG evaluations carry unrelated questions and evaluation biases, and that under a fair framework the reported gains shrink a lot. So hold the pro-graph numbers to that test too. The strongest pro-GraphRAG results going around come from a vendor blog that used a language model as its judge with no ground truth, which is exactly the method that bias work flags as inflating results. That is not evidence. It is a marketing piece with a chart in it. Our own scoreboard grades deterministically against the data, with fixed rules, for exactly this reason, and it had better be able to print a loss.

The graph is one source among several

Here is the shape of the choice in Fabric, and it is a real choice, not a formality.

A data agent, Fabric's name for an AI agent grounded in your data, can be pointed at several kinds of source. A lakehouse or warehouse over SQL. An eventhouse, for streaming and time series data. A Power BI semantic model, for the ready-made totals. And the ontology and its knowledge graph, for the traversals. Attaching the graph is one option among those. It is not the default, and it is not always the best.

So the design choice is per question, not per platform. If most of your volume is "what is this customer's contract end date," that is a lookup, and a data agent over the semantic model or straight SQL will answer it faster and cheaper than any walk. Send the traversals to the graph and send the totals somewhere else. That is the same split the semantic model concept drew, now with a price for getting it wrong.

Honest status as of July 2026: the graph engine itself is generally available. The ontology item is in preview. Pointing a data agent at a graph or an ontology is in preview. The tooling is still changing. The trade-off is not.

The one trap

The trap is reading this page as permission to skip the graph.

Twenty concepts of graphs winning, then one page of graphs losing, and it is tempting to treat the newest thing as the true thing. Don't. Nothing here says skip the graph. It says that "which customers does this breach put at risk" still cannot be answered without one, at any budget, on any model. No amount of cheap vector retrieval will build a path that exists in no single chunk. The graph is not overrated. It is sent to the wrong place, again and again, by people who bought it for the multi-hop questions and then sent it everything.

So don't pick one stack for everything. Pick per question, and be able to prove the pick. If your scoreboard has never once printed that the graph lost, your problem is not that the graph never loses.

Every agent on this page, cheap or expensive, right or wrong, had one thing in common: it sat still until somebody typed. The next one doesn't wait to be asked.