Permissions set the ceiling, not model quality
Most complaints that the model is not good enough turn out, on inspection, to be complaints that the assistant cannot see the document containing the answer.
- Most complaints that the model is not good enough are complaints that the assistant cannot reach the document holding the answer.
- Over-broad access produces a leak. Over-tight access produces apparent stupidity. They look nothing alike from outside and have the same root cause.
- Take twenty poor answers, find the document that answers each, and check whether retrieval could reach it under the asking user's identity.
- Budget the access review into every expansion. The first deployment only avoided it by choosing a corpus somebody had already curated.
When an internal assistant gives a weak answer, the reported cause is almost always the model. The fix proposed is almost always a better model. Both are frequently wrong, and the substitution is expensive because a model swap is a visible, fundable action while a permissions audit is neither.
An assistant grounded in your own material can only answer from what it is allowed to retrieve. Retrieval inherits the access controls of the underlying store. If the store's controls are wrong, you get one of two failures, and they look nothing alike from the outside.
The two failures
Over-broad controls produce a leak. A document library that everyone can technically read, because permissions were never tightened after a migration, becomes a library the assistant will happily quote to anyone who asks the right question. The assistant did not break anything. It exercised an access grant that had been sitting there unexercised, because no human had the patience to read every file they were entitled to read.
Over-tight controls produce apparent stupidity. The answer exists, in a system the assistant cannot reach, or behind a group membership the service account does not hold. The user asks a reasonable question, gets a vague answer, and concludes the tool is weak. They are right about their experience and wrong about the cause, and they will tell their colleagues the tool is weak.
The answer exists in a system retrieval cannot see. The user is right about their experience and wrong about the cause, and tells colleagues the tool is weak.
The assistant answers from material this user could have read themselves.
Nothing to see and nothing that should be seen. The only quadrant nobody complains about.
An access grant that had sat unexercised because no human had the patience to read every file they were entitled to. The assistant does have that patience.
The test
Take twenty questions the assistant answered poorly. For each one, find the document that contains the answer and check whether the retrieval layer could reach it under the asking user's identity. Our experience is that the majority of poor answers resolve to a reachability problem rather than a reasoning one. Until you have run this, a model upgrade is a guess with a purchase order attached.
Why this is a scaling issue specifically
The first deployment is usually scoped to one well-governed corpus that somebody curated by hand. That is why it worked. The second deployment reaches into a store nobody has audited since the last reorganisation, and the ceiling arrives immediately. The cost of the second deployment is not the model. It is the access review that the first deployment avoided by choosing an easy corpus.
Budget the access review as part of every expansion, not as a remediation project after the first incident. It is the same work at a quarter of the cost and none of the reputational damage.