The problem with RAG is knowing when it found nothing
Vector search always returns a first result. A good system knows when that result does not help.
The first RAG that seems to work almost always fools you. RAG is a system that searches document passages to help a model answer. You ask about something in the collection, search finds a similar passage, the model answers, and it seems like memory is solved.
The real test comes when the answer is not there.
Vector search turns text into lists of numbers and looks for the closest ones. Cosine similarity measures how much those numbers point in the same direction. It sorts results, but does not check whether the subject exists. If the collection has no answer, a first result still appears. Its score is often high enough to look like proof.
Then the model writes using the passage it received. The error does not look like an error. It looks like a confident answer built on a passage that does not help.
Similar does not mean relevant
"Most similar" compares options inside the collection. It does not say what is true. If I search for something that is not there, the system still picks the closest result. The best of bad options is still bad.
So it is not enough to ask which passage ranked first. You also need to know if its score passed a limit that means relevance for that collection. Without this limit, RAG never says "I don't know." It only swaps one guess for another.
But the limit does not come from intuition. Every collection has its own noise floor.
The noise floor
The noise floor is the score a random passage tends to get in your collection. It is not the score from a tutorial or a provider test. It is the score in your collection. Long documents, repeated terms, similar names, and standard wording can raise similarity, even when a passage does not answer the question.
Without measuring this, any cutoff is a guess. A low limit accepts poor context. A high limit makes the system say "I don't know" even when an answer was there.
In practice, test questions whose answers you know are missing and check their scores. Then test questions with answers and see where the good scores appear. The range between these groups helps set the limit.
Reranking and the right to say no
An embedding is a numeric representation of text that helps find related subjects. Reranking is a second check: it compares the question and passage together. The names are similar, but the steps do different things.
The reranker can lower the rank of a passage that uses similar words but does not answer the question. It also helps separate a passage that only mentions the subject from one with the needed information. In many systems, this second step improves perceived quality more than changing the model that writes the final answer.
Most of all, the system needs the option to say "nothing here helps." Without it, every RAG tries to justify the least wrong passage.
Portuguese and passage size
Portuguese content also changes the results. An embedding model trained only on English may compare terms, word forms, and writing styles poorly. This does not appear as a system error. It appears as a mediocre answer that seems fine until someone who knows the subject reads it.
Passage size matters as much as the model. A large passage dilutes the information: the question matches one part, but the model receives several ideas together. A small passage can lose earlier context. The pronoun, definition, or condition may be in the previous paragraph.
That is why I treat passage size as part of the product, not a search setup detail. If a passage lacks enough context to stand as a citation, it should not be delivered alone. If it has too much, it becomes noise presented as a source.
Where it breaks
An old document still in search can be worse than a missing document. The system finds it, gives it a high score, and the model treats it as true. An absence can become "I don't know." An old document can produce a wrong answer with a source.
Too much caution also gets in the way. A limit set to never fail can reject an answer that was in the collection. Calibration is a tradeoff: accept some risk of a poor match or frustrate someone by rejecting an answer the system had. Pretending both can be avoided only hides the choice.