Your fallback probably is not a fallback
An ordered list of models looks like resilience, but it only becomes fallback when the next step can do the same work.
Fallback (an alternative path) for AI seems simple. You choose an AI model (a program that generates answers) and put another after it. Maybe you add a third for emergencies. If the first one fails, the system tries the next.
That list is almost always misleading.
The mistake is treating the model name as a measure of safety. In practice, a usage limit, subscription, capability, tool, or task type can fail. If the next step depends on the same things, the list fails for the same reason.
Redundancy is not a different name
The first trap is counting different names as protection. Two clients may use the same subscription. Two models may consume the same limit. Two routes may depend on the same account or permission.
When the limit runs out, both stop.
That is why I think about fallback by the resources and limits each option uses, not by name. Instead of asking "do I have two models?", I ask: "if this limit runs out, does the next one have an independent limit?" If not, the list only looks deeper.
This applies to the supplier, project, account type, and any step that can deny the request before the model answers. The system does not need private details. It needs to record which options depend on the same point.
The same task
The second trap looks like success. The first model edits files. The second only returns text. The first one fails, and the second replies with a plan or a formatted patch. The automation marks the work as done.
But nothing changed.
The next step only counts as fallback if it can do the same task. If the work needs a tool, a file edit, image reading, or a call to another system, the fallback needs those same capabilities. Lower quality may be acceptable. Silently changing the task is not.
A clear error is better than hiding a loss of capability. The error stops the process. A false success moves on and presents absence as finished work.
When capability changes, the route needs to change too. It can switch to text only, ask for review, or block. It cannot keep the same agreement.
Temporary failure or permanent loss
The third trap is confusing a temporary failure with a lost resource. A rate limit (a limit on how many requests can be made during a period) says the request did not go through at that moment. It does not say the resource is gone for good. An unauthorized or not found response points to a different problem.
When the router (a program that chooses which model gets each request) treats every failure the same way, it takes a healthy option out of use. A temporary spike becomes false proof that the chosen path is dead. The system then uses a worse option, even after the first one has recovered.
That is why fallback needs to record the cause and destination. "It went to the second model" is not enough. I need to know whether it was a limit, an outage, missing permission, a missing resource, or a task the model could not do. Without that, the choices cannot be adjusted.
One passage point
In my experience, the simplest way to track this is a single gateway (a service that receives requests and sends them on). Every call goes through it.
It needs to know the requested task, the required capabilities, which options are independent, and which model answered.
Recording the model that answered is essential. Without it, you may think the system is using the first option. In reality, it may have used the last one for weeks. The product still works, but the margin is gone.
A working fallback needs an agreement: what capabilities the task requires, which options are independent, which failures allow another try, and which require a stop.
Where it breaks
A single gateway can also be the only point of failure. If it stops, everything that depends on it stops too. Treat it as an important part of the system, with monitoring and a manual way to handle critical work.
Records of model choices are sensitive data too. They show what you send out, when you send it, and what kind of supplier receives it. Store them as operational data, not loose notes.
Automatic model selection can hide a decline in service. Without checking usage data, you may only find out the best option stopped when the last one stops too.