Most guides to buying customer service AI are written for a generic enterprise. Airlines are not one.

An airline's demand spikes faster than its staffing can. A weather event in one hub can multiply contact volume in an hour, and every one of those contacts is time-critical, because the customer is standing in a terminal watching a departure board. The systems of record are older and stricter than a typical retail stack. And the outcome the passenger wants is rarely an answer. It is a seat.

So the evaluation questions are different too. Here is what actually matters, and what to ask.

What does this platform do on the worst day of the year?

Every platform demos well on a Tuesday in April.

The question is what happens during a mass disruption event: a hub closure, a ground stop, a canceled bank of flights. Ask for the behavior under load rather than the architecture diagram. Specifically:

• What happens to queue depth and response time when volume multiplies?
• Can the platform reach out first, before the passenger contacts you, and can that outbound message hand straight into the process that fixes the problem?
• When a passenger is already in a conversation and their flight status changes underneath them, does the conversation know?

That last one is where platforms diverge most visibly in an evaluation. A disruption is not a question with an answer, it is a situation that keeps changing.

Which systems does it have to talk to, and what does it do when they say no?

An airline's stack is the constraint. Any platform you evaluate will have to reach a passenger service system, and probably an NDC layer, a loyalty system, a baggage system, and a payments processor.

Ask three things about every one of those connections.

What does the integration actually do? Reading a booking is a very different capability from changing one. Many platforms can look up a PNR. Far fewer can rebook against it, apply the fare rules that govern whether that passenger is entitled to the change, and issue what needs issuing.

What happens when the downstream system is slow or down? Ask what the platform does when the PSS is under the same load you are. One that assumes its integrations always answer will fail at exactly the moment you need it.

Who can change the integration, and how fast? Fare rules change. Waiver policies change during an event, sometimes hourly. If updating that logic requires a professional-services ticket, you do not have an operational tool, you have a project.

What counts as resolved?

This is the question most evaluations get wrong, and three different words get used for it: deflection, containment, and resolution. They count different events. We have written the three definitions out properly elsewhere and linked them below.

The buyer's version is one question. Ask any platform which of the three its headline number measures, then ask how it is calculated and who can see the working. If the answer is a dashboard number with no method behind it, you cannot manage against it and you certainly cannot report it upward.

Then ask the follow-up that separates them. Can that number be improved by making it harder to reach a person? For two of the three, it can.

What does the AI agent do when it should not act?

Rebooking is a transaction with money and entitlement attached. So the interesting question is not whether the AI agent can rebook. It is what stops it from rebooking something it should not.

Ask to see the boundary. Where does the agent stop, and why? Can it be prevented from promising a refund the fare class does not support? When it hands to a person, does that person get the context, or does the passenger start over?

An agent that never hands off is not a better agent. It is an agent with the brakes disconnected.

Is the suggestion the passenger actually wants?

When a passenger is rebooked during a disruption, the itinerary a reservation system returns first is not always the one they want. A passenger with a tight connection and checked bags has different constraints than one with hand luggage.

A platform that presents options in the order the API returned them is passing your operational logic straight through to your passenger.

Ask whether the platform can re-rank options against what it knows about that specific passenger, and ask who controls that ranking logic.

What happens after go-live?

Most of the platform evaluation industry is organized around deployment. Almost none of it is organized around month four.

This is where the real difference shows up, so ask the day-two questions:

• When the agent gets something wrong, how do you find out? Do you find out from a passenger complaint, or from the platform?
• Can you test a change before it reaches a passenger, and can you undo it quickly if it goes badly?
• Can you see why the agent did what it did in a specific conversation, or only that it did it?
  

A platform your team can improve every week beats a platform that launched better and then stopped changing.

What will your risk owners ask?

Bring them in before you shortlist, not after. Security, legal, and compliance can veto a deal in week ten over something that was knowable in week one.

The questions they will have: where does passenger data go, what is retained and for how long, what happens to information the model generates rather than receives, and what your obligations look like under the AI regulation applicable in the markets you fly. Get those answers in writing during evaluation. A platform that finds them difficult in a sales cycle will not find them easier in production.

A shortlist, honestly

If you take one thing from this: weight the day-two questions more heavily than the demo.

Every serious platform will show you a good disruption demo. Fewer will show you what happens on the worst day of the year, who can change the logic on that day, what their resolution number actually counts, and how your team makes the agent better in month four than it was in month one.

That last one is where the value is, and it is the hardest thing to see in a sales cycle.

If you want the worked example, Ada ran an airline disruption end to end in a recorded session, available on demand: eligibility validated against the passenger's situation, options re-ranked, and a rebooking confirmed with seat preferences intact. It is written up here.