Join 3,497 readers in helping fund MetaFilter (Hide)


Why are airline computer reservation systems so slow?
February 19, 2014 2:53 PM   Subscribe

The computer reservation system that airline personnel at airports use to rebook flights for customers, change seats, and so on seem markedly slower than when the customer uses the internet to pick a seat, or book or change a flight. Why is that?

I've noticed it in the past, but was reminded of this mystery again when our flight out of Santo Domingo was canceled recently and we had to go to the airline desk to rebook on another flight. After the agent confirmed that there were two seats available on the next flight, it took her another fifteen to twenty minutes to book us on it. She didn't ask us for any additional information than she already had on her computer screen, so presumably that wasn't the cause of the delay, nor was she markedly less competent than her colleagues. I've noticed this in many different countries, including the US, and even noticed it for (what I think of as) relatively straightforward tasks such as changing seat assignments in the system. Tasks that take me about ten minutes or less to accomplish online seem to take them twice or thrice as long.

I'm old enough to remember pre-internet travel, when booking an airline ticket required a festive family visit to a travel agent and a meeting that could last over two hours, but things have progressed considerably since then, and now picking a seat on a flight requires nothing more complicated than looking at an image of the plane online and seeing which seats are available. But apparently not for the airline agents.

So, what're they actually doing on those monitors?
posted by lassie to Computers & Internet (24 answers total) 42 users marked this as a favorite
 
They don't use GUI interfaces-- they are all text. So instead of seeing a plane picture and picking a seat, you type in the code to show the plane, then type a code to identify open seats, then type a code to select that seat. Source: looked over the shoulder of an agent making a change for me this Saturday in Tampa airport.
posted by holyrood at 2:56 PM on February 19 [2 favorites]


Seriously? Why would they do that?
posted by lassie at 3:09 PM on February 19


This article explains that a little bit.
posted by jessamyn at 3:17 PM on February 19


Because it works and it would cost more to implement, bug-fix, train, and troubleshoot with a new system. Lots of industries use legacy text-based programs on the back end — it takes a lot less effort to keep an old system going than to make something new, even if the something new is just another interface to the same data.
posted by stopgap at 3:17 PM on February 19 [3 favorites]


because they replace their terminals with shiny new dells, but then use a terminal emulator to connect to some ancient 90s at the newest mainframe type system that runs it all.

the local library, and many other municipal systems like the DMV in my area were doing this until like, at most a couple years ago. big corporations move slooooow with this stuff, especially if the system itself doesn't handle credit cards but an outside card processing machine does so they aren't forced to upgrade.
posted by emptythought at 3:18 PM on February 19 [2 favorites]


They use systems like Sabre, which has been around since 1960 (!). They're still around because they work and replacing them would be enormously expensive.

Some airlines have more up-to-date systems. For example, there was a lot of consternation when United bought Continental. United had a rather modern, point-and-click based system, but moved everyone over to Continental's older, mainframe-based text system. More info here.
posted by zsazsa at 3:23 PM on February 19 [2 favorites]


Seriously? Why would they do that?

Because it exposes the more complex functionality that isn't available to people clicking around on a website. At the desk/gate-agent level, when dealing with real-time connection and seat information, that kind of access (and reliability) is important.
posted by holgate at 3:23 PM on February 19 [3 favorites]


3270 emulation. Ah the good old days. It works and these back-end hosts are massive and allow thousands of terminals to connect to them.
posted by nostrada at 3:24 PM on February 19 [3 favorites]


Dilbert asked the same question on Christmas Eve 1995.

Text based interfaces aren't slower by definition. In fact many things are significantly faster to do with a command line than a GUI, especially if you're familiar with the system and there is significant lag. I can be typing in the next command while the previous one is still running, while with a graphical or web based system you are frequently forced to wait for the round-trip to the server to complete.
posted by autopilot at 3:28 PM on February 19 [4 favorites]


Thanks, everyone! This is all pretty fascinating.

From zsazsa's link, I learned the term "green screen" which describes the non-GUI interface you're all referencing. That, in turn, has led me to a large number of articles on the issue. I also see that Sabre has a GUI interface, but I'm guessing (as many of you point out) that airlines would prefer not to spend additional money on the upgrade when they can use systems they already own.

holgate, besides reliability, could you give me an example of the type of complex functionality you're talking about? I saw the same rationale offered for the green screen interface in a few articles I read, and am wondering under what circumstances it would be useful for an agent to have greater control over the system than a customer. I thought the whole point of the system is that, once a ticket has been sold or a seat assigned, nobody can override that, no?
posted by lassie at 3:46 PM on February 19


Oh, and best answers to everyone! I just learned a ton about something that's been puzzling me for a while.
posted by lassie at 4:00 PM on February 19


I don't think it's that you *couldn't* implement complex functionality in a GUI-based interface, it's more about the cost of implementing and deploying that new interface, as well as training people on it. The issue about what level of control a customer vs. an agent of the airline should have is separate, regardless of what sort of user interface the system presents. For that one, I would assume that airlines see a business need for this level of control, so they have it.
posted by Aleyn at 4:37 PM on February 19


under what circumstances it would be useful for an agent to have greater control over the system than a customer.

Complex re-routing on multi-leg itineraries, especially if it requires some non-standard connections or putting passengers on different carriers (not alliance / code-shared) or changing classes or overriding conditions on tickets because of circumstances. I've been in situations with cancellations and delays on international itineraries where the check-in or gate agents did some fairly nifty work on the terminal to get me from A to B via Q and J instead of C when the customer-facing rebooking system wasn't in a position to do so.

As Aleyn says, all of that functionality could be baked into a GUI, but the benefits of doing so become increasingly marginal for operations that aren't suited to the same kind of abstraction as pointing-and-clicking on a seat plan.
posted by holgate at 6:44 PM on February 19 [3 favorites]


First, a skilled agent should often be able to use these systems faster than you would be able to use a point-and-click web site.

Second, you probably couldn't use a web site to rebook your canceled flight out of Santo Domingo unless you wanted to pay for it. The rebooking involved unchecking you from a flight that had been checked in, restoring the value of your electronic ticket, quite possibly a ticket reissue, and documenting why they were overriding restrictions to get you on that other flight (your initial fare was probably not valid for any of the available seats on your new flight). Then a lot of checking of business rules that happens behind the scenes (on some airlines it takes minutes to hours after you press purchase for a ticket to be issued) has to be done while you wait.

Third, humans are notorious for overestimating the amount of time they wait.

I thought the whole point of the system is that, once a ticket has been sold or a seat assigned, nobody can override that, no?

The point of the system is to sell tickets and make money. Often this requires overriding the usual rules of the system, but it's the airline who gets to decide when that is, not the customer.
posted by grouse at 6:49 PM on February 19 [3 favorites]


Another thing that I haven't seen explicitly mentioned here is that SABRE (or is it just "Sabre" now?), as it currently exists, provides something really valuable: uptime, which is thrown into doubt whenever you replace a system that is known to work reliably with a newer, flashier system.

This is also why National Weather Service bulletins, and nearly all of your bank transactions, show up in all caps: upgrading to a system that supports prettier, more complex output is full of unknowns. Nobody wants to be the one who approved a system upgrade that didn't let anyone book flights to Chicago for eight hours without any warning.
posted by one more dead town's last parade at 7:05 PM on February 19 [8 favorites]


I worked as a consultant for one of the big reservation systems in the late 1990's, wrapping some of the legacy code with Java-based GUIs. I haven't worked in that industry since 1999, but I doubt if it has changed that much, so I'll offer a couple of observations on top of what has already been said by the other commenters. Hopefully this will explain the dynamic and why, in 2014, people in the industry are still using green screen interfaces.

#1 Most of the reservation systems in the world (by most I mean all but one) are written in a dialect of IBM assembler called TPF that runs on IBM mainframes. The code base for my client's reservation system dated back to the 1960's - you could see comments in the source code about "this is kludgy, will fix later" that had been written thirty years previously (and of course the code had never been modified since). So: think millions and millions and millions of lines of assembler.

#2 The travel agents grew up interacting with the green screens using lots of obscure two letter and three letter codes to get things done.

#3 There were all kinds of documented/undocumented special features in the system that almost certainly don't get passed through to any of the GUI interfaces wrapping these systems, like one-off features for specific partners (e.g. hotel chains or rental car companies). Don't assume that the GUI interface being faster is with equivalent functionality; the GUI is exposing a subset of the functionality.

#4 Reservation systems have hundreds and hundreds of "downstream" customers like travel agencies, hotel chains, rental car companies, and other airlines who, in the 1990's, had written their own software using screen scraping techniques. Changing even one character on a green screen could break dozens or hundreds of third party applications. It was like cement; it would take months of negotiation to orchestrate the change in layout on a green screen if you wanted to, say, make a field one character wider.

#5 Unscheduled down time is very expensive. The reservation system I worked on measured downtime in millions of dollars per hour of lost revenue and in the years I worked with them, they had less than 20 minutes of unscheduled downtime per year (uptime was actually a measured goal in their corporate strategic plan). As you might imagine, the best way to achieve that kind of uptime was to release new code very rarely and change as little as possible.
posted by kovacs at 8:05 PM on February 19 [50 favorites]


Also, hinted at here, but I didn't see called out specifically:

What you do on the web site usually applies to something that will happen hours days or weeks in the future. What the gate agents are doing applies to something that needs to have an immediate impact.

When it is happening days in the future, there is a greater chance that you are the only person in the middle of booking a seat on a given flight. If that is the case, the system can quickly verify that it can provide your chosen seat. On the other hand, if you are being rebooked for a cancelled flight, chances are, you aren't the only person being booked into the new flight. The system has to move more deliberately and may have to wait for another rebooking in progress at another window to reach completion before verifying both seats. And then there is the fact that the agent may not just be rebooking you, they may be shuffling other stuff around to make room for you.
posted by Good Brain at 11:02 PM on February 19 [1 favorite]


*waves to kovacs* Not often I cross fellow consultants who have worked on airline systems.

I worked for the IT department of an airline whose name is Air Country, where "country" is the one I live in currently, abbreviation AF. (I don't type it out because I know AskMe turns up in search results.) My job was to test their migration from their own internal system, to a very similar one managed by Amadeus. In matter of fact, AF's flight reservation system is the parent of Amadeus' flight reservation system. ("Flight" reservation specifically.) Migration simplified (outsourced, sort of, since AF and Amadeus are nonetheless related) their own management, and provided a more direct link to Amadeus' other reservation systems, as other commenters here have mentioned.

My job was to test that nothing was broken... which is a huge simplification of matters. AF has partnerships with other airlines around the world, not all of whom use Amadeus - some use Sabre, others use Sita. Those three are the main reservation systems nowadays, though there are others.

Customer interfaces, where a customer means someone like you buying a ticket, are vastly simplified for useability reasons. It would be impossible to train customers to use airline systems. It took me six months to learn how to use AF's check-in application, which was a green-screen that ran on a mainframe. Yep, IBM 3270. Even after six months, I was still learning specifics about it, and I used it every single day.

And that's just the check-in application. Then there's the reservation application. Everything is based on codes, and for very good reason. Once you memorize the codes, "speak the system's language", you're golden. Yesterday I took AF6205 NCE-ORY class U with PNR X8RHO5 (not actually that). AF's in-France, odd-numbered flights go to Paris, the even ones go out of it. NCE is the IATA code for Nice airport, ORY is the IATA code for Paris Orly. U is a sub-group of the catch-all economy class Y (the entire alphabet is used). PNR is Passenger Name Record, and is what customers know as "booking reference". As you can see, they're only 6 characters. Codes are reused over time, since there's no need to have a permanent booking reference. That saves database space, processing time, and has the added benefit of being grokkable by customers, as well as being unique enough that any number of people around the world can be travelling at the same time and no one's PNR will be identical to another's. Keep in mind, this is an IATA standardized code that is used by reservation systems around the world: all of that has to be managed. A PNR for an AF flight cannot be identical to a PNR for a flight reserved on a Sita system, for instance. The different systems also have to communicate with each other.

Flight numbers are the same way. As you've probably noticed, they're short. The 08:50 NCE-ORY flight is always AF6205. The 18:00 ORY-NCE flight is always AF6236. Same reason: database space, processing time, ease of comprehension. Two-letter airline codes are also IATA standard.

What about all those reservation classes? That's one key reason airline reservation systems are complex. There's a lot more to reservation classes than just first-class, business, and economy. Payment type, customer type (company, private, employee), time the ticket was purchased, whether the flight was overbooked, cancellations, reservation changes, plane type, all that enters into play. And in order for interfaces with sales applications to work properly, a "simple" change of reservation has to be done properly. First your PNR is looked up on the reservation system. But then the reservation agent actually goes into the check-in system, because you're at the airport and need to get on a plane. You're not just reserving a new flight, you're modifying an entire workflow, in essence. On the check-in system, the agent starts out with the company code (if you flew on a codeshare, this is not as straightforward as it may seem), then the flight number, then the flight date, then they look for your PNR, check your reservation class, and see which spots are attributable on the other flight: same story. Company code, flight number, flight date, and for the new flight, they look for spots available according to your class. Remember: there are 26 class codes, it's not just first/business/economy. The type of plane also enters into consideration. If the plane was changed due to scheduling, mechanical issues, whatever, then there's a whole new level of complexity being managed by the check-in and reservation systems. Those all have to work together quickly and efficiently. The agent also has to tell the system that, say, a class U is now a class W, which is okay because they're both Y (economy), for instance. And we haven't even gotten into luggage, which is very complex to manage when you change reservations at the airport. It's last-minute. There is a huge logistical chain in place for luggage management. That chain has to be modified, reliably. A complex reservation system is complex because it needs to be in order to be accurate.

I've only really scratched the surface here, but hopefully it gives a bit of a look behind the scenes into the panoply of considerations from the business side.
posted by fraula at 3:19 AM on February 20 [51 favorites]


Again, thanks to all of you, and particularly to kovacs and fraula for your insider take on this question.
posted by lassie at 10:35 AM on February 20


Hey guys; small world. I too worked for one of the big Airlines as a consultant in the 90's and got my time with full frontal TPF.

One minor clarification: TPF (Transaction Processing Facility) is actually the operating system that runs the reservation systems. At least, some people call it an operating system, but what it really is is the thinnest veneer of a real-time kernel that could be painted on an IBM mainframe's bare hardware and get the job done, for the sake of speed and reliability. Kovacs is not exaggerating...guys were always showing when they found a stick of code that hadn't been touched since before they were born (late 60s for me, and I found a lot of code). TPF does have it's own, demented, variant of the IBM Macroassembler called BAL, but in these more refined times IBM offers C/C++ and Java environments, and apparently Perl runs, but such a thing is almost to evil to contemplate.

Another minor clarification: Almost all of those reservation terminals you see that haven't been converted to PCs are not IBM 3270 type, they're Airline Control Protocol (ALC) aka 'SITA P1024B' attached. The terminals speak a multidrop, bisync, serial protocol with 6-bit characters and next to no error detection or correction unlike almost any other networking protocol ever deployed. Every terminal in a "row" (which means different things depending on the layout of the airport) is daisy chained together and connected to an ALC PAD (nee 'controller'). Each terminal get's a time-slice to squirt out any message it has and listen if there was a message for him (book this seat, ack it's book...that sort of thing). The PADs then connected to the mainframes over X.25.

My own project was on the team that replaced the PADs with an ALC TPAD emulator on top of SVR4 Unix that would first give us a monitoring and management framework for the ALC gear and eventually let us replace those clunky terminals with clunky but cheap PCs and translate IP messages to ALC for the back end. Monitoring ended up being a huge driver. Since there wasn't much in the way of debugging built into most of the ALC gear we were using, if a terminal started acting up (or more usually, all the terminals on a multi-drop leg), they called for one of the ALC techs to come down. The primary tool was a little box that had 2 probes and a speaker (and presumably some small circuit). The tech would clamp this thing on the wire string that terminal was part of. Remember this was low-speed serial, message based and multi-drop, so each message "type" have a particular content (consequently sound) and length, and each terminal was suppose to only talk when it was their turn (or slot in the timing). These guys were experienced enough that listening to the burps of static from the messages passing, they could tell pretty much what the problem was ("uh oh...terminal 2 is sending a No Data message instead of a Request...better replace him", "terminal 8 timing is off and he's crapping on terminal 9..."). A big part of my role was creating a debugging, logging and reporting framework so every problem didn't involve dispatching a tech to a wiring closet. Across the carriers network, that ended up being a huge amount of money.

Perhaps most interesting for a networking technology around 50 years old, there's still a thriving TPF/ALC industry. You can buy ALC terminals and PADs, IBM fully supports TPF on it's latest mainframes, and lot's of people make a living as TPF programmers. You can even make your Cisco router act as a PAD (it's hidden in the IBM feature set, at least as of IOS 12.4...haven't looked in a long time).
posted by kjs3 at 11:56 AM on February 20 [46 favorites]


Of course I can't find the article that I want, but there's some ideas about usability vs different types of customers/users. The short version is that airline agents and people changing their reservation inside the airport are typically captive audiences, even if the system is not-great, you don't have much choice, and for most of the end customers the sting of a bad trip will have worn off by the time you go to buy your next tickets. Compare with search engines and online booking where people can leave quickly, which are usually the upper end of usability, and employee timesheet systems which are the absolute low end.
posted by anaelith at 10:30 AM on February 23 [1 favorite]


The interesting thing about systems that were over-engineered and kludged to death to deal with every possible use case and are incredibly arcane and complex because the problem domain is arcane and complex is just how amazing it is to reach that point of near absolute stability where downtime is extremely rare and your focus is simply on keeping it alive, and how desperately you want to cling to that system.

In many cases, the risk of disruption is too costly, and realistically, any slick system no matter well thought-out is not going to have this progressive lineage of people thinking through all of the less common glitches or fixing bugs they're embarrassed to find from their own contributions that go quietly unreported but are only recognized based on deep institutional knowledge and experience in the problem domain.

As complexity increases, getting it right with a full rebuild is essentially impossible because even if we had perfect documentation and specification standards today, we didn't before, and no, we don't today, not even close.

I've been there with much simpler systems that are still mission critical to a business. You reach a point where the same build whizzes by through one or two years of production with absolute stability and it becomes increasingly nerve wracking to deploy new features or fix bugs that are easy enough to adjust for outside of the code, perhaps even seamlessly, without anyone else really knowing how that all works. Documentation is the most expensive part of software development and often the expense is postponed and paid by simply suffering in lost productivity.

And at the end of the day these old school systems are simply badass and people fall in love with their efficiency and familiarity and can train new hires holistically, wrapping the whole job around the user interface and making them masters of it vs. conditioning them to expect regular changes like you might see with a web browser or any consumer software.

And then we can't even really agree on user interface design to this day when it comes to highly complex systems that require a shitload of quick access to switches and knobs and doo-hickeys. Ultimately you have to kind of embrace the clutter and muscle memory is expensive to attain and pays dividends until you force the employee to re-learn it again.
posted by lordaych at 4:55 AM on March 31


wrapping the whole job around the user interface and making them masters of it

Also many of the people who use these interfaces aren't customer facing. Flight bookers of course are, but you'll notice how they're just kind of short and abrupt and kind of ignore you and clickedy-clack-clack until it's time for you to exist again. Seems cold and they don't have to do that, I kind of make it a game to be super nice and get them to warm up, but ultimately they're just masters of this system and they're holding a little stack of information in their brain swirling around pushing and popping information in and out of the system to make things come together. Almost like wizardry just coordinating a multiple step reservation / rescheduling in real-time, now build a better one and it must be more fault tolerant and better in every possible way or it will immediately be pilloried and never spoke of again unless it's forced into adoption through legislation and nobody is gonna do that with this particular software space.
posted by lordaych at 5:02 AM on March 31


I also worked in the airline business in the 90s. I didn't work directly on check-in systems, but data feeds they generated were vital to things I did work on. As people say the systems are spaghetti coded in assembler, and on occasions I delved into the assembler to try to work out what possible sequences of events could give rise to the various category codes the system gave to passengers.

The kinds of events those systems have to deal with can be very convoluted. For example, someone turns up who doesn't have a booking on this flight at all, but a ticket on another airline for another flight, of a kind that we will accept if we can. Fine, we check him in, take his bags. Then he doesn't show up for boarding. Ok, for security reasons we have to offload his bags now. Perhaps the guy eventually travels on another flight. Then later on we somehow have to settle up with the other airline that he was booked on, about who gets how much money. Also we have to produce data feeds for people who want to forecast how many people might actually fly in the future, and we have to have some kind of classification for this person what was not supposed to fly, then he was, and then he didn't.

On top of that this system is running 24x7 in airports in different timezones across the world, and if it is down for half an hour, parts of the airline stop functioning and there is great gnashing of teeth by passengers and staff alike.

To ever migrate off it to a new system that didn't break anything would be monumentally difficult and expensive.

Airlines were among the first industries to make heavy use of IT for critical functions, and their legacy systems are from a whole different era than most industries. It's kind of a wonder it all works as well as it does, and with such uptime.
posted by philipy at 12:55 PM on April 8


« Older I'm working on a project at wo...   |  How would I go about finding o... Newer »

You are not logged in, either login or create an account to post comments