Why Hotels Should Use AB Testing To Drive Direct Bookings

If you asked me two years ago if hotels need to use A/B Testing I probably would have given an answer like I hear today from a lot of Hoteliers:

“A/B Testing for hotels is useful I am sure, but it seems complicated and expensive so it is probably most applicable for big hotel chains.”


I was dead wrong!


A/B Testing For Every Hotel

A/B Testing is easy, cheap and makes sense for even the smallest of hotels.  Let me tell you why.


Before we go any further, let me tell you that Hotel Res Bot does NOT provide A/B testing tools for your website.  Hotel Res Bot uses A/B Testing on behalf of our clients within our automated email platform.  We help our clients develop different email templates that our Hotel Email Reservation Assistant Bot (Hera Bot) uses to send out offers.  Every open, click and booking is tracked as part of our A/B Testing process.


A/B Testing Results
Sample A/B Testing Results by Variant for Hotel Bookings

What is A/B Testing?

A/B Testing is using the scientific method to see if changes you make to most commonly your website, are effective in generating more bookings.  Simply put, in addition to the “Champion” or current “A” version of your website, you create a “Challenger” version called the “B variant”.

Then, you test the variants against each other by sending a random 50% of the website visitors to the “A Variant” 50% to the “B Variant”.  This is perfect when you are planning a new website.  It allows you to compare the conversion figures and see if your new website actually helps you get more bookings; or as is too often the case; less bookings.


How much does AB Testing Cost?

In addition to some free tools and free trials, a basic A/B Testing tool which also hosts the pages for you can run something like 99 USD a month.  If we assume a new hotel website starts at 5.000 EUR, we are talking about less than 2% of the overall cost.

Unbounce Offering


If you can get 1 additional booking by testing that could cover the entire cost of a basic testing platform. Of course, because A/B Testing is so critical for all ecommerce companies there are more powerful and more expensive platforms out there which you can grow into; depending on your needs.


A/B Testing is Easy

Don’t let anyone tell you that A/B Testing is hard to do.  It’s not.  Anyone who thinks that it is hard to do, probably hasn’t tried it yet.  It is easier than you think.  With the A/B Testing platform we use we can create a new dynamic (optimized for desktop and mobile) webpage in minutes.  It automatically tracks all conversions and even does the math to let you know when the difference is the conversion difference is real or just based on chance.


Changing Without Testing Is Insane

When you think about it, it is completely crazy to spend €10.000 EUR or more on a new website and then just release it; without seeing if it converts better than your previous one.  The same thing goes for replacing your booking engine.

Can anyone tell me why hotels don’t test the new booking engine vs the old one to see which one converts better?  Just because you used a respectable agency and selected a well known internet booking engine does not mean that your conversion will increase.  All too often conversion actually decreases.  But this is often even not seen, because we are looking at the wrong numbers.

AB Testing Process
A/B Testing Process

 Cohort Analysis

When implementing a new booking engine most hotels tend to focus on total booking numbers.  The ongoing trend toward online bookings means that we should expect growth on the booking engine.  Additionally, seasonality can completely skew the numbers.  Cohort analysis is the solution to this problem.


It is critical to avoid the temptation of lying to yourself with top line figures.  You need to look at each day, week or month as a separate group and compare the conversion on website A vs. website B.  If you don’t break down into time periods, you could easily skew the data.


The HiPPO Problem

Without A/B Testing, design topics tend to dominated by the Highest Paid Person’s Opinion (HiPPO).  Like a hippopotomus, they tend to throw their weight around and usually they get their way.  Unfortunately, your HiPPO looks nothing like the market segment you are focussing on.  Or said another way, the overlap of your customer’s needs and what your HiPPO thinks is usually nothing.


I have been both the HiPPO and the person trying to convine a HiPPO and both positions are uneviable.


HiPPOs usually have become successful by being able to make decisions in the face of uncertainty.  To be successful in the future, HiPPOs have to change the way they think.  They need to go from DECIDING WHICH solution wins, to generating new ideas to test and implementing them.


OTAs Use AB Testing

I can tell you from my own experience.  The bad news is that your favourite design will rarely be the one that achieves the highest conversion.  If you want proof of this – just look at Booking.com.  I don’t know anyone that says the like the “look” of the interface.  But I can guarantee you, Booking.com is contantly testing the conversion impact of changes.  Their website is the result of the best converting design.  Is it a work of art? No – it the most powerful hotel booking converting machine ever created.  Built by A/B Testing; not by a HiPPO.


How Powerful is A/B Testing?

When I build my first ResBot I was extremely dissapointed with the results.  Less than 1/3 of the people that started interacting with bot were able to complete their request.  This was a small catastrophe and we needed to fix the problem; immediately.


One of my co-founders suggested Unbounce as a landing page platform with built in A/B Testing capabilities.  Within 2 days we had migrated out bot to Unbounce and began testing variants.  Although the journey took many trips through the A/B Testing Process before we were anything close to satisfied, it was immediately apparent that we were able to dramatically change the conversion with extremely minimal design changes.


We were able to move from under 30% to over 95% conversion.  This translated into a 270%+ increase in revenue.


AB Testing For Hotels With Minimal Design Changes

We brainstormed a ton of ideas and tested many of them.  The changes with the biggest impact were things like:

  • The colour of the booking button (red is bad, green for example is good)
  • The length of the text (the shorter the better as long as critical information is not missing)
  • “Du” vs. “Sie” (German informal or formal word for “You”) for our Bot “Du” converted better
  • With pictures or without (with was better)


Hotels Need To Start A/B Testing

Hotels need to start using this powerful tool that OTAs have been using for years.  If hotels continue to ignore A/B Testing it is akin to just giving the OTAs the keys of the kingdom.  OTAs now represent 29% of European Hotel Revenue up from 22% in 2013 (according to Roland Schegg’s 2018 European Hotel Distribution Study available for download at e-tourism monitor).


Based on the success we have had using A/B Testing for Hotel ResBot, it would be crazy for a hotel to implement a new website, or a new booking engine without testing.  In fact, we don’t make ANY changes without testing anymore and I hope you do the same in the future.


About Hotel Res Bot

Hotel Res Bot does not provide generic A/B Testing services for websites.  We do passionately believe in helping hotels drive their direct distribution.  A/B is a tool that Hoteliers and especially their agency and booking engine partners need to start using if they want effectively compete against the OTAs.

We believe in AB Testing so strongly that we made it a core part of our Hotel Email Reservation Assistant Bot (HeraBot).  HeraBot automatically reads and responds to email booking request with perfectly formatted emails; within seconds.  Combined with A/B testing, hotels can not save time and money by automating their email booking processes, but increase revenue through improved conversion.  For more information check out our website.



User Testing by Sharon Showalter from the Noun Project

Screen Capture Unbounce courtesy of Unbounce

A/B Testing Process by Hotel ResBot

Sample A/B Testing Results by Hotel Res Bot