Every year, Agoda’s data science team organizes an internal competition called Machine Learning Cup, in which participants use machine learning to solve business problems. The competition allows Agodans to test their data science skills, network and learn from one another, and ultimately contribute to a real-world problem.
We chatted with the organizer and the winners to find out more about this year’s competition and how they approached the challenge.
The 2022 Machine Learning Cup
This year, participants were asked to predict how hotel room prices would change leading up to check-in and were judged based on how accurate their predictions were for each day. The loss function used was the root mean square percentage error (RMSPE).
“One of the surprising findings was that many hotels appear not to do any revenue management. They set a price and never change it, regardless of what happens to demand. This suggests an opportunity for Agoda to use its price intelligence to advise hotels on pricing. The winning team used a machine learning package called LightGBM to forecast how prices would change from the latest date,” said Robert Tayon, Head of Data Science Innovation and the competition organizer.
He continues, “To encourage teams outside of Data Science to enter the competition, we also offered a separate prize for the best entry from a non-Data Science team. This time, best non-DS team came from the Customer Experience Group (CEG). The CEG analytics team regularly forecasts incoming call volume to determine appropriate staffing levels, so it appears that expertise was also helpful in predicting how prices would change for hotel room nights.”
First Place: Pekka Aalto, Principal Data Scientist
“For the challenge, I first looked at the given price data to understand what it’s all about and how it behaves. Then, I thought about how to phrase the problem in terms of model inputs and outputs and finally built the model to predict relative price change compared to the base price for each hotel. The model I used is LightGBM (Light Gradient Boosting Machine), a machine learning package.
A tip from me? I’d advise people to think about how to create a simple baseline solution for the given task. After you get your baseline, you can already participate. Then if you have more time and energy, try to improve from there.
In Agoda, we have a culture of appreciating data. The data we have is good quality and easily accessible for everybody. That makes working with all kinds of machine learning tasks smooth.”
Second Place: Yan Kang Lee, Senior Data Scientist
“During the initial data analysis phase, I made three observations when analyzing prices for historic hotel check-in dates. First, I realized that many hotels kept their prices constant, meaning that reporting the last-known price will lead to highly accurate results.
Second, some hotels followed a static pricing pattern, such as raising or lowering their prices a few days before check-in. These patterns can be adopted to provide a stronger prediction than simply using the last-known price. Finally, the remaining hotels primarily adopted a dynamic pricing strategy, and it was not easy to model effective features for such hotels.
Still, the last-known hotel price remains a strong baseline for price prediction of such hotels. Combining these three observations, I used the baseline of reporting the most recent known price for every hotel as my strategy, except for hotels where a known static pricing pattern is used, in which case that static pricing pattern is adopted instead.
From this challenge, I learned that before performing any modeling or prediction, I should perform robust data analysis and establish a clear baseline, as this can often yield highly effective results. I also learned other strategies from the other contestants for data handling and data modeling, which will be useful for future tasks or Machine Learning Cups.”
Third Place: Shawn Xiao, Senior Data Scientist
I spent most of the time understanding the data, discussing approaches with other data scientists, and deciding on the baseline models. I felt most of the work would be to find or create predictive features. There is no need to try models beyond LightGBM because if the metric is RMSE (Root Mean Square Error), there is no better way than using yesterday’s price to predict today’s price from my experience with stock data.
For me, it was very interesting to look at the pricing team’s data. I’m always interested in other teams’ datasets and solutions. Data scientists generally have very different skill sets; there is always something you can learn from one another. And this challenge was a good way of doing so!
One tip that I have for anyone that will participate next year is to practice with Kaggle and do more exploratory data analysis (EDA). Kaggle is a popular platform for data science competitions, and it has a lot of online resources that are very useful to any Data scientist”.
From left to right: Hongyi Lin, Saruta Tantisiriwat, Akeamorn Puengpreeda and Adam Muhammad
“We are a group of data analysts working under the Customer Experience Group. Our work primarily focuses on analyzing customer data/experiments and developing dashboards/reports to support business decisions. We decided to join the competition to refresh our ML knowledge, connect with and learn from other Agodans, and ultimately have fun!
We started off by discussing together to understand the problem and spent quite a decent amount of time brainstorming many ideas. After we reframed our question, we split the tasks into two approaches: Heuristic and Machine Learning. A pair of us worked on each approach and we came back to discuss the result, fine-tune the model further and get the best-performing one.
Through this activity, we’ve learned how Agoda data scientists tackle problems and seen other teams using tools and algorithms that we hadn’t experienced before. Hopefully, we can integrate some knowledge that we gain from this competition into our day-to-day jobs.”
Agoda Machine Learning Cup is part of the effort to upskill our tech talents and foster the tech community at Agoda. Stay tuned to learn more about other initiatives, including Codegoda, Agoda Tech Talks and Agoda Engineering blogs.
Senior Machine Learning Engineer (Bangkok based, relocation provided)
Lead Data Scientist/ Lead Machine Learning Engineer (Bangkok based, relocation provided)
Lead/ Staff Data Scientist – Pricing Optimization (Bangkok based, relocation provided)