Estimating the Impact of Website Changes on Conversion Rates
(2023) DABN01 20231Department of Economics
Department of Statistics
- Abstract
- This study sought to evaluate the historical impact of changes to an ordering page of an online travel agency on its conversion rates. Data gathered from the website over a year, detailing aspects such as travel dates, prices, itineraries, number of passengers, travel time, and carriers, was analyzed. External data sources were also included, with the dataset covering 12 changes to the website's layout and payment process. The changes' effectiveness was assessed using three methods: comparing conversion rates before and after the changes, a modified linear regression model, and the Double Machine Learning (DML) method with Random Forests as the base learners. The analysis revealed that the only modification with a statistically significant... (More)
- This study sought to evaluate the historical impact of changes to an ordering page of an online travel agency on its conversion rates. Data gathered from the website over a year, detailing aspects such as travel dates, prices, itineraries, number of passengers, travel time, and carriers, was analyzed. External data sources were also included, with the dataset covering 12 changes to the website's layout and payment process. The changes' effectiveness was assessed using three methods: comparing conversion rates before and after the changes, a modified linear regression model, and the Double Machine Learning (DML) method with Random Forests as the base learners. The analysis revealed that the only modification with a statistically significant positive impact on conversion rates was related bug fixing. Most changes did not significantly affect conversion rates, and some even demonstrated a non-significant negative impact. The DML method proved a useful tool in this context, outperforming simpler comparison methods with better control for confounding variables and reducing potential bias in Average Treatment Effect (ATE) estimation. However, estimates from the DML model were sensitive to the analysis time window. This study suggests future website design should focus on user-friendly and intuitive design, clear and detailed information provision, and careful evaluation of changes' potential impact on user experience. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9134717
- author
- Jarco, Jan LU
- supervisor
-
- Simon Reese LU
- organization
- course
- DABN01 20231
- year
- 2023
- type
- H1 - Master's Degree (One Year)
- subject
- keywords
- website design, causal inference, observational study, average treatment effects, double machine learning
- language
- English
- id
- 9134717
- date added to LUP
- 2023-11-21 12:54:07
- date last changed
- 2023-11-21 12:54:07
@misc{9134717, abstract = {{This study sought to evaluate the historical impact of changes to an ordering page of an online travel agency on its conversion rates. Data gathered from the website over a year, detailing aspects such as travel dates, prices, itineraries, number of passengers, travel time, and carriers, was analyzed. External data sources were also included, with the dataset covering 12 changes to the website's layout and payment process. The changes' effectiveness was assessed using three methods: comparing conversion rates before and after the changes, a modified linear regression model, and the Double Machine Learning (DML) method with Random Forests as the base learners. The analysis revealed that the only modification with a statistically significant positive impact on conversion rates was related bug fixing. Most changes did not significantly affect conversion rates, and some even demonstrated a non-significant negative impact. The DML method proved a useful tool in this context, outperforming simpler comparison methods with better control for confounding variables and reducing potential bias in Average Treatment Effect (ATE) estimation. However, estimates from the DML model were sensitive to the analysis time window. This study suggests future website design should focus on user-friendly and intuitive design, clear and detailed information provision, and careful evaluation of changes' potential impact on user experience.}}, author = {{Jarco, Jan}}, language = {{eng}}, note = {{Student Paper}}, title = {{Estimating the Impact of Website Changes on Conversion Rates}}, year = {{2023}}, }