Explainable housing price prediction with determinant analysis

• Published in: International journal of housing markets and analysis Vol. 16; no. 5; pp. 1021 - 1045
• Main Authors: Teoh, Ean Zou, Yau, Wei-Chuen, Ong, Thian Song, Connie, Tee
• Format: Journal Article
• Language: English
• Published: Bingley Emerald Publishing Limited 24-08-2023; Emerald Group Publishing Limited
• Subjects: Basements; Bathrooms; Datasets; Decision making; Decision trees; Housing; Housing prices; Independent variables; Learning algorithms; Likelihood ratio; Machine learning; Predictions; Regression analysis; Regression models; Support vector machines; Multinomial logistic regression; House price prediction; Housing market analysis; Determinant analysis; SHAP analysis; Regression model
• ISSN: 1753-8270; 1753-8289; 1753-8270
• DOI: 10.1108/IJHMA-02-2022-0025

Purpose
This study aims to develop a regression-based machine learning model to predict housing price, determine and interpret factors that contribute to housing prices using different data sets available publicly. The significant determinants that affect housing prices will be first identified by using multinomial logistics regression (MLR) based on the level of relative importance. A comprehensive study is then conducted by using SHapley Additive exPlanations (SHAP) analysis to examine the features that cause the major changes in housing prices.
Design/methodology/approach
Predictive analytics is an effective way to deal with uncertainties in process modelling and improve decision-making for housing price prediction. The focus of this paper is two-fold; the authors first apply regression analysis to investigate how well the housing independent variables contribute to the housing price prediction. Two data sets are used for this study, namely, Ames Housing dataset and Melbourne Housing dataset. For both the data sets, random forest regression performs the best by achieving an average R2 of 86% for the Ames dataset and 85% for the Melbourne dataset, respectively. Second, multinomial logistic regression is adopted to investigate and identify the factor determinants of housing sales price. For the Ames dataset, the authors find that the top three most significant factor variables to determine the housing price is the general living area, basement size and age of remodelling. As for the Melbourne dataset, properties having more rooms/bathrooms, larger land size and closer distance to central business district (CBD) are higher priced. This is followed by a comprehensive analysis on how these determinants contribute to the predictability of the selected regression model by using explainable SHAP values. These prominent factors can be used to determine the optimal price range of a property which are useful for decision-making for both buyers and sellers.
Findings
By using the combination of MLR and SHAP analysis, it is noticeable that general living area, basement size and age of remodelling are the top three most important variables in determining the house’s price in the Ames dataset, while properties with more rooms/bathrooms, larger land area and closer proximity to the CBD or to the South of Melbourne are more expensive in the Melbourne dataset. These important factors can be used to estimate the best price range for a housing property for better decision-making.
Research limitations/implications
A limitation of this study is that the distribution of the housing prices is highly skewed. Although it is normal that the properties’ price is normally cluttered at the lower side and only a few houses are highly price. As mentioned before, MLR can effectively help in evaluating the likelihood ratio of each variable towards these categories. However, housing price is originally continuous, and there is a need to convert the price to categorical type. Nonetheless, the most effective method to categorize the data is still questionable.
Originality/value
The key point of this paper is the use of explainable machine learning approach to identify the prominent factors of housing price determination, which could be used to determine the optimal price range of a property which are useful for decision-making for both the buyers and sellers.