Long term water demand forcasting : use of Monte Carlo cross validation for the best model selection

Selection and validation of any statistical models are very crucial in modelling and forecasting problems. In multiple regression analysis of forecasting long term water demand, various models are developed with a variety of predictor variables. Moreover, multiple regression models can take different forms such as linear, semi-log and log-log. In this paper, an effective but simple procedure named Monte Carlo cross validation (MCCV) is applied and compared to the most widely used leave-one-out validation (LOO) to select the best multiple regression model to forecast water demand. Unlike LOO validation, MCCV leaves out a major part of the sample during validation. Both methods are also used for estimating the prediction ability of the selected model on future samples. The advantage of MCCV is that it can reduce the risk of over fitting the model by avoiding an unnecessary large model. In this paper, MCCV and LOO are applied to the water demand data set for the Blue Mountains, NSW in Australia for single dwelling residential sector. The results show that MCCV has the ability to select an appropriate water demand forecasting model. It is also found that, MCCV assesses the prediction ability of the selected model with a higher degree of accuracy. Furthermore, the model selected by MCCV provides less uncertainty when forecasting long term water demand.