4 ms·
This comparison was not intended as an academic paper. However, we are confident that predicting and evaluating performance over more than 55k series from the M
by maxmc 4y ago
This comparison was not intended as an academic paper. However, we are confident that predicting and evaluating performance over more than 55k series from the M international forecasting competitions (standard benchmarks in the field) is a good start. After corresponding with the author, we included additional data sets from the electricity domain like Ercot and ETTM2. The dataset that did not finish training shows that Neuralprophet simply does not scale. (We let the training run for over 73 hours with 96 CPUs -as NeuralProphet implementation is restricted to CPU-, and we canceled it after investing 288 USD in it) Suggestions to strengthen our experiments (without spending hundreds of dollars) are highly welcomed.
Regarding the hyperparameter selection, we went beyond the
original NeuralProphet paper, tunning and actively trying to help improve its performance. We even made a PR fixing NP bugs in the process.