It might have been 4pm on the Thursday before Easter (14 April) but this talk attracted close to 30 attendees on Zoom – testament to the drawing power of the speaker and title!
professor Johannes Schmidt-Hieber of the University of Twente in the Netherlands presented a topic of interest to academics and Government statisticians from several states. I’ve been wondering about this topi for a long time too.
Johannes began by noting the fundamental difference between machine learning and statistics. In machine learning, there’s an objective function to be minimised on the training data over some parameters (as always let’s call them theta). In statistics, there’s one more key ingredient – the distribution of the data.
Johannes then focused in on two machine learning methods. First was neural networks and deep learning. He showed that they could be written in the form of a statistical model, much like a nonparametric regression, along with a two-part formula for the prediction error. Secondly, working in the opposite direction, he described how multivariate adaptive regression splines could also be represented by a sparse neural network.
These innovations raised the question then – is it going to be possible to defy the bias-variance trade-off that seems to be unavoidable in statistics? Johannes answered No, while pointing to some interesting literature and debate around this topic from the late 2010s.
In conclusion, a thought-provoking way to lead into the Easter holidays!
aliceinstatisticsland 