Abstract: To specify a function determined by a feedforward ReLU neural network, one usually gives a list of parameters (weights and biases). However, multiple different choices of parameters can determine the same function; in other words, the map that assigns functions to parameters is not injective. Furthermore, the degree to which injectivity fails is very inhomogeneous across the space of parameters. We define the "functional dimension" of a point in parameter space -- a measure of the dimension of the space of functions that can be realized by perturbing the parameter. I will discuss functional dimension and some of its properties. This talk is based on ongoing joint work with Eli Grigsby, Rob Meyerhoff and Chenxi Wu.
TAG-DS is a hybrid seminar and will be available in-person at the UW Mathematics Department as well as online on Zoom. You can find the link to the Zoom meeting here. If you would like to be added to our mailing list, you can do so by visiting this page.