CMX Lunch Seminar

Learning with artificial neural networks relies on the complexity of the representable functions and their parameters. In this talk I present results on the maximum and expected number of linear regions of the functions represented by neural networks with maxout units. In the first part, I discuss counting formulas and sharp upper bounds for the number of linear regions, with connections to Minkowski sums of polytopes. This is based on work with Yue Ren and Leon Zhang. In the second part, I discuss the behavior for generic parameters and present upper bounds on the expected number of regions given a probability distribution over the parameters and parameter initialization strategies. This is based on work with Hanna Tseran.