3:40 PM - 4:00 PM
[1Q3-J-2-02] Construction of pooling layer by skip connection and analysis based on expressive power of these models
Keywords:deep neural networks, skip connection, model design, deep learning, expressive power
We showed that widely used these structures can be understood as a composition of affine functions and concatenated activation functions.
Moreover, we show the followings:
(i) the pooling layer explicitly decreases expressive power,
(ii) there is no deference in expressive power between addition and concatenation as skip connection for fully connected neural networks, and
(iii) the single activation block has superior expressive power compared to the multiple activation block.
These results propose one guideline for design of neural networks models.