Overview

Let’s quickly go through the new models related to network compression published at CVPR 2019, ICLR 2019 and ICML 2019. Some works needs to be read and understood more carefully.

CVPR 2019

CVPR is more kind of tending to solve problems in practical applications, while ICLR and ICML are more close to theoretical explanations.

1. Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression^[1]

• Institutes: Xiamen University, Peng Cheng Laboratory (Shenzhen, China), Beihang University, Huawei Noahs Ark Lab, University of Buffalo and BestImage of Tencent Technology (Shanghai)

• Notes

1. Investigate CNN compression from a novel interpretable perspective. Discover that importance of feature maps depend on sparsity and richness (using the proposed Kernel sparsity and Entropy metric);

Overview

In this post, we are going to look at the following neural network models: MobileNet v1^[1] & v2^[2], SqueezeNet^[3], ShuffleNet v1^[4] & v2^[5], NasNet^[6]. We consider the following questions:

1. What in the world do they look like?

2. Why are they fast? Why are they small? Which one is better and Why?

3. Why the authors design them like that?

So, let’s try to solve these doubts step by step.