We chose this paper to give us context: this is a buoy in the sea of papers and improvements in image recognition that took place after the 2012 ImageNet challenge showed such remarkable improvements. This paper is easy to follow and has a very appealing motivation: to understand what’s going on in CNNs.
After discussing the methodology of this paper, we took a detour to learn about the ImageNet dataset, looked at an interesting t-SNE of CNN codes from ImageNet images, and to read about the guy who tried to memorize all the ImageNet classes. We then returned to read the results and methodology. It was especially interesting to us how the authors used their visualization tool to debug a CNN: they changed the architecture to fix some inherent problems and, in doing so, achieved the (at the time) top score in the world in the ImageNet challenge. All in all, this is a really good paper for providing a lot of high- level context about CNNs and the ImageNet challenge.