VAADER Reading Group #5 / Joseph Faye / Dense-Gated Blocks, towards layers relationships in densely connected blocks

ResNet, Highway networks to DenseNet, adding more inter-layer connections besides the direct connection in adjacent layers, emerged as popular approaches to strengthen feature propagation among different layers.  However, dense connections cause much redundancy especially in the case of DenseNet.  Another aspect is that for many dense connections from previous layers, the role played by the mainstream module is unclear. To address these issues, authors introduce a gating mechanism, inspired by SENeT to model the layer relationships in densely connected blocks.

Tags: deep learning informatique