Asosiy kontentga oʻtish
Maqola

Regularization of Neural Networks using DropConnect

Li WanCIMS - Courant Institute of Mathematical Sciences [New York] (251, Mercier Street, New York, NY 10012 - United States)Matthew D. ZeilerCIMS - Courant Institute of Mathematical Sciences [New York] (251, Mercier Street, New York, NY 10012 - United States)Sixin ZhangIRIT-APO - Algorithmes Parallèles et Optimisation (Institut de recherche en informatique de Toulouse - IRIT 2 rue Charles Camichel 31071 Toulouse Cedex 7 - France)Yann LecunCIMS - Courant Institute of Mathematical Sciences [New York] (251, Mercier Street, New York, NY 10012 - United States)Rob FergusCIMS - Courant Institute of Mathematical Sciences [New York] (251, Mercier Street, New York, NY 10012 - United States)
2013en
Ochiq kirishYashil

Litsenziya: cc-by

ABI

Annotatsiya

We introduce DropConnect, a generalization of Dropout (Hinton et al., 2012), for regular-izing large fully-connected layers within neu-ral networks. When training with Dropout, a randomly selected subset of activations are set to zero within each layer. DropCon-nect instead sets a randomly selected sub-set of weights within the network to zero. Each unit thus receives input from a ran-dom subset of units in the previous layer. We derive a bound on the generalization per-formance of both Dropout and DropCon-nect. We then evaluate DropConnect on a range of datasets, comparing to Dropout, and show state-of-the-art results on several image recognition benchmarks by aggregating mul-tiple DropConnect-trained models. 1.

Hali tarjima qilinmagan

Identifikatorlar

Iqtiboslar va manbalar

3 ta iqtibos0 ta foydalanilgan manba