Published on Sat May 20 2017

Speedup from a different parametrization within the Neural Network algorithm

Michael F. Zimmer

A different parametrization of the hyperplanes is used in the neural network algorithm. As demonstrated on several autoencoder examples it significantly outperforms the usual parametriation.

0
0
0
Abstract

A different parametrization of the hyperplanes is used in the neural network algorithm. As demonstrated on several autoencoder examples it significantly outperforms the usual parametrization, reaching lower training error values with only a fraction of the number of epochs. It's argued that it makes it easier to understand and initialize the parameters.

Mon Oct 30 2017
Machine Learning
Empirical analysis of non-linear activation functions for Deep Neural Networks in classification tasks
We provide an overview of several non-linear activation functions in a neural network architecture that have proven successful in many machine learning applications. We conduct an empirical analysis on the effectiveness of using these function on the MNIST classification task. We conclude by providing a list of the optimal neural network architecture.
0
0
0
Mon May 02 2016
Machine Learning
Some Insights into the Geometry and Training of Neural Networks
Neural networks have been successfully used for classification tasks in arapidly growing number of practical applications. Despite their popularity, there are still many aspects of training and classification that are not well understood.
0
0
0
Sun Jan 14 2018
Machine Learning
Fix your classifier: the marginal value of training the last weight layer
Neural networks are commonly used as models for classification for a wide variety of tasks. We argue that this classifier can be fixed, up to a global scale constant, with little or no loss of accuracy for most tasks.
0
0
0
Wed May 29 2019
Machine Learning
Intrinsic dimension of data representations in deep neural networks
Deep neural networks progressively transform their inputs across multiple layers. What are the geometrical properties of the representations learned by these networks? We study the intrinsic dimensionality of data-representations.
1
0
0
Sun Jun 24 2012
Machine Learning
Practical recommendations for gradient-based training of deep architectures
This chapter is meant as a practical guide with recommendations for some of the most commonly used hyper-parameters. Overall, it describes elements of the practice used to successfully and efficiently train and debug large-scale and often deep neural networks.
1
0
1
Mon Dec 14 2015
Neural Networks
On non-iterative training of a neural classifier
A new algorithm, was discovered, which separates points in n-dimension by planes in such a manner that no two points are left un-separated by at least one plane. By using this new algorithm we show that there are two ways of classification by a neural network, for a large
0
0
0