# Deep Learning

http://www.cs.toronto.edu/~bonner/courses/2014s/csc321/lectures/lec5.pdf DNN: gradient check, learning rate, initialization(not zero, asymmetric), local minimum, plateau momentenm (exponential average of previous gradients, pointing to the same direction), saddle point() Tips for training Regularization: norm penalties, early stopping, data augmentation, drop out (hidden units cannot co-adapt, generally used, test time:expectation, geometric average for signel layer) Auto encoder: sparse AE, denoising… Continue reading Deep Learning