data science · machine learning

Deep Learning – NLP

https://zhuanlan.zhihu.com/p/49271699 Home https://jalammar.github.io/ Neural Language Model predict the next work, replace HMM, RNN LSTM different architectures stateful LSTM: memorize last batch. dependent stateless LSTM: update parameter in batch one, when batch two, initialize hidden states and cell states to zero. batch to batch. independent in different batches Word2Vec: CBOW, skip-grams | Glove (cannot solve the… Continue reading Deep Learning – NLP

data science · machine learning

Interpretability of ML

https://github.com/jphall663/awesome-machine-learning-interpretability https://christophm.github.io/interpretable-ml-book/index.html Global Interpretability Partial Dependence and Partial Dependence Plot (PDP) Individual Conditional Expectation (ICE) Total and two-way H Statistics Global Feature importance using permutation Global surrogate model Local Interpretability Local Interpretable Model-agnostic Explanations (LIME) Shapley additive explanation An intuitive way to understand the Shapley value is the following illustration: The feature values enter a… Continue reading Interpretability of ML

machine learning

Deep Learning Foundations

Neural Networks   Click to access lec5.pdf DNN: gradient check, learning rate, initialization(not zero, asymmetric), local minimum, plateau momentenm  (exponential average of previous gradients, pointing to the same direction), saddle point() Tips for training Regularization: norm penalties, early stopping, data augmentation, drop out (hidden units cannot co-adapt, generally used, test time:expectation, geometric average for signel… Continue reading Deep Learning Foundations

machine learning

Kinds of Machine Learning

Kinds of learning: Based on the information available: Supervised learning, Reinforcement learning, Unsupervised learning Based on the role of the learner: Passive learning, Active learning Scenarios: Membership Query Synthesis: the learner may request labels for any unlabeled instance in the input space Stream-Based Selective Sampling: it can first be sampled from the actual distribution, and then… Continue reading Kinds of Machine Learning

machine learning

Feature Engineering

Data Pre-processing(Transformation) Normalization sigmoid normalization 0-1 normalization ((bla – min(bla)) / ( max(bla) – min(bla) )) z-score Gaussian normalization (Gaussian kernel) Box-cox transformation log transformation Tukey’s Ladder of Powers   Feature Engineering image speech text time series: entropy, approximate entropy, sample entropy Data Visualization Statistics Histogram Density estimation: kernal density estimation (Parzen–Rosenblatt window) Feature Selection… Continue reading Feature Engineering

data science · machine learning

Classification

Elements of a model Objective Model structure (e.g. variables, formula, equation, parameters) Model assumptions Parameter estimates and interpretation Model fit (e.g. goodness-of-fit tests and statistics) Model selection LDA generative model model p(x|y) as multivariate Gaussian, Both classes have the same covariance matrix, Σ QDA Each class has their own Σ Naive Bayes generative model Assume the xj… Continue reading Classification