ENGIMY.IO - CHEATSHEET
ML × QUICK REFERENCE
ML
AI / ML REFERENCE

MACHINE
LEARNING ALGORITHMS.

A comprehensive reference for machine learning algorithms – supervised, unsupervised, reinforcement, evaluation metrics, and best practices. All in one place.

REFERENCE v1.0

ML Algorithms Quick Reference

Everything you need day‑to‑day – algorithms, use cases, and evaluation metrics.

ML Categories

Supervised Learning
  • Labeled data (input → output)
  • Regression – continuous output
  • Classification – categorical output
  • Examples: Linear Regression, Logistic Regression, Decision Trees, SVM, Random Forest, XGBoost, Neural Networks
Unsupervised Learning
  • Unlabeled data (find patterns)
  • Clustering – group similar points
  • Dimensionality Reduction – reduce features
  • Anomaly Detection – find outliers
  • Examples: K‑Means, DBSCAN, PCA, t‑SNE, Autoencoders
Semi‑Supervised Learning
  • Small labeled + large unlabeled
  • Self‑training, co‑training
Reinforcement Learning
  • Agent + environment (reward signals)
  • Q‑Learning, Deep Q‑Learning
  • Policy Gradients, Actor‑Critic

Regression Algorithms

Linear Regression

  • y = w₀ + w₁x₁ + w₂x₂ + ...
  • Finds linear relationship
  • Minimises MSE (Ordinary Least Squares)
  • Assumes linearity, independence, homoscedasticity
  • Pros: simple, interpretable
  • Cons: linear only, sensitive to outliers

Ridge Regression (L2 Regularisation)

  • Adds L2 penalty: λΣwᵢ²
  • Prevents overfitting
  • Shrinks coefficients but doesn't zero them

Lasso Regression (L1 Regularisation)

  • Adds L1 penalty: λΣ|wᵢ|
  • Performs feature selection
  • Zeroes out less important features

Polynomial Regression

  • y = w₀ + w₁x + w₂x² + ...
  • Capture non‑linear relationships
  • Beware of overfitting

Support Vector Regression (SVR)

  • Finds hyperplane with max margin
  • Uses ε‑insensitive loss
  • Works with kernel trick
  • Good for non‑linear data

Classification Algorithms

Logistic Regression

  • Binary classification (0/1)
  • Log‑odds: log(p/(1‑p)) = w·x
  • Output probability (sigmoid)
  • Pros: probabilistic, interpretable
  • Cons: assumes linear decision boundary

k‑Nearest Neighbours (k‑NN)

  • Memory‑based, lazy learner
  • Class predicted by majority vote of k neighbours
  • Distance metric: Euclidean, Manhattan, Minkowski
  • Pros: simple, no training
  • Cons: slow, sensitive to scale, k selection

Decision Trees (CART, ID3, C4.5)

  • Tree‑based, recursive partitioning
  • Splits based on impurity (Gini, Entropy)
  • Easily interpretable
  • Pros: interpretable, handles non‑linear
  • Cons: prone to overfitting, unstable

Random Forest

  • Ensemble of decision trees
  • Bagging (Bootstrap Aggregating)
  • Random feature selection
  • Pros: reduces overfitting, feature importance
  • Cons: less interpretable, larger

Support Vector Machines (SVM)

  • Maximises margin between classes
  • Kernel trick: linear, polynomial, RBF, sigmoid
  • Pros: effective in high dimensions
  • Cons: less interpretable, sensitive to parameters

Naïve Bayes

  • Bayes' Theorem with independence assumption
  • Types: Gaussian, Multinomial, Bernoulli
  • Pros: fast, simple, works with small data
  • Cons: strong independence assumption

XGBoost / Gradient Boosting

  • Sequential additive models
  • Gradient descent to optimise loss
  • Regularisation prevents overfitting
  • Pros: state‑of‑the‑art performance
  • Cons: complex, hyperparameter tuning

Neural Networks (MLP, CNN, RNN, LSTM)

  • Multi‑layer perceptron (fully‑connected)
  • Convolutional (image, spatial)
  • Recurrent (sequence, time series)
  • Pros: powerful, universal approximator
  • Cons: data hungry, interpretability, hyperparameters

Clustering Algorithms

K‑Means

  • Partitions data into k clusters
  • Centroid‑based, iterative
  • Minimises inertia (within‑cluster SSE)
  • Pros: fast, simple
  • Cons: k must be known, sensitive to initialisation, clusters are spherical

DBSCAN

  • Density‑based clustering
  • Finds core points, border points, noise
  • No need to specify k
  • Pros: handles arbitrary shapes, outliers
  • Cons: sensitive to ε and minPts

Hierarchical Clustering

  • Agglomerative (bottom‑up) or Divisive (top‑down)
  • Dendrogram visualisation
  • Linkage: single, complete, average, Ward
  • Pros: hierarchical structure
  • Cons: computationally expensive O(n³)

Gaussian Mixture Models (GMM)

  • Probabilistic, assumes Gaussian components
  • EM (Expectation‑Maximisation) algorithm
  • Pros: soft assignment, flexible
  • Cons: requires covariance matrix estimation

Dimensionality Reduction

Principal Component Analysis (PCA)

  • Finds directions of maximum variance
  • Linear, unsupervised
  • Projects data onto principal components
  • Pros: reduces dimensions, decorrelates
  • Cons: linear only, interpretability loss

t‑Distributed Stochastic Neighbour Embedding (t‑SNE)

  • Non‑linear, preserves local structure
  • Great for visualisation (2D/3D)
  • Pros: excellent for visualisation
  • Cons: computationally expensive, non‑deterministic

Linear Discriminant Analysis (LDA)

  • Supervised, finds axes that separate classes
  • Maximises between‑class variance
  • Also used for classification

Autoencoders (Neural)

  • Neural network, unsupervised
  • Encoder → bottleneck → decoder
  • Learns compressed representation
  • Non‑linear, can be used for denoising

Ensemble Methods

Bagging
  • Bootstrap samples, multiple models
  • Random Forest (trees)
  • Bagged Decision Trees
  • Reduces variance
Boosting
  • Sequential, weighted samples
  • AdaBoost
  • Gradient Boosting (XGBoost, LightGBM, CatBoost)
  • Reduces bias
Stacking
  • Multiple base learners + meta‑learner
  • Combines predictions
Voting
  • Hard (majority) or soft (probability)
  • Combine multiple classifiers
  • Simple ensemble

Evaluation Metrics

Regression Metrics
  • MSE – Mean Squared Error
  • RMSE – Root Mean Squared Error
  • MAE – Mean Absolute Error
  • R² – Coefficient of Determination
  • Adjusted R² – R² with penalty for features
Classification Metrics
  • Accuracy – (TP+TN)/Total
  • Precision – TP/(TP+FP)
  • Recall – TP/(TP+FN)
  • F1 Score – 2*(P*R)/(P+R)
  • AUC‑ROC – Area Under ROC Curve
  • Log Loss – Cross‑Entropy
Clustering Metrics
  • Inertia – within‑cluster SSE
  • Silhouette Score – cohesion + separation
  • Davies‑Bouldin – average similarity
  • Calinski‑Harabasz – variance ratio
Confusion Matrix
  • TP – True Positives
  • TN – True Negatives
  • FP – False Positives
  • FN – False Negatives

Confusion Matrix Structure

                    Predicted
                    +---------+---------+
              +     |    TP   |    FP   |
              A     +---------+---------+
              C     |    FN   |    TN   |
              T     +---------+---------+

Hyperparameter Tuning

  • Grid Search – exhaustive over parameter grid
  • Random Search – random samples, more efficient
  • Bayesian Optimisation – probability model
  • Cross‑Validation – k‑fold (5, 10) to evaluate
  • Early Stopping – stop training when validation stops improving

Regularisation Techniques

  • L1 (Lasso) – Σ|wᵢ|, feature selection
  • L2 (Ridge) – Σwᵢ², weight decay
  • Elastic Net – L1 + L2 combined
  • Dropout – random neuron deactivation (NN)
  • Early Stopping – stop before overfitting
  • Data Augmentation – create more data

Feature Engineering

  • Scaling – StandardScaler, MinMaxScaler, RobustScaler
  • Encoding – One‑hot, Label, Ordinal
  • Imputation – mean, median, mode, KNN, iterative
  • Feature Selection – filter, wrapper, embedded
  • Polynomial Features – interaction terms
  • Interaction Terms – x₁*x₂
  • Log Transform – handle skewness

Model Selection Guide

Problem Recommended Algorithms
Regression (linear) Linear Regression, Ridge, Lasso
Regression (non‑linear) Polynomial Regression, SVR, Random Forest, XGBoost, Neural Net
Binary Classification Logistic Regression, SVM, Random Forest, XGBoost, Neural Net
Multi‑Class Classification Logistic Regression (OvR), SVM, Random Forest, XGBoost, Neural Net
Small data (< 1000) Logistic Regression, SVM, Naïve Bayes
Large data (> 100k) XGBoost, LightGBM, Neural Net (with GPU)
Text data Naïve Bayes, Logistic Regression, SVM, Transformers
Image data CNN (ResNet, EfficientNet, etc.)
Time series ARIMA, LSTM, Prophet, XGBoost (with lags)
Clustering (spherical) K‑Means
Clustering (arbitrary shape) DBSCAN
Outlier detection Isolation Forest, DBSCAN, One‑Class SVM
Dimensionality reduction (visual) t‑SNE, UMAP
Dimensionality reduction (feature) PCA, LDA (supervised)
📌 Quick Reference
Supervised: Linear/Logistic Regression, SVM, Decision Trees, Random Forest, XGBoost, Neural Nets
Unsupervised: K‑Means, DBSCAN, PCA, t‑SNE, Autoencoders
Regression metrics: MSE, RMSE, MAE, R²
Classification metrics: Accuracy, Precision, Recall, F1, AUC‑ROC, Log Loss
Regularisation: L1 (Lasso), L2 (Ridge), Elastic Net, Dropout
Tuning: Grid Search, Random Search, Bayesian Optimisation, k‑fold CV
Ensemble: Bagging (Random Forest), Boosting (XGBoost), Stacking, Voting
← Back to All Cheatsheets