SCIKIT‑LEARN
ML CHEATSHEET.

Installation

# Via pip
pip install -U scikit-learn

# Via conda
conda install scikit-learn

# Check version
import sklearn
print(sklearn.__version__)

---

Data Preparation

Train/Test Split

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

Feature Scaling

from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler

scaler = StandardScaler()          # mean=0, std=1
X_scaled = scaler.fit_transform(X)

scaler = MinMaxScaler()            # range [0,1]
X_scaled = scaler.fit_transform(X)

scaler = RobustScaler()            # robust to outliers
X_scaled = scaler.fit_transform(X)

Encoding Categorical Variables

from sklearn.preprocessing import OneHotEncoder, LabelEncoder

# One‑hot for features
encoder = OneHotEncoder(sparse_output=False)
X_encoded = encoder.fit_transform(X_categorical)

# Label encode for target
le = LabelEncoder()
y_encoded = le.fit_transform(y)

# ColumnTransformer for mixed types
from sklearn.compose import ColumnTransformer
preprocessor = ColumnTransformer([
    ('num', StandardScaler(), numerical_cols),
    ('cat', OneHotEncoder(), categorical_cols)
])

Imputing Missing Values

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(strategy='mean')      # or median, most_frequent, constant
X_imputed = imputer.fit_transform(X)

---

Supervised Learning Models

Classification

from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNB

models = {
    'Logistic': LogisticRegression(random_state=42),
    'Decision Tree': DecisionTreeClassifier(random_state=42),
    'Random Forest': RandomForestClassifier(n_estimators=100, random_state=42),
    'Gradient Boosting': GradientBoostingClassifier(random_state=42),
    'SVM': SVC(kernel='rbf', random_state=42),
    'KNN': KNeighborsClassifier(n_neighbors=5)
}

Regression

from sklearn.linear_model import LinearRegression, Ridge, Lasso, ElasticNet
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.svm import SVR
from sklearn.neighbors import KNeighborsRegressor

models = {
    'Linear': LinearRegression(),
    'Ridge': Ridge(alpha=1.0),
    'Lasso': Lasso(alpha=1.0),
    'ElasticNet': ElasticNet(alpha=1.0, l1_ratio=0.5),
    'Decision Tree': DecisionTreeRegressor(random_state=42),
    'Random Forest': RandomForestRegressor(n_estimators=100, random_state=42),
    'Gradient Boosting': GradientBoostingRegressor(random_state=42),
}

---

Unsupervised Learning

Clustering

from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering

kmeans = KMeans(n_clusters=3, random_state=42)
labels = kmeans.fit_predict(X)

dbscan = DBSCAN(eps=0.5, min_samples=5)
labels = dbscan.fit_predict(X)

agg = AgglomerativeClustering(n_clusters=3)
labels = agg.fit_predict(X)

Dimensionality Reduction

from sklearn.decomposition import PCA
from sklearn.manifold import TSNE

pca = PCA(n_components=2)
X_pca = pca.fit_transform(X)

tsne = TSNE(n_components=2, random_state=42)
X_tsne = tsne.fit_transform(X)

---

Model Evaluation

Classification Metrics

from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score,
    confusion_matrix, classification_report, roc_auc_score, roc_curve
)

y_pred = model.predict(X_test)

accuracy_score(y_test, y_pred)
precision_score(y_test, y_pred, average='weighted')
recall_score(y_test, y_pred, average='weighted')
f1_score(y_test, y_pred, average='weighted')

confusion_matrix(y_test, y_pred)
print(classification_report(y_test, y_pred))

# For binary classification
roc_auc_score(y_test, model.predict_proba(X_test)[:, 1])

Regression Metrics

from sklearn.metrics import (
    mean_squared_error, mean_absolute_error, r2_score
)

y_pred = model.predict(X_test)

mse = mean_squared_error(y_test, y_pred)
mae = mean_absolute_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

Cross‑Validation

from sklearn.model_selection import cross_val_score, cross_validate

scores = cross_val_score(model, X, y, cv=5, scoring='accuracy')
print(f'Mean accuracy: {scores.mean():.3f} ± {scores.std():.3f}')

# Multiple metrics
scores = cross_validate(model, X, y, cv=5,
    scoring=['accuracy', 'precision_macro', 'recall_macro'],
    return_train_score=True)

---

Hyperparameter Tuning

Grid Search

from sklearn.model_selection import GridSearchCV

param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [None, 10, 20],
    'min_samples_split': [2, 5, 10]
}

grid_search = GridSearchCV(
    RandomForestClassifier(random_state=42),
    param_grid,
    cv=5,
    scoring='accuracy',
    n_jobs=-1
)
grid_search.fit(X_train, y_train)

print(grid_search.best_params_)
print(grid_search.best_score_)

Random Search

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import randint, uniform

param_dist = {
    'n_estimators': randint(50, 300),
    'max_depth': [None, 10, 20, 30],
    'min_samples_split': randint(2, 20),
    'max_features': ['sqrt', 'log2', None]
}

random_search = RandomizedSearchCV(
    RandomForestClassifier(random_state=42),
    param_distributions=param_dist,
    n_iter=50,
    cv=5,
    scoring='accuracy',
    n_jobs=-1,
    random_state=42
)
random_search.fit(X_train, y_train)

Bayesian Optimization (Optuna / scikit-optimize)

# Optuna example
import optuna

def objective(trial):
    n_estimators = trial.suggest_int('n_estimators', 50, 300)
    max_depth = trial.suggest_int('max_depth', 5, 30)
    min_samples_split = trial.suggest_int('min_samples_split', 2, 20)
    clf = RandomForestClassifier(
        n_estimators=n_estimators,
        max_depth=max_depth,
        min_samples_split=min_samples_split,
        random_state=42
    )
    return cross_val_score(clf, X_train, y_train, cv=3).mean()

study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=50)
print(study.best_params)

---

Pipelines

Chain preprocessing and modelling steps.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', RandomForestClassifier(random_state=42))
])

pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)

# With column transformer (mixed types)
preprocessor = ColumnTransformer([
    ('num', StandardScaler(), numerical_cols),
    ('cat', OneHotEncoder(), categorical_cols)
])

pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('classifier', LogisticRegression(random_state=42))
])

---

Feature Selection

from sklearn.feature_selection import SelectKBest, f_classif, RFE

# Univariate selection
selector = SelectKBest(f_classif, k=10)
X_selected = selector.fit_transform(X, y)

# Recursive Feature Elimination
estimator = RandomForestClassifier(random_state=42)
rfe = RFE(estimator, n_features_to_select=10)
X_selected = rfe.fit_transform(X, y)

# Feature importance from tree models
model = RandomForestClassifier(random_state=42)
model.fit(X, y)
importances = model.feature_importances_

---

Model Persistence

# Using pickle
import pickle
with open('model.pkl', 'wb') as f:
    pickle.dump(model, f)

with open('model.pkl', 'rb') as f:
    model = pickle.load(f)

# Using joblib (more efficient for large models)
import joblib
joblib.dump(model, 'model.joblib')
model = joblib.load('model.joblib')

---

Common Utilities

Make a Pipeline with Custom Transformers

from sklearn.base import BaseEstimator, TransformerMixin

class CustomTransformer(BaseEstimator, TransformerMixin):
    def __init__(self, param=1):
        self.param = param
    def fit(self, X, y=None):
        return self
    def transform(self, X):
        return X ** self.param

Generate Synthetic Data

from sklearn.datasets import make_classification, make_regression, make_blobs

X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)
X, y = make_regression(n_samples=1000, n_features=20, noise=0.1, random_state=42)
X, y = make_blobs(n_samples=500, centers=3, n_features=2, random_state=42)

---

Model Selection Strategy

1. Define problem – classification, regression, clustering.
2. Split data – train/test (and validation).
3. Preprocess – scale, encode, impute.
4. Start with simple baseline – LogisticRegression, LinearRegression, etc.
5. Evaluate – cross‑validation with appropriate metrics.
6. Try more complex models – RandomForest, XGBoost, etc.
7. Tune hyperparameters – GridSearchCV / RandomizedSearchCV.
8. Ensemble – voting, stacking, bagging.
9. Test on holdout set – final evaluation.

---

Best Practices

• Always split data – before preprocessing to avoid data leakage.
• Use pipelines – to encapsulate preprocessing and model.
• Scale features – for models that rely on distance (SVM, KNN, linear models).
• Feature engineering – domain knowledge often beats complex models.
• Start simple – baseline is important.
• Use cross‑validation – to get robust performance estimates.
• Monitor for overfitting – compare train/validation scores.
• Use feature importance – for interpretability.
• Check class imbalance – use stratification or class_weight.
• Log and version experiments – use MLflow, Weights & Biases, or simple logging.
• Keep models simple – if performance is sufficient.
• Use n_jobs=-1 – for parallelisation when possible.

---

Common Pitfalls

• Data leakage – fitting scalers on full data before split.
• Overfitting – not using cross‑validation or validation set.
• Ignoring class imbalance – use balanced metrics (F1, ROC‑AUC).
• Not tuning hyperparameters – default parameters are rarely optimal.
• Using accuracy on imbalanced data – misleading.
• Not handling missing values – sklearn models require no NaNs.

---

Quick Model Selection Reference

Task	Model	When to Use
Classification	LogisticRegression	Linear, binary/multiclass, interpretable
Classification	RandomForestClassifier	Non‑linear, good default, feature importance
Classification	GradientBoostingClassifier	High performance, slower training
Classification	SVC (RBF)	Non‑linear, small datasets
Classification	KNeighborsClassifier	Simple, non‑parametric, small datasets
Regression	LinearRegression	Linear relationships, interpretable
Regression	Ridge/Lasso	Regularisation, feature selection (Lasso)
Regression	RandomForestRegressor	Non‑linear, good default
Regression	GradientBoostingRegressor	High performance
Clustering	KMeans	Compact, spherical clusters
Clustering	DBSCAN	Density‑based, arbitrary shapes
Dimensionality	PCA	Linear, interpretable components
Dimensionality	t‑SNE	Visualisation, non‑linear