Machine Learning (ML)
A field of artificial intelligence focused on building models that learn from data to make predictions or decisions.
Types of Machine Learning
- Supervised Learning - Uses labeled data for training.
- Unsupervised Learning - Finds patterns in unlabeled data.
- Reinforcement Learning - Learns via rewards and penalties.
Example
Spam email classification using supervised learning.
Manifold Learning
A type of nonlinear dimensionality reduction technique that assumes data lies on a lower-dimensional manifold.
Types of Manifold Learning
- t-SNE - Visualizes high-dimensional data.
- Isomap - Preserves geodesic distances.
Example
Used in visualizing high-dimensional word embeddings.
Markov Chain
A stochastic model describing a sequence of possible events where the probability of each event depends only on the previous state.
Types of Markov Models
- Discrete-Time Markov Chain - Has fixed time steps.
- Continuous-Time Markov Chain - Evolves continuously over time.
Example
Used in speech recognition systems.
Markov Decision Process (MDP)
A mathematical framework used in reinforcement learning to model decision-making.
Components of MDP
- State Space - Set of possible states.
- Action Space - Set of possible actions.
- Transition Model - Probability distribution of state changes.
Example
Used in robot navigation problems.
Maximum Likelihood Estimation (MLE)
A statistical method for estimating the parameters of a model by maximizing the likelihood function.
Types of Estimation
- Point Estimation - Provides a single best guess.
- Interval Estimation - Provides a range of possible values.
Example
Used in fitting Gaussian distributions to data.
Mean Absolute Error (MAE)
A regression loss function that calculates the average absolute difference between actual and predicted values.
Types of Regression Errors
- MAE - Measures absolute differences.
- RMSE - Penalizes large errors more heavily.
Example
Used to evaluate house price prediction models.
Mean Squared Error (MSE)
A regression loss function that calculates the average squared difference between actual and predicted values.
Types of Regression Metrics
- MSE - Penalizes large errors more than MAE.
- R-Squared - Measures the proportion of variance explained.
Example
Used in deep learning models for loss calculation.
Types of Meta-Learning
- Few-Shot Learning - Learns from a small number of examples.
- Hyperparameter Optimization - Learns the best model parameters.
Example
Used in transfer learning and model adaptation.
Minimax Algorithm
An optimization algorithm used in game theory and adversarial machine learning.
Types of Minimax Algorithms
- Alpha-Beta Pruning - Improves efficiency by eliminating unnecessary branches.
- Expectimax - Used in probabilistic games.
Example
Used in chess AI to make optimal moves.
Monte Carlo Methods
A class of algorithms that rely on repeated random sampling to obtain numerical results.
Types of Monte Carlo Methods
- Monte Carlo Integration - Used for numerical integration.
- Markov Chain Monte Carlo (MCMC) - Used for probabilistic modeling.
Example
Used in reinforcement learning for value estimation.
Multi-Class Classification
A classification task where there are more than two possible output classes.
Types of Multi-Class Classification
- One-vs-All (OvA) - Trains one classifier per class.
- One-vs-One (OvO) - Trains a classifier for each class pair.
Example
Used in digit recognition with MNIST dataset.
Multi-Label Classification
A classification task where each instance can belong to multiple classes simultaneously.
Types of Multi-Label Classification
- Binary Relevance - Trains one classifier per label.
- Label Powerset - Transforms the problem into a multi-class task.
Example
Used in tagging images with multiple objects.
Multi-Layer Perceptron (MLP)
A class of neural networks composed of multiple layers of neurons with nonlinear activation functions.
Types of Neural Networks
- Feedforward Neural Network - Information moves in one direction.
- Recurrent Neural Network (RNN) - Includes cycles for sequential processing.
Example
Used in credit scoring and medical diagnosis.
Multi-Task Learning (MTL)
A machine learning paradigm where a model is trained to solve multiple related tasks simultaneously.
Types of Multi-Task Learning
- Hard Parameter Sharing - Shared hidden layers for all tasks.
- Soft Parameter Sharing - Each task has its own model but shares information.
Example
Used in natural language processing for text classification and sentiment analysis.
Types of Mutual Information
- Shannon Mutual Information - Measures shared information using entropy.
- Conditional Mutual Information - Measures shared information given a third variable.
Example
Used in selecting the most informative features in a dataset.
Maximum Entropy Model
A probabilistic model that makes the least assumptions while satisfying known constraints.
Types of Maximum Entropy Models
- Log-Linear Models - Uses exponentiated linear functions.
- Conditional Maximum Entropy - Models conditional probabilities.
Example
Used in natural language processing for part-of-speech tagging.
Minimum Description Length (MDL)
A principle stating that the best model is the one that compresses data most efficiently.
Types of MDL
- Two-Part Code - Balances model complexity and accuracy.
- Normalized MDL - Uses probability distributions for encoding.
Example
Used in model selection and feature engineering.
Manifold Hypothesis
A hypothesis stating that high-dimensional data lie on a low-dimensional manifold.
Types of Dimensionality Reduction
- PCA - Projects data onto principal components.
- t-SNE - Preserves local structure for visualization.
Example
Used in feature engineering and data compression.
Model Compression
A set of techniques to reduce the size of machine learning models while preserving accuracy.
Types of Model Compression
- Pruning - Removes unimportant weights.
- Quantization - Reduces precision of model parameters.
Example
Used in deploying deep learning models on mobile devices.
Model Evaluation
The process of assessing the performance of a machine learning model.
Types of Model Evaluation Metrics
- Accuracy - Measures correct predictions.
- Precision & Recall - Evaluate class-specific performance.
Example
Used in comparing different classifiers on the same dataset.
Model Generalization
The ability of a machine learning model to perform well on unseen data.
Types of Generalization
- Interpolation - Performance on data within the training distribution.
- Extrapolation - Performance on data outside the training distribution.
Example
Used in deep learning models to ensure they do not overfit to training data.
Model Regularization
A technique to prevent overfitting by adding constraints to a model.
Types of Regularization
- L1 Regularization (Lasso) - Shrinks some weights to zero.
- L2 Regularization (Ridge) - Reduces the magnitude of weights.
Example
Used in logistic regression and neural networks.
Monte Carlo Methods
A class of algorithms that use randomness to solve deterministic problems.
Types of Monte Carlo Methods
- Monte Carlo Integration - Used for numerical approximations.
- Monte Carlo Tree Search - Used in AI for decision-making.
Example
Used in reinforcement learning and probabilistic simulations.
Manifold Learning
A type of unsupervised learning for dimensionality reduction.
Types of Manifold Learning
- t-SNE - Preserves local structure in high-dimensional data.
- Isomap - Uses geodesic distances for dimensionality reduction.
Example
Used in visualization of high-dimensional datasets.
Metric Learning
A technique in machine learning where the model learns a similarity function between data points.
Types of Metric Learning
- Supervised Metric Learning - Uses labeled data to define distances.
- Unsupervised Metric Learning - Learns similarity without labels.
Example
Used in face recognition systems.
Mixture of Experts (MoE)
A machine learning model that combines multiple specialized sub-models.
Types of MoE
- Soft MoE - Uses probability-based selection of experts.
- Hard MoE - Assigns each input to a single expert.
Example
Used in large-scale language models like GPT.
Mini-Batch Gradient Descent
A gradient descent optimization method that updates model weights using small batches of training data.
Types of Gradient Descent
- Batch Gradient Descent - Uses the entire dataset for updates.
- Stochastic Gradient Descent - Updates weights per sample.
Example
Used in training deep learning models efficiently.
Model Drift
A phenomenon where a trained model's performance degrades over time due to changing data distributions.
Types of Model Drift
- Concept Drift - The relationship between input and output changes.
- Covariate Shift - The input data distribution changes.
Example
Seen in fraud detection models as fraud patterns evolve.
Model Selection
The process of choosing the best machine learning model based on evaluation metrics.
Types of Model Selection Techniques
- Cross-Validation - Splits data into training and validation sets.
- Grid Search - Tunes hyperparameters by testing multiple configurations.
Example
Used in optimizing machine learning pipelines.
Markov Chains
A stochastic process where the next state depends only on the current state.
Types of Markov Chains
- Discrete-Time Markov Chain - Works in discrete steps.
- Continuous-Time Markov Chain - Evolves continuously over time.
Example
Used in language models and reinforcement learning.
Model Interpretability
The ability to understand and explain a machine learning model’s predictions.
Types of Interpretability
- Local Interpretability - Explains individual predictions.
- Global Interpretability - Explains overall model behavior.
Example
SHAP and LIME are used to interpret black-box models.
Multi-Label Classification
A classification task where each instance can belong to multiple categories.
Types of Multi-Label Approaches
- Binary Relevance - Treats each label as a separate binary classification.
- Classifier Chains - Uses dependencies between labels for prediction.
Example
Used in text categorization where an article can have multiple topics.
Multi-Task Learning (MTL)
A learning paradigm where a model is trained to perform multiple related tasks simultaneously.
Types of MTL
- Hard Parameter Sharing - Shares parameters across tasks.
- Soft Parameter Sharing - Uses task-specific parameters with some shared knowledge.
Example
Used in NLP models that handle translation, summarization, and sentiment analysis together.
Memory-Augmented Neural Networks (MANNs)
Neural networks enhanced with external memory to store and retrieve information efficiently.
Types of MANNs
- Neural Turing Machines (NTM) - Mimics traditional computers with read/write memory.
- Differentiable Neural Computers (DNC) - Improves memory access patterns.
Example
Used in reinforcement learning and question-answering systems.
Model Compression
Techniques used to reduce the size and computational requirements of machine learning models.
Types of Model Compression
- Quantization - Reduces precision of weights.
- Pruning - Removes unnecessary connections.
Example
Used in deploying deep learning models on mobile devices.
Types of Meta-Learning
- Gradient-Based Meta-Learning - Learns optimal model initialization.
- Memory-Based Meta-Learning - Uses past experiences for learning.
Example
Used in few-shot learning tasks.
Multi-Modal Learning
A method where models learn from multiple types of data, such as text, images, and audio.
Types of Multi-Modal Fusion
- Early Fusion - Combines raw features from different modalities.
- Late Fusion - Combines separate model outputs.
Example
Used in AI assistants that process speech and text together.
Multi-Armed Bandit Problem
A decision-making problem where an agent must balance exploration and exploitation.
Types of Multi-Armed Bandits
- ε-Greedy Algorithm - Chooses the best action most of the time but explores randomly.
- Upper Confidence Bound (UCB) - Selects actions based on uncertainty estimates.
Example
Used in online advertising and A/B testing.
Model Deployment
The process of integrating a trained machine learning model into a production environment.
Types of Deployment
- Batch Deployment - Processes data in intervals.
- Online Deployment - Responds in real-time to inputs.
Example
Used in recommendation systems for streaming services.
Types of Mutual Information
- Discrete Mutual Information - Used for categorical variables.
- Continuous Mutual Information - Used for continuous variables.
Example
Used in feature selection and information theory.
Machine Learning (ML)
ML is a subset of AI that enables machines to learn patterns from data and make predictions or decisions without explicit programming.
Types of ML
- Supervised Learning
- Unsupervised Learning
- Reinforcement Learning
Example
Spam detection in emails using classification models.
Deep Learning (DL)
DL is a subset of ML that uses artificial neural networks to process complex data and perform high-level computations.
Example
Image recognition in self-driving cars.
Generative AI (Gen AI)
Gen AI refers to AI models that generate new content, including text, images, and code, using trained knowledge bases.
Example
AI models like ChatGPT and Stable Diffusion that generate text and images.