Training Machine Learning Models

The process of training an Machine Learning model involves providing an ML algorithm with training data to learn from. The learning algorithm finds patterns in the training data that maps the input data attributes to the target (what to predict)  and outputs an ML model that captures these patterns.
The term "ML model" refers to the object (instance) of an ML algorithm that is created by the training process. ML model is used to make predictions on unseen future data.

To understand the abstract overview of how machine learning works, lets take an example of fitting a linear equation:
To fit this equation, you need to predict value of m and b given (x, y) pair of inputs. Any machine learning model (algorithm) is also a kind of complex equation. Similarly, training a ML model is also like fitting a complex equation to predict model parameters given a set of input features.

Training dataset consists of pairs of input feature and target value. Once the model is trained, it learns the model parameters and can predict target values only given the feature values. Instead, training works differently for Reinforcement Learning with reward-punishment mechanism.

There are mainly two approaches to train Machine Learning Model:
  1. Closed-Form Equation Approach
  2. Iterative Optimization Approach

1. Closed-Form Equation Approach

   This approach involves solving a some kind of mathematical equation to find the coefficients of variables. To train a model, it directly computes the model parameters that best fit the model to the training set (i.e., the model parameters that minimize the cost function over the training set).

2. Iterative Optimization Approach

   This approach is also known as Gradient Descent (GD), that gradually tweaks the model parameters to minimize the cost function over the training set, eventually converging to the same set of parameters.

Online Learning Vs Offline Learning

Machine learning models can be trained in two ways: Online or Offline.
  • Online learning involves incremental training of ML model with mini-batches of dataset.
  • Offline learning involves training ML model on whole dataset between certain time intervals.
  • Online trained model is called dynamic model and offline trained model is called static model.
  • With online learning, model is up-to-date and continuously trained with incoming data.
  • Model is not up-to-date which is trained offline.

Comments