Machine Learning Archive

What is a Support Vector?

In this post, we will develop an understanding of support vectors, discuss why we need them, how to construct them, and how they fit into the optimization objective of support vector machines. A support vector machine classifies observations by constructing a hyperplane that separates these observations. Support vectors are observations that lie on the

What is a Kernel in Machine Learning?

In this post, we are going to develop an understanding of Kernels in machine learning. We frame the problem that kernels attempt to solve, followed by a detailed explanation of how kernels work. To deepen our understanding of kernels, we apply a Gaussian kernel to a non-linear problem. Finally, we briefly discuss the construction

Regularization in Machine Learning

In this post, we introduce the concept of regularization in machine learning. We start with developing a basic understanding of regularization. Next, we look at specific techniques such as parameter norm penalties, including L1 regularization and L2 regularization, followed by a discussion of other approaches to regularization. What is Regularization? In machine learning, regularization

Introduction to the Hypothesis Space and the Bias-Variance Tradeoff in Machine Learning

In this post, we introduce the hypothesis space and discuss how machine learning models function as hypotheses. Furthermore, we discuss the challenges encountered when choosing an appropriate machine learning hypothesis and building a model, such as overfitting, underfitting, and the bias-variance tradeoff. The hypothesis space in machine learning is a set of all possible

Types of Machine Learning: A High-Level Introduction

In machine learning, we distinguish between several types and subtypes of learning and several learning techniques. Broadly speaking, machine learning comprises supervised learning, unsupervised learning, and reinforcement learning. Problems that do not fall neatly into one of these categories can often be classified as semi-supervised learning, self-supervised learning, or multi-instance learning. In supervised learning,

Logistic Regression in Python

In this post, we are going to perform binary logistic regression and multinomial logistic regression in Python using SKLearn. If you want to know how the logistic regression algorithm works, check out this post. Binary Logistic Regression in Python For this example, we are going to use the breast cancer classification dataset that comes

The Softmax Function and Multinomial Logistic Regression

In this post, we will introduce the softmax function and discuss how it can help us in a logistic regression analysis setting with more than two classes. This is known as multinomial logistic regression and should not be confused with multiple logistic regression which describes a scenario with multiple predictors. What is the Softmax

The Sigmoid Function and Binary Logistic Regression

In this post, we introduce the sigmoid function and understand how it helps us to perform binary logistic regression. We will further discuss the gradient descent for the logistic regression model (logit model). In linear regression, we are constructing a regression line of the form y = kx + d. Within the specified range,

Linear Regression in Python

This post is about doing simple linear regression and multiple linear regression in Python. If you want to understand how linear regression works, check out this post. To perform linear regression, we need Python’s package numpy as well as the package sklearn for scientific computing. Furthermore, we import matplotlib for plotting. Simple Linear Regression

The Coefficient of Determination and Linear Regression Assumptions

We’ve discussed how the linear regression model works. But how do you evaluate how good your model is. This is where the coefficient of determination comes in. In this post, we are going to discuss how to find the coefficient of determination and how to interpret it. What is the Coefficient of Determination? The