Gradient Descent is the backbone of optimization and deep learning. In this blog post, we dig into the math behind gradient descent, Polyak's heavy ball method, and Nesterov's accelerated gradient descent method. We perform these methods on an example function and see them in action!
Behind important things like machine learning, finance, and operations research lies an important concept: optimization. Optimization is the process of finding the best solution to a problem from all possible solutions. In this blog post, we will discuss the basics of unconstrained optimization, a fundamental concept in optimization theory. We will specifically discuss a few necessary and sufficient conditions for optimality, and consider a few examples.
Maximum likelihood estimation (MLE) stands as a cornerstone in the realm of statistical inference, offering a powerful method for estimating the parameters of a probability distribution. In this blog post, we explore the concept of MLE, its mathematical underpinnings, and consider a few examples.
In this blog post we talk about the deep reinforcement learning algorithm proximal policy optimization.
In this blog post we begin a brief introduction to reinforcement learning by talking about the Markov decision process and the Bellman equation.
In this blog post we build upon previous blog posts. We will be dicussing slice surfaces and slice genus.
In this blog post we build upon a previous post. We will be discussing braids, Alexander's theorem, and Markov's theorem.
Knot theory is a branch of topology that studies the properties of mathematical knots. In this blog post, we will briefly explore the basics of knot theory and define some common terms used in the field. This blog post is a high level introduction to knot theory that is meant to not only be accessible to a general audience, but also lays a foundation for more technical topics.
Kernel Density Estimators (KDEs) are statistical tools used for estimating the probability density function (PDF) of a continuous random variable. They provide a smooth, non-parametric representation of the underlying distribution of data by placing a kernel (smooth, symmetric function) at each data point and summing up these kernels to form a continuous probability density estimate. KDEs are particularly useful in exploratory data analysis and visualizing the distribution of data without making assumptions about the underlying parametric form.
XGBoost is a powerful machine learning algorithm that is used in many data science competitions, combining ideas from gradient boosting, random forests, and other important topics. In this blog post, we'll discuss the math behind XGBoost, exploring its inner workings and understanding how it stands out among other machine learning algorithms. We'll also shed light on the importance of regularization and draw comparisons between XGBoost and random forests through a practical Python example.
Gradient boosting is a clever way to improve classification and regression tasks. The main idea is to perform gradient descent to minimize the loss function of our problem. In this post, we will discuss the math behind gradient boosting and work through an example comparing gradient boosting and random forests.
Random forests are a powerful machine learning tool that can be used for classification and regression. In this blog post, we will discuss how random forests work for classification, and we will work through an example of building a random forest for classification.
Decision trees for classification are foundational for many popular machine learning algorithms such as random forests and XGBoost. In this article, we learn the math behind decision trees and construct an example tree using the Gini index.
Feature selection is a very important part of machine learning. In this article, we will explore feature selection using $L^1$ regularization and stepwise feature removal, and talk about how $L^1$ regularization often outperforms stepwise feature removal.
An introduction to Akaike information criterion (AIC), and Bayesian information criterion (BIC).
An introduction to linear regression, model accuracy, and regularization.
A cool visualization of the prime spiral in Python using Plotly.
An introductory tutorial into CVXPY and convex linear optimization with two worked through examples.
A quick tutorial on how to create one-factor ANOVA tables in R without any external packages.
