Kobus Esterhuysen – Data Science Blog


I specialize in Reinforcement Learning, Sequential Decision Analytics, Data Science, Decision Science, AI, Stochastic Optimization, and Machine Learning. I help digitally aware companies achieve more efficient, and even optimal, operations. I have been successful because of my system-oriented focus, passion, extensive quantitative background in multiple industries, and having worked for Fortune 500 companies, small firms, as well as non-profits. What is unique about my approach is that I rely on a three-fold perspective on client problems deriving from three master’s degrees in the areas of engineering, business, and data science.
Portfolio of Projects | LinkedIn | Consulting Site

Deriving Secret Sales Volumes with Bayesian Inference

Using RxInfer to decouple two signals
Retail Industry
Bayesian Inference
Active Inference
RxInfer
Julia
This use case of active inference comes from the retail industry. We use the published sales volume of a company over 250 time intervals. They state that this volume is…
Oct 12, 2024
Kobus Esterhuysen

Gaussian Multivariate Bayesian State Inference

Using RxInfer to estimate the hidden state vector
Aerospace Industry
Bayesian Inference
Active Inference
RxInfer
Julia
Multivariate Bayesian State estimation is performed using the RxInfer Julia package. The hidden states are a vector of three Gaussians.
Oct 11, 2024
Kobus Esterhuysen

Using RxInfer to perform Bayesian regression

Learning parameters for a linear regression model
Medical Industry
Bayesian Inference
Bayesian Regression
RxInfer
Julia
RxInfer.jl is a Julia package designed for automated Bayesian inference using reactive message passing on factor graphs[1]. This powerful tool offers several advantages over…
Oct 1, 2024
Kobus Esterhuysen

Powell Unified Framework – Code Conventions

Some Python code conventions that make it easier to map between the mathematics and the code
Powell Unified Framework
Reinforcement Learning
Python
Dr. Warren Powell’s unified framework shows great promise for unifying the formalisms of at least a dozen different fields. Using his framework enables easier access to…
Apr 21, 2023
Kobus Esterhuysen

Temporal Difference 4 – Control with Q-Learning in Reinforcement Learning

Find the optimal policy using Q-Learning
Reinforcement Learning
TD
Q_Learning
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Feb 4, 2022
Kobus Esterhuysen

Temporal Difference 3 – Control with Sarsa in Reinforcement Learning

Find the optimal policy using Sarsa
Reinforcement Learning
TD
Sarsa
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Feb 3, 2022
Kobus Esterhuysen

Temporal Difference 2 – Estimation of the Action-value Function in Reinforcement Learning

Find action values under a given policy
Reinforcement Learning
TD
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Feb 2, 2022
Kobus Esterhuysen

Temporal Difference 1 – Estimation of the State-value Function in Reinforcement Learning

Find state values under a given policy
Reinforcement Learning
TD
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Feb 1, 2022
Kobus Esterhuysen

Monte Carlo 6 – Off-Policy Control with Importance Sampling in Reinforcement Learning

Find the optimal policy using Weighted Importance Sampling
Reinforcement Learning
MC
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Jan 27, 2022
Kobus Esterhuysen

Monte Carlo 5 – Off-Policy Estimation of the Action-value Function in Reinforcement Learning

Find action values under a given policy
Reinforcement Learning
MC
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Jan 26, 2022
Kobus Esterhuysen

Monte Carlo 4 – Control with Epsilon-Soft Policies in Reinforcement Learning

Find the optimal policy using epsilon-greeady policies
Reinforcement Learning
MC
Control
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Jan 22, 2022
Kobus Esterhuysen

Monte Carlo 3 – Control with Exploring Starts in Reinforcement Learning

Find the optimal policy using exploring starts
Reinforcement Learning
MC
Control
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Jan 20, 2022
Kobus Esterhuysen

Monte Carlo 2 – Estimation of the Action-value Function in Reinforcement Learning

Find action values under a given policy
Reinforcement Learning
MC
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Jan 19, 2022
Kobus Esterhuysen

Monte Carlo 1 – Estimation of the State-value Function in Reinforcement Learning

Find state values under a given policy
Reinforcement Learning
MC
OpenAI Gym
In a Markov Decision Process (Figure 1) the agent and environment interacts continuously.
Jan 14, 2022
Kobus Esterhuysen

Control of a Cart-Pole Dynamic System with Ray RLlib

Reinforcement Learning (RL) to control the balancing of a pole on a moving cart
Control
Reinforcement Learning
Ray RLlib
The cart-pole problem can be considered as the “Hello World” problem of Reinforcement Learning (RL). It was described by Barto (1983). The physics of the system is as follows:
Nov 29, 2021
Kobus Esterhuysen

SageMaker Regression on Abalone Data

Estimation of the age of an abalone using readily available measurements
Fishing Industry
Regression
AWS-Sagemaker
Linear-Learner
What is Abalone? It is a large marine gastropod mollusk that lives in coastal saltwater and is a member of the Haliotidae family. Abalone is often found around the waters of…
May 21, 2021
Kobus Esterhuysen

Semantic Segmentation on Oxford-IIIT Pet Dataset

Semantic Segmentation with Amazon SageMaker
Computer Vision
Semantic Segmentation
SageMaker
ResNet-50
FCN
Python
There are 4 prominent computer vision tasks: 1. Image Classification 2. Object Detection 3. Semantic Segmentation 4. Instance Segmentation
Apr 8, 2021
Kobus Esterhuysen

Training an Agent to play Pong using Reinforcement Learning

Using a video image for state information, RL is used to teach an Agent to play Pong
Gaming Industry
Reinforcement Learning
DQN
OpenAI Gym
TensorFlow
TF-Agents
Python
The Atari 2600 is a home video game console from Atari, Inc. It was released in 1977. It is credited with popularizing the use of microprocessor-based hardware and games…
Feb 19, 2021
Kobus Esterhuysen

Reinforcement Learning helps with Investor Decisions

RL is used to provide optimal investment decisions for a typical home owner
Investment Industry
Reinforcement Learning
DDPG
OpenAI Gym
TensorFlow
TF-Agents
Python
This post describes a simple way of how Reinforcement Learning (RL) can be applied to the problem of “homeowner investment”. This does not represent a sophisticated model.…
Feb 8, 2021
Kobus Esterhuysen

Control of a Cart-Pole Dynamic System with TF-Agents

Reinforcement Learning (RL) to control the balancing of a pole on a moving cart
Control
Reinforcement Learning
TensorFlow
TF-Agents
Python
The cart-pole problem can be considered as the “Hello World” problem of Reinforcement Learning (RL). It was described by Barto (1983). The physics of the system is as follows:
Jan 19, 2021
Kobus Esterhuysen

Social Media Marketing in the Aerospace Industry

The Use of Social Media Marketing in the Aerospace Industry
Marketing Industry
social media marketing
Deep Learning
word embedding
sentence embedding
This report considers the role of social media marketing in the aerospace industry. What is not always obvious from casual readings on this subject is the highly technical…
Jun 17, 2019
Kobus Esterhuysen

Response to Cathy O’Neil’s Weapons of Math Destruction

Bias in data leads to bias in model behavior
Data Ethics
After reading Cathy O’Neil’s Weapons of Math Destruction, I put down a few comments. Overall, I was impressed by O’Neil’s insight, honesty, sense of responsibility, and…
Dec 16, 2017
Kobus Esterhuysen

The Cathedral and the Bazaar for Data Science

Ideas from the Open Source Movement is still relevant for Data Science
Open Source
The title, “The Cathedral and the Bazaar”, captures the essence of the open source movement: Do not build software using exclusive, highly compensated, highly qualified…
Dec 16, 2017
Kobus Esterhuysen

Resampling Methods

Departing from strict theoretical principles, basing inferences on statistics obtained from repeated resampling of the same sample
Statistical Methods
Resampling
Parametric procedures compare observed statistics to sampling distributions based on solid theoretical principles. However, there are always conditions on the data before…
Jun 21, 2017
Kobus Esterhuysen

Welcome To My Data Science Blog

Welcome! This is a quick introduction …
Jan 13, 2017
Kobus Esterhuysen
No matching items