2.1 General Idea

In supervised learning, we have a set of input features \(X\) and a collection of true values \(y\) associated with each of these features. Our goal is to find some function \(f\) such that

\[ f(X) = y + \epsilon \in \mathbb{R} \] where \(\epsilon\) is some (hopefully) small error term. There are two classes of supervised learning problems, regression and classification. Regression models are trained to give a real number as the output of \(f\). Classification models are used to determine the category associated with a collection of input features.

2.2 Calendar Spread Regression Example

Let us consider an example of supervised learning applied to the Chicago Corn ZZ calendar spreads. We define the value of the calendar spread as

\[ S_{Z_i, Z_{i+1}} := P_{Z_i} - P_{Z_{i+1}} \] where \(P_{Z_i}\) represents the price of the \(Z_i\) contract. As input features, we consider the front month price \(P_{Z_i}\) and stock-to-usage percentages as determined from the monthly WASDE publication. In this example we have WASDE data going back to July 1998. The spread data between two consecutive WASDE reports is connected to the data from the earlier report. The idea is then to take the WASDE data as a true representation of the underlying fundamentals and to use these fundamental inputs to model the value of the corn ZZ spread.

As a first step in the modeling of the data, it is always a good idea to visually inspect the data. Intuitively we expect to see a decreasing spread with increasing stock-to-usage. Fundamentally we also expect this effect to become smaller with increasing stock-to-usage where the spread approaches the fully carry value as the stock-to-usage approaches infinity. This behaviour is characteristic of exponential decay. The plot below confirms our intuition. The current values of the WASDE stock-to-usage and C ZZ spread is indicated by the vertical and horizontal lines respectively. The blue curve with shadow shows a loess fit to the underlying data.

From above notice that the fit has decreasing slope and captures the general expected results. What is also clear from the above plot is that the relationship is not linear.

A fundamental idea in the machine learning literature is the concept of a training and testing set. To get a better idea of how a model will perform out of sample it is important to test your models on data that was not used in the construction of the model. A good rule of thumb is to use 20% of your dataset for out of sample testing. The above dataset consists of 199 data points. We pick 39 points at random and keep them for testing purposes. The remaining 160 data points are used in the training process.

In order to capture the general non-linear behaviour in the data, we use a kNN-algorithm. The algorithm is pretty intuitive, given an input value \(x\) for the United States Corn Stock-to-Usage, it looks at those \(k\) Nearest Neighbour datapoints in the training set which are closest to \(x\). The model then chooses the mean spread associated with each of those \(k\) data points as the output value.

An important step in training any machine learning algorithm is cross validation. Cross validation splits observations into two sets, one used for training and the other for testing. Each observation in the complete dataset belongs to one, and only one set. This is done to prevent leakage from one set into the other. If this is not done it would defeat the purpose of testing on unseen data. One of the more popular cross validation schemes is called k-fold cross validation. The figure below illustrates the k train/test splits carried out by a k-fold cross validation, where \(k=5\). The process goes as follows:

• The dataset is partitioned into \(k\) subsets
• For \(i = 1, \dots, k\)
  • The machine learning algorithm is trained on all subsets excluding \(i\)
  • The fitted machine learning algorithm is tested on \(i\)

Five fold cross validation - (Image taken from Advances in Financial Machine Learing)

The plot below shows the model represented by the dashed black line. The actual and predicted spreads are indicated by the red and blue dots respectively. Notice that the model does a pretty decent job in describing the nonlinear relationship within the data. One big drawdown of this type of model is that it does not generalize well to test data outside of the range of the training data.

The interested reader can have a look at the links below for some more examples of research problems where we used regression models

• Corn price vs Stock-to-Usage
• Soybean price vs Stock-to-Usage
• Wheat price vs Stock-to-Usage

The idea behind these three write-ups was the explore the links between global supply and demand and the outright price of the commodities. Once these models have been created we use them to test the feature sensitivity associated with individual input features. In this way, you can find possible critical points in the fundamentals which might cause significant price moves. These results are then combined with rigorous fundamental forecasting and stress testing to prod the balance sheets of the main producing and consuming nations. The stressed fundamentals are then passed to the models. If these stressed fundamentals coincide with significant price moves as predicted by the model it adds a layer of conviction to a trade. This type of quantamental approach plays a major part in how we will eventually size up a position.

2.3 Using classification to help determine bet sizing

This section will be a little bit more technical, but it illustrates an idea that is used within the Polar Star Quantitative Commodity Fund when sizing positions. Within the Polar Star Quantitative Commodity Fund, we have a strategy that takes systematic positions in calendar spreads. A calendar spread is a futures position where we take up opposing positions on the front and deferred sides of the futures curve of a single commodity. From the previous section, it should be clear that it is important to label your trading data correct when dealing with supervised learning. In the following, we introduce the idea of meta-labeling.

For a detailed discussion on how to label time series data for use in machine learning applications, the work of Marcos Lopez de Prado is invaluable. This is a must-read book for anybody interested in applying machine learning to time series data. In chapter 3 Marcos introduces the concept of meta-labeling.

The first step is in deciding the side of your bet, i.e. to take a long or short position. Below we suppose this has already been done either by some quantitative or discretionary method. What is left, is to decide the size of the bet, which includes the possibility of no bet at all. Below we simplify the problem to either taking the bet or not, i.e. we transform the problem to binary classification. The labeling is determined from looking at the return of the spread between successive rebalance periods. If the return is positive and above some predefined threshold it is labeled with a 1 (true), otherwise it is labeled with a 0 (false). This labeling defines the y or correct output of the classification function f we are trying to determine. For the input features X of these models we use a collection of lagged time series calculations which include

• autocorrelations,
• momentum,
• trend,
• mean reverting indicators as well as
• the side of the bet.

Our aim is then to fund the funciton \(f\) such that

\[ f(X) = y \in \left\{ 0, 1 \right\} \]

where the inpute features are

\[ X = \left(\begin{array}{ccccc} \text{autocorrelations}(t_1) & \text{momentum}(t_1) & \text{trend}(t_1) & \dots & \text{side}(t_1) \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \text{autocorrelations}(t_n) & \text{momentum}(t_n) & \text{trend}(t_n) & \dots & \text{side}(t_n) \end{array}\right) \]

The idea is that certain time series properties might have some predictive power in forecasting whether a bet will be profitable or not. Sticking with corn we consider the results of a simple Random Forest Classification model on corn calendar spreads using the scheme outlined above. Naturally, we have also used

• training and testing sets,
• k-fold cross validation, and
• hyperparameter tuning

in the construction of the classification model. Because we are dealing with time series data there are a couple of finer details and technicalities involved in the splitting between testing and training sets as well as the cross validation. These details amount to removing all lookahead bias and possible data leakage from out of sample data into the training data. For details on this, the book by Marcos Lopez de Prado serves as a great place to start.

The results of the model can be summarised in the confusion matrix below. The columns represent the true values, which in this case is a 1 if the position realised a profit and 0 if it realised a loss. The model predictions, of whether to take the bet or not, is shown on the rows.

The confusion matrix can also be visualised using the image below. Here the left and right halves of the box represents the actual positive and negative classes respectively. The abbreviations refer to

• TP: True Positive
• TN: True Negative
• FP: False Positive
• FN: False Negative

Here, true positives refer to the positive predictions that turned out to be true. A similar argument holds for the true negatives. The false positives refer to those entries marked positive but which should have been classified as negatives. The same argument holds for the false negatives.

A visualisation of the confusion matrix - (Image taken from Advances in Financial Machine Learing)

From the confusion matrix below we can define a couple of useful quantities that are valuable in evaluating binary classifiers. Define Precision as the ratio between the TP are and the area in the ellipse. Define Recall as the ratio between the TP area and the area in the left rectangle. Accuracy is the ratio of the ellipse to the rectangle. A general rule of thumb is that decreasing the FP area comes at the cost of increasing the FN area. This is because higher precision typically means fewer calls, hence lower recall. There is also a combination of precision and recall that maximises the overall efficiency of a classifier, this is called the F1-score and is defined as the harmonic mean between precision and recall.

The table below shows the performance of the classifier according to the measures defined above. The columns “No Mask” and “With Meta-Labeling” show compare the results after adding the machine learning classifier to the original side of the bet. Notice the measured improvements in accuracy and precision. Adding the mask will result in greater profits and lower drawdowns. It also comes with the added benefit of reducing the turnover in your portfolio thereby reducing the overall trading costs.

Another way to inspect the quality of these binary classification models is to use receiver operating characteristic (ROC) curves. These plots show the true positive rate (recall) as a function of false positive rate. The closer this graph gets to the top left corner of the graph the better classifier it is. For this reason, a measure that is often used to gauge the quality of a classifier is the area under the curve (AUC). The closer this number is to 1 to better classifier it is. A completely random classifier, basically a coin flip, yields the dashed line with slope 1 and AUC of 1/2. The hope is that our classifier will at least be better than a coin flip. The plot below shows the receiver operator curve of your meta-labeling model on the corn calendar spreads. We can see that it performs better than a random coin flip.

Some other interesting classification problems include:

• For a given front month price, will the curve be in backwardation or contango?
• Given the current rainfall data over the United States corn-producing area, should we expect a yield that is greater or less than the USDA?
• Given the global balance sheets for wheat, will the US vs Europe wheat trade yield a positive or negative roll?

Basically any question that can be framed so that it has a yes or no answer can be modeled using a classification algorithm.