Source Link: https://algotrading101.com/learn/what-is-overfitting-in-trading/

Overfitting, also called curve fitting, is the process of designing a trading system that adapts so closely to historical data that it becomes ineffective or irrelevant.

When you overfit your strategy, it will give you false confidence that your strategy will be profitable.

How to Overfit

Overfitting is done by adapting strategies to market noise instead of signals. Noise is a distraction, while signals are vital information.

In this illustration below, we have three charts with the same data. We will create a model that fits the shape of the data, which will be used to predict future data points.

The data that forms a U shape is our signal.

The data in a straight line have poor predictive abilities, and we describe it as an underfitted model.

The other data, which intercepts every data point, is the perfect model. 

In the illustration, you will see that this model will have very poor predictive value unless future data points follow the past perfectly, an overfitted model.

In the same example, the middle chart describes the general shape of the data points. There are some errors, which is fine. We need our models to have a certain degree of error. This means that the model does not rigidly follow the past. 

This model can adapt to minor changes in the future. Thus, it is a good fit or robust model.

Overfitting: The Demonstration

Let’s put the theory into actual action.

Let’s curve fit a basic trading robot called Belinda in this demonstration.

Step 1

Run an optimization for Belinda by varying three variables: sma_short, sma_long, and atr_period.

Step 2

Run the optimization from 1st April 2014 to 1st January 2015.

Step 3

Determine the parameter values that produce the best objective function.

Step 4

Run a backtest to see the performance and equity curve in detail using the same backtest dates as before: 1st April 2014 to 1st Jan 2015. You should see a profit of $3,549.18.

Test Belinda with the optimized parameter values using data from the future. Run the backtest in the future period: 1st Jan 2015 to 1st Oct 2015.

Backtest Performance Manipulation

Here’s how you manipulate backtest performance:

Step 1

Run an optimization and find the parameter values that offer the best results.

Step 2

Run a backtest using these parameter values and the same dates used in the optimization.

In this result, the $10K was turned into $2.5 million in 9 months!

Is overfitting bad?

Yes, because it will lead to poor performance in the future. 

In the financial markets, the past does not predict the future. Thus, choosing a strategy that is too close to the past data will result in inflexibility to adapt to the changes in the future.

want to learn how to algo trade so you can remove all emotions from trading and automate it 100%? click below to join the free discord and then join the bootcamp to get started today.