Solution - Forecasting SPY

Solution - Forecasting SPY#

# Output from previous execution
DP EP CAPE BP T10YR CREDIT VIX TSPREAD
date
1993-02-28 2.7997 4.100613 15.8223 42.574932 6.0200 221.00 13.16 3.0230
1993-03-31 2.7790 4.677947 16.0443 35.236082 6.0240 218.60 12.53 3.0670
1993-04-30 2.8574 4.033836 15.5631 39.158868 6.0090 216.10 12.42 3.0530
1993-05-31 2.7867 3.191707 16.3376 38.678734 6.1490 211.10 13.47 3.0390
1993-06-30 2.8183 3.147158 16.2361 38.659296 5.7760 218.40 11.26 2.6960
... ... ... ... ... ... ... ... ...
2025-06-30 1.2426 3.800547 34.7994 19.081785 4.2280 186.17 16.73 -0.0631
2025-07-31 1.2199 3.719947 35.2735 18.676927 4.3740 169.20 16.72 0.0368
2025-08-31 1.2035 3.731719 36.2103 18.898590 4.2284 174.16 15.36 0.0893
2025-09-30 1.1683 3.595376 37.3821 18.299601 4.1503 164.17 16.28 0.2180
2025-10-31 1.1392 3.516434 38.1525 17.893569 4.0775 167.25 17.44 0.2768

393 rows × 8 columns

# Output from previous execution
  Mean DP EP ALL
date        
1993-03-31 1.1% 1.6% 0.6% 0.3%
1993-04-30 1.7% 1.6% 0.8% 0.9%
1993-05-31 0.2% 1.7% 0.6% 0.7%
  Mean DP EP ALL
date        
2025-08-31 0.9% 0.4% 0.5% 0.1%
2025-09-30 0.9% 0.4% 0.5% -0.3%
2025-10-31 0.9% 0.4% 0.5% -0.2% 
# Output from previous execution
../_images/df1cc3e5b097142726cf3552049b2096878fd0b2440f2598ca73089668f69ebc.png 
# Output from previous execution
  Passive Mean DP EP ALL
date          
1993-03-31 2.2% 2.4% 3.7% 1.4% 0.7%
1993-04-30 -2.6% -4.2% -4.2% -2.0% -2.3%
1993-05-31 2.7% 0.7% 4.5% 1.6% 1.8%
  Passive Mean DP EP ALL
date          
2025-08-31 2.1% 1.9% 0.9% 1.0% 0.3%
2025-09-30 3.6% 3.3% 1.5% 1.8% -1.0%
2025-10-31 2.4% 2.2% 0.9% 1.1% -0.6% 
# Output from previous execution
  Mean DP EP ALL
OOS-Rsquared 0.0% 1.6% 1.6% 6.6%
Corr. between forecast and target -3.6% 8.2% 8.0% 23.9%
../_images/1353176950b9d40241248b1d83cd9d3ee5763d573101699e37ad8a4d6d3bb0dd.png ../_images/dbb1e36e620540c771febb67e1a3a5c9f4b6fe87cad2a46d31d82cd6fb67ea5c.png ../_images/b84fc609173024fe2e3aab56985fa1efb96fe4ddda14677012bbce1e371c2072.png ../_images/4ab93d7867e705e418b83763966c8d36c54de9721ea98ace573af9c62d7579da.png
  Mean Vol Sharpe Min Max
Passive 11.3% 14.8% 76.6% -16.5% 12.7%
Mean 10.2% 15.0% 68.2% -23.3% 15.3%
DP 12.1% 15.6% 78.0% -22.7% 22.3%
EP 12.1% 16.1% 74.9% -21.9% 17.9%
ALL 23.1% 27.4% 84.4% -63.5% 62.5%
  Skewness Kurtosis VaR (0.05) CVaR (0.05) Max Drawdown Peak Bottom Recover Duration (to Recover)
Passive -59.5% 98.1% -7.0% -9.4% -50.8% 2007-10-31 2009-02-28 2012-03-31 1613 days
Mean -57.6% 282.9% -6.9% -9.5% -51.7% 2000-08-31 2002-09-30 2013-07-31 4717 days
DP -46.0% 610.1% -6.0% -9.9% -66.1% 2007-10-31 2009-02-28 2012-02-29 1582 days
EP -47.4% 440.4% -6.1% -10.7% -65.2% 2007-10-31 2009-02-28 2012-08-31 1766 days
ALL 38.0% 2312.6% -6.9% -14.4% -69.7% 2019-12-31 2020-03-31 2020-12-31 366 days
  alpha SPY r-squared Treynor Ratio Info Ratio
Passive -0.00% 1.00 100.00% 11.32%
Mean -0.67% 0.96 89.85% 10.62% -14.10%
DP 1.24% 0.96 83.64% 12.60% 19.65%
EP 0.56% 1.02 86.82% 11.87% 9.64%
ALL 9.04% 1.24 45.03% 18.58% 44.52% 
# Output from previous execution
  Mean DP EP ALL
date        
1998-02-28 1.6% 2.3% 3.2% 1.6%
1998-03-31 1.7% 2.8% 3.3% 1.2%
1998-04-30 1.7% 3.0% 1.4% 1.7%
  Mean DP EP ALL
date        
2025-08-31 0.9% 0.4% 0.5% 0.0%
2025-09-30 0.9% 0.4% 0.5% -0.4%
2025-10-31 0.9% 0.4% 0.5% -0.3% 
# Output from previous execution
  Passive Mean DP EP ALL
date          
1998-02-28 6.9% 11.0% 16.1% 21.9% 10.9%
1998-03-31 4.9% 8.1% 13.5% 16.0% 5.6%
1998-04-30 1.3% 2.2% 3.9% 1.8% 2.2%
  Passive Mean DP EP ALL
date          
2025-08-31 2.1% 1.9% 0.8% 1.0% 0.0%
2025-09-30 3.6% 3.3% 1.4% 1.7% -1.5%
2025-10-31 2.4% 2.2% 0.9% 1.1% -0.7% 
# Output from previous execution
  Mean DP EP ALL
OOS-Rsquared 0.0% -0.4% -0.2% -6.8%
Corr. between forecast and target -5.4% -2.0% -1.4% 4.8%
../_images/8e52e8525fdc688f0162096fa58fbc41280055615db48a68a054cf1efbb888bc.png ../_images/f12dd23bd25e62f19a3eb5e764f90e1301ee48742a13839b5729dbdf7405fa1d.png ../_images/fb9660184d0f419e1a4e303cfebf353001866fd4c96d8306d5df22d1437e2828.png ../_images/7f01779b08c9e2cc7e6c0617cc68bde7afd68fb8ec3575b9cab62bc3e464d50d.png
  Mean Vol Sharpe Min Max
Passive 9.9% 15.4% 64.8% -16.5% 12.7%
Mean 8.6% 15.2% 56.3% -23.3% 15.3%
DP 8.6% 16.8% 51.4% -33.5% 16.1%
EP 9.1% 17.0% 53.4% -28.2% 21.9%
ALL 14.9% 36.5% 40.8% -68.9% 54.6%
  Skewness Kurtosis VaR (0.05) CVaR (0.05) Max Drawdown Peak Bottom Recover Duration (to Recover)
Passive -56.3% 84.8% -7.6% -9.8% -50.8% 2007-10-31 2009-02-28 2012-03-31 1613 days
Mean -64.5% 297.0% -7.3% -9.8% -51.7% 2000-08-31 2002-09-30 2013-07-31 4717 days
DP -108.5% 878.1% -6.6% -11.3% -55.2% 2007-10-31 2009-02-28 2013-07-31 2100 days
EP -75.8% 623.5% -6.9% -11.6% -64.3% 2007-10-31 2009-02-28 2013-10-31 2192 days
ALL -121.9% 1390.5% -10.4% -26.4% -83.6% 2000-03-31 2011-09-30 2019-04-30 6969 days
  alpha SPY r-squared Treynor Ratio Info Ratio
Passive 0.00% 1.00 100.00% 9.94%
Mean -0.81% 0.94 90.46% 9.08% -17.24%
DP -1.09% 0.98 79.93% 8.83% -14.52%
EP -0.95% 1.01 82.93% 9.00% -13.48%
ALL 3.03% 1.20 25.27% 12.48% 9.59% 
# Output from previous execution

/var/folders/zx/3v_qt0957xzg3nqtnkv007d00000gn/T/ipykernel_67656/944787420.py:17: FutureWarning: Series.ravel is deprecated. The underlying array is already 1D, so ravel is not necessary.  Use `to_numpy()` for conversion to a numpy array instead.
  y = spy.squeeze('columns').ravel()
/opt/miniconda3/envs/env-finm-2025/lib/python3.10/site-packages/sklearn/neural_network/_multilayer_perceptron.py:780: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (500) reached and the optimization hasn't converged yet.
  warnings.warn(
  Mean OLS Random Forest NN
date        
1993-03-31 1.1% 0.3% 0.8% -0.8%
1993-04-30 1.7% 0.9% 0.8% 0.4%
1993-05-31 0.2% 0.7% 0.8% 0.2%
  Mean OLS Random Forest NN
date        
2025-08-31 0.9% 0.1% 0.8% 0.7%
2025-09-30 0.9% -0.3% 0.4% -0.3%
2025-10-31 0.9% -0.2% -0.1% -0.1% 
# Output from previous execution
../_images/378b9a7a454fcb6a6b98792d70c12b762c4104734581593ee511536e1e7550ee.png 
# Output from previous execution
  Passive Mean OLS Random Forest NN
date          
1993-03-31 2.2% 2.4% 0.7% 0.9% -0.9%
1993-04-30 -2.6% -4.2% -2.3% -1.0% -0.5%
1993-05-31 2.7% 0.7% 1.8% 1.1% 0.3%
  Passive Mean OLS Random Forest NN
date          
2025-08-31 2.1% 1.9% 0.3% 0.8% 0.8%
2025-09-30 3.6% 3.3% -1.0% 0.8% -0.5%
2025-10-31 2.4% 2.2% -0.6% -0.1% -0.1% 
# Output from previous execution
  Mean OLS Random Forest NN
OOS-Rsquared 0.0% 6.6% 31.9% 28.4%
Corr. between forecast and target -3.6% 23.9% 70.0% 54.1%
../_images/c5bb0f5786638a13375c5dd76a079745f97a4edda6471399338136efd2d14e45.png ../_images/49aa435d0f1ef70bf030e54380edce5554dacfe42ba191d8a0ed7d17db14de59.png ../_images/b50c738bcd3bcddc0b193058f9f5ff862d11df1ea392790b93ca8aa468298588.png ../_images/60f5b82d4839403cbf88d61ada07f6bffe9f7b48d217326255a6189eaa723f23.png
  Mean Vol Sharpe Min Max
Passive 11.3% 14.8% 76.6% -16.5% 12.7%
Mean 10.2% 15.0% 68.2% -23.3% 15.3%
OLS 23.1% 27.4% 84.4% -63.5% 62.5%
Random Forest 26.4% 20.1% 131.3% -4.1% 57.2%
NN 29.7% 20.7% 143.4% -14.8% 61.8%
  Skewness Kurtosis VaR (0.05) CVaR (0.05) Max Drawdown Peak Bottom Recover Duration (to Recover)
Passive -59.5% 98.1% -7.0% -9.4% -50.8% 2007-10-31 2009-02-28 2012-03-31 1613 days
Mean -57.6% 282.9% -6.9% -9.5% -51.7% 2000-08-31 2002-09-30 2013-07-31 4717 days
OLS 38.0% 2312.6% -6.9% -14.4% -69.7% 2019-12-31 2020-03-31 2020-12-31 366 days
Random Forest 574.9% 4382.6% -1.3% -2.0% -6.3% 2007-10-31 2008-03-31 2008-10-31 366 days
NN 416.1% 2951.5% -2.4% -4.3% -16.6% 2019-11-30 2020-03-31 2020-04-30 152 days
  alpha SPY r-squared Treynor Ratio Info Ratio
Passive -0.00% 1.00 100.00% 11.32%
Mean -0.67% 0.96 89.85% 10.62% -14.10%
OLS 9.04% 1.24 45.03% 18.58% 44.52%
Random Forest 22.54% 0.34 6.38% 76.72% 115.67%
NN 24.76% 0.44 9.82% 67.55% 125.68% 
# Output from previous execution

OLS
Random Forest
NN
  Mean OLS Random Forest NN
date        
1998-03-31 1.6% 1.2% -1.2% -4.9%
1998-04-30 1.7% 1.7% -0.7% -6.8%
1998-05-31 1.7% 1.0% 0.3% -2.5%
  Mean OLS Random Forest NN
date        
2025-08-31 0.9% 0.0% 0.4% -0.3%
2025-09-30 0.9% -0.4% -0.6% -1.3%
2025-10-31 0.9% -0.3% -1.6% -0.1% 
# Output from previous execution
  Mean OLS Random Forest NN
OOS-Rsquared 0.0% -6.8% -17.2% -69.2%
Corr. between forecast and target -6.9% 4.7% -15.2% -9.7%
../_images/12125398086d40b6746fe879c5ecd3ae31f9566301b0e30cd15b689d68cc867e.png ../_images/cabd18024b40840451906a886a5e15064a5d8507d068b9efca8d8e0ff6d188f9.png ../_images/d5d4452291de417df31446b1af417ae4d6bead2fa172a82490b1915977f257c9.png ../_images/73fbba2b09be0ee44306e4ece43d26e0b9b34d08955e32b970aed2a16a240228.png
  Mean Vol Sharpe Min Max
Passive 9.1% 15.1% 60.3% -16.5% 12.7%
Mean 9.2% 14.5% 63.7% -15.8% 12.5%
OLS 6.6% 15.3% 43.0% -20.9% 20.6%
Random Forest 1.5% 15.5% 9.6% -22.8% 16.5%
NN 2.6% 15.0% 17.2% -23.8% 15.9%
  Skewness Kurtosis VaR (0.05) CVaR (0.05) Max Drawdown Peak Bottom Recover Duration (to Recover)
Passive -50.4% 79.9% -7.7% -9.5% -50.8% 2007-10-31 2009-02-28 2012-03-31 1613 days
Mean -47.9% 71.7% -6.9% -8.9% -47.2% 2007-10-31 2009-02-28 2012-08-31 1766 days
OLS -59.1% 654.2% -5.7% -11.2% -43.5% 2007-10-31 2011-09-30 2014-05-31 2404 days
Random Forest -132.2% 704.1% -7.3% -13.2% -63.6% 2000-03-31 2012-05-31 2018-08-31 6727 days
NN -112.0% 757.2% -6.1% -11.8% -56.6% 2000-03-31 2002-11-30 2014-10-31 5327 days
  alpha SPY r-squared Treynor Ratio Info Ratio
Passive 0.00% 1.00 100.00% 9.13%
Mean 0.54% 0.95 98.90% 9.70% 35.70%
OLS 1.27% 0.58 33.10% 11.31% 10.12%
Random Forest -2.94% 0.48 22.47% 3.06% -21.59%
NN -0.44% 0.33 11.14% 7.80% -3.11%