BVAR Hyper-parameter Selection using EViews (Part 1)

Bayesian VAR models are often used for forecasting. One of the decisions the forecaster faces is the choice of prior and corresponding hyper-parameter values.

One method of hyper-parameter selection is to do a grid search over a range of possible hyper-parameters and see which combination provides the best forecasting model.

Many different criteria can be used to define the best forecasting model; for example, the model that has minimum RMSE or perhaps minimum Theil U statistic. A measure of directional accuracy could be chosen, too, or even, say, an information criterion, such as AIC.

In the EViews code below, I demonstrate how the BVAR hyper-parameter selection for forecasting can be done using Theil's U as the selection criteria. The prior type is Minnesota, but this can easily be changed. The variable of interest to forecast is real GDP growth (Y).

The program uses simulated data (that mimics the dynamics of macroeconomic data) and the best model chosen by the algorithm turns out to be fairly similar to the underlying data generating process.

I've tinkered around with this program using a much finer grid search - estimating approximately 2.5 million models - and EViews doesn't crash or appear to give any problems. Other software, may or may not run out of memory - this is something I've yet to investigate, but will do shortly.

' -------------------------------------------------------------------------------------------------------
' PROGRAM:    BVAR_Selection.prg
' PURPOSE:   To demonstrate BVAR hyper-parameter selection.
' AUTHOR:   Graeme Walsh
' DATE:     25 / 06 / 2016
' --------------------------------------------------------------------------------------------------------

' ***********************************************
' Pre-liminary set-up
' ************************************************

' Create undated workfile with 1000 observations
  wfcreate(wf=BVAR_SELECT) u 1000

' ***********************************************
' Simulate data
' ************************************************

' Simulate four data series to mimic macroeconomic data
'  Real GDP Growth (Y)
' Unemployment Rate (U)
' Short-Term Interest Rate (R)
' Inflation (P)

' Simulate error processes
  series e1 = nrnd
  series e2 = nrnd
  series e3 = nrnd
  series e4 = nrnd

' Simulate AR(2) processes
  smpl @first @first+2
  series Y = 0
  series U = 0
  series R = 0

  smpl @first+2 @last
  series Y = 3.20 + 0.22* Y(-1) + 0.15 * Y(-2) + e1
  series U = 6.2 + 1.58 * U(-1) - 0.64 * U(-2) + e2
  series R = 6.0 + 1.18 * R(-1) - 0.23 * R(-2) + e3

' Simulate AR(4) process
  smpl @first @first+4
  series P = 0

  smpl @first+4 @last
  series P = 4.10 + 0.46 * P(-1) + 0.31 * P(-2) + 0.16 * P(-3) + 0.01 * P(-4) + e4

  smpl @all

' Delete first 250 observations (these are not representative of the true processes)
  graph sim1.line(m) Y U R P
  pagestruct(start=251)
  graph sim2.line(m) Y U R P

' ***********************************************
' Inspect simulated data
' ************************************************

' Impulse Response Functions
  var VAR_Y
  VAR_Y.ls 1 2 Y
  freeze(IMP_Y) VAR_Y.impulse(20, imp=unit, se=a)
 
  var VAR_U
  VAR_U.ls 1 2 U
  freeze(IMP_U) VAR_U.impulse(20, imp=unit, se=a)
  
  var VAR_R
  VAR_R.ls 1 2 R
  freeze(IMP_R) VAR_R.impulse(20, imp=unit, se=a)
  
  var VAR_P
  VAR_P.ls 1 4 P
  freeze(IMP_P) VAR_P.impulse(20, imp=unit, se=a)
  
  graph IRFs.merge IMP_Y IMP_U IMP_R IMP_P 

' SOME NOTES (Basic IRF interpretation)

' Momentum:
'  Momentum is the tendency to continue moving in the same direction.
' The momentum effect can offset the force of regression (convergence) toward the mean and can allow a variable to move away from its historical mean, for some time, but not indefinitely.

' Persistence:
'  A persistence variable will hang around where it is and converge slowly only to the historical mean.

' Following a one unit shock, the unemployment rate and short-term interest rate (3-month treasury) are carried further from their historical mean.
' This is the momentum effect.
' The IRFs also show that the unemployment rate overshoots to a greater extent than does the short-term interest rate.

' We also see that all of the variables return to their historical means (none of them "blow up"), although they each do this at different rates.
' For example, GDP growth returns to its historical mean after about 6 periods following a shock, the unemployment rate returns to its historical mean after about 18 periods, but inflation and short-term interest take longer than 20 periods to return to their historical means.
' In this sense, GDP growth is the least persistent of the four variables while inflation can be said to be highly persistent.

' ************************************************
' Bayesian VAR
' ************************************************

' Bayesian VAR Model
  var bmod

' Hyper-Parameter Descriptions
'  L1 = Overall tightness hyper-parameter (0-1) [small, prior dominates sample]
'  L2 = Relative cross-variable weight hyper-parameter (0-1) [0 = univariate]
'  L3 = Lag decay hyper-parameter (>0) [1 or 2]

' Hyper-parameter specification (i.e. define the grid to be searched)
  scalar L1_MIN = 0.1
  scalar L1_MAX = 1
  scalar L1_STEP = 0.05
 
  scalar L2_MIN = 0.1
  scalar L2_MAX = 1
  scalar L2_STEP = 0.05
 
  scalar L3_MIN = 0.1
  scalar L3_MAX = 3.5
  scalar L3_STEP = 0.05

' Iteration counter
  scalar ITER = 1

' Matrix to store the Theil U statistics
  matrix(25000,5) MTHEIL 

' A nested for-loop to search the space of models
  for !l1=L1_MIN to L1_MAX step L1_STEP
   for !l2=L2_MIN to L2_MAX step L2_STEP
    for !L3=L3_MIN to L3_MAX step L3_STEP
       bmod.bvar(prior=lit,L1=!l1,L2=!l2,L3=!l3,Mu1=0) 1 4 Y U R P
       bmod.fit _f
       scalar NEW_THEIL = @theil(Y,Y_f)      
        if ITER=1 then
        scalar OLD_THEIL = NEW_THEIL
        copy bmod best
       else
        if NEW_THEIL < OLD_THEIL then
         copy bmod best
         scalar OLD_THEIL = NEW_THEIL
        endif
       endif
       MTHEIL(ITER,1) = !l1
       MTHEIL(ITER,2) = !l2
       MTHEIL(ITER,3) = !l3
       MTHEIL(ITER,4) = OLD_THEIL
       MTHEIL(ITER,5) = NEW_THEIL
       scalar ITER = ITER + 1
    next
   next
  next

' Best BVAR model for forecasting chosen on basis of hyper-parameters that minimize Theil's U is stored as var object BEST.

' ************************************************
' END OF PROGRAM
' ************************************************