Failure to fit complicated models: handle errors gracefully?

[chm-von-tla]

Consider the following example.

using ARCHModels
X = BG96[773:1272]
fit(EGARCH{1,1,1},X,meanspec=AR{1},dist=StdT)

This fails instantly with the following error:

ERROR: AssertionError: isfinite(phi_d) && isfinite(gphi)
Stacktrace:
  [1] bisect!(ϕdϕ::LineSearches.var"#ϕdϕ#6"{Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}}, Vector{Float64}, Vector{Float64}, Vector{Float64}}, alphas::Vector{Float64}, values::Vector{Float64}, slopes::Vector{Float64}, ia::Int64, ib::Int64, phi_lim::Float64, display::Int64)
    @ LineSearches ~/.julia/packages/LineSearches/Ki4c5/src/hagerzhang.jl:503
  [2] (::LineSearches.HagerZhang{Float64, Base.RefValue{Bool}})(ϕ::Function, ϕdϕ::LineSearches.var"#ϕdϕ#6"{Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}}, Vector{Float64}, Vector{Float64}, Vector{Float64}}, c::Float64, phi_0::Float64, dphi_0::Float64)
    @ LineSearches ~/.julia/packages/LineSearches/Ki4c5/src/hagerzhang.jl:201
  [3] HagerZhang
    @ ~/.julia/packages/LineSearches/Ki4c5/src/hagerzhang.jl:101 [inlined]
  [4] perform_linesearch!(state::Optim.BFGSState{Vector{Float64}, Matrix{Float64}, Float64, Vector{Float64}}, method::Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Optim.Flat}, d::Optim.ManifoldObjective{NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}})
    @ Optim ~/.julia/packages/Optim/TNmSw/src/utilities/perform_linesearch.jl:56
  [5] update_state!(d::NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, state::Optim.BFGSState{Vector{Float64}, Matrix{Float64}, Float64, Vector{Float64}}, method::Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Optim.Flat})
    @ Optim ~/.julia/packages/Optim/TNmSw/src/multivariate/solvers/first_order/bfgs.jl:110
  [6] optimize(d::NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, initial_x::Vector{Float64}, method::Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Optim.Flat}, options::Optim.Options{Float64, Nothing}, state::Optim.BFGSState{Vector{Float64}, Matrix{Float64}, Float64, Vector{Float64}})
    @ Optim ~/.julia/packages/Optim/TNmSw/src/multivariate/optimize/optimize.jl:57
  [7] optimize
    @ ~/.julia/packages/Optim/TNmSw/src/multivariate/optimize/optimize.jl:33 [inlined]
  [8] #optimize#89
    @ ~/.julia/packages/Optim/TNmSw/src/multivariate/optimize/interface.jl:116 [inlined]
  [9] _fit!(garchcoefs::Vector{Float64}, distcoefs::Vector{Float64}, meancoefs::Vector{Float64}, ::Type{EGARCH{1, 1, 1, T} where T<:AbstractFloat}, ::Type{StdT}, meanspec::AR{1, Float64}, data::Vector{Float64}; algorithm::Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Optim.Flat}, autodiff::Symbol, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ARCHModels ~/.julia/packages/ARCHModels/iIiJW/src/univariatearchmodel.jl:371
 [10] fit(::Type{EGARCH{1, 1, 1, T} where T<:AbstractFloat}, data::Vector{Float64}; dist::Type{StdT}, meanspec::Type{AR{1, T} where T}, algorithm::Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Optim.Flat}, autodiff::Symbol, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ARCHModels ~/.julia/packages/ARCHModels/iIiJW/src/univariatearchmodel.jl:431
 [11] top-level scope
    @ REPL[2]:1

The data was specifically chosen to demonstrate this error. The error however isn't related to just that specific data; it happens often when EGARCH (and sometimes TGARCH) models are combined with AR mean specifications and StdT/StdSkewT distribution of errors

At present, my code that depends on ARCHModels is handling this error in a very naive manner:

try
    fit(EGARCH{1,1,1}, data, meanspec=AR{1}, dist=StdT)
catch e
    if isa(e,AssertionError)
        @warn ("Could not fit model, falling back to GARCH{1,1}")
        fit(GARCH{1,1}, data, meanspec=AR{1}, dist=StdT)
    else
        rethrow()
    end
end

It would be very helpful if
- 1) this error was handled "gracefully" by ARCHModels, perhaps falling back to simpler models like in my code snippet,
or maybe even better if
- 2) the source of the error could be found and corrected.

In either case sadly I cannot be of much help. For case 1 I'm not familiar enough with the codebase in order to suggest remedies and for case 2 I do not have the required expertise/familiarity with optimizers in order to help

[s-broda]

Thanks for the MWE! Unfortunately I haven't managed to fix this in general. In this particular example, you can do

using Optim
using ARCHModels
X = BG96[773:1272]
fit(EGARCH{1,1,1}, X, meanspec=AR{1}, dist=StdT, algorithm=BFGS(linesearch = Optim.BackTracking(order=3)))

but I'm not sure if this is more robust in general (feel free to test). It is definitely a bit slower for some models. One idea would indeed be to catch the error and then try the other line search algorithm. If your testing reveals that this helps, then we could put that into the package (although it still feels a bit hacky).

Cheers
Simon

[chm-von-tla]

Hello Simon,

thank you for your input. I will try to familiarize myself with the Optim package, in order to test whether different values to the algorithm argument of fit can help us overcome this problem (of course taking performance into account).

One idea would indeed be to catch the error and then try the other line search algorithm. If your testing reveals that this helps, then we could put that into the package (although it still feels a bit hacky).

If a robust general solution to the problem cannot be found, I think that catching the exception, falling back to another algorithm and outputting a warning is the better alternative [1]. I fully agree that it is hacky, but in my opinion it is much more preferable than throwing an exception. What are your thoughts on this?

I will contact you again, after I have done some testing.

Best regards,

Charis

====================================================================

[1]: This is done by other packages as well. If you run this piece of code:

using ARCHModels

for i in 1:length(BG96)-100
    fit(TGARCH{1,1,1},BG96[i:i+100],meanspec=AR{1},dist=StdT)
end

the LineSearches package outputs the following warning about ten times

┌ Warning: Failed to achieve finite new evaluation point, using alpha=0
└ @ LineSearches /home/chm/.julia/packages/LineSearches/Ki4c5/src/hagerzhang.jl:148