Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: cannot import a model in a function #114

Open
albop opened this issue Sep 8, 2017 · 12 comments
Open

BUG: cannot import a model in a function #114

albop opened this issue Sep 8, 2017 · 12 comments

Comments

@albop
Copy link
Member

albop commented Sep 8, 2017

This is a bit frightening. Are we doing something wrong with the function generation ?

julia> import Dolo

julia> function solve_julia(fname)
            model = Dolo.Model(fname)
            dr = Dolo.time_iteration(model)
       end
solve_julia (generic function with 1 method)

julia> sol = solve_julia("neoclassical.yaml")

ERROR: MethodError: no method matching controls_lb(::Dolo.Model{Symbol("##836")}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1})
The applicable method may be too new: running in world age 22573, while current world is 22637.
Closest candidates are:
  controls_lb(::Dolo.Model{Symbol("##836")}, ::AbstractArray{T,1} where T, ::AbstractArray{T,1} where T, ::Any) at :0 (method too new to be called from this world context.)
  controls_lb(::Dolo.Model{Symbol("##836")}, ::AbstractArray, ::AbstractArray, ::Any) at :0 (method too new to be called from this world context.)
  controls_lb(::Type{Dolang.Der{0}}, ::Dolo.Model{Symbol("##836")}, ::AbstractArray{T,1} where T, ::AbstractArray{T,1} where T, ::Any) at :0 (method too new to be called from this world context.)
  ...
Stacktrace:
 [1] #time_iteration#141(::Bool, ::Int64, ::Float64, ::Bool, ::Dict{Any,Any}, ::Function, ::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.CartesianGrid{1}, ::Dolo.ConstantDecisionRule{1}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:199
 [2] time_iteration(::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.CartesianGrid{1}, ::Dolo.ConstantDecisionRule{1}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:167
 [3] #time_iteration#145(::Dict{Any,Any}, ::Array{Any,1}, ::Function, ::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.ConstantDecisionRule{1}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:257
 [4] (::Dolo.#kw##time_iteration)(::Array{Any,1}, ::Dolo.#time_iteration, ::Dolo.Model{Symbol("##836")}, ::Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}, ::Dolo.ConstantDecisionRule{1}) at ./<missing>:0
 [5] #time_iteration#148(::Dict{Any,Any}, ::Array{Any,1}, ::Function, ::Dolo.Model{Symbol("##836")}) at /home/pablo/.julia/v0.6/Dolo/src/algos/time_iteration.jl:277
 [6] solve_julia(::String) at ./REPL[3]:3


@sglyon
Copy link
Member

sglyon commented Sep 8, 2017

Oh man this could be tricky to figure out.

I honestly don't know too much about how the new world/old world stuff works. Let me ask around and see if we can figure out a fix

@albop
Copy link
Member Author

albop commented Sep 8, 2017

could be related to the scope where the functions are evaluated

@sglyon
Copy link
Member

sglyon commented Sep 8, 2017

One idea would be to have Dolo.@Model "neoclassical.yaml" just emit the generated code as opposed to what we do now where Dolo.Model("neoclassical.yaml") builds the code and then calls evalon it.

That would almost certainly work , but might make it so the methods are only callable within the same scope as when the macro is called. Not sure if that is an issue or not...

@albop
Copy link
Member Author

albop commented Sep 8, 2017

it is of course an issue...

@quinnj
Copy link

quinnj commented Sep 8, 2017

That's the approach I took w/ defining custom json formatting for types here. I'm not sure what you mean "methods are only callable within the same scope"? In my case, my macro expands to a generated function that overloads a JSON2 function for the new type, so there aren't any issues calling it from any scope.

@sglyon
Copy link
Member

sglyon commented Sep 8, 2017

Ok cool, thanks for the tip/reference. We'll take a look at that and see if we can use something similar.

Here's an example of what I meant and was worried about:

Suppose that Dolo.@Model "fname.yaml" did something like this:

begin
    Dolo.controls_lb(::Model{:fname}, ...) = nothing
    Model{:fname}(constructor_args...)
end

If the macro were just to place this code at the call site @albop function above would look like this after macro expansion:

function solve_julia(fname)
    model = begin
        Dolo.controls_lb(::Model{:fname}, ...) = nothing
        Model{:fname}(constructor_args...)
    end
    dr = Dolo.time_iteration(model)
end

I was worried that the particular method of Dolo.controls_lb defined by that macro would only be callable from within the solve_julia function and not visible elsewhere in Julia (e.g. within the Dolo.time_iteration function). I thought about it a bit more and don't think that this will actually be an issue...

@quinnj
Copy link

quinnj commented Sep 8, 2017

It seems like you should just move Dolo.@Model "fname.yml" to a top-level expression, not inside a function, or at least have a macro that defines the type/functions at the top-level, then have a separate construct/macro/whatever that you call in the function that actually defined the instance.

@albop
Copy link
Member Author

albop commented Sep 8, 2017

I don't totally understand how the macro thing solves the initial problem. @sglyon and @quinnj : aren't you assuming the model filename is constant, so that the macro can read the file, expand the code and replace it in the function ? So you can't pass the filename to the model_julia function, can you ?
I confess, I don't understand the error message very well...

@sglyon
Copy link
Member

sglyon commented Sep 8, 2017

Well, I found a solution to this problem, but it isn't necessarily pretty:

julia> function solve_julia(fname::String)
           model = Dolo.Model(fname)
           Base.invokelatest(Dolo.time_iteration, model)
       end
solve_julia (generic function with 1 method)

julia> solve_julia("neoclassical.yaml")
------------------------------------------------------------------
It    ηₙ=|xₙ-xₙ₋₁|    λₙ=ηₙ/ηₙ₋₁      Time            Newton steps
------------------------------------------------------------------
0     NaN             NaN             0.00e+00        0
1     2.03e-02        NaN             6.44e+00        3
2     1.01e-02        4.96e-01        2.34e-02        2
3     7.15e-03        7.11e-01        1.75e-02        2
4     5.21e-03        7.29e-01        8.76e-02        2
5     3.87e-03        7.43e-01        3.25e-02        2
6     2.92e-03        7.53e-01        2.62e-02        2
7     2.22e-03        7.62e-01        2.10e-02        2
8     1.71e-03        7.69e-01        1.82e-02        2
9     1.38e-03        8.05e-01        3.08e-02        2
10    1.16e-03        8.44e-01        1.77e-02        2
11    9.80e-04        8.44e-01        6.17e-02        2
12    8.27e-04        8.43e-01        2.30e-02        2
13    6.97e-04        8.43e-01        2.98e-02        2
14    5.87e-04        8.42e-01        2.06e-02        2
15    4.94e-04        8.42e-01        1.82e-02        2
16    4.15e-04        8.41e-01        1.18e-02        1
17    3.49e-04        8.41e-01        1.64e-02        1
18    2.93e-04        8.40e-01        1.02e-02        1
19    2.46e-04        8.39e-01        1.09e-02        1
20    2.06e-04        8.39e-01        1.57e-02        1
21    1.73e-04        8.38e-01        4.84e-02        1
22    1.45e-04        8.37e-01        1.18e-02        1
23    1.21e-04        8.37e-01        1.27e-02        1
24    1.01e-04        8.36e-01        1.31e-02        1
25    8.46e-05        8.35e-01        1.52e-02        1
26    7.06e-05        8.35e-01        1.52e-02        1
27    5.89e-05        8.34e-01        1.24e-02        1
28    4.91e-05        8.34e-01        1.45e-02        1
29    4.09e-05        8.33e-01        1.48e-02        1
30    3.41e-05        8.33e-01        1.63e-02        1
31    2.84e-05        8.32e-01        1.19e-02        1
32    2.36e-05        8.32e-01        1.26e-02        1
33    1.96e-05        8.31e-01        3.98e-02        1
34    1.63e-05        8.31e-01        1.11e-02        1
35    1.36e-05        8.31e-01        1.76e-02        1
36    1.13e-05        8.30e-01        7.41e-03        1
37    9.34e-06        8.30e-01        5.79e-03        1
38    7.75e-06        8.30e-01        8.91e-03        1
39    6.43e-06        8.29e-01        7.76e-03        1
40    5.33e-06        8.29e-01        5.85e-03        1
41    4.42e-06        8.29e-01        4.34e-03        1
42    3.66e-06        8.29e-01        6.00e-03        1
43    3.03e-06        8.28e-01        1.09e-02        1
44    2.51e-06        8.28e-01        1.01e-02        1
45    2.08e-06        8.28e-01        1.67e-02        1
46    1.72e-06        8.28e-01        6.07e-03        1
47    1.42e-06        8.27e-01        9.77e-03        1
48    1.18e-06        8.27e-01        4.97e-03        1
49    9.72e-07        8.27e-01        1.16e-02        1
50    8.04e-07        8.27e-01        6.40e-03        1
51    6.64e-07        8.26e-01        5.38e-03        1
52    5.49e-07        8.26e-01        6.05e-03        1
53    4.53e-07        8.26e-01        1.09e-02        1
54    3.74e-07        8.26e-01        6.81e-03        1
55    3.09e-07        8.25e-01        8.97e-03        1
56    0.00e+00        0.00e+00        1.97e-03        0
------------------------------------------------------------------
Results of Time Iteration Algorithm
 * Complementarities: true
 * Discretized Process type: Dolo.DiscretizedProcess{Dolo.CartesianGrid{1}}
 * Decision Rule type: Dolo.CubicDR{Dolo.CartesianGrid{1},Dolo.CartesianGrid{1},1,2}
 * Number of iterations: 56
 * Convergence: true
   * |x - x'| < 1.0e-07: true

Notice that I use Base.invokelatest(Dolo.time_iteration, model) when calling time iteration. I don't understand what is going on well enough to know why that works or how to get around it, but at least that does give us a starting place...

@albop
Copy link
Member Author

albop commented Sep 8, 2017 via email

@sglyon
Copy link
Member

sglyon commented Sep 8, 2017

I would like to brainstorm a bit more and figure out a different way to do this.

invokelatest isn't a magic solution and has drawbacks as highlighted in the docstring:

help?> Base.invokelatest
  invokelatest(f, args...)

  Calls f(args...), but guarantees that the most recent method of f will be executed.
  This is useful in specialized circumstances, e.g. long-running event loops or
  callback functions that may call obsolete versions of a function f. (The drawback
  is that invokelatest is somewhat slower than calling f directly, and the type of
  the result cannot be inferred by the compiler.)

@albop
Copy link
Member Author

albop commented Sep 8, 2017

Here is a reduced form version of our problem: <edit: I removed the generated macro>

struct Model{T}
end

code = """
function squareit(t::Model{T}, x) where T
    x^2
end
"""

function applyfun(model::Model)
    squareit(model, 0.1)
end

function reduced_problem(fname, cc)
    model = Model{fname}()
    eval(parse(cc))
    applyfun(model)
end

reduced_problem(15, code)

It produces the error:

MethodError: no method matching squareit(::Model{15}, ::Float64)
The applicable method may be too new: running in world age 23129, while current world is 23130.
Closest candidates are:
  squareit(::Model{T}, ::Any) where T at none:2 (method too new to be called from this world context.)
reduced_problem(::Int64, ::String) at bug.jl:28
include_string(::String, ::String) at loading.jl:515
include_string(::String, ::String, ::Int64) at eval.jl:30
include_string(::Module, ::String, ::String, ::Int64, ::Vararg{Int64,N} where N) at eval.jl:34
(::Atom.##49#52{String,Int64,String})() at eval.jl:50
withpath(::Atom.##49#52{String,Int64,String}, ::String) at utils.jl:30
withpath(::Function, ::String) at eval.jl:38
macro expansion at eval.jl:49 [inlined]
(::Atom.##48#51{Dict{String,Any}})() at task.jl:80

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants