pyimport_conda yields PyObject NULL

[cscherrer]

Hi,

I'm hitting a bug I'm having some trouble replicating, hope you might be able to steer me in the right direction.

At a very high level, the problem happens when I

1. Import sympy.stats in the standard way within my package
2. using Soss (my package) externally
3. Make a call involving stats

Result:

julia> Soss.stats
PyObject NULL

In some cases, this triggers a segfault.

---

For context, I've tried this in both Julia 1.2 and 1.3.

Also, the following works with no trouble:

# in a file
module M
import PyCall
import SymPy
using SymPy: Sym, sympy, symbols

export SymPy, stats, symbols

stats = PyCall.pyimport_conda("sympy.stats", "sympy")
SymPy.import_from(stats)

import Distributions.Normal
export Normal
Normal(μ::Sym, σ::Sym) = stats.Normal(:Normal, μ,σ) |> SymPy.density
Normal(μ,σ) = Normal(promote(μ,σ)...)
end

using Distributions
using .M

stats.Normal(:Normal, symbols(:μ),1) |> SymPy.density
Normal(symbols(:μ),1)

# in the REPL
julia> stats.Normal(:Normal, symbols(:μ),1) |> SymPy.density
NormalDistribution(μ, 1)

julia> Normal(symbols(:μ),1)
NormalDistribution(μ, 1)

The code in symbolic.jl takes this same approach. But when I try from a fresh environment, ...

(Soss#dev) pkg> add Soss#dev
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `https://github.com/cscherrer/Soss.jl.git`
 Resolving package versions...
  Updating `~/Soss#dev/Project.toml`
  [8ce77f84] + Soss v0.6.0 #dev (https://github.com/cscherrer/Soss.jl.git)
  Updating `~/Soss#dev/Manifest.toml`
  ... (other dependencies)
  [8f4d0f93] + Conda v1.3.0
  [24249f21] + SymPy v1.0.7

julia> using Soss
[ Info: Recompiling stale cache file /home/chad/.julia/compiled/v1.2/Soss/vXX0U.ji for Soss [8ce77f84-9b61-11e8-39ff-d17a774bf41c]

# *LOTS* OF THESE
WARNING: eval into closed module SymPy:
Expr(:block, #= Symbol("/home/chad/.julia/packages/SymPy/1Cwgd/src/utils.jl"):252 =#, Expr(:call, :P, Expr(:parameters, Expr(:..., :kwargs)), Expr(:::, :ex, :SymbolicObject), Expr(:..., :args)) = Expr(:block, #= Symbol("/home/chad/.julia/packages/SymPy/1Cwgd/src/utils.jl"):252 =#, Expr(:call, Expr(:call, :getproperty, PyCall.PyObject(o=0x00007f6c2c085778), "P"), Expr(:parameters, Expr(:..., :kwargs)), :ex, Expr(:..., :args))))
  ** incremental compilation may be fatally broken for this module **

julia> Soss.stats
PyObject NULL

Any ideas?

[jverzani]

I'm guessing, but am pretty confident, this all needs to be done within a `__init__` call or otherwise engineered, as adding a method isn't allowed. I recall I had a similar issue. My workaround is a bit unsatisfactory, but I pre-generate the commands I want and load them rather than doing so dynamically. See https://github.com/JuliaPy/SymPy.jl/blob/master/src/importexport.jl for the generated list.

…

On Tue, Oct 15, 2019 at 12:24 PM Chad Scherrer ***@***.***> wrote: Hi, I'm hitting a bug I'm having some trouble replicating, hope you might be able to steer me in the right direction. At a very high level, the problem happens when I 1. Import sympy.stats in the standard way within my package 2. using Soss (my package) externally 3. Make a call involving stats Result: julia> Soss.stats PyObject NULL In some cases, this triggers a segfault. ------------------------------ For context, I've tried this in both Julia 1.2 and 1.3. Also, the following works with no trouble: # in a file module M import PyCall import SymPy using SymPy: Sym, sympy, symbols export SymPy, stats, symbols stats = PyCall.pyimport_conda("sympy.stats", "sympy") SymPy.import_from(stats) import Distributions.Normal export Normal Normal(μ::Sym, σ::Sym) = stats.Normal(:Normal, μ,σ) |> SymPy.density Normal(μ,σ) = Normal(promote(μ,σ)...) end using Distributions using .M stats.Normal(:Normal, symbols(:μ),1) |> SymPy.density Normal(symbols(:μ),1) # in the REPL julia> stats.Normal(:Normal, symbols(:μ),1) |> SymPy.density NormalDistribution(μ, 1) julia> Normal(symbols(:μ),1) NormalDistribution(μ, 1) The code in symbolic.jl <https://github.com/cscherrer/Soss.jl/blob/925a28105a819198f59a96980fa48415748e51f8/src/symbolic.jl> takes this same approach. But when I try from a fresh environment, ... (Soss#dev) pkg> add Soss#dev Updating registry at `~/.julia/registries/General` Updating git-repo `https://github.com/JuliaRegistries/General.git` Updating git-repo `https://github.com/cscherrer/Soss.jl.git` Resolving package versions... Updating `~/Soss#dev/Project.toml` [8ce77f84] + Soss v0.6.0 #dev (https://github.com/cscherrer/Soss.jl.git) Updating `~/Soss#dev/Manifest.toml` ... (other dependencies) [8f4d0f93] + Conda v1.3.0 [24249f21] + SymPy v1.0.7 julia> using Soss [ Info: Recompiling stale cache file /home/chad/.julia/compiled/v1.2/Soss/vXX0U.ji for Soss [8ce77f84-9b61-11e8-39ff-d17a774bf41c] # *LOTS* OF THESE WARNING: eval into closed module SymPy: Expr(:block, #= Symbol("/home/chad/.julia/packages/SymPy/1Cwgd/src/utils.jl"):252 =#, Expr(:call, :P, Expr(:parameters, Expr(:..., :kwargs)), Expr(:::, :ex, :SymbolicObject), Expr(:..., :args)) = Expr(:block, #= Symbol("/home/chad/.julia/packages/SymPy/1Cwgd/src/utils.jl"):252 =#, Expr(:call, Expr(:call, :getproperty, PyCall.PyObject(o=0x00007f6c2c085778), "P"), Expr(:parameters, Expr(:..., :kwargs)), :ex, Expr(:..., :args)))) ** incremental compilation may be fatally broken for this module ** julia> Soss.stats PyObject NULL Any ideas? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#301?email_source=notifications&email_token=AADG6TFAX5NA5PAVJTHRO7DQOXVCLA5CNFSM4JA7AIZ2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HR5ST2Q>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADG6TFVMQ6BKD6GJLKOOTLQOXVCLANCNFSM4JA7AIZQ> .

-- John Verzani Department of Mathematics College of Staten Island, CUNY [email protected]

[cscherrer]

Thanks, but I think I'm missing a lot of context here. I'm guessing by __init__ you mean the class initialization method from Python. I've used this from Python, but never from Julia. It's not clear to me how the pieces connect. Do I need a line line one of the ones you have in importexport.jl? Or do I need to build something that writes code like this? I looked for a macro that writes this code but don't see anything. Is it a build script somewhere?