Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a flag which delays the init until IJulia cell #480

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

xzackli
Copy link

@xzackli xzackli commented Apr 21, 2020

Here's another attempt to allow PyPlot to work in PackageCompiler sysimages, for #476. The goal is to delay the init code until Jupyter cell execution, by adding init to the pre-execution hooks. It does this only if a new global flag delay_init_until_cell is set to true (false by default) and it finds a Jupyter environmental variable. You can set that flag to true in the PackageCompiler configuration script.

This PR only changes when the initialization occurs for Jupyter. This should not change any behavior unless delay_init_until_cell is set to true in i.e. the PackageCompiler config script.

You have to specify a script.jl for PackageCompiler,

# file: script.jl
using PyPlot
PyPlot.delay_init_until_cell[] = true

and make the image with IJulia and the above script.jl,

using PackageCompiler
create_sysimage(
    [:PyCall, :IJulia, :PyPlot]; 
    sysimage_path = "mysys.dylib",
    script = "script.jl")

If you use this same sysimage in the REPL, you have to call ion() manually, probably because of the early init order of being in a sysimage.

It would be nice if there were a way for a package to know that it is in a sysimage so that one could avoid these caveats, but I'm not sure how to probe the order of init execution.

@xzackli
Copy link
Author

xzackli commented May 8, 2020

I've given this some testing by using it for two weeks, and it seems to work fine. I've marked it ready for review -- please let me know if you have any comments, and I can try to make modifications.

@stevengj
Copy link
Member

stevengj commented May 9, 2020

I really don't like using environment variables for this sort of thing; they "leak" into subprocesses, and in general can't be relied upon. And note the IJulia might not be used from Jupyter — it could be used from any other program that implements the Jupyter messaging protocol.

@tfiers
Copy link

tfiers commented Sep 16, 2022

Tested this on my windows laptop, it works well. Thanks @xzackli !

tfiers added a commit to tfiers/Sciplotlib.jl that referenced this pull request Sep 20, 2022
that was only bc I used PyPlot fork (pr 480) in sysimg

JuliaPy/PyPlot.jl#480
tfiers added a commit to tfiers/phd that referenced this pull request Sep 20, 2022
@tfiers
Copy link

tfiers commented Sep 22, 2022

On second thought, this solution gives a problem when creating custom packages that depend on PyPlot, and that expect PyPlot's __init__ to have been called in their own __init__ -- which is normally the case, but not with this patch.

My specific situation: I had custom packages: PkgA ← PkgB ← PyPlot,
with only PyPlot in the sysimg.

Loading PkgA yielded the error:
Failed to precompile PkgA, .. error during initialization of module PkgB:
..
LoadError: InitError: ArgumentError: ref of NULL PyObject

This was because I accessed a property of PyPlot.plt in PkgB's __init__; but plt was not initialized yet, as PyPlot's init wasn't called during precompilation (it normaly is, but not with this patch).

@stevengj
Copy link
Member

It seems like a better way to do this would be to (a) use the LazyPyModule type for all the Python modules (plt, matplotlib, …) and (b) update the LazyPyModule instantiation code to call init.

This way, it would initialize PyPlot lazily the first time you tried to access anything in Matplotlib, regardless of whether it is in an IJulia cell or somewhere else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants