Conda recipe for Linux

[gidiko]

The intention is to combine this Linux recipe for libelemental with the OS X recipe in #151

Updated/made a few modifications to the 0.86.xx one (most notably the addition of lib64/ for finding the libs; previously only lib/ was needed there). This snapshot seems to work nicely with our libskylark, but surely it could be made more generic and improved after being combined with the OS X conda recipe.

Some notes:

• Building the recipe (builds OK with few warnings in my machine):
  [gkollias@dccxl001 Elemental]$ conda build conda

• Putting the resulting tarball in a dir - will play the role of "local" conda channel (alternatively upload e.g. to an Anaconda repository):
  cp /u/gkollias/anaconda2/conda-bld/linux-64/libelemental-0.87.5-0.tar.bz2 /u/gkollias/conda/channel/linux-64/

• Indexing the channel:
  conda index /u/gkollias/conda/channel/linux-64/

• Installing libelemental (to be used with an Anaconda installation):
  conda install libelemental=0.87.5 --channel file:///u/gkollias/conda/channel

• Using libelemental's python bindings (the following just works!):
  In [1]: import El; A = El.Matrix(); El.Uniform(A, 100, 100); triplet = El.SVD(A);

[rhl-]

This looks very straightforward. Is there some way to test that this works?

[rhl-]

Perhaps you can edit the Travis builds to perform the installation itself after the existing build ?

[gidiko]

The short answer:
You need a Miniconda or Anaconda installation for this.

The long answer:
You need a Miniconda or Anaconda installation for actually building with the conda command the recipe (which is a couple of files) into a tarball containing ... 459 files in all (for the above snapshot): .h, .hpp, .py, .so, executables etc. The produced binaries are automatically patched by patchelf during this conda build, so working over an existing build is not a clean option after all.

Then you can "test" (the quotes stand for "manually") this built tarball by conda installing it and checking the python bindings (more or less along the lines shown above) or going into the *conda installation's lib/, include/ etc directories and linking against or using the libelemental-artifacts that landed there.

Now, for doing real testing you can start populating test-related sections in meta.yaml http://conda.pydata.org/docs/building/meta-yaml.html (roughly checking artifacts under Examples/ and Tests/ as in the "actual" build - I mean from what I see towards the end of the Travis trace - not the "conda" one (which as noted does not happen as of now).

Going one step further, actually for automating the whole process you can "upload" the recipe (I mean after the additions of tests or other tweaks and first of all after the merge with the OS X recipe) to conda-forge so they do the conda build and make the build readily available for installation for all anaconda users (so you do the 3-4 steps under the "ADD A RECIPE" part in https://conda-forge.github.io/ and if it all works OK, down the path a user will do something like conda install libelemental and automatically get the software to use it like in the single-line python session above).

[rhl-]

Reposting as the appropriate user:

Thanks for the detailed response.

Would you be willing to produce something, such as a travis build, or links to an outside automated CI tool, that demonstrate that this package is building? We also are baking a project jenkins instance, to build every PR and master and do some things not doable in the travis/appveyor environment. We could try and host this sort of thing there.

This doesn't need to gate the merge to master, but, we would definitely appreciate any help in supporting the existence of this package going forward.

As an example, we have the RHEL spec files hooked up to Fedora provided CI that builds on every commit to master: https://copr.fedorainfracloud.org/coprs/rhl/elemental/packages/ (hopefully this will move into a elemental group in the near future).

[poulson]

Why disable El::Quad?

[poulson]

I would recommend not enabling the Hybrid build by default: is there a good reason to do so?

[poulson]

Is there a reason to avoid GMP/MPFR/MPC?

[gidiko]

@rhl
The right way that integrates also CI infrastructure is conda-forge that I was talking about... I will be attempting to put the set of updated recipes for libskylark in the next few days and in doing so I will also include a custom configuration for libelemental... so get a feeling how these work together... or perhaps try something along the line of travis downloading a miniconda, then building the recipes, etc... so I certainly will come back to this... you may also give it a try as well for the libelemental recipe alone at conda-forge in the meantime...

@poulson

1. quadmath: GPL "issue" (we are also packing the software in an installer)
2. hybrid: no particular reason ;)
3. default mpc lib in conda comes as libmpc.a and this does not work in the relevant build

But now after the latest commits (libelemental also builds OK after them) i.e.:

1. enabled quadmath
2. disabled hybrid
3. added conda-forge as a repo [a], conda updated the mpc package [b] and then libmpc.so lands in conda's lib dir and things work with mpc enabled :)
   [a] conda config --add channels conda-forge
   [b] conda update mpc

[rhl-]

This all seems good to me.

[poulson]

@gidiko Thank you for the very useful response! For what it's worth, libquadmath is released under the LGPL and not the GPL, so it should be okay to include. Elemental's SuiteSparse modifications are under the LGPL as well.

[gidiko]

Thanks for pointing this out @poulson ... basically was reading http://libelemental.org/documentation/dev/build.html under libquadmath and had gotten the impression it was under GPL only... but now I see https://github.com/gcc-mirror/gcc/blob/master/libquadmath/COPYING.LIB which clearly says LGPL

[poulson]

@gidiko Thank you for letting me know that the documentation was out of date on that important issue. I have removed this misleading information.

[rhl-]

@gidiko is there any reason we can't merge this?

[grlee77]

I was recently checking to see if there was already a recipe for this on conda-forge and came across this PR. Great to see that someone has already made a recipe!

Is there an intention to make a PR over at https://github.com/conda-forge/staged-recipes/ as well?

A couple of other comments:
1.) The optional METIS dependency is now available through the conda-forge channel as well (version 5.1.0)
2.) I recently opened a PR for the optional QD (quad-double) dependency here:
conda-forge/staged-recipes#2408

If you have any feedback on how that one is compiled and if it looks compatible with Elemental, I would appreciate it. Also, if any of you are interested in being a co-maintainer on METIS, QD or other related packages, please let me know. Mainly this gives you commit rights to the conda-forge feedstocks so that updated recipes/builds can be made as new versions are released.