Windows 10 support

[Yattabyte]

Hi there, I'm having an issue compiling this library on windows 10. I'm using Intel OneAPI 2023s next-gen compilers (ICX/ICPX), and have visual studio 2022 installed. My environment is git-bash/cygwin. I've disabled MPI, OpenMP, slate, scotch, parmetis, basically trying to get the minimal viable version to work with Intel MKL's BLAS+LAPACK implementation.

For reference, I've managed to compile this library successfully with the above configuration on MacOS Catalina & Monterey, as well on RHEL6 & RHEL7 linux. On those operating systems I target Clang, GCC-11, ICC, and ICX when available, and this library works out of the box there.

I've encountered three sets of issues that are blocking me from compiling on windows. Two of them I was able to solve, though the third I cannot. My position on my development team is not one that uses these types of libraries, though I'm responsible for preparing them, deploying them, and ensuring our code can link against them, so my knowledge is limited with regards to troubleshooting BLAS/LAPACK or even fortran issues.

1. All uses of std::iota and std::partial_sum require including the <numeric> header on windows. I adjusted the code as such, leading to the next error.
2. The NoInit class seems to be missing some boilerplate to work on windows, appearing to also have non-viable implicit default/move/copy ctors:

In file included from strumpack\src\sparse\fronts\FrontalMatrixDense.cpp:30:
In file included from strumpack\src\sparse\fronts/FrontalMatrixDense.hpp:36:
In file included from C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\include\random:17:
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\include\vector(832,27): error : no matching conversion for static_cast from '_Alty' (aka 'NoInit<float>') to '_Rebind_alloc_t<_Alty, _Container_proxy>' (aka 'NoInit<std::_Container_proxy>')
        auto&& _Alproxy = _GET_PROXY_ALLOCATOR(_Alty, _Getal());
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\include\xmemory(1352,42): note: expanded from macro '_GET_PROXY_ALLOCATOR'
#define _GET_PROXY_ALLOCATOR(_Alty, _Al) static_cast<_Rebind_alloc_t<_Alty, _Container_proxy>>(_Al)
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
strumpack\src\sparse\fronts\FrontalMatrixDense.cpp(550,18): note: in instantiation of member function 'std::vector<float, strumpack::NoInit<float>>::~vector' requested here
  template class FrontalMatrixDense<float,int>;
                 ^
strumpack\src\sparse\fronts\FrontalMatrixDense.cpp(550,18): note: in implicit destructor for 'strumpack::FrontalMatrixDense<float, int>' first required here
strumpack\src\sparse\fronts\FrontalMatrixDense.cpp(550,18): note: in instantiation of template class 'strumpack::FrontalMatrixDense<float, int>' requested here
strumpack\src\misc/Tools.hpp(52,30): note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'NoInit<float>' to 'const NoInit<std::_Container_proxy>' for 1st argument
  template<typename T> class NoInit {
                             ^
strumpack\src\misc/Tools.hpp(52,30): note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'NoInit<float>' to 'NoInit<std::_Container_proxy>' for 1st argument
strumpack\src\misc/Tools.hpp(52,30): note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was **provided**

• I took a stab at solving this after some research and aide from my IDE (look to the ifdef _WIN32 block):

  template<typename T> class NoInit {
  public:
    using value_type = T;
#ifdef _WIN32
    template<typename U>
    struct rebind { using other = NoInit<U>; };
    NoInit() noexcept { }
    NoInit(const NoInit&) noexcept { }
    template <class U> NoInit (const NoInit<U>&) noexcept { }
    void deallocate(T* p) noexcept { ::operator delete(p); }
    std::size_t max_size() const noexcept
    { return std::numeric_limits<std::size_t>::max() / sizeof(value_type); }
    template<class U> void destroy(U* p) { p->~U(); }
    T* address(value_type& x) const noexcept { return &x; }
    const T* address(const value_type& x) const noexcept { return &x; }
#endif
    value_type* allocate(std::size_t n)
    { return static_cast<value_type*>(::operator new(n*sizeof(value_type))); }
    void deallocate(T* p, std::size_t) noexcept { ::operator delete(p); }
    //template <class U, class ...Args> void construct(U* /*p*/) { }
    template <class U, class ...Args> void construct(U* p, Args&& ...args) {
      ::new(p) U(std::forward<Args>(args)...);
    }
  };

3. The BLASLAPACKWrapper.cpp appears to not correctly resolve the appropriate BLAS/LAPACK functions in Intel MKL's BLAS/LAPACK implementation on windows, I think. Here are some excerpts of the errors I've encountered:

BLASLAPACKWrapper.cpp(885,39): error : use of undeclared identifier 'SDOTU'; did you mean 'SDOT'?
      return STRUMPACK_FC_GLOBAL(sdot,SDOTU)(&n_, x, &incx_, y, &incy_);
             ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
BLASLAPACKWrapper.cpp(1562,34): error : use of undeclared identifier 'DGETRS'
      STRUMPACK_FC_GLOBAL(dgetrs,DGETRS)(&t, &n, &nrhs, a, &lda, ipiv, b, &ldb, &info);
                                 ^
BLASLAPACKWrapper.cpp(1569,34): error : use of undeclared identifier 'CGETRS'
      STRUMPACK_FC_GLOBAL(cgetrs,CGETRS)(&t, &n, &nrhs, a, &lda, ipiv, b, &ldb, &info);
                                 ^

• There are many more like this, like CUNGLQ, ZUNGLQ, ZORGLQ, etc. My compiler eventually just gives up on outputting errors and just terminates.

Is there anything I should try?

[pghysels]

If you can create a pull request with the changes (include , and the boilerplate for NoInit), I'd be happy to include this.

For the BLAS/LAPACK errors, can you check the definition of STRUMPACK_FC_GLOBAL?
This should be defined by CMake in the file StrumpackFortranCInterface.h in the build directory.
For instance, for me it is defined as:

#define STRUMPACK_FC_GLOBAL(name,NAME) name##_

[Yattabyte]

The contents of StrumpackFortranCInterface.h on windows (at least in my environment):

#ifndef STRUMPACK_FC_HEADER_INCLUDED
#define STRUMPACK_FC_HEADER_INCLUDED

/* Mangling for Fortran global symbols without underscores. */
#define STRUMPACK_FC_GLOBAL(name,NAME) NAME

/* Mangling for Fortran global symbols with underscores. */
#define STRUMPACK_FC_GLOBAL_(name,NAME) NAME

/* Mangling for Fortran module symbols without underscores. */
#define STRUMPACK_FC_MODULE(mod_name,name, mod_NAME,NAME) mod_NAME##_mp_##NAME

/* Mangling for Fortran module symbols with underscores. */
#define STRUMPACK_FC_MODULE_(mod_name,name, mod_NAME,NAME) mod_NAME##_mp_##NAME

#endif

However, if I replace it with the one generated on MacOS (that appears as you've described with #define STRUMPACK_FC_GLOBAL(name,NAME) name##_), the compilation gets much further but hits several linking issues:

(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol myslapmr_ referenced in function "void __cdecl strumpack::blas::lapmr(bool,int,int,float *,int,int const *)" (?lapmr@blas@strumpack@@YAX_NHHPEAMHPEBH@Z)
(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol mydlapmr_ referenced in function "void __cdecl strumpack::blas::lapmr(bool,int,int,double *,int,int const *)" (?lapmr@blas@strumpack@@YAX_NHHPEANHPEBH@Z)
(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol myclapmr_ referenced in function "void __cdecl strumpack::blas::lapmr(bool,int,int,class std::complex<float> *,int,int const *)" (?lapmr@blas@strumpack@@YAX_NHHPEAV?$complex@M@std@@HPEBH@Z)
(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol myzlapmr_ referenced in function "void __cdecl strumpack::blas::lapmr(bool,int,int,class std::complex<double> *,int,int const *)" (?lapmr@blas@strumpack@@YAX_NHHPEAV?$complex@N@std@@HPEBH@Z)
(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol sgeqp3tol_ referenced in function "int __cdecl strumpack::blas::geqp3tol(int,int,float *,int,int *,float *,float *,int,int &,float,float)" (?geqp3tol@blas@strumpack@@YAHHHPEAMHPEAH00HAEAHMM@Z)
(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol dgeqp3tol_ referenced in function "int __cdecl strumpack::blas::geqp3tol(int,int,double *,int,int *,double *,double *,int,int &,double,double)" (?geqp3tol@blas@strumpack@@YAHHHPEANHPEAH00HAEAHNN@Z)
(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol cgeqp3tol_ referenced in function "int __cdecl strumpack::blas::geqp3tol(int,int,class std::complex<float> *,int,int *,class std::complex<float> *,class std::complex<float> *,int,int &,float,float)" (?geqp3tol@blas@strumpack@@YAHHHPEAV?$complex@M@std@@HPEAH00HAEAHMM@Z)
(BLASLAPACKWrapper.obj) : : error LNK2019: unresolved external symbol zgeqp3tol_ referenced in function "int __cdecl strumpack::blas::geqp3tol(int,int,class std::complex<double> *,int,int *,class std::complex<double> *,class std::complex<double> *,int,int &,double,double)" (?geqp3tol@blas@strumpack@@YAHHHPEAV?$complex@N@std@@HPEAH00HAEAHNN@Z)
(MatrixReordering.obj) : : error LNK2019: unresolved external symbol genrcm_ referenced in function "void __cdecl strumpack::WRAPPER_rcm<int>(class std::vector<int,class std::allocator<int> > &,class std::vector<int,class std::allocator<int> > &,class std::vector<int,class std::allocator<int> > &)" (??$WRAPPER_rcm@H@strumpack@@YAXAEAV?$vector@HV?$allocator@H@std@@@std@@00@Z)
(MatrixReordering.obj) : : error LNK2019: unresolved external symbol amdbar_ referenced in function "void __cdecl strumpack::ordering::WRAPPER_amd<int>(int,int *,int *,int *,int *)" (??$WRAPPER_amd@H@ordering@strumpack@@YAXHPEAH000@Z)
(MatrixReordering.obj) : : error LNK2019: unresolved external symbol ordmmd_ referenced in function "class strumpack::SeparatorTree<int> __cdecl strumpack::ordering::mmd_reordering<int>(int,int const *,int const *,class std::vector<int,class std::allocator<int> > &,class std::vector<int,class std::allocator<int> > &)" (??$mmd_reordering@H@ordering@strumpack@@YA?AV?$SeparatorTree@H@1@HPEBH0AEAV?$vector@HV?$allocator@H@std@@@std@@1@Z)

I hope my development environment isn't too unusual, but seeing as I mostly target linux and get MacOS for basically free, I've attempted to get my Windows environment as similar as possible with cygwin and git-bash.

[pghysels]

Some Fortran compilers add an underscore to routine names, and some don't (some use capital names, others lower case).
CMake detects that your Fortran compiler does not add an underscore, so it just defines NAME.
But it looks like the BLAS/LAPACK you are linking to was compiled with a compiler that does add an an underscore, so you need to use name##_ to link to these routines. But then there is a problem linking to Fortran routines in STRUMPACK, such as my?lapmr, ?geqp3tol, genrcm, amdbar and ordmmd.
I have not seen this issue before.

Do you have another Fortran compiler on your system that you can try?

Another thing to try is to compile OpenBLAS with your Fortran compiler yourself. OpenBLAS provides both BLAS and LAPACK. Then you can also compile ScaLAPACK and link it to that OpenBLAS (in case you want MPI support).

[Yattabyte]

On my production servers I only have Intel OneAPI's ifx fortran compiler, which I would think should be compatible with their own blas/lapack libraries in MKL since they all come from the same vendor in the same package. I'll try the latest version of OneAPI on my windows test environment in case it works any better.

[Yattabyte]

Same issue after updating visual studio and OneAPI's mkl and fortran compiler to latest version

[pghysels]

Can you try openblas?