MacOS Sonoma iconv library is not compatible with gnu libiconv - Move from native MacOS iconv to gnu libiconv?

[fredowski]

Output of brew config

Fredo:pspp fritz$ brew config
HOMEBREW_VERSION: 4.1.18-51-g930f24b
ORIGIN: https://github.com/Homebrew/brew
HEAD: 930f24b63fdbf5858358fdd6a7eb6bdacd078be8
Last commit: 21 hours ago
Core tap JSON: 05 Nov 18:01 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 4
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: quad-core 64-bit broadwell
Clang: 14.0.0 build 1400
Git: 2.37.1 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 8.1.2 => /usr/bin/curl
macOS: 12.7-x86_64
CLT: 14.2.0.0.1.1668646533
Xcode: N/A
Fredo:pspp fritz$

Output of brew doctor

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  [email protected]

Description of issue

I maintain the brew pspp tap. pspp uses iconv and gettext. The pspp regression fails on MacOS Sonoma because the new Apple iconv library behaves not according to what iconv is supposed to do. Our pspp regression tests the response to an invalid UTF-8 character. The discussion about the regression problem for pspp is here:

fredowski/homebrew-pspp#5 (comment)

Now it seems difficult to use the gnu libiconv library as gettext uses the native iconv library. Are there reasons to stick with the "buggy" Apple native library or would it be an idea to move brew to the gnu libiconv? The problem is discussed at other places also:

Homebrew/homebrew-core#142490 (comment)
https://gitlab.gnome.org/GNOME/gtk-osx/-/merge_requests/44

[kencu]

because the symbols in a system-installed libiconv and an external libiconv are different (upstream design decision), there is opportunity for link issues to arise when the “wrong” libiconv is found at link time opportunistically.

This issue causes headaches for macports, who made this change some years back… I won’t say often, but certainly not rarely either. Some of these issues, like building stack for example, have proven hard to resolve.

commercialhaskell/stack#6182

It also happens to folks when the build their own code, pulling in the system headers but the external library gets linked in opportunistically. Many of these link failures point out build system issues with header search paths and library search paths that indeed need fixing, but not all are trivial to fix, and they do cause frustrations.

I came to (unscientifically) believe it was the most-reported complaint against macports in StackOverflow, for example.

[fredowski]

Thanks for your thoughts! I think if the differences between gnu libiconv and Apple iconv are considered as bugs at Apple, then one could hope that the problem of the differences would vanish in the future. If Apple decides to have a different behaviour as feature, then this would mean to have platform specific code in the applications. I do not think that this would be a proper way.

pspp uses iconv directly and it uses gettext which uses iconv via libintl. Is it possible to build pspp such that is uses gnu libiconv while gettext uses native iconv? Or is there a reason not to go that path?

[kencu]

I’m sorry not to be an expert on that issue to help you, and perhaps someone else will be, but I think you could try linking your pspp against the keg-only gnu libiconv and see how that fares.

There are examples in the repo where other formulae force gnu tools to be used instead of system tools. The exact method depends on whether pkgconfig is used to find the paths to the includes and libs or not, but here’s a formula that forces a gnu library for sqlite to be found, for example:

https://github.com/Homebrew/homebrew-core/blob/418d57e7f7b95774fbf0608d80b667c2e0b91ab9/Formula/lib/libspatialite.rb#L65C5-L65C5

I think it’s worth a test to see if that solves the pspp issue (until Apple fixes theirs, presumably).