Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support FreeBSD + avoid hard-coding excluded library versions #45

Open
wants to merge 4 commits into
base: release/2.0
Choose a base branch
from

Conversation

ararslan
Copy link

@ararslan ararslan commented Dec 5, 2021

Summary/motivation of changes:

  • FreeBSD has sys/auxv.h but it does not have getauxval, and its equivalent does not have AT_PLATFORM. Luckily, we don't actually need this at all on FreeBSD since, unlike Linux, libraries aren't put into platform-prefixed directories.
  • bash is not installed by default and when it is, it does not live in /bin. Thus the integration tests shebang has been changed to use /usr/bin/env bash, which should work across platforms.
  • BSD diff, which macOS also uses, does not support the --color flag, so that now enabled only on Linux.
  • Rather than including versioned shared libraries in the exclusion list, we can list unversioned library names and strip any version number from the library currently being inspected. This accounts for cases like FreeBSD's libc.so.7, which would have to be added as another special case alongside Linux's libc.so.6 otherwise.

With these changes, libtree builds and passes all tests on FreeBSD 12.2 on x86_64.

Summary/motivation of changes:
- FreeBSD has `sys/auxv.h` but it does not have `getauxval`, and its
  equivalent does not have `AT_PLATFORM`. Luckily, we don't actually
  need this at all on FreeBSD since, unlike Linux, libraries aren't put
  into platform-prefixed directories.
- `bash` is not installed by default and when it is, it does not live in
  `/bin`. Thus the integration tests shebang has been changed to use
  `/usr/bin/env bash`, which should work across platforms.
- BSD `diff`, which macOS also uses, does not support the `--color`
  flag, so that now enabled only on Linux.
- Rather than including versioned shared libraries in the exclusion
  list, we can list unversioned library names and strip any version
  number from the library currently being inspected. This accounts for
  cases like FreeBSD's `libc.so.7`, which would have to be added as
  another special case alongside Linux's `libc.so.6` otherwise.
@haampie
Copy link
Owner

haampie commented Dec 6, 2021

Thanks Alex! This is great!

I noticed a few more things, but that shouldn't block this PR:

  1. There is a $PLATFORM variable that can be interpolated in rpaths, not sure where it is taken from if not from getauxval
  2. FreeBSD has $OSNAME, $OSREL variables but not $LIB in rpaths
  3. FreeBSD has multiple config files for search paths: /etc/ld.so.conf and /etc/ld-elf.so.conf, is the latter included by the former?

@ararslan
Copy link
Author

ararslan commented Dec 6, 2021

Ah, good notes!

  1. I didn't realize FreeBSD had $PLATFORM, I wonder in what cases it's used. I looked at the RTLD source and this, as well as $OSNAME and $OSREL, come from uname. So e.g.

    #include <sys/utsname.h>
    // ...
    struct utsname* uts;
    uname(uts);
    printf("$PLATFORM: %s\n$OSNAME: %s\n$OSREL: %s\n",
           uts->machine, uts->sysname, uts->release);

    In case you're curious, the source I was looking at is here. I'll amend the platform substitution part of the PR and add the OS name and release to the regex substitutions.

  2. I guess I should move the $LIB substitution into a Linux-only conditional.

  3. I can't find any information or documentation about /etc/ld.so.conf, I can only find /etc/ld-elf.so.conf. So I guess the latter is the only one we'd need to care about, then?

With this change, we use `getauxval` to get the platform on Linux when
that function is available, and when it isn't, we use the POSIX `uname`
to retrieve that info. The `utsname` struct gets passed around as well
so that substitutions other than `$PLATFORM` can be applied downstream
with only one additional function parameter.
@ararslan
Copy link
Author

ararslan commented Dec 7, 2021

Okay I figured out the BSD-specific substitutions but I don't love how I did it (passing around a utsname object in addition to platform, which may come from the utsname). Do you know whether there are cases where getauxval(AT_PLATFORM) and .machine from utsname won't be the same? If they'll always be equal, we should be able to just use utsname everywhere and drop getauxval.

@haampie
Copy link
Owner

haampie commented Dec 7, 2021

Hm, another option is to work with a list of regexes + replacements Vector{Pair{Regex,String}} std::vector<std::pair<std::regex,std::string>>> and populate them in main and pass them to tree, if that makes life easier.

For me AT_PLATFORM / .machine are equal strings but they're not the same pointer (I hoped the one would call the other :p)

@ararslan
Copy link
Author

ararslan commented Dec 7, 2021

Oh true. I can implement that here if you think that'd be a better solution. I don't have much of an opinion.

@haampie
Copy link
Owner

haampie commented Dec 7, 2021

FYI, Linus Torvalds wrote this 17 years ago:

/* This yields a string that ld.so will use to load implementation
   specific libraries for optimization.  This is more specific in
   intent than poking at uname or /proc/cpuinfo.  */

In Linux it goes roughly like this:

#define ELF_PLATFORM <something in some arch/.../include/.../elf.h>
const char *k_platform = ELF_PLATFORM;
copy_to_user(u_platform, k_platform, len)
NEW_AUX_ENT(AT_PLATFORM, (elf_addr_t)(unsigned long)u_platform);

@haampie
Copy link
Owner

haampie commented Dec 7, 2021

Meanwhile though ...

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/include/asm/elf.h#n238

/* I'm not sure if we can use '-' here */
#define ELF_PLATFORM       ("x86_64")

😆

@haampie haampie changed the base branch from master to release/2.0 December 11, 2021 14:15
@haampie haampie added the 2.x label Dec 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants