Wednesday, 14 June 2023

Re: Symbols files for C++ libraries for Ubuntu main

On Fri, Jun 9 2023 at 12:10:07 -0700, Steve Langasek
<steve.langasek@ubuntu.com> wrote:
> Hi Seb,
>
> On Fri, Jun 09, 2023 at 02:27:02PM +0200, Sebastien Bacher wrote:
>> I would like to ask if there is any chance the MIR team would
>> reconsider
>> their position on the topic (at least until the day we have a
>> somewhat
>> working solution we can use)?
>
>> which also included those types of changes
>
>> - _Znam@Base 2.0~b2-0ubuntu3
>> + _Znaj@Base 2.0~b2-0ubuntu4
>> +#MISSING: 2.0~b2-0ubuntu4# _Znam@Base 2.0~b2-0ubuntu3
>
>> I personally don't understand why we have those symbols existing on
>> armhf
>> which don't exist on amd64. Nor why _Znam@Base is becoming
>> _Znaj@Base nor
>> how we are supposed to handle such cases
>
> Passing them through c++filt may help explain:
>
> $ echo _Znam@Base | c++filt
> operator new[](unsigned long)@Base
> $ echo _Znaj@Base | c++filt
> operator new[](unsigned int)@Base
> $
>
> There are various C++ functions whose signature will change based on
> the
> architecture word length.
>
> .symbols files support various kinds of globbing etc to be able to
> express
> this logically (e.g., you could say '_Zna[mj]@Base' instead of
> listing two
> different symbols as optional), but as you've found, it's an onerous,
> iterative process to identify all the ways C++ symbols vary across
> architectures and then encode this in a .symbols file. And in this
> case,
> the symbol isn't part of the library's public ABI anyway, this is
> just a
> function from the base C++ library!
>
>> 4. doing those tweaks need to be done manually since it's not only
>> applying
>> the diff but also adding optional keyword at places, I got one
>> wrong and it
>> failed to build again
>
>> add one more symbol specific to risvc
>>
>> http://launchpadlibrarian.net/647875197/libcupsfilters_2.0~b2-0ubuntu6_2.0~b2-0ubuntu7.diff.gz
>
> Yep.
>
>> I understand the motivation for wanting a symbol file but I agree
>> with what
>> Robie, what's the benefit. In that case we spent a few hours to end
>> up with
>> a .symbols which as over 150 '(optional)' entries, that doesn't
>> protect us
>> much better than just not having a .symbols or using -c0 but still
>> has a
>> high cost.
>
> I wouldn't say that it doesn't protect you. It's a pain to set up
> initially
> and as you note, you can even have to do further fix-ups as a result
> of
> toolchain changes, as the set of template functions and other C++
> sugar from
> outside of the library that gets exported as ELF symbols can change.
> It
> DOES have a high cost, but in the end it provides the same level of
> protection against accidental ABI breakage as it does for C libraries.
>
> It would be nice to have better consistent tooling around ABI
> checking for
> C++ libraries. I think the KDE team had some tools around automating
> the
> generation of symbols files - it does require two passes, first to
> build on
> all archs and then to merge the results. But in principle that's
> better
> than whack-a-mole.
>
> We could also consider using abi-compliance-checker instead of
> symbols files
> for C++ libraries. There is a dh-acc debhelper addon, but I've never
> used
> it. We are currently using abi-compliance-checker for the ABI
> analysis of
> armhf for the move to 64-bit time_t; it's unmaintained upstream, but
> it does
> seem to work pretty well - the vast majority of issues we've
> encountered
> with it, when trying to run it over the entire Ubuntu archive, have
> been due
> to header files that #include headers from packages they don't depend
> on, or
> collections of headers that can't all be included together. Both of
> these
> are issues of much less significance when it's the maintainer doing
> the
> work. It would require the same sort of two-pass setup process as
> the KDE
> tools, though, and if it has to be done per-arch (probably), it's more
> awkward to set up because a-c-c .dump files aren't ascii that you can
> scrape
> from the build logs of failed builds - but it might be a more
> reliable tool
> over the long term than dpkg-gensymbols for C++ libraries.

In the Mir (not MIR ☺) team we've periodically been annoyed by
maintaining symbols files for C++¹, and have experimented with both
abi-compliance-checker and abigail. None of those experiments have
ended up sticking, though, for reasons which I'm not fully aware of.
Alan Griffiths and MichaƂ Sawicz did most of that investigation; I'll
see if they can help shed light on problems we encountered.

If we *can* get (one of) the ABI checking tools working they'll be more
valuable than a symbols file anyway, as they actually check that ABI
didn't change rather than just that the symbol strings in the DSO match.

¹: and we're playing on *easy* mode, where we, as upstream, use a
linker script to export only symbols we intend to export.

>
> Downside: symbols files also let you track what version of a package a
> symbol was added in, which lets packages' versioned dependencies on
> libraries be no stricter than actually necessary.
>
>
> I don't speak for the MIR team, I have no objection to them relaxing
> the
> requirement of .symbols files for C++ libraries in main. Just
> offering some
> suggestions on how we can do a better job of automating C++ ABI
> checks than
> we're doing today.
>
> Cheers,
> --
> Steve Langasek Give me a lever long enough and a
> Free OS
> Debian Developer to set it on, and I can move the
> world.
> Ubuntu Developer
> https://www.debian.org/
> slangasek@ubuntu.com
> vorlon@debian.org
> --
> ubuntu-devel mailing list
> ubuntu-devel@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel