Wednesday, 2 August 2023

Re: Changes in the container stack

Hi Seth,

Em 01/08/2023 23:02, Seth Arnold escreveu:
> On Tue, Aug 01, 2023 at 06:45:59PM -0300, Lucas Kanashiro wrote:
>> With that in mind, we, the Server team, identified the need to decouple the
>> application (what users really use) and the library (what is used by rdeps)
>> in a way that, on the one hand, we can keep following upstream projects
>> without worrying about breaking changes and on the other hand, we keep the
>> library (-dev package) stable to avoid breakages of packages sync'ed from
>> Debian or already in stable releases during SRUs.
> Hello Lucas, I'm very concerned about this plan, I hope you can assuage my
> fears.

Thanks for raising your concerns. I hope I can help you clear them out :)

> This sounds like we'll introduce a new problem, where third-party
> applications built against the libraries will become incompatible with
> the main application.

That _might_ happen (depending on how those apps are interacting) if
they are using the -dev package in the archive to build their app.
However, from what I have seen in the Go ecosystem, people will not be
doing that, they will be using the vendor code from upstream, pinning
the version they want.

For the packages in the archive depending on -dev packages, I have seen
API changes in generic functions used by other packages (software that
does not interact directly with the containers stack but re-use some
exported functions). So making those rdeps use an old version of the
library does not seem an issue most of the time, considering there is no
interaction during runtime among those components.

> If the ecosystems are changing external library ABIs too often, I can
> only shudder in fear at how quickly internal library ABIs are changing.
>
> What gives us the confidence that the internal ABIs are actually more
> stable and reliable than the external ABIs?

All the breakages I have seen so far are related to API changes (TBH not
too complex changes), the main problem is that those changes require
backporting upstream patches which usually slow down our velocity of
updates (in the end, we cannot follow upstream releases because SRU
process takes too long). I haven't seen weird ABI changes.

For the container stack itself, I do some manual testing with all the
packages and there are also DEP-8 tests which guarantee that those 3
pieces of software work well together before I perform any upload to the
archive. Thinking about third-party apps interacting with the stack,
usually, those upstream projects follow the latest (or considerably new)
version of docker/containerd/runc, so to avoid ABI breakages would be
better to follow upstream closely. Keeping an old version of those
software is more likely to cause ABI issues.

As I mentioned, this ecosystem moves fast, if we do not try to follow
upstream we will not be able to provide value to those users. This is a
problem I have seen, some users are preferring to use, for instance, the
Docker debian package provided by the Docker upstream maintainers
instead of ours, because we are sometimes way behind them. I have some
plans in mind to change this scenario, the decoupling I presented here
is just the first step to improve users' experience.

I hope my answer makes sense to you Seth. Let me know if it does not.

Cheers!

--
Lucas Kanashiro


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel