On Tue, Jul 04, 2017 at 01:22:05PM -0700, Steve Langasek wrote:
> On Tue, Jul 04, 2017 at 04:31:32PM +0000, James Page wrote:
> > Mathias and I where discussing some challenges around fitting the Ceph
> > build with any level of parallelism in to a virtualized launchpad builder
> > (as used for the majority of architectures today).
>
> > Basically the current memory spec only permits a max-parallel of 2 (on
> > Xenial amd64). Ceph is a big C++ project so takes some time to build with
> > such a low level of parallelism.
>
> > Would it be possible to nudge up the default memory allocation on the
> > builders? I use a 16G machine for test builds of i386/amd64 and I've not
> > seen any memory exhaustion with parallel=8.
>
> > Or maybe have a mechanism to allow a source package to request a higher
> > spec of builder?
>
> You should probably consult the Launchpad team on this, since the Launchpad
> builder configuration is owned by them and not by the Ubuntu developers
> directly.
>
> As far as raising the default memory allocation, understand that there is a
> tradeoff between being able to parallelize /within/ a build and being able
> to parallelize across /all/ builds. If the sole justification for this
> request is that it would speed up the build of one particular package, I
> don't think that's a sufficient reason to reduce the capacity of the
> launchpad build farm as a whole.
Indeed - increasing the size of a single builder would mean reducing the
number of builders we can fit on scalingstack, and I don't think that's
a good trade-off globally. The current size seems to be a sweet spot,
even if it means that some individual packages are a bit slower to
build.
The build farm spends a lot of time on lots of relatively small builds,
so parallelising across all builders tends to win as a strategy.
> Providing a mechanism to request a builder of a different size seems ok, but
> would need to be designed/implemented in Launchpad; and how do you prevent
> this from being abused?
Our experience is that the build farm delivers best overall performance
when it's divided into as small a number of pools as possible, and as
Dimitri says we pre-boot builder VMs before knowing what build is going
to be dispatched to them, so having some larger builders would require
dividing the build farm into more pools: we'd have to have amd64+i386
(small) and amd64+i386 (large) rather than just amd64+i386, for
instance. We'd be very resistant to this.
--
Colin Watson [cjwatson@ubuntu.com]
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
