On 4 July 2017 at 21:22, Steve Langasek <steve.langasek@ubuntu.com> wrote:
>
> Hi James,
>
> On Tue, Jul 04, 2017 at 04:31:32PM +0000, James Page wrote:
> > Hi All
>
> > Mathias and I where discussing some challenges around fitting the Ceph
> > build with any level of parallelism in to a virtualized launchpad builder
> > (as used for the majority of architectures today).
>
> > Basically the current memory spec only permits a max-parallel of 2 (on
> > Xenial amd64). Ceph is a big C++ project so takes some time to build with
> > such a low level of parallelism.
>
> > Would it be possible to nudge up the default memory allocation on the
> > builders? I use a 16G machine for test builds of i386/amd64 and I've not
> > seen any memory exhaustion with parallel=8.
>
> > Or maybe have a mechanism to allow a source package to request a higher
> > spec of builder?
>
> You should probably consult the Launchpad team on this, since the Launchpad
> builder configuration is owned by them and not by the Ubuntu developers
> directly.
>
> As far as raising the default memory allocation, understand that there is a
> tradeoff between being able to parallelize /within/ a build and being able
> to parallelize across /all/ builds. If the sole justification for this
> request is that it would speed up the build of one particular package, I
> don't think that's a sufficient reason to reduce the capacity of the
> launchpad build farm as a whole.
>
> Providing a mechanism to request a builder of a different size seems ok, but
> would need to be designed/implemented in Launchpad; and how do you prevent
> this from being abused?
>
Yeah, if we make all builders x4 the size, it may mean we will have 4x
less /builders/.
However debhelper 10 defaults to parallel builds, so I wonder if
having bigger instances will be a net gain or not.
As far as I understand, from previous face to face conversations, the
builder VMs are pre-booted (allocated) and join Launchpad pool of
builders idling. Thus a builder is already there, before it knows
which package it will build and whether the package is a distro
package or a ppa one.
My concern was that launchpad builders should scale with the queue
sizes, as their utilisation is not 100%. Such that e.g. launchpad
allocation of builders quota could be potentially shared with the ADT
testing quota. As in ADT we frequently see the tests failing due to
hitting ADT quotas, at the same time launchpad builders sit there
idling.
If there was some capacity to auto scale builders, when the queue is
short it would be nicer probably to have less builders, but maybe a
few beefier instances.
But this is very hand-wavy, as changes to builders will need a chunk
of engineering on launchpad builder side.
--
Regards,
Dimitri.
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
