Friday, 16 February 2018

Re: autopkgtest-build-lxd failing with bionic

On Fri, Feb 16, 2018 at 08:15:35PM +0100, Julian Andres Klode wrote:
> > I think the network-online.target is the better thing to key on.
>
> I think we should just grep the apt output and retry if it fails with
> connection error messages.

The problem is a general one though. It's not specific to apt. Any time
we use automation on a container or VM, we need to wait until it's
finished booting.

In uvtool this is what "uvt-kvm wait" provides, which currently waits
for upstart runlevel 2 or systemd runlevel 5 and then asks cloud-init
(since a script might also have asked cloud-init to do things it expects
done when the container is "ready"). Of course that's cloud-init
specific.

The script may need fixing, but Ubuntu should agree upon a general
answer to the common question. Even if the answer provides multiple
specified points if multiple points in time are appropriate to solve
different problems.

So, to me, retrying apt is a hacky workaround.

Robie

Re: autopkgtest-build-lxd failing with bionic

On Fri, Feb 16, 2018 at 11:12:32AM -0800, Steve Langasek wrote:
> On Fri, Feb 16, 2018 at 11:52:05AM +0000, Iain Lane wrote:
> > On Thu, Feb 15, 2018 at 09:55:47PM +0100, Martin Pitt wrote:
> > > Hello Iain, all,
>
> > > Iain Lane [2018-02-15 18:48 +0000]:
> > > > There's a patch attached here which fixes the problem for me. I'm not
> > > > sure if there's a better way to do this - basically it starts
> > > > network-online.target and waits for it to become active, with a timeout.
> > > > Review appreciated.
>
> > > I wouldn't pick on any of these: network-online.target is a sloppily defined
> > > shim for SysV init backwards compatibility, and may not ever get started (in
> > > fact, that's the goal ☺); and the container might not use networkd, so I
> > > wouldn't use s-n-wait-online either. I think querying
>
> > Interesting. I thought that it was the systemd way to say 'I am online
> > now' --- i.e. nm-online or systemd-networkd-wait-online, which is the
> > question I wanted to get a positive answer to. I can see that the SysV
> > implementation isn't great, but it's not clear to me that it was ill
> > defined for this case.
>
> > > [ -n "$(ip route show to 0/0)" ]
>
> > This is better though, and works too. Please take a look at the attached
> > patch. Thanks! :-)
>
> Actually no, this is racy, because the route comes up before DNS resolution
> is in place.
>
> It's also not forwards-compatible with ipv6-only deploys.
>
> I think the network-online.target is the better thing to key on.

I think we should just grep the apt output and retry if it fails with
connection error messages. This should be fine until I have an improved
solution in apt itself, one of

(1) "there are no transient errors"
(2) one source must have updated
(3) all sources must have updated

Not sure on details. Could be an option for all three.

--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en

--
ubuntu-devel mailing list
[email protected]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: autopkgtest-build-lxd failing with bionic

On Fri, Feb 16, 2018 at 11:52:05AM +0000, Iain Lane wrote:
> On Thu, Feb 15, 2018 at 09:55:47PM +0100, Martin Pitt wrote:
> > Hello Iain, all,

> > Iain Lane [2018-02-15 18:48 +0000]:
> > > There's a patch attached here which fixes the problem for me. I'm not
> > > sure if there's a better way to do this - basically it starts
> > > network-online.target and waits for it to become active, with a timeout.
> > > Review appreciated.

> > I wouldn't pick on any of these: network-online.target is a sloppily defined
> > shim for SysV init backwards compatibility, and may not ever get started (in
> > fact, that's the goal ☺); and the container might not use networkd, so I
> > wouldn't use s-n-wait-online either. I think querying

> Interesting. I thought that it was the systemd way to say 'I am online
> now' --- i.e. nm-online or systemd-networkd-wait-online, which is the
> question I wanted to get a positive answer to. I can see that the SysV
> implementation isn't great, but it's not clear to me that it was ill
> defined for this case.

> > [ -n "$(ip route show to 0/0)" ]

> This is better though, and works too. Please take a look at the attached
> patch. Thanks! :-)

Actually no, this is racy, because the route comes up before DNS resolution
is in place.

It's also not forwards-compatible with ipv6-only deploys.

I think the network-online.target is the better thing to key on.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
[email protected] [email protected]

Re: autopkgtest-build-lxd failing with bionic

On Thu, Feb 15, 2018 at 12:00:41PM -0800, Steve Langasek wrote:
> It's a bit odd to be "start"ing a target in this manner. Is it even
> necessary to start the target, or would it be sufficient to just check
> is-active in a loop?

Yeah, it is - it needs to be pulled in by something to get started, but
in this case it's not so we do the same thing in code. It's like this so
you don't end up blocking the boot unnecessarily waiting for the network
to be "up" when nothing needs it to be.

Doesn't matter any more though for this case. :-)

--
Iain Lane [ [email protected] ]
Debian Developer [ [email protected] ]
Ubuntu Developer [ [email protected] ]

Re: autopkgtest-build-lxd failing with bionic

From a194535b9bb2d5fbfc8cefa3b047c57fa40736a1 Mon Sep 17 00:00:00 2001
From: Iain Lane <[email protected]>
Date: Thu, 15 Feb 2018 16:21:59 +0000
Subject: [PATCH] lxd: Wait until we have a default route.

We execute `apt-get update' more or less as soon as the container is
started. In some situations this is too early: it can be before network
is fully working.

When building lxd containers or using autopkgtest-virt-lxd, wait until
we have a default route before proceeding.
---
tools/autopkgtest-build-lxd | 14 +++++++++++++-
virt/autopkgtest-virt-lxd | 10 ++++++++++
2 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/tools/autopkgtest-build-lxd b/tools/autopkgtest-build-lxd
index 623d5eb..b3141d6 100755
--- a/tools/autopkgtest-build-lxd
+++ b/tools/autopkgtest-build-lxd
@@ -68,7 +68,7 @@ setup() {
lxc exec "$CONTAINER" -- chmod 644 /etc/apt/apt.conf.d/01proxy
fi

- # wait until it is booted: lxc exec works and we get a numeric runlevel
+ # wait until it is booted: lxc exec works, we get a numeric runlevel and networking is up
timeout=60
while [ $timeout -ge 0 ]; do
timeout=$((timeout - 1))
@@ -81,6 +81,18 @@ setup() {
exit 1
}

+ while [ $timeout -ge 0 ]; do
+ timeout=$((timeout - 1))
+ if lxc exec "$CONTAINER" -- sh -ec 'test -n "$(ip route show to 0/0)"'; then
+ break
+ fi
+ sleep 1
+ done
+ [ $timeout -ge 0 ] || {
+ echo "Timed out waiting for network to come up" >&2
+ exit 1
+ }
+
ARCH=$(lxc exec "$CONTAINER" -- dpkg --print-architecture </dev/null)
DISTRO=$(lxc exec "$CONTAINER" -- sh -ec 'lsb_release -si 2>/dev/null || . /etc/os-release; echo "${NAME% *}"' </dev/null)
CRELEASE=$(lxc exec "$CONTAINER" -- sh -ec 'lsb_release -sc 2>/dev/null || awk "/^deb/ {sub(/\\[.*\\]/, \"\", \$0); print \$3; quit}" /etc/apt/sources.list' </dev/null)
diff --git a/virt/autopkgtest-virt-lxd b/virt/autopkgtest-virt-lxd
index f8e9a4d..6e7ec65 100755
--- a/virt/autopkgtest-virt-lxd
+++ b/virt/autopkgtest-virt-lxd
@@ -100,6 +100,16 @@ def wait_booted():
continue
out = out.strip()
if out.split()[-1].isdigit():
+ adtlog.debug('waiting for network')
+ VirtSubproc.check_exec(['lxc', 'exec', container_name, '--',
+ 'sh', '-ec',
+ '''while true; do
+ if test -n "$(ip route show to 0/0)"; then
+ break;
+ fi;
+ sleep 1;
+ done'''],
+ timeout=30)
return

adtlog.debug('wait_booted: runlevel "%s", retrying...' % out)
--
2.14.1

On Thu, Feb 15, 2018 at 09:55:47PM +0100, Martin Pitt wrote:
> Hello Iain, all,
>
> Iain Lane [2018-02-15 18:48 +0000]:
> > There's a patch attached here which fixes the problem for me. I'm not
> > sure if there's a better way to do this - basically it starts
> > network-online.target and waits for it to become active, with a timeout.
> > Review appreciated.
>
> I wouldn't pick on any of these: network-online.target is a sloppily defined
> shim for SysV init backwards compatibility, and may not ever get started (in
> fact, that's the goal ☺); and the container might not use networkd, so I
> wouldn't use s-n-wait-online either. I think querying

Interesting. I thought that it was the systemd way to say 'I am online
now' --- i.e. nm-online or systemd-networkd-wait-online, which is the
question I wanted to get a positive answer to. I can see that the SysV
implementation isn't great, but it's not clear to me that it was ill
defined for this case.

> [ -n "$(ip route show to 0/0)" ]

This is better though, and works too. Please take a look at the attached
patch. Thanks! :-)

Cheers,

--
Iain Lane [ [email protected] ]
Debian Developer [ [email protected] ]
Ubuntu Developer [ [email protected] ]

Re: Launchpad i386 build: Memory exhausted

On Fri, Feb 16, 2018 at 09:21:56AM +0100, Cesare Falco wrote:
> I'm packaging latest MAME 0.194 and it builds fine on my PPA.
> Unfortunately, link fails for i386 due to virtual machine memory issues:
>
> /usr/bin/ld: final link failed: Memory exhausted
[...]
> Does anyone know whether virtual machines for official Ubuntu releases have
> more resources, specifically more memory?

PPA builds and Ubuntu builds use 100% identical virtual machines.

Anyway, it's not that the virtual *machine* is out of memory; it's that
any single 32-bit process can only address a certain amount of that
memory. As far as I'm aware, there's no option but to continue trying
to persuade the linker to take less memory.

--
Colin Watson [[email protected]]

--
ubuntu-devel mailing list
[email protected]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Launchpad i386 build: Memory exhausted

Hello everyone,

I'm packaging latest MAME 0.194 and it builds fine on my PPA. Unfortunately, link fails for i386 due to virtual machine memory issues:
/usr/bin/ld: final link failed: Memory exhausted
You can find a full log for build here:

and also here in case I got struck by the greatest idea and try a new build :)

Please note I've already put great care in tweaking linker options to reduce memory consumption, with no luck. I've already faced this issue in backporting, but it never occured with current release. It's worth noting that the default compiler in Artful (gcc 7) is also the recommended compiler by upstream developers, so forcing a different release doesn't look an option either.

It would be nice to have a working i386 build on my PPA, but what I care more is uploading Mame 0.194 to Bionic.

Does anyone know whether virtual machines for official Ubuntu releases have more resources, specifically more memory? Should I try uploading the tarball and see? What happens if I get the same memory issue, would I be able to downgrade to 0.193?

I'm looking forward for any kind advice, thank you in advance! :)

Cheers,
Cesare