Thanks for looking into this.
On Sat, Dec 14, 2019 at 01:39:28AM +0000, Dimitri John Ledkov wrote:
> And like all of foundations retrying it due to all of us being blocked
> by it http://autopkgtest.ubuntu.com/packages/c/cyrus-imapd/focal/arm64
You've done the right thing by investating the problem. But the number
of retries there is concerning. Please, developers, *look at the history
and the failure* before retrying and assess if it is likely to pass.
Hammering the infrastructure to hit a small possibility of a pass is not
a good use of our resources - and in this case, after analysing, is
rather unfriendly to www.unicode.org itself.
> make[3]: Entering directory '/srv/dovecot.git/src/lib'
> test -f UnicodeData.txt || wget
> http://www.unicode.org/Public/UNIDATA/UnicodeData.txt
> --2019-12-12 18:14:51-- http://www.unicode.org/Public/UNIDATA/UnicodeData.txt
> Resolving squid.internal (squid.internal)... 91.189.89.11
> Connecting to squid.internal (squid.internal)|91.189.89.11|:3128... connected.
> Proxy request sent, awaiting response... 503 Service Unavailable
> 2019-12-12 18:15:51 ERROR 503: Service Unavailable.
The proxy is supposed to work. I think it actuaully does. What looks to
happen to me from a quick investigation is that there's rate limiting on
www.unicode.org. Multiple tests running in parallel will fetch that same
file lots of times, and presumably that trips that rate limiting. That
results in an error 503. This seems like unfriendly behaviour on the
part of the tests really, and that web server is well within its rights
to respond in the way that it does. Also from the perspective of the
tests relying on volatile external data doesn't seem like the most
sensible thing to be doing. Even if this file weren't in the archive
they would be advised to ship a static copy of it.
(You could make an argument about how the proxy might return from its
cache instead, but I think the proxy is also within its rights to *not*
do that if it decides for any reason that it doesn't want to - and in
any case even the use of a proxy is an implementation detail; e.g. AFAIK
ci.debian.net doesn't use one.)
Try fetching that file multiple times from your own connection, without
any proxy involved. It will stop working after a few tries, or at least
it does here. I just get a hang - the connection never completes -
instead of a 503 though, so presumably the 503 is generated by squid.
> Well luckily we have unicode data packaged in the archive, so I
> uploaded a change to the autopackage to use UnicodeData.txt from
> unicode-data package.
Did you forward that change to Debian? That is clearly applicable there
too, so please make sure it is done. (And the package would otherwise be
in sync.)
> Overall it still points at unreliable network in our autopkgtest
> infra, but at least cyrus-impad should not be holding up any packages
> in proposed migration. Seems like it's all green now.
Don't think that's the right analysis, but I think it is the right fix
nevertheless.
Cheers,
--
Iain Lane [ iain@orangesquash.org.uk ]
Debian Developer [ laney@debian.org ]
Ubuntu Developer [ laney@ubuntu.com ]
