> Hi Andreas,
>
> Andreas Hasenack wrote on 08/03/2023:
>> Hi,
>>
>> taking this as a sample:
>>
>> https://autopkgtest.ubuntu.com/packages/f/fdroidserver/lunar/ppc64el
>> <https://autopkgtest.ubuntu.com/packages/f/fdroidserver/lunar/ppc64el>
>>
>> It looks like something got slower since a few days ago, and tests
>> started being killed due to a timeout ("kind: install" timeout, which
>> has a default of 3000s in autopkgtest).
>>
>> By the end of February 2023, the passing tests were taking about 20min.
>> Then a little bit more up to 26min in March 2nd, and then they all
>> started to fail.
>>
>> Looking at the logs, we see apt-get install being interrupted by the
>> autopkgtest timeout mid package download:
>>
>> Get:242 http://ftpmaster.internal/ubuntu <http://ftpmaster.internal/ubuntu> lunar/universe ppc64el apksigner all 31.0.2-1 [438 kB]
>> Get:243 http://ftpmaster.internal/ubuntu <http://ftpmaster.internal/ubuntu> lunar/main ppc64el openjdk-17-jdk-headless ppc64el 17.0.6+10-1 [217 MB]
>> autopkgtest-virt-ssh [04:23:09]: ------- nova console-log 766e17f7-5365-4cdf-8202-35b6b88a7cdc (adt-lunar-ppc64el-fdroidserver-20230303-012357-lrg-root5) ------
>
> For some reason a few days ago the connection between te bos01 and bos02
> datacenters to ftpmaster.internal got extremely slow. Download speeds
> are of the order of 100kB/s, sometimes even less, while downloading from
> the Internet (archive.ubuntu.com) works just fine (so it is not storage).
>
> AIUI the connection from bos0[12] to ftpmaster passes through an haproxy
> and then an IPsec tunnel crossing an ocean. While maybe not ideal, the link
> was ~10 times faster up to a few days ago.
>
> I filed a ticket about it with the details I could gather. If that's not
> sorted out Soon(tm) we can workaround the issue by switching to
> us.archive.ubuntu.com or by bumping the timeouts.
This is half worked around now: one of the two datacenters which host
the ppc64/s390x/arm64 autopkgtest VMs now has fast connectivity to
ftpmaster.internal, and the other is not as slow as before (perhaps
some link is less loaded now?). So we shouldn't be hitting as many
timeouts as before.
I'm trying to have the other datacenter get the same treatment.
Paride
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
