Saturday 20 April 2024

+1 maintenance report (Apr 15-Apr 20, 2024)

Hi there,

I was on my second +1 maintenance shift last week. I spent most of my time working through the update_excuses page from time to time. I spent some time analyzing package-revdep pairs from the non-amd64-nbs list, but couldn't derive the kind of conclusions that I expected to (I must admit that I learnt to think more clearly about NBS in the process).

I list the universe packages that I picked up from the update_excuses regressions and my work on each of them.

=== siconos ===
I came across this package in the NBS list. Tests that use the Python interface fail with 4.4.0+dfsg-4build1 because the Python interface isn't installed (though it is generated), making it akin to FTBFS. I filed a bug report [1]. The upstream setup.py depends on distutils and needs to be migrated to use setuptools, for which I submitted a merge proposal [2]. The bug report has been marked 'wont fix' and siconos binaries removed from noble.

[1] https://bugs.launchpad.net/ubuntu/+source/siconos/+bug/2061719
[2] https://code.launchpad.net/~pushkarnk/ubuntu/+source/siconos/+git/siconos/+merge/464496

=== ogre-next ===
Upstream changes to ogre-next, that apparently enable ogre projects co-exist with ogre-next projects, led to a massive clean-up of debian/rules in version 2.3.3+dfsg-0ubuntu1. The previous version let cmake find OGRE-Next projects by installing a patched .cmake file, which was removed by the latest version. I submitted a bug report [4] and a simple merge proposal [5]. I later withdrew the MP in favor of another MP [6], submitted a few hours before [5].

[4] https://bugs.launchpad.net/ubuntu/+source/ogre-next/+bug/2062378
[5] https://code.launchpad.net/~pushkarnk/ubuntu/+source/ogre-next/+git/ogre-next/+merge/464628
[6] https://code.launchpad.net/~j-rivero/ubuntu/+source/ogre-next/+git/ogre-next/+merge/464598

=== nodejs ===
I found the nodejs autopkgtests failing because of a stale python3-distutils dependency in debian/control/tests. Reported the bug [7] and submitted an MP [8]. Thanks bdrung for sponsoring.

[7] https://bugs.launchpad.net/ubuntu/+source/nodejs/+bug/2061946
[8] https://code.launchpad.net/~pushkarnk/ubuntu/+source/nodejs/+git/nodejs/+merge/464487

=== dgit ====
With version 11.7, the manpages-format test fails because it cannot find the tag2upload.5 manpage installed by the git-debpush binary. On investigation, I found tag2upload.5.gz being wrongly installed under man1 (instead of man5). I created a bug report [9] and found a Debian commit [10] that fixes this issue in version 11.8. It might be worth keeping [9] open until we sync with the latest version (11.9 for now) in Debian.

 [9] https://bugs.launchpad.net/ubuntu/+source/dgit/+bug/2062955
[10] https://salsa.debian.org/dgit-team/dgit/-/commit/7e33c8decf6438727a5b2fe15443b3c2f8f1cd43

=== tcpxtract ===
This is an interesting case where an assertion failure can be consistently reproduced on arm64, albeit only with the 1.0.1-17ubuntu1 binaries in the proposed pocket (as of today). The test command is a simple one and I cannot reproduce the failure locally (noble/arm64 on Pi5). Created a bug report [11]. Ran out of guesswork here (maybe I am missing something very fundamental)!

[11] https://bugs.launchpad.net/ubuntu/+source/tcpxtract/+bug/2063015


Other than the above investigations, I also did some autopkgtest requests with the right targets, to get things off the update_excuses list:

pyfai vs. pocl/5.0-2.1build3, which also needed silx/2.0.0+dfsg-1build1
dotnet8 vs. ltt-control -
gnucobol vs. gmp
nauty vs. gmp
normaliz vs. gmp
debcong/1.5.86ubuntu1 vs. rsyslog/8.2312.0-3ubuntu9
python-eventlet vs cinder

As a part of the NBS investigations, I looked into the following package-revdep pairs,
thanks to a guidance document from doko:

libpcap0.8 vs. aircrack-ng - thanks to schopin!
libcups2 vs. libqt5printsupport5t64
libcups2 vs. libqt6printsupport6
libcups2 vs. libqt6printsupport6t64
libcups2 vs. openjdk-8-jre-headless
jdupes vs. libjodycode
libgeoip1 vs. geoip-database
libgeoip1 vs. python3-geoip
libgeoip1 vs. subnetcalc

Thank you!
Pushkar

Tuesday 16 April 2024

Re: Searching for autopkgtest regressions in Noble

I forgot to mention but I'd like to thank Brian Murray for helping me with the autopkgtest SQLite database and also suggestions for improvements. And sharing the results for other teams was his idea as well :)

Em 16/04/2024 18:54, Lucas Kanashiro escreveu:
Hi everyone,

As Noble release is approaching and in the final part of the cycle we had some big changes in the archive (time_t transition, xz's CVE fix), the Server team decided to compare the autopkgtest results from before the end of February, more precisely 2024-02-28 (a guess of a date before the time_t work started), and now (2024-04-16). This comparison could show us any potential regression due to big changes in the archive, and allow us to try to address those issues before the release.

What we did is basically get the latest test result of the packages in all architectures before the reference date (2024-02-28) and compare with the latest test run (2024-04-16) of the packages on the same architecture. We are using the autopkgtest SQLite database available here [1]. I am calling it "bad news" when the tests of a package in a given architecture was passing before the reference date and now they are not.

The script I used to do this is available here [2]. And I used the mapping of packages and teams [3] to get the list of packages. The output of the script is a JSON file that looks like the following for one package:
    
    "adsys": {
        "arm64": {
            "before": {
                "result": "all tests passed",
                "test_run_id": "20240227_182345_d0549@",
                "triggers": "samba/2:4.19.5+dfsg-1ubuntu1",
            },
            "after": {
                "result": "at least one test failed",
                "test_run_id": "20240416_122755_08ec0@",
                "triggers": "sssd/2.9.4-1.1ubuntu6 c-ares/1.27.0-1.0ubuntu1 samba/2:4.19.5+dfsg-4ubuntu9",
            }
        }
    },
    

Attached are the output of the scripts for the following teams:
    
- foudantions-bugs
- desktop-packages
- kernel-package
- ubuntu-server 

The Server team is already going through the list to check if there is any real regression requiring some work. Keep in mind that not all packages listed there are necessarily real problems, maybe the test failed because of a bad trigger, or autopkgtest infra issue, so manual check is required to make sure this is a real regression. It is also important to note that the script always gets the latest test result before the reference date, so the failure being analysed could be a flaky test for instance, too.

If you have any question or suggestion on this let me know.

I hope that's useful for other teams.

--   Lucas Kanashiro

--   Lucas Kanashiro

Searching for autopkgtest regressions in Noble

Hi everyone,

As Noble release is approaching and in the final part of the cycle we had some big changes in the archive (time_t transition, xz's CVE fix), the Server team decided to compare the autopkgtest results from before the end of February, more precisely 2024-02-28 (a guess of a date before the time_t work started), and now (2024-04-16). This comparison could show us any potential regression due to big changes in the archive, and allow us to try to address those issues before the release.

What we did is basically get the latest test result of the packages in all architectures before the reference date (2024-02-28) and compare with the latest test run (2024-04-16) of the packages on the same architecture. We are using the autopkgtest SQLite database available here [1]. I am calling it "bad news" when the tests of a package in a given architecture was passing before the reference date and now they are not.

The script I used to do this is available here [2]. And I used the mapping of packages and teams [3] to get the list of packages. The output of the script is a JSON file that looks like the following for one package:
    
    "adsys": {
        "arm64": {
            "before": {
                "result": "all tests passed",
                "test_run_id": "20240227_182345_d0549@",
                "triggers": "samba/2:4.19.5+dfsg-1ubuntu1",
            },
            "after": {
                "result": "at least one test failed",
                "test_run_id": "20240416_122755_08ec0@",
                "triggers": "sssd/2.9.4-1.1ubuntu6 c-ares/1.27.0-1.0ubuntu1 samba/2:4.19.5+dfsg-4ubuntu9",
            }
        }
    },
    

Attached are the output of the scripts for the following teams:
    
- foudantions-bugs
- desktop-packages
- kernel-package
- ubuntu-server 

The Server team is already going through the list to check if there is any real regression requiring some work. Keep in mind that not all packages listed there are necessarily real problems, maybe the test failed because of a bad trigger, or autopkgtest infra issue, so manual check is required to make sure this is a real regression. It is also important to note that the script always gets the latest test result before the reference date, so the failure being analysed could be a flaky test for instance, too.

If you have any question or suggestion on this let me know.

I hope that's useful for other teams.

--   Lucas Kanashiro

Monday 15 April 2024

Re: pastebinit default target on Ubuntu

Sergio Durigan Junior kirjoitti 15.4.2024 klo 20.51:
> On Monday, April 15 2024, Robie Basak wrote:
>
>> Reason to keep it dpaste.com:
>>
>> People have complained that the login requirement makes it unusable for
>> helping Ubuntu users at large who don't necessarily have an Ubuntu SSO
>> account.
>
> The requirement for login is really a pain. I find myself avoiding
> paste.ubuntu.com most of the time because of it, especially if I know
> that the target audience might not even have a Launchpad account.

+1

>> Reason to keep it paste.ubuntu.com:
>>
>> I'm not keen on relying on third party services when not necessary,
>> especially ad-supported ones. I have no reason to distrust the current
>> operator, but in general, these things tend to go wrong sooner or later.
>
> dpaste.com also runs a proprietary backend, so I'm -1 on using it.
> There's dpaste.org, which is FLOSS and doesn't seem to load any ads.

dpaste.org seems like a fine alternative, so +1 here too


--
t


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: pastebinit default target on Ubuntu



On Tue, 16 Apr 2024 at 14:37, Steve Langasek <steve.langasek@ubuntu.com> wrote:
On Mon, Apr 15, 2024 at 04:42:37PM -0400, St├ęphane Graber wrote:
> > And if there are issues with the usability of paste.ubuntu.com, uh, we own
> > that service?  So let's work with our IS team to make it fit for purpose.
> > (I don't know why it currently requires a login to *view* paste contents;
> > that seems straightforwardly a bug that we should just get sorted.)

> That's because pastebin servers are frequently abused as a way to get
> free mass storage.

> It's not very practical to require login to post to a pastebin as the
> whole point is for a tool like "pastebinit" to work without needing
> user configuration as it's commonly used as a debug tool on cloud
> instances and other random servers random than a user's personal
> system.

> With that in mind, a bunch of folks noticed that you could abuse a
> service like paste.ubuntu.com by pushing large files (base64 encoded
> or the like) and then retrieve them with a very trivial amount of html
> parsing (if no raw option is offered directly).

> There are obviously alternatives to this, but they tend to require a
> bunch more server side logic, basically trying to find the right set
> of restrictions to both poster and reader so that legitimate users can
> use the service normally while abusers get sufficiently annoyed to
> stay away from it.

The current behavior of paste.ubuntu.com, and what I assumed was the driver
for moving away from this as a default, was that it requires a login to VIEW
the contents of the pastebin.  AFAICS this is not justifiable on the basis
of preventing abuse with illicit/illegal pastes, that's already addressed by
requiring login on the submission side.

I think the current behaviour is to require login for at least one of submission or view, so a paste created while logged in can be viewed anonymously and a paste created anonymously (e.g. by pastebinit, which I don't think supports logging in?) requires a login to view.

Cheers,
mwh

Re: pastebinit default target on Ubuntu

On Mon, Apr 15, 2024 at 04:42:37PM -0400, Stéphane Graber wrote:
> > And if there are issues with the usability of paste.ubuntu.com, uh, we own
> > that service? So let's work with our IS team to make it fit for purpose.
> > (I don't know why it currently requires a login to *view* paste contents;
> > that seems straightforwardly a bug that we should just get sorted.)

> That's because pastebin servers are frequently abused as a way to get
> free mass storage.

> It's not very practical to require login to post to a pastebin as the
> whole point is for a tool like "pastebinit" to work without needing
> user configuration as it's commonly used as a debug tool on cloud
> instances and other random servers random than a user's personal
> system.

> With that in mind, a bunch of folks noticed that you could abuse a
> service like paste.ubuntu.com by pushing large files (base64 encoded
> or the like) and then retrieve them with a very trivial amount of html
> parsing (if no raw option is offered directly).

> There are obviously alternatives to this, but they tend to require a
> bunch more server side logic, basically trying to find the right set
> of restrictions to both poster and reader so that legitimate users can
> use the service normally while abusers get sufficiently annoyed to
> stay away from it.

The current behavior of paste.ubuntu.com, and what I assumed was the driver
for moving away from this as a default, was that it requires a login to VIEW
the contents of the pastebin. AFAICS this is not justifiable on the basis
of preventing abuse with illicit/illegal pastes, that's already addressed by
requiring login on the submission side.

If requiring authentication on the SUBMISSION side is sufficient reason to
change the default pastebin, then that of course isn't something we should
second-guess; we don't need to be reinvesting anonymous ftp servers. But in
that case, I think there should have been a discussion about who the default
behavior is for, because for my part it makes the default behavior much
worse.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

Re: pastebinit default target on Ubuntu

On Mon, Apr 15, 2024 at 04:42:37PM -0400, Stéphane Graber wrote:
> On Mon, Apr 15, 2024 at 4:14 PM Steve Langasek
> <steve.langasek@ubuntu.com> wrote:
> > And if there are issues with the usability of paste.ubuntu.com, uh, we own
> > that service? So let's work with our IS team to make it fit for purpose.
> > (I don't know why it currently requires a login to *view* paste contents;
> > that seems straightforwardly a bug that we should just get sorted.)
>
> That's because pastebin servers are frequently abused as a way to get
> free mass storage.
>
> It's not very practical to require login to post to a pastebin as the
> whole point is for a tool like "pastebinit" to work without needing
> user configuration as it's commonly used as a debug tool on cloud
> instances and other random servers random than a user's personal
> system.
>
> With that in mind, a bunch of folks noticed that you could abuse a
> service like paste.ubuntu.com by pushing large files (base64 encoded
> or the like) and then retrieve them with a very trivial amount of html
> parsing (if no raw option is offered directly).

I'll add that (from memory) it wasn't just being abused as free mass
storage in general, it was very very dodgy stuff that required urgent
takedown enforcement. We talked IS down from making it require a login
to use the service at all and this was the compromise.

--
Colin Watson (he/him) [cjwatson@ubuntu.com]

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel