Friday 9 June 2023

Re: git-ubuntu build

On Wed, Jun 07, 2023 at 09:27:47PM -0700, Bryce Harrington wrote:
> On Wed, Jun 07, 2023 at 06:41:14PM -0700, Steve Langasek wrote:

> > As git-ubuntu sees increasing use, including for such things as requests for
> > sponsorship of Debian merges, I've had an itch to scratch regarding the
> > complexity of passing correct flags to dpkg-buildpackage, so I spent some
> > time prototyping a git-ubuntu wrapper for it.

> > bzr-builddeb hasn't been useful for general work on Ubuntu packages for
> > quite a while, but the behavior of this wrapper is inspired by it.
> > Hopefully some of you will find using this tool pleasantly familiar!

> > The intent is that this will eventually become a git-ubuntu subcommand,
> > though there are some namespace questions to sort out first - the obvious
> > name for such a command IMHO would be 'git-ubuntu build' but that already
> > exists and does other things.

> It does not exist, actually! I recall we dropped it a few years ago,
> see see f2dc622e.

$ git-ubuntu help
usage: git-ubuntu [-h] <command> ...
git-ubuntu: error: argument <command>: invalid choice: 'help' (choose from 'build', 'build-source', 'import-local', 'import-ppa', 'lint', 'review', 'clone', 'export-orig', 'import', 'importer-service-broker', 'importer-service-poller', 'importer-service-worker', 'merge', 'prepare-upload', 'queue', 'remote', 'submit', 'tag')
^^^^^
$

So, it does *partially* exist, even though running 'git-ubuntu build'
returns an error :-)

> I recall at the time it was intended to one day bring it back, but the
> plan was to reimplement - as you're doing - but also build it up from
> first principles with ample test case coverage. The original subcommand
> lacked tests, but tried to do a bit too much (it included wrapping lxd,
> running lint, etc.) and the lack of tests made maintenance a bit scary.
> So, I'd encourage making matching tests as you go. :-)

Yes, Robie and I have talked about this; adding tests was not a blocker for
including it in a separate sandbox/ directory but will be for landing it as
a git-ubuntu subcommand. I'll be working on adding tests shortly.

> That said, though, I've wondered if 'build' may not necessarily be the
> ideal jargon, anyway. Since the (prepare-upload args) step can trigger
> a git push, and because this is done principly when uploading, it feels
> more like a submission-style workflow than a build; "build" also implies
> you're creating some form of artifact for local use, which in this case
> you're not, really.

> So, I'd suggest that even though 'git-ubuntu build' is not used, you may
> still want to think more anyway about if there's a better term.

Related commands:

- dpkg-buildpackage
- debuild
- gbp buildpackage
- bzr builddeb
- sbuild

So "build" is quite a strong theme in the existing tools.

An interesting point, though, is that this makes me realize I would only
ever care about the sauce this provides when preparing a source package for
upload to Ubuntu, and not when I was trying to do a binary package build.
Another previous git-ubuntu subcommand was 'git-ubuntu build-source'. Do
you think that would be a better fit?

I don't think 'submission-style' quite describes what I'm expecting to
achieve here. I might want to build a source package, then do a variety of
things with it before actually uploading it; e.g. run a debdiff against the
previous package to be sure it's actually a clean diff at that level, take
the source artifacts and do a test build, push to a ppa before pushing to
the Ubuntu archive, manually mangle the .changes file (rare, but I happen
to have just done this for a series of SRUs so that they would have complete
changelogs but not link to a whole list of unrelated bugs in the SRU
process), etc. And the target of the 'git push' may or may not be something
that we want to merge immediately, may or may not want to raise an MP for
immediately.

> > Why this is useful:

> > - the syntax 'dpkg-buildpackage $(git-ubuntu prepare-upload args)' is
> > onerous and repetitive - but we want to encourage inclusion of these
> > headers in .changes files, as this lets us automate closure of git-ubuntu
> > MPs
> > - there are certain options that can be inferred as correct for any
> > git-ubuntu repo (-i -I)
> > - orig.tar.xz should be reconstituted or downloaded when needed, without
> > extra commands (we have pristine-tar branches in git-ubuntu which often
> > save having to do a duplicate download; having to clone a git repository
> > *and* apt source the package is meh)
> > - getting the correct options to dpkg-buildpackage by hand for a package
> > merge is tedious; this automates -v and -sa arguments.

> Very cool, and I'll have to look at what you're doing to automate the
> -sa arguments, I hadn't figured out a good solution there.

> Btw, you may already be doing this but I've found in scripting this
> myself that it's worthwhile to check the $(git ubuntu prepare-upload
> args) run for exit codes before passing its output along; there are
> situations where it fails.

Well, I'm calling into the python module directly, and it appears that
gitubuntu.prepare_upload.push() will raise some AssertionErrors but only
returns on success, so this seems handled.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org