Saturday 23 April 2022

Re: pv (a pipeline progress indicator) in main?

During kernel package builds compressing debs often takes a very long time without any feedback.

I was considering to start patching dpkg-deb to fork and compress things via pv to get progress output in the build logs.

I can't remember where, but it would be nice for pv to output info after a delay. I.e. if operation is quick don't bother, but after like 10s start showing progress output would be nice.

On Fri, 22 Apr 2022, 17:58 Bryce Harrington, <bryce.harrington@canonical.com> wrote:
The Ubuntu Server team is looking at several potential items to promote
to main, including cli admin tools that might have broad usefulness.
One of these we're on the fence about and would like broader input.

pv, 'Pipe Viewer' is a command line utility that essentially copies
stdin to stdout, and displays an animated progress bar.

Standard example is compressing a large file, e.g.:

  $ pv Mail/spam.assassin  | gzip > /tmp/spam.gz
  31.3MiB 0:00:01 [31.3MiB/s] [===========>                             ] 31% ETA 0:00:02

pv can also be used in the middle of pipelines (although since it
doesn't know the stream's size it can't estimate % progress):

  $ mysqldump -uroot -p database1 | pv | gzip -9 > database1.sql.gz
  53.7MiB 0:00:01 [29.7MiB/s] [ <=>                                                     ]

Overview: https://catonmat.net/unix-utilities-pipe-viewer
Man page: https://linux.die.net/man/1/pv
LP page:  https://launchpad.net/ubuntu/+source/pv


Googling indicates that pv comes up very commonly as a general purpose
solution to displaying progress, although there do appear to be
main-provided solutions for at least some common situations.  For simply
copying files, there is already rsync which has --progress and --status
options.  For creating tarballs, tar has a --checkpoint option, though
it's not fancy.  For copying streams, dd is in main, which has a
status=progress option that animates the bytes copied (but not %'s or
visual bars).

That said, pv looks like it would be a relatively light addition to
main; it's written in C, appears to have an active upstream, and looks
pretty self-contained.  A MIR for pv looks like it would be reasonably
straightforward to file.  With it in main, other packages could rely on
having it available for providing progress info, and would make it more
at hand for scripting, tutorials/howto's, tech support, etc.


Does this look useful enough to you that it should be made available by
default?  Are there alternatives you feel would be better to look at?
Or other considerations that need made before deciding?

Thanks,
Bryce





--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel