Thursday, 23 May 2019

Re: mlocate - what is it good for?

On Thu, 23 May 2019 at 09:07, Brian Murray <brian@ubuntu.com> wrote:
The Ubuntu Foundations team was recently looking at an issue with
mlocate[1] and the effect it has on all users of Ubuntu. While that
specific issue is fixable there are also issues[2,3] with keeping
PRUNEFS and PRUNEPATHS current in updatedb.conf. So we ended up
questioning the usefulness of installing mlocate by default on systems
at all. We believe that find is an adequate replacement for mlocate but
want to hear from you about use cases where it may not be.

I think this is a cattle vs pet sort of thing as well: on a system that being used for development or adminned by multiple humans, locate is quite useful. And the overhead of having to install it for systems like this seems acceptable (users like these are exactly the sort of users for whom installing a package is easy!).

On systems that are part of a fleet of automatically maintained machines, all locate does is waste IO bandwidth -- which might well be a shared resource if the systems are VMs.

My vote would be for removing it from the default install.

Cheers,
mwh

Re: mlocate - what is it good for?

On Thu, May 23, 2019 at 08:25:20PM +0200, Sebastien Bacher wrote:
> Le 22/05/2019 à 21:47, Julian Andres Klode a écrit :
> > containing about 1435134 files. mlocate foo takes 1s, find / -mount
> > -name '*foo*' takes about 7-9 secs, or 19 seconds with all mount points
> > (but there is a davfs mount of an internet server, so things might be

> Well here on a few years old 80GB intel ssd doing

> $ time find / -mount -name disco-desktop-amd64.iso

> Takes over 30 seconds (on a machine which not doing any other work)
> where mlocate takes around 1 second on the same machine... and I do
> personally find 30 seconds to be too long.

I agree that 30 seconds is too long, but as a user of find I find this usage
surprising, why would you be searching the whole root disk for the file? I
usually know a file like that is going to be in one of a few subtrees (my
homedir or /tmp or /var/lib/libvirt or something) and would be doing a more
targeted search.

Also since the locate database is only updated once a day, locate doesn't
help with this if the file I've lost is one I've downloaded today, and it
seems likely to me that files I'm looking for that aren't part of the system
data are fairly likely to fall into this category.

Anyway, agreed that 30s is too long. Would you be annoyed if the solution
to that 30s being too long was "get tired of waiting, run 'locate', get told
it's not installed, apt install mlocate, run 'locate' again"?

> Note that nautilus has a mlocate search provider which we have been
> using by default until bionic, to avoid depending on tracker. Now we use
> tracker so it's less important but some users do uninstall tracker
> because they don't like having an indexer (and it still does have some
> bugs) so it's nice to have the UI still responsive for those users.

Well, I don't think this is an argument for keeping mlocate installed by
default on desktops, because effectively this means that you have TWO
indexers on your desktop system - both tracker and mlocate. It looks like
nautilus currently depends on tracker, so I'm not sure how one would
uninstall it and usefully fall back to the mlocate backend anyway, but at
most I'd say this should be expressed as 'Depends: tracker | mlocate' in
nautilus, and not have mlocate kept around on the system updating its
database daily just in case a user removes tracker.

Thanks,
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

Re: mlocate - what is it good for?

I don't know if anyone has said it already but I want to applaud you for asking rather than just doing.


From: ubuntu-server <ubuntu-server-bounces@lists.ubuntu.com> on behalf of Steve Langasek <steve.langasek@ubuntu.com>
Sent: Thursday, May 23, 2019 12:48:35 PM
To: jim
Cc: ubuntu-server@lists.ubuntu.com; ubuntu-devel@lists.ubuntu.com
Subject: [EXTERNAL] Re: mlocate - what is it good for?
 
Hi Jim,

On Wed, May 22, 2019 at 03:51:14PM -0400, jim wrote:

> Seems to me for an individual user such
> as I (my laptop runs Ubuntu 18.04), locate
> and (the cumbersome) find are sufficient.
> To what extent does decision-making balance
> net ops and large system administrators as
> opposed to individual users?

Currently, this is seeded as part of ubuntu-standard, which is common to all
Ubuntu flavors.  If the discussion led to a determination that this still
made sense to include by default on desktops but not servers, or vice versa,
we could seed this as part of one Ubuntu flavor or the other.

My own sense is that this is not a server vs desktop thing; there are users
of locate, to be sure, but I believe they are a very small minority on both
desktop and server (small on desktop because the user will generally use the
gui instead; small on server because most server use is not interactive at
the shell).  I don't think the benefit of having locate available by default
justifies the daily disk thrashing / energy usage on every Ubuntu machine
everywhere.  I think it's not onerous for those who want to use locate to
manually install it the first time they need it on a machine.

But this thread is here to be a sanity check on that opinion, to understand
if the locate db being available by default either affects so many users, or
has so great an impact on some use cases, that we should consider leaving it
enabled by default despite the daily disk usage for everyone not using it.

Harriscomputer

Leroy Tennison
Network Information/Cyber Security Specialist
E: leroy@datavoiceint.com


2220 Bush Dr
McKinney, Texas
75070
www.datavoiceint.com
 

This message has been sent on behalf of a company that is part of the Harris Operating Group of Constellation Software Inc. These companies are listed here.

If you prefer not to be contacted by Harris Operating Group please notify us.

 

This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message.

 

> On 5/22/19 3:47 PM, Julian Andres Klode wrote:
> > On Wed, May 22, 2019 at 01:19:48PM -0600, Neal McBurnett wrote:
> > > I use mlocate multiple times a day.
> > > Find is way too slow and inconvenient for finding files in a big
> > > set of filesystems, compared to properly configuring mlocate.
> > Specifically, the filesystem must be huge or on a slow medium. It might make
> > sense to move it out of standard and elsewhere, as I don't think it's
> > necessarily needed everywhere, such as laptops.
> >
> > Consider my laptop, fairly standard, 512 GB NVME SSD, about 250G allocated,
> > containing about 1435134 files. mlocate foo takes 1s, find / -mount
> > -name '*foo*' takes about 7-9 secs, or 19 seconds with all mount points
> > (but there is a davfs mount of an internet server, so things might be
> > screwed up a bit).
> >
> > 19s to find something is perfectly workable, also you don't usually
> > find from /, but you have an idea where things are, so it will be much
> > faster.
> >
> > I think mlocate only really makes sense on data storage servers with
> > huge disks, or on machines with HDDs. I therefore do not think the
> > overhead of building the index is warranted for most users. It might
> > make sense to keep mlocate in always-on tasks, like servers, but get
> > rid of it from desktop scenarios.


--
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                   https://www.debian.org/
slangasek@ubuntu.com                                     vorlon@debian.org

Re: mlocate - what is it good for?

Le 22/05/2019 à 21:47, Julian Andres Klode a écrit :
> containing about 1435134 files. mlocate foo takes 1s, find / -mount
> -name '*foo*' takes about 7-9 secs, or 19 seconds with all mount points
> (but there is a davfs mount of an internet server, so things might be

Well here on a few years old 80GB intel ssd doing

$ time find / -mount -name disco-desktop-amd64.iso

Takes over 30 seconds (on a machine which not doing any other work)
where mlocate takes around 1 second on the same machine... and I do
personally find 30 seconds to be too long.

Note that nautilus has a mlocate search provider which we have been
using by default until bionic, to avoid depending on tracker. Now we use
tracker so it's less important but some users do uninstall tracker
because they don't like having an indexer (and it still does have some
bugs) so it's nice to have the UI still responsive for those users.

Cheers,
Sebastien Bacher


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: mlocate - what is it good for?

Hi Jim,

On Wed, May 22, 2019 at 03:51:14PM -0400, jim wrote:

> Seems to me for an individual user such
> as I (my laptop runs Ubuntu 18.04), locate
> and (the cumbersome) find are sufficient.
> To what extent does decision-making balance
> net ops and large system administrators as
> opposed to individual users?

Currently, this is seeded as part of ubuntu-standard, which is common to all
Ubuntu flavors. If the discussion led to a determination that this still
made sense to include by default on desktops but not servers, or vice versa,
we could seed this as part of one Ubuntu flavor or the other.

My own sense is that this is not a server vs desktop thing; there are users
of locate, to be sure, but I believe they are a very small minority on both
desktop and server (small on desktop because the user will generally use the
gui instead; small on server because most server use is not interactive at
the shell). I don't think the benefit of having locate available by default
justifies the daily disk thrashing / energy usage on every Ubuntu machine
everywhere. I think it's not onerous for those who want to use locate to
manually install it the first time they need it on a machine.

But this thread is here to be a sanity check on that opinion, to understand
if the locate db being available by default either affects so many users, or
has so great an impact on some use cases, that we should consider leaving it
enabled by default despite the daily disk usage for everyone not using it.

> On 5/22/19 3:47 PM, Julian Andres Klode wrote:
> > On Wed, May 22, 2019 at 01:19:48PM -0600, Neal McBurnett wrote:
> > > I use mlocate multiple times a day.
> > > Find is way too slow and inconvenient for finding files in a big
> > > set of filesystems, compared to properly configuring mlocate.
> > Specifically, the filesystem must be huge or on a slow medium. It might make
> > sense to move it out of standard and elsewhere, as I don't think it's
> > necessarily needed everywhere, such as laptops.
> >
> > Consider my laptop, fairly standard, 512 GB NVME SSD, about 250G allocated,
> > containing about 1435134 files. mlocate foo takes 1s, find / -mount
> > -name '*foo*' takes about 7-9 secs, or 19 seconds with all mount points
> > (but there is a davfs mount of an internet server, so things might be
> > screwed up a bit).
> >
> > 19s to find something is perfectly workable, also you don't usually
> > find from /, but you have an idea where things are, so it will be much
> > faster.
> >
> > I think mlocate only really makes sense on data storage servers with
> > huge disks, or on machines with HDDs. I therefore do not think the
> > overhead of building the index is warranted for most users. It might
> > make sense to keep mlocate in always-on tasks, like servers, but get
> > rid of it from desktop scenarios.


--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

Re: mlocate - what is it good for?

On Wed, 22 May 2019, Seth Arnold wrote:

> Anyway, I believe locate can have great value to all our users,
> experienced or brand new, huge systems or small systems. I'd like
> us to keep it in default installs.
>
IMO, find has its place and so does mlocate. I still use locate myself fairly
regularly (perhaps a couple/few times a month?) on my laptop and I have been
known to adjust PRUNEPATHS. As said by another elsewhere, even if it was moved
out of standard we would still want to address the bugs so I'd just assume it
stay in standard.

--
Jamie Strandboge | http://www.canonical.com

Re: mlocate - what is it good for?

On Wed, May 22, 2019 at 11:25:45PM -0700, Bryan Quigley wrote:
> The mentioned Debian bug #880507 is one of the advantages of moving from
> cron to systemd units -see proposed unit file in [1]. The timers in
> systemd are really perfect for this...
>
> I use locate all the time. I really like how much simpler it is then
> find. I don't have to think to usually get what I'm looking for. I'm
> going to experiment with using grep <filename> /var/lib/dpkg/info/*.list as
> the majority of files I run locate for are from packages..
>
> From a product perspective it does make sense to remove:
> * Desktop has tracker installed by default. The majority of users will
> use that instead as it's integrated with the GUI.

Did this change? It's not installed by default in bionic; and no meta
package depends on it, but it seems to have moved to main in disco.

I think both tracker and locate can offer a terrible user experience for
a lot of users, due to I/O load after resume/boot; but tracker also has
a ton of CPU usage extracting stuff from documents and putting the text
into its xapian (?) db.

> * The majority of cloud/server deployments aren't interactive these days.
> It's a waste there.

ack - also, it sucks to have this I/O load once a day.

>
> They key group I can see wanting it are developers - who can install it.

ack


--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel