On Mon, Nov 13, 2017 at 2:27 AM, Robie Basak <robie.basak@ubuntu.com> wrote:
Hi Mark,
On Tue, Nov 07, 2017 at 08:11:27PM +0000, Mark Stosberg wrote:
> Some time around the 15.04 release, a policy change was made to quit
> making some logging persistent by default.
To help me understand the problem, exactly what logging was persistent
previously, and isn't persistent now? Can you please give us an example?
Any data in systemd-journal that is not flush to /var/log/* is gone forever on power fail due
to the journal being hosted on ephemeral tmpfs (/run/log/journal);
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
root@t-nonvocational-azzie:/var/log# ls -al /var/log/
total 500
drwxrwxr-x 8 root syslog 4096 Nov 13 15:19 .
drwxr-xr-x 12 root root 4096 Nov 10 20:43 ..
drwxr-xr-x 2 root root 4096 Nov 10 20:43 apt
-rw-r----- 1 syslog adm 2884 Nov 13 15:19 auth.log
-rw-r--r-- 1 root root 7678 Nov 13 15:12 boot.log
-rw-rw---- 1 root utmp 0 Nov 10 20:43 btmp
-rw-r--r-- 1 syslog adm 153678 Nov 13 15:12 cloud-init.log
-rw-r--r-- 1 root root 3870 Nov 13 15:12 cloud-init-output.log
drwxr-xr-x 2 root root 4096 Nov 30 2016 dist-upgrade
-rw-r--r-- 1 root adm 30483 Nov 13 15:12 dmesg
-rw-r--r-- 1 root root 0 Nov 13 15:12 dmesg.0
drwxr-xr-x 2 root root 4096 Nov 10 20:41 fsck
-rw-r----- 1 syslog adm 52797 Nov 13 15:13 kern.log
drwxr-xr-x 2 landscape root 4096 Nov 13 15:12 landscape
-rw-rw-r-- 1 root utmp 292292 Nov 13 15:19 lastlog
-rw-r----- 1 syslog adm 55736 Nov 13 15:17 syslog
-rw-r--r-- 1 root root 148851 Nov 13 15:12 udev
drwxr-xr-x 2 root root 4096 May 8 2017 unattended-upgrades
drwxr-xr-x 2 root root 4096 Nov 13 15:19 upstart
-rw-rw-r-- 1 root utmp 5376 Nov 13 15:19 wtmp
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
ubuntu@x-undiluvial-edith:/var/log$ ls -al /var/log/
total 312
drwxrwxr-x 7 root syslog 4096 Nov 13 15:15 .
drwxr-xr-x 13 root root 4096 Nov 10 14:25 ..
drwxr-xr-x 2 root root 4096 Nov 10 14:24 apt
-rw-r----- 1 syslog adm 3181 Nov 13 15:17 auth.log
-rw------- 1 root utmp 0 Nov 10 14:24 btmp
-rw-r--r-- 1 syslog adm 127616 Nov 13 15:15 cloud-init.log
-rw-r--r-- 1 root root 4271 Nov 13 15:15 cloud-init-output.log
drwxr-xr-x 2 root root 4096 Oct 20 10:35 dist-upgrade
drwxr-xr-x 2 root root 4096 Nov 10 14:22 fsck
-rw-r----- 1 syslog adm 52555 Nov 13 15:15 kern.log
-rw-rw-r-- 1 root utmp 292292 Nov 13 15:16 lastlog
drwxr-xr-x 2 root root 4096 Aug 23 02:06 lxd
-rw-r----- 1 syslog adm 82360 Nov 13 15:17 syslog
drwxr-x--- 2 root adm 4096 Sep 20 14:13 unattended-upgrades
-rw-rw-r-- 1 root utmp 2688 Nov 13 15:16 wtmp
boot.log includes services that start on boot,
These service starts are not part of syslog; ssh for example in boot.log shows
it starting (but any service may include more information here in case of failure).
# grep -i openssh boot.log
* Starting OpenSSH server [ OK ]
$ journalctl -o short-precise --no-pager -u ssh
-- Logs begin at Mon 2017-11-13 15:15:13 UTC, end at Mon 2017-11-13 15:17:01 UTC. --
Nov 13 15:15:33.816052 x-undiluvial-edith systemd[1]: Starting OpenBSD Secure Shell server...
Nov 13 15:15:33.846646 x-undiluvial-edith sshd[1249]: Server listening on 0.0.0.0 port 22.
Nov 13 15:15:33.847790 x-undiluvial-edith sshd[1249]: Server listening on :: port 22.
Nov 13 15:15:33.848100 x-undiluvial-edith systemd[1]: Started OpenBSD Secure Shell server.
Nov 13 15:16:35.015383 x-undiluvial-edith sshd[1317]: Accepted publickey for ubuntu from 10.5.0.10 port 54030 ssh2: RSA SHA256:0cuEJGbgH9aUy12t0jKluNEwLeAF4SIbvimjeM1OLWw
Nov 13 15:16:35.018911 x-undiluvial-edith sshd[1317]: pam_unix(sshd:session): session opened for user ubuntu by (uid=0)
/var/log/udev is another.
In general when systems fail to boot, users may not have console access but may retain power control.
Issuing a shutdown or even hard stop will still preserve data that's been written to the filesystem for later
inspection. With the bulk if boot and service status and output information included in the journald held in
tmpfs; we lose that data unless there is some way to flush that data to persistent storage.
I'm +1 for having journald be persistent by default; it certainly makes debugging boot failures
much easier since one has actual logs to examine versus nothing at all.
Thanks,
Robie
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
