[baseimage]: Run fsck filesystem check support prior mounting filesystem (#4431)

* Run fsck filesystem check support prior mounting filesystem

If the filesystem become non clean ("dirty"), SONiC does not run fsck to
repair and mark it as clean again.

This patch adds the functionality to run fsck on each boot, prior to the
filesystem being mounted. This allows the filesystem to be repaired if
needed.

Note that if the filesystem is maked as clean, fsck does nothing and simply
return so this is perfectly fine to call fsck every time prior to mount the
filesystem.

How to verify this patch (using bash):

Using an image without this patch:

Make the filesystem "dirty" (not clean)
[we are making the assumption that filesystem is stored in /dev/sda3 - Please adjust depending of the platform]
[do this only on a test platform!]

dd if=/dev/sda3 of=superblock bs=1 count=2048
printf "$(printf '\\x%02X' 2)" | dd of="superblock" bs=1 seek=1082 count=1 conv=notrunc &> /dev/null
dd of=/dev/sda3 if=superblock bs=1 count=2048

Verify that filesystem is not clean
tune2fs -l /dev/sda3 | grep "Filesystem state:"

reboot and verify that the filesystem is still not clean
Redo the same test with an image with this patch, and verify that at next reboot the filesystem is repaired and becomes clean.

fsck log is stored on syslog, using the string FSCK as markup.

build_debian.sh

@@ -165,6 +165,10 @@ sudo chmod +x $FILESYSTEM_ROOT/etc/initramfs-tools/scripts/init-premount/arista-
 sudo cp files/initramfs-tools/resize-rootfs $FILESYSTEM_ROOT/etc/initramfs-tools/scripts/init-premount/resize-rootfs
 sudo chmod +x $FILESYSTEM_ROOT/etc/initramfs-tools/scripts/init-premount/resize-rootfs
 
+# Hook into initramfs: run fsck to repair a non-clean filesystem prior to be mounted
+sudo cp files/initramfs-tools/fsck-rootfs $FILESYSTEM_ROOT/etc/initramfs-tools/scripts/init-premount/fsck-rootfs
+sudo chmod +x $FILESYSTEM_ROOT/etc/initramfs-tools/scripts/init-premount/fsck-rootfs
+
 ## Hook into initramfs: after partition mount and loop file mount
 ## 1. Prepare layered file system
 ## 2. Bind-mount docker working directory (docker overlay storage cannot work over overlay rootfs)

files/image_config/platform/rc.local

@@ -340,4 +340,10 @@ if [ -f $FIRST_BOOT_FILE ]; then
     firsttime_exit
 fi
 
+# Copy the fsck log into syslog
+if [ -f /var/log/fsck.log.gz ]; then
+    gunzip -d -c /var/log/fsck.log.gz | logger -t "FSCK"
+    rm -f /var/log/fsck.log.gz
+fi
+
 exit 0

files/initramfs-tools/fsck-rootfs

@@ -0,0 +1,34 @@
+#!/bin/sh
+
+case $1 in
+    prereqs)
+        exit 0
+        ;;
+esac
+
+# Extract kernel parameters
+root_val=""
+set -- $(cat /proc/cmdline)
+for x in "$@"; do
+    case "$x" in
+        root=*)
+            root_val="${x#root=}"
+            ;;
+    esac
+done
+
+# Check the filesystem we are using
+if [ ! -z $root_val ]; then
+    fstype=$(blkid -o value -s TYPE $root_val)
+    case "$fstype" in
+        ext4)
+            cmd="fsck.ext4 -v -p"
+            ;;
+        ext3)
+            cmd="fsck.ext3 -v -p"
+            ;;
+    esac
+    if [ ! -z "$cmd" ]; then
+        $cmd $root_val 2>&1 | gzip -c > /tmp/fsck.log.gz
+    fi
+fi

files/initramfs-tools/union-mount.j2

@@ -74,5 +74,12 @@ then
   mount -t tmpfs -o rw,nosuid,nodev,size=${varlogsize}M tmpfs ${rootmnt}/var/log
   [ -f ${rootmnt}/host/disk-img/var-log.ext4 ] && rm -rf ${rootmnt}/host/disk-img/var-log.ext4
 else
+  [ -f ${rootmnt}/host/disk-img/var-log.ext4 ] && fsck.ext4 -v -p ${rootmnt}/host/disk-img/var-log.ext4 2>&1 \
+                                                  | gzip -c >> /tmp/fsck.log.gz
   [ -f ${rootmnt}/host/disk-img/var-log.ext4 ] && mount -t ext4 -o loop,rw ${rootmnt}/host/disk-img/var-log.ext4 ${rootmnt}/var/log
 fi
+
+## fscklog file: /tmp will be lost when overlayfs is mounted
+if [ -f /tmp/fsck.log.gz ]; then
+    mv /tmp/fsck.log.gz ${rootmnt}/var/log
+fi