matthew/sonic-buildimage
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[201803] [logrotate] Enhance robustness (#2941)

Browse Source 
* [logrotate] Decrease frequency to every 10 minutes; kill any lingering logrotate processes

* [logrotate] Decrease usable space to 90%; Delete all *.1.gz files as firstaction
This commit is contained in:
Joe LeVeque 2019-05-25 18:00:33 -07:00 committed by lguohan
parent a78812b6bc
commit 769d3b303c
2 changed files with 10 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
files/image_config/cron.d/logrotate
View File

@ -1,2 +1,3 @@
# Attempt to rotate system logs once per minute
* * * * * root /usr/sbin/logrotate /etc/logrotate.conf > /dev/null 2>&1
# Attempt to rotate system logs once every 10 minutes.
# First kill any logrotate process(es) if they are still running, as they're most likely hung
*/10 * * * * root /usr/bin/pkill -9 logrotate > /dev/null 2>&1; /usr/sbin/logrotate /etc/logrotate.conf > /dev/null 2>&1

10
files/image_config/logrotate/logrotate.d/rsyslog
View File

@ -44,7 +44,7 @@
compress
delaycompress
nosharedscripts
prerotate
firstaction
# Adjust NUM_LOGS_TO_ROTATE to reflect number of log files that trigger this block specified above
NUM_LOGS_TO_ROTATE=8
@ -57,14 +57,18 @@
VAR_LOG_SIZE_KB=$(df -k /var/log | sed -n 2p | awk '{ print $2 }')
# Limit usable space to 95% of the partition minus the reserved space for other logs
USABLE_SPACE_KB=$(( (VAR_LOG_SIZE_KB * 95 / 100) - RESERVED_SPACE_KB))
# Limit usable space to 90% of the partition minus the reserved space for other logs
USABLE_SPACE_KB=$(( (VAR_LOG_SIZE_KB * 90 / 100) - RESERVED_SPACE_KB))
# Set our threshold so as to maintain enough space to write all logs from empty to full
# Most likely, some logs will have non-zero size when this is called, so this errs on the side
# of caution, giving us a bit of a cushion if a log grows quickly and passes its rotation size
THRESHOLD_KB=$((USABLE_SPACE_KB - (NUM_LOGS_TO_ROTATE * LOG_FILE_ROTATE_SIZE_KB * 2)))
# First, delete any *.1.gz files that might be left around from a prior incomplete
# logrotate execution, otherwise logrotate will fail to do its job
find /var/log/ -name '*.1.gz' -type f -exec rm -f {} +
while true; do
USED_KB=$(du -s /var/log | awk '{ print $1; }')