Commit Graph

7 Commits

Author SHA1 Message Date
Danny Allen
818ab7fdaa [core_cleanup] Fix issue where core_cleanup job runs too frequently (#3659)
Signed-off-by: Danny Allen <daall@microsoft.com>
2019-10-24 17:04:16 +00:00
Danny Allen
ba77de12ac [cron.d] Add cron job to periodically clean-up core files (#3449)
* [cron.d] Create cron job to periodically clean-up core files
* Create script to scan /var/core and clean-up older core files
* Create cron job to run clean-up script

Signed-off-by: Danny Allen <daall@microsoft.com>

* Update interval for running cron job

* Respond to feedback

* Change syslog id
2019-09-13 17:52:10 +00:00
Joe LeVeque
8ae67c4c5d [logrotate] Enhance robustness (#2942)
* [logrotate] Decrease frequency to every 10 minutes; kill any lingering logrotate processes

* [logrotate] Delete all *.1.gz files as firstaction; Remove note about init-system-helpers < 1.47 workaround

However, continue to send SIGHUP directly to rsyslogd process
because 'service rsyslog rotate' still doesn't work properly with
init-system-helpers version 1.48
2019-05-29 00:53:13 +00:00
Joe LeVeque
9d5adb993a [System logs]: Improvements to prevent filling /var/log partition (#865)
- Force log rotation at size thresholds only (no longer also rotating logs daily), allowing for more consistent archived log size
- Eliminate remaining duplicate log messages
- Cron facility now only logs to cron.log (was also logging to syslog)
- Debug, mail, news and user log facilities only log to syslog; no longer creating separate log files for these facilities
- Cron job that calls logrotate every minute now uses the main /etc/logrotate.conf file so as to check/rotate all logs every minute, not just the logs specified in the rsyslog file. Also redirecting output of this command to /dev/null to prevent "(CRON) info (No MTA installed, discarding output)" messages in cron.log due to lack of a mail service
- Delete archive files based on remaining /var/log partition space. Note that this solution currently requires a minimum /var/log partition size of 32MB to function correctly
- Update sonic-sairedis and sonic-swss submodules to incorporate recording file name changes
- Add .screen file to .gitignore (unrelated)
2017-08-10 16:24:57 -07:00
Joe LeVeque
f49cac086f Remove extra trailing newlines at EOF (#804)
Files now end with a single newline
2017-07-12 20:54:37 -07:00
Joe LeVeque
f28790ce9a [System logs]: Fix logrotate bugs (#535)
- rsyslog logs were being rotated regardless of whether they exceeded their maixmum size. This was due to "-f" flag passed to logrotate in cron job.

- After rotation, /var/log/syslog was never written to again. Instead, logs were written to /var/log/syslog.1. This was due to rsyslog not properly closing the file descriptor to the pre-rotated log.

- Also brought back time-related rotation via the new(er) maxsize option, which performs a boolean OR operation. If the log exceeds the maxsize OR the log hasn't been rotated in the specified, it will be rotated. Using the older size option, the time-based rotation was ignored.

- Also addresses issue #528
2017-04-24 17:55:05 -07:00
Joe LeVeque
6c202919b3 [System logs]: Eliminate duplicate log messages and attempt rotation more frequently (#520)
* Rename 'ACSFileFormat' -> 'SONiCFileFormat'

* Rename '00-acs.conf' -> '00-sonic.conf'

* Add logrotate.d and systemd-journald config files to image

* Log all SONiC process messages to /var/log/syslog; prevent duplicate logging to /var/log/messages

* Do not redirect cron and daemon logs to their own files, let them log to /var/log/syslog

* Log all teamd messages to /var/log/teamd.log; Add more SONiC program names to SONiC rules clause

* Remove duplicate code by condensing quagga programs into a list; Fix teamd log rule

* Kernel and LPR messages no longer getting duplicated to their own log files

* Now calling logrotate every minute via cron job

* Need full path to logrotate in cron job

* Add '.log' suffix to wildcards, otherwise logrotate will rotate already-rotated logs (e.g., bgpd.log.1.1.1.1.1...)

* Add microsecond granularity to syslog messages

* Don't overwrite system crontab, instead, install additional logrotate crontab file into /etc/cron.d

* Removed incomplete concept of per-process SONiC logs. We can revisit again later
2017-04-21 08:22:44 -07:00