Nginx Log Rotation Management Explained

Your web server’s activity, performance, and errors can be understood more effectively with the help of Nginx’s various log files. The access log keeps track of all requests made to the server, whereas the error log records any errors the server may encounter during its operation. By thoroughly analyzing these logs, you can easily identify issues, monitor traffic patterns, and enhance your website’s overall performance.

Why Log Rotation is Necessary

As your website becomes more popular and more people access your server, your log files can quickly become overwhelming, taking up valuable disk space. Log rotation is a necessary process that involves archiving, compressing, and deleting old log files. This practice is crucial for multiple reasons: it prevents your log files from becoming unmanageable, optimizes your disk space, enhances your system performance, and makes it easier to analyze your logs.

Log Rotation Methods

There are multiple ways to rotate Nginx log files. Let’s explore the most common methods:¹²

Manual Log Rotation: To manually rotate log files in Nginx, you need to stop the server, rename the log files, and start it again. While this approach offers complete control, it can be time-consuming and susceptible to human error. Besides, it’s not practical for large-scale deployments or frequent log rotations.
Logrotate Utility: A better alternative is to use Logrotate, a widely used utility that automates log file rotation. With Logrotate, you can configure rules for rotation frequency, retention policies, compression, and more via a simple configuration file. It can also perform post-rotation actions like notifying processes to reopen log files, ensuring hassle-free log management. Logrotate is a trusted choice and is available on most Unix-like systems.
Third-Party Log Rotation Tools: For advanced features and flexibility, third-party log rotation tools like cronolog, rotatelogs, and logadm can be considered. These tools offer additional customization options such as custom log file naming conventions, integration with external services, and more. Always choose a reputable tool that meets your needs when evaluating third-party options.

Log Rotation with logrotate Utility

The logrotate utility is a powerful tool for automating log file rotation. Let’s walk through the process of setting up log rotation for Nginx using logrotate:

Step 1: Install logrotate (if not already installed)

Depending on your Linux distribution, logrotate may already be installed. If not, install it using system’s the package manager. For Ubuntu or Debian this is the command to use:³

sudo apt-get install logrotate

Step 2: Create a logrotate configuration file for Nginx

Navigate to the logrotate configuration directory:

cd /etc/logrotate.d

Create a new configuration file for Nginx, such as nginx:

sudo nano nginx

Inside the file, add the following configuration (modify paths and options as per your setup):⁴

/var/log/nginx/*.log {
    daily
    rotate 7
    missingok
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        /etc/init.d/nginx reload > /dev/null
    endscript
}

Explaining each configuration option:
- daily: Rotates the logs on a daily basis.
- rotate 7: Keeps up to 7 rotated log files.
- missingok: Does not produce an error if the log file is missing.
- compress: Compresses the rotated log files using gzip.
- delaycompress: Delays compression of the previous log file until the next rotation cycle.
- notifempty: Disable rotation for empty log files.
- create 0640 www-data adm: Sets the ownership and permissions of the newly created log files.
- sharedscripts: Executes the post-rotate script only once for all matching log files.
- postrotate and endscript: The commands within these tags are executed after log rotation. In this example, we reload Nginx to ensure it starts writing to the new log files.

Step 3: Save and exit the configuration file.

Step 4: Test log rotation manually

You can manually test the log rotation process to verify that it’s working correctly before waiting for the next scheduled rotation. Execute the following command to simulate the log rotation:

sudo logrotate -f /etc/logrotate.d/nginx

This command forces logrotate to run using the specified configuration file. Check that the log files are rotated as expected, and Nginx continues writing to the new log file.

Step 5: Set up a log rotation schedule

By default, logrotate runs daily through a cron job. You can adjust the schedule by modifying the corresponding cron file. For instance, on Ubuntu or Debian, you can edit the cron file using the following command:⁵

sudo nano /etc/cron.daily/logrotate

Adjust the frequency according to your requirements. Save the file and exit.

That’s it! You have successfully set up log rotation for Nginx using the logrotate utility. Logrotate will now handle the rotation of your Nginx log files automatically, based on the configuration you defined.

Remember to periodically check the log rotation process and monitor disk space usage to ensure smooth log management and prevent any issues caused by log files consuming excessive disk space.

Best Practices for Nginx Log Rotation

To optimize log rotation with Nginx, consider the following best practices:

Determine an appropriate log rotation frequency and retention policy based on your server’s traffic and disk space availability.
Set correct file permissions and ownership to ensure proper access and security.

Handle log rotation during high traffic periods to minimize disruption to your website’s visitors.
Archive and compress rotated log files to save disk space while preserving data integrity.
Monitor the log rotation process to ensure it runs smoothly and catch any potential issues promptly.

Advanced Techniques and Troubleshooting

If you require more advanced log rotation capabilities, you may want to consider using compression algorithms like gzip to compress log files and further decrease disk space usage. It’s also important to have a log rotation strategy that considers each of your Nginx instances individually if you have multiple instances. If you encounter any common log rotation problems, such as failures or misconfigurations, you can troubleshoot them by examining error messages, permissions, or log file locking within logrotate.

References

Jun, C. M., & Jun, C. M. (2021, December 31). Rotating logs with Logrotate in Linux | Baeldung on Linux. Baeldung on Linux. https://www.baeldung.com/linux/rotating-logs-logrotate ↩︎
Henry-Stocker, S. (2020, March 18). Manually rotating log files on Linux. Network World. https://www.networkworld.com/article/968476/manually-rotating-log-files-on-linux.html ↩︎

Parthiban, P. (2023, June 14). How to empty, delete, or rotate log files in Linux? Atatus Blog – for DevOps Engineers, Web App Developers and Server Admins. https://www.atatus.com/blog/empty-delete-or-rotate-log-files/ ↩︎
Ubiq. (2020, December 14). How to Configure NGINX Log Rotation – UBIQ BI. Ubiq BI. https://ubiq.co/tech-blog/how-to-configure-nginx-log-rotation/ ↩︎
Mulonda, Y. (2022, November 4). Cron & logrotate: How To Use Cron To Automate Tasks. Medium. https://blog.devgenius.io/cron-logrotate-how-to-use-cron-to-automate-tasks-a93069d9185c ↩︎