truncate log files in the given directory with spezified size in Gigabytes.

README.md

Truncate Large Files

This Python script searches a specified directory and truncates all files with the .log extension that exceed a certain size.

Requirements

• Python 3.6 or higher

Usage

Run the script with the following arguments:

• --directory: The directory to search for large .log files.
• --max-size: The maximum file size in GB. .log files that exceed this size will be truncated. The default value is 5 GB.

Example:

python app.py --directory /var/lib/docker/containers/ --max-size 10

This example searches the directory /var/lib/docker/containers/ and truncates all .log files that are larger than 10 GB.

Logging

The script logs its activities in the file /var/log/truncate_files.log. It logs both successful truncations and errors.

Warning

Please be careful when using this script. It truncates .log files irreversibly to free up storage space. Make sure you don't lose any important data.

Run in Docker

You can also run this script in a Docker container. Use the provided Dockerfile and build a container.

# docker build -t truncate_logs:latest .

Run the container with the required arguments:

# docker run -it --rm -v /var/lib/docker/containers/:/containers:rw -v /var/log:/app/log:rw truncate_logs:latest ./app.py --directory /containers/ --max-size 10

Run Docker Container in Cronjob

You can also set up a cron job to run the Docker container regularly. To do this, edit the crontab file:

# crontab -e

Add the following line to run the container every 6 hours:

0 */6 * * * docker run -it --rm -v /var/lib/docker/containers/:/containers:rw -v /var/log:/app/log:rw truncate_logs:latest ./app.py --directory /containers/ --max-size 10

License

MIT License

Copyright (c) 2024 Günter Bailey

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

direct translation from github Copilot.

app.py

#!/usr/bin/env python3
import os
import logging


class TruncateLargeFiles:
    def __init__(self, directory: str = '/var/lib/docker/containers/', max_size: int = (5 * 1024 * 1024 * 1024)):
        self.directory = directory
        self.max_size = max_size

    @staticmethod
    def calculate_bytes(size: int = 0) -> str:
        return f"{size / (1024 * 1024 * 1024):.2f} GB"

    def truncate_file(self, filepath: str):
        try:
            file_size = os.path.getsize(filepath)
            if file_size > self.max_size:
                with open(filepath, 'w') as file:
                    file.truncate(0)
                logging.info(f"File '{filepath}' truncated from {self.calculate_bytes(int(file_size))} successfully.")
            else:
                logging.debug(f"Skipping file '{filepath}'.")
        except OSError as e:
            logging.error(f"Error: {e}")

    def truncate_large_files(self):
        try:
            for root, dirs, files in os.walk(self.directory):
                for filename in files:
                    filepath = os.path.join(root, filename)
                    if filename.endswith('.log'):
                        self.truncate_file(filepath=filepath)
                    else:
                        logging.debug(f"Skipping non-log file '{filename}'.")
        except OSError as e:
            logging.error(f"Error: {e}")


if __name__ == "__main__":
    import argparse
    from pathlib import Path

    parser = argparse.ArgumentParser(description='Truncate large log files in a directory.')
    parser.add_argument('--directory', metavar='DIRECTORY', type=str, default='/var/lib/docker/containers/',
                        help='the directory to search for large log files')
    parser.add_argument('--max-size', dest='max_size', metavar='MAX_SIZE', type=int, default=5,
                        help='the maximum size of files to truncate in GB (default: 5)')
    parser.add_argument('--log', dest='log_file', type=str, default='./log/truncate_files.log',
                        help='the log file to write logs to (absolute path) (default: truncate_files.log)')
    args = parser.parse_args()

    if not Path(args.log_file).exists():
        Path(args.log_file).parent.mkdir(parents=True, exist_ok=True)

    logging.basicConfig(filename=args.log_file, level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

    calculated_size = int(args.max_size) * 1024 * 1024 * 1024  # Convert GB to bytes

    TruncateLargeFiles(directory=args.directory, max_size=calculated_size).truncate_large_files()

Dockerfile

FROM python:3.11-slim-bullseye
MAINTAINER Gbailey

RUN mkdir -p "/app/log" && \
    chmod -R 775 "/app"

WORKDIR "/app"

COPY app.py /app/
COPY README.md /app/
RUN chmod a+x app.py

VOLUME ["/app/log"]

CMD ["./app.py", "--help"]

Dockerfile to run this script on server < Python3.6

FROM python:3.11-slim-bullseye
MAINTAINER Gbailey

RUN mkdir -p "/app/log" && \
    chmod -R 775 "/app"

WORKDIR "/app"

COPY app.py /app/
COPY README.md /app/
RUN chmod a+x app.py

VOLUME ["/app/log"]

CMD ["./app.py", "--help"]

On first time, docker build: docker build -t truncate_logs:latest .

command to run in cronjob:

45 */6 * * * root docker run -it --rm -v /var/lib/docker/containers/:/containers:rw -v /var/log:/app/log:rw truncate_logs:latest ./app.py --directory /containers/ --max-size 10