A Python package which supports global logfmt formatted logging.

Overview

Python Logfmter

pre-commit test

A Python package which supports global logfmt formatted logging.

Install

$ pip install logfmter

Usage

Before integrating this library, you should be familiar with Python's logging functionality. I recommend reading the Basic Logging Tutorial.

This package exposes a single Logfmter class that can be integrated into the standard library logging system similar to any logging.Formatter.

The provided formatter will logfmt encode all logs. Key value pairs are provided via the extra keyword argument or by passing a dictionary as the log message.

Basic

import logging
from logfmter import Logfmter

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(Logfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error("hello", extra={"alpha": 1}) # at=ERROR msg=hello alpha=1
logger.error({"token": "Hello, World!"}) # at=ERROR token="Hello, World!"

Customize

You can subclass the formatter to change its behavior.

import logging
from logfmter import Logfmter


class CustomLogfmter(Logfmter):
    """
    Provide a custom logfmt formatter which formats
    booleans as "yes" or "no" strings.
    """

    @classmethod
    def format_value(cls, value):
        if isinstance(value, bool):
            return "yes" if value else "no"

	return super().format_value(value)

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(CustomLogfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error({"example": True}) # at=ERROR example=yes

Development

Required Software

Refer to the links provided below to install these development dependencies:

Getting Started

Setup

$ <runtimes.txt xargs -n 1 pyenv install -s
$ direnv allow
$ pip install -r requirements/dev.txt
$ pre-commit install
$ pip install -e .

Tests

Run the test suite against the active python environment.

$ pytest

Run the test suite against the active python environment and watch the codebase for any changes.

$ ptw

Run the test suite against all supported python versions.

$ tox

Publishing

Build

$ python -m build

Upload

$ twine upload dist/*
Comments
  • Question: How to include default / reserved attributes in the log message?

    Question: How to include default / reserved attributes in the log message?

    Thank you for making this library available. I have been looking at the tests and docs but could not find out how I can include some of the keys you defined as "RESERVED" in my message, such as asctime. I would like to say::

    >>> logger.info("hello")
    "asctime=2022-04-20 msg='hello'"
    

    without having to pass asctime in as an extra. I have tried the below in my dictconfig but did not work::

            'logfmt': {
                'format': 'on=%(asctime)s level=%(levelname)s module=%(name)s msg=%(message)s',
                'class': 'logfmter.Logfmter',
            },
    

    Is this possible?

    opened by meitham 3
  • Include Native Log Record Attributes

    Include Native Log Record Attributes

    We should be able to configure the logger to always include some default keys. For example, a user may want to always include a time or logger key.

    This functionality is seen in the python-json-logger.

    opened by jteppinette 1
  • Add Python 11 Official Support / Testing

    Add Python 11 Official Support / Testing

    I assume this will work on Python 11 just fine, but let's add the Python 11 image to the test scripts and see if there are any improvements that can be made.

    opened by jteppinette 0
  • Update Structure

    Update Structure

    We should move the logfmter directory inside src as suggested by:

    • https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
    • https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure%3E

    This is highly recommend and will also support the usage of importlib import mode as recommended here.

    opened by jteppinette 0
  • Exclude Keys

    Exclude Keys

    Some third party libraries may make log calls with keys which we don't want to log. We should be able to exclude these keys from the final log line.

    For example, when overriding uvicorn logging. The color_message key is currently printed (it is used by their custom color formatter). If we could exclude this key, then we could prevent the following output:

    at=INFO msg="Started server process [97819]" color_message="Started server process [%d]"
    at=INFO msg="Waiting for application startup."
    at=INFO msg="Application startup complete."
    at=INFO msg="Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)" color_message="Uvicorn running on %s://%s:%d (Press CTRL+C to quit)"
    
    opened by jteppinette 0
Releases(v0.0.6)
  • v0.0.6(Apr 22, 2022)

  • v0.0.5(Apr 20, 2022)

    • Add support for include native log record attributes in the final log output.
    • Add support for overriding the date format used when formatting the asctime attribute.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Mar 30, 2022)

    • Fix the usage documentation
    • Escape newline characters in all logged values. Previously, you could generate multi-line log statements. This should never be the case.
    • Add support for auto-generating exc_info parameters. If the log record is generated with exc_info, as in when using logging.exception(...), the log message will contain properly formatted exception and traceback information.
    • Add support for logging with an empty message dictionary.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 12, 2022)

  • v0.0.1(Feb 6, 2022)

The new Python SDK for Sentry.io

sentry-python - Sentry SDK for Python This is the next line of the Python SDK for Sentry, intended to replace the raven package on PyPI. from sentry_s

Sentry 1.4k Dec 31, 2022
Integrates a UPS monitored by NUT into OctoPrint

OctoPrint UPS This OctoPrint plugin interfaces with a UPS monitored by NUT (Network UPS Tools). Requirements NUT must be configured by the user. This

Shawn Bruce 11 Jul 05, 2022
Fancy console logger and wise assistant within your python projects

Fancy console logger and wise assistant within your python projects. Made to save tons of hours for common routines.

BoB 5 Apr 01, 2022
Soda SQL Data testing, monitoring and profiling for SQL accessible data.

Soda SQL Data testing, monitoring and profiling for SQL accessible data. What does Soda SQL do? Soda SQL allows you to Stop your pipeline when bad dat

Soda Data Monitoring 51 Jan 01, 2023
Display tabular data in a visually appealing ASCII table format

PrettyTable Installation Install via pip: python -m pip install -U prettytable Install latest development version: python -m pip install -U git+https

Jazzband 924 Jan 05, 2023
Discord-Image-Logger - Discord Image Logger With Python

Discord-Image-Logger A exploit I found in discord. Working as of now. Explanatio

111 Dec 31, 2022
A python library used to interact with webots robocup game web logs

A python library used to interact with webots robocup game web logs

Hamburg Bit-Bots 2 Nov 05, 2021
Key Logger - Key Logger using Python

Key_Logger Key Logger using Python This is the basic Keylogger that i have made

Mudit Sinha 2 Jan 15, 2022
Progressbar 2 - A progress bar for Python 2 and Python 3 - "pip install progressbar2"

Text progress bar library for Python. Travis status: Coverage: Install The package can be installed through pip (this is the recommended method): pip

Rick van Hattem 795 Dec 18, 2022
Ultimate Logger - A Discord bot that logs lots of events in a channel written in python

Ultimate Logger - A Discord bot that logs lots of events in a channel written in python

Luca 2 Mar 27, 2022
A Fast, Extensible Progress Bar for Python and CLI

tqdm tqdm derives from the Arabic word taqaddum (تقدّم) which can mean "progress," and is an abbreviation for "I love you so much" in Spanish (te quie

tqdm developers 23.7k Jan 01, 2023
Translating symbolicated Apple JSON format crash log into our old friends :)

CrashTranslation Translating symbolicated Apple JSON format crash log into our old friends :) Usage python3 translation.py -i {input_sybolicated_json_

Kam-To 11 May 16, 2022
Rich is a Python library for rich text and beautiful formatting in the terminal.

Rich 中文 readme • lengua española readme • Läs på svenska Rich is a Python library for rich text and beautiful formatting in the terminal. The Rich API

Will McGugan 41.5k Jan 07, 2023
This is a key logger based in python which when executed records all the keystrokes of the system it has been executed on .

This is a key logger based in python which when executed records all the keystrokes of the system it has been executed on

Purbayan Majumder 0 Mar 28, 2022
Docker container log aggregation with Elasticsearch, Kibana & Filebeat

Epilog Dead simple container log aggregation with ELK stack Preface Epilog aims to demonstrate a language-agnostic, non-invasive, and straightfo

Redowan Delowar 23 Oct 26, 2022
Lazy Profiler is a simple utility to collect CPU, GPU, RAM and GPU Memory stats while the program is running.

lazyprofiler Lazy Profiler is a simple utility to collect CPU, GPU, RAM and GPU Memory stats while the program is running. Installation Use the packag

Shankar Rao Pandala 28 Dec 09, 2022
HTTP(s) "monitoring" webpage via FastAPI+Jinja2. Inspired by https://github.com/RaymiiOrg/bash-http-monitoring

python-http-monitoring HTTP(s) "monitoring" powered by FastAPI+Jinja2+aiohttp. Inspired by bash-http-monitoring. Installation can be done with pipenv

itzk 39 Aug 26, 2022
A basic logging library for Python.

log.py 📖 About: A basic logging library for Python with the capability to: save to files. have custom formats. have custom levels. be used instantiat

Sebastiaan Bij 1 Jan 19, 2022
Track Nano accounts and notify via log file or email

nano-address-notifier Track accounts and notify via log file or email Required python libs

Joohansson (Json) 4 Nov 08, 2021
A watchdog and logger to Discord for hosting ScPrime servers.

ScpDog A watchdog and logger to Discord for hosting ScPrime servers. Designed to work on Linux servers. This is only capable of sending the logs from

Keagan Landfried 3 Jan 10, 2022