kyle's datadog python vision/proposal

not for production use

See examples/comprehensive.py for a mostly working example of the proposed API.

📈 🐶 ❤️ 🐍

The Datadog Python products are great but the Python offering is fragmented.

One has to configure and initialize 4 different clients (metrics, logs, tracing, profiling) to get a cohesive experience.

It's time to unify and provide a great user experience out of the box for users.

proposed API

from datadog import DDClient, DDConfig

# Options are
#  - type-checked + validated
#  - available as corresponding environment vars
ddcfg = DDConfig(
        agent_url="localhost",
        datadog_site="us1.datadoghq.com",
        service="my-python-service",
        env="prod",
        version="0.01",
        tracing_enabled=True,
        tracing_patch=True,
        tracing_modules=["django", "redis", "psycopg2"],
        tracing_sampling_rules=[("my-python-service", "prod", 0.02)],
        profiling_enabled=True,
        security_enabled=True,
        runtime_metrics_enabled=True,
)
ddclient = DDClient(config=ddcfg)

# metrics
ddclient.gauge()
ddclient.measure()
ddclient.count()
ddclient.flush_metrics()

# logs
ddclient.log()
ddclient.warning()
ddclient.exception()
ddclient.info()
ddclient.debug()
log = ddclient.getLogger()
ddclient.DDLogHandler()  # or datadog.DDLogHandler()
ddclient.flush_logs()

# tracing
ddclient.trace()
ddclient.patch()
ddclient.flush_traces()

# profiling
ddclient.profiling_start()
ddclient.profiling_stop()
ddclient.flush_profiles()

package structure

+datadog
|
|- DDClient
|- DDConfig

`ddtrace-run`

I propose datadog-run which will install a default DDClient, initialized only via environment variable to datadog.client. Essentially sitecustomize.py would just be something like:

import datadog
from datadog import DDConfig, DDClient


_DEFAULT_CONFIG = dict(
  tracing_patch=True,  # different from the default when using the library manually
  # ... rest of defaults
)

datadog.client = DDClient(DDConfig(default_config=_DEFAULT_CONFIG))

open questions/concerns

What API is exposed for flushing data?
- Unified for entire client?
  - Reuse connections/batch data for performance.
- Must allow both automatic + manual strategies
  - Buffer size
  - Flush period
What to use to locate an agent?
- UDS vs HTTP(S) support
- URL is weird/not intuitive with unix sockets
Should config values store whether they are user defined?

kyle's vision of how datadog's python client should look

Related tags

Overview

kyle's datadog python vision/proposal

📈 🐶 ❤️ 🐍

proposed API

package structure

`ddtrace-run`

open questions/concerns

Owner

Kyle Verhoog

Jupyter Notebook extension leveraging pandas DataFrames by integrating DataTables and ChartJS.

Squidpy is a tool for the analysis and visualization of spatial molecular data.

Learning Convolutional Neural Networks with Interactive Visualization.

Tools for writing, submitting, debugging, and monitoring Storm topologies in pure Python

649 Pokémon palettes as CSVs, with a Python lib to turn names/IDs into palettes, or MatPlotLib compatible ListedColormaps.

An(other) implementation of JSON Schema for Python

Numerical methods for ordinary differential equations: Euler, Improved Euler, Runge-Kutta.

Cartopy - a cartographic python library with matplotlib support

This is simply repo for line drawing rendering using freestyle in Blender.

Schema validation just got Pythonic

A simple project on Data Visualization for CSCI-40 course.

I've demonstrated the working of the decision tree-based ID3 algorithm. Use an appropriate data set for building the decision tree and apply this knowledge to classify a new sample. All the steps have been explained in detail with graphics for better understanding.

CPG represent!

Simple addon for snapping active object to mesh ground

Gallery of applications built using bqplot and widget libraries like ipywidgets, ipydatagrid etc.

3D Vision functions with end-to-end support for deep learning developers, written in Ivy.

This GitHub Repository contains Data Analysis projects that I have completed so far! While most of th project are focused on Data Analysis, some of them are also put here to show off other skills that I have learned.

VDLdraw - Batch plot the log files exported from VisualDL using Matplotlib

Advanced_Data_Visualization_Tools - The present hands-on lab mainly uses Immigration to Canada dataset and employs advanced visualization tools such as word cloud, and waffle plot to display relations between features within the dataset.

With Holoviews, your data visualizes itself.