Async boto3 with Autogenerated Data Classes

Overview

awspydk

Async boto3 with Autogenerated JIT Data Classes


Motivation

This library is forked from an internal project that works with a lot of backend AWS APIs, and I got tired of having to constantly parse the returned responses before being able to work with them for (most of the time), a few seconds. Any API-driven application that uses boto3 tends to suffer from being syncronous only. So this library solves a few major problems:

  • Enables both sync and async from the same client.

  • Client can be called implicitly without needing initialization, i.e. you can directly use AwsClient without needing to initialize.

  • Dynamically generates class functions based on the boto3.client child functions, and allows you to call them. This is useful in repl or ipython environments where type-hints are always helpful, especially when the names are so long.

  • Automatically initializes the aws client if its not initialized from the defaults, simply by calling it.

  • Translates all results into Automatically Generated Dataclasses through lazycls.

    • Can be disabled by setting aws.config.AutoCls = False

Quickstart

pip install --upgrade awspydk
from aws import AwsClient

# Sync Method
buckets = AwsClient.v1.s3_list_buckets(as_cls=True)

# Async Method
buckets = await AwsClient.v1.async_s3_list_buckets(as_cls=True)

"""
Both yield the same results.
The underlying classes are auto-generated from Pydantic BaseModels, so anything you can do with Pydantic Models, you can do with these.

{
    'Buckets': [
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 8, 25, 16, 42, 46, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 17, 54, 56, tzinfo=tzutc()), Name='...',
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 3, 4, 20, 10, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 1, 20, 50, 33, tzinfo=tzutc()), Name='...'),
        AwsS3Bucket(CreationDate=datetime.datetime(2021, 9, 2, 4, 2, 28, tzinfo=tzutc()), Name='...')
    ],
    'Owner': AwsS3Owner(DisplayName='...', ID='...')
}
"""

## Change Regions
AwsClient.reset(region='us-west-1')

## Change the defaut clients created
from aws.config import DefaultClients

## Modify to only create ec2 client
DefaultClients = {
    'ec2': 'ec2'
}

## Reset implicitly
AwsClient.reset()


BotoKwargs = {
    'AWS_PROFILE': ...,
}

## Reset Explicitly
AwsClient.reset(clients=DefaultClients, boto_kwargs=BotoKwargs)

Client Defaults

These are found in aws.config

AwsRegion = envToStr('AWS_REGION', 'us-east-1')
AutoCls = envToBool('AWSSDK_AUTOCLS', 'true')

## These are the default clients that will be autogenerated.
## Key is the shorthand, value is the actual AWS API Name in boto3
DefaultClients = {
    'ec2': 'ec2', 
    'ecr': 'ecr', 
    'r53' :'route53', 
    'acm': 'acm',
    'elb': 'elb',
    'elbv2': 'elbv2',
    'asg': 'autoscaling',
    's3': 's3'
}

## These are the default resources that will be autogenerated.
## Key is the shorthand, value is the actual AWS Resource API Name in boto3
DefaultResources = {
    'Ec2': 'ec2',
    'S3': 's3',
    'Iam': 'iam'
}

# These are the default filter args for querying
DefaultFilterArgs = {
    'string_only': True,
    'remove_null': True
}
Owner
Chief Architect @ Growth Engine
Telegram bot to scrape images from the reddit universe

Telegram bot to scrape images from the reddit universe

XD22 3 Sep 30, 2022
GitHub Activity Generator - A script that helps you instantly generate a beautiful GitHub Contributions Graph for the last year.

GitHub Activity Generator A script that helps you instantly generate a beautiful GitHub Contributions Graph for the last year. Before 😐 😶 😒 After ?

1 Dec 30, 2021
Framework for Telegram users and chats investigating.

telegram_scan Fantastic and full featured framework for Telegram users and chats investigating. Prerequisites: pip3 install pyrogram; get api_id and a

71 Dec 17, 2022
Multi-purpose bot made with discord.py

PizzaHat Discord Bot A multi-purpose bot for your server! ℹ️ • Info PizzaHat is a multi-purpose bot, made to satisfy your needs, as well as your serve

DTS 28 Dec 16, 2022
Python client and module for BGP Ranking

Python client and module for BGP Ranking THis project will make querying BGP Ranking easier. Installation pip install pybgpranking Usage Command line

D4 project 3 Dec 16, 2021
Rhythm bot clone for discord written in Python and uses YouTube to get media files.

Tunebot About Rhythm bot clone for discord written in Python and uses YouTube to get media files. Usage You need a .env file within the same directory

1 Oct 21, 2021
SpotPlay2YouPlay - Converts new additions to a Spotify playlist to a matching Youtube playlist

SpotPlay2YouPlay - Converts new additions to a Spotify playlist to a matching Youtube playlist, can also be configured to converting whole playlists with the refresh fun

9 Mar 06, 2022
Adriano's Diets Consulting Bot - Parses and extracts informations about your diet (files in the Adriano's format).

Adriano's Diets Consulting Bot - Parses and extracts informations about your diet (files in the Adriano's format).

Marco A. 2 Feb 07, 2022
A telegram string extractor bot

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.com/FayasNoushad/String-Extract-Bot/blob/main/LIC

Fayas Noushad 12 Jul 19, 2022
Telegram-Voice Recoginiton Project (Python)

Telegram-Voice Recoginiton Project (Python) It is a telegram bot that analyses voice messages and convert it to text and reply back response on bot's

Krishnadev P Melevila 1 Jan 28, 2022
Diablo II Resurrected helper

Diablo II Resurrected 快捷施法辅助 功能: + 创建守护进程,注册全局热键 alt+/ 启用和关闭功能 (todo: 播放声音提示) + 按 x 强制移动 + 按 1 ~ 0 快捷施法到鼠标区域 使用 编辑配置 settings.py 技能信息做如下定义: SKILLS:

Wan 2 Nov 06, 2022
Reddit bot for r/khiphop

khiphop-bot Description This project is a collection of scripts that better the state of the r/khiphop subreddit, which represents Korean Hip-Hop and

1 Dec 21, 2021
Scheduled Block Checker for Cardano Stakepool Operators

ScheduledBlocks Scheduled Block Checker for Cardano Stakepool Operators Lightweight and Portable Scheduled Blocks Checker for Current Epoch. No cardan

SNAKE (Cardano Stakepool) 4 Oct 18, 2022
This is a translator that i made by myself in python with the 'googletrans' library

Translator-Python This is a translator that i made by myself in python with the 'googletrans' library This application completely made in python allow

Thadeuks 2 Jun 17, 2022
Red-mail - Advanced email sending library for Python

Red Mail Next generation email sender What is it? Red Mail is an advanced email

Mikael Koli 313 Jan 08, 2023
0-1 knapsack with an additional constraint of maximum number of items used

extended_knapsack 0-1 knapsack with an additional constraint of maximum number of items used. Usage pip3 install extended_knapsack from extended_knaps

Lion Ralfs 1 Jan 06, 2022
A GUI Weather Application written with Python

weather-box - A GUI Weather Application written with Python Made with ❤️ by Suresh Mishra

Suresh Mishra 2 Dec 18, 2021
Personal Discord Python Bot based on Discord.py

Personal Discord bot using the discord.py library by Rapptz

2 Dec 14, 2022
Scrapes an instagram user's photos and videos

Instagram Scraper instagram-scraper is a command-line application written in Python that scrapes and downloads an instagram user's photos and videos.

7.3k Nov 18, 2022
A modern, easy to use, feature-rich, and async ready API wrapper improved and revived from original discord.py.

A Python API wrapper that is improved and revived from the original discord.py

Orion 19 Nov 06, 2021