The elegance of Airflow + the power of AWS

Overview

Orkestra

The elegance of Airflow + the power of AWS

Docs Codecov

PyPI PyPI - Downloads PyPI - License PyPI - Python Version GitHub issues Mentioned in Awesome CDK

examples/hello_orkestra.py

list: logger.info(item.dict()) return [item.dict()] * 10 @compose(model=Item, is_map_job=True, **default_args) def double_price(item: Item, context): item.price = item.price * 2 return item.dict() @compose(**default_args) def assert_false(event, context): assert False @compose(**default_args) def do_nothing(event, context): logger.info({"doing": "nothing"}) @compose(**default_args) def say_hello(event, context): return "hello, world" @compose(**default_args) def say_goodbye(event, context): return "goodbye" @compose(**default_args) def random_int(event, context): return random.randrange(100) @compose(**default_args) def random_float(event, context): return float(random_int(event, context)) dag() ">
import random
from typing import *
from uuid import uuid4

from aws_lambda_powertools import Logger, Tracer
from pydantic import BaseModel

from orkestra import compose
from orkestra.interfaces import Duration


def dag():
    (
        generate_item
        >> add_price
        >> copy_item
        >> double_price
        >> (do_nothing, assert_false)
        >> say_hello
        >> [random_int, random_float]
        >> say_goodbye
    )


class Item(BaseModel):
    id: str
    name: str
    price: Optional[float] = None

    @classmethod
    def random(cls):
        return cls(
            id=str(uuid4()),
            name=random.choice(
                [
                    "potato",
                    "moon rock",
                    "hat",
                ]
            ),
        )


logger = Logger()

tracer = Tracer()


default_args = dict(
    enable_powertools=True,
    timeout=Duration.seconds(6),
)


@compose(**default_args)
def generate_item(event, context):
    logger.info("generating random item")
    item = Item.random().dict()
    logger.info(item)
    tracer.put_metadata("GenerateItem", "SUCCESS")
    return item


@compose(model=Item, **default_args)
def add_price(item: Item, context):
    price = 3.14
    logger.info(
        "adding price to item",
        extra={
            "item": item.dict(),
            "price": price,
        },
    )
    item.price = price
    return item.dict()


@compose(model=Item, **default_args)
def copy_item(item: Item, context) -> list:
    logger.info(item.dict())
    return [item.dict()] * 10


@compose(model=Item, is_map_job=True, **default_args)
def double_price(item: Item, context):
    item.price = item.price * 2
    return item.dict()


@compose(**default_args)
def assert_false(event, context):
    assert False


@compose(**default_args)
def do_nothing(event, context):
    logger.info({"doing": "nothing"})


@compose(**default_args)
def say_hello(event, context):
    return "hello, world"


@compose(**default_args)
def say_goodbye(event, context):
    return "goodbye"


@compose(**default_args)
def random_int(event, context):
    return random.randrange(100)


@compose(**default_args)
def random_float(event, context):
    return float(random_int(event, context))


dag()

app.py

#!/usr/bin/env python3
from aws_cdk import core as cdk

from examples.hello_orkestra import generate_item


class HelloOrkestra(cdk.Stack):
    def __init__(self, scope, id, **kwargs):

        super().__init__(scope, id, **kwargs)

        generate_item.schedule(
            self,
            expression="rate(5 minutes)",
            state_machine_name="hello_orkestra",
        )


app = cdk.App()


app.synth()

state machine

Owner
Stephan Fitzpatrick
“Talk is cheap. Show me the code.”
Stephan Fitzpatrick
A simple url uploader bot with permenent thumbnail support

URL-Uploader A simple url uploader bot with permenent thumbnail support Scrapped some code from @SpEcHIDe's AnyDLBot Repository Please fork this repos

Fayas Noushad 40 Nov 29, 2021
A Telegram Bin Checker Bot made with python for check Bin valid or Invalid. 💳

Bin Checker Bot A Telegram Bin Checker Bot made with python for check Bin valid or Invalid. 📌 Deploy On Heroku 🏷 Environment Variables API_ID - Your

Chamindu Denuwan 20 Dec 10, 2022
Tools to download and aggregate feeds of vaccination clinic location information in the United States.

vaccine-feed-ingest Pipeline for ingesting nationwide feeds of vaccine facilities. Contributing How to Configure your environment (instructions on the

Call the Shots 26 Aug 05, 2022
An API that allows you to get full information about TikTok videos

TikTok-API An API that allows you to get full information about TikTok videos without using any third party sources and only the TikTok API. ##API onl

FC 13 Dec 20, 2021
Sakura: an powerfull Autofilter bot that can be used in your groups

Sakura AutoFilter This Bot May Look Like Mwk_AutofilterBot And Its Because I Like Its UI, That's All Sakura is an powerfull Autofilter bot that can be

PaulWalker 12 Oct 21, 2022
PepeSniper is an open-source Discord Nitro auto claimer/redeemer made in python.

PepeSniper is an open-source Discord Nitro auto claimer made in python. It sure as hell is not the fastest sniper out there but it gets the job done in a timely and stable manner. It also supports ho

Unknown User 1 Dec 22, 2021
Download videos from Youtube and other platforms through a Telegram Bot

ytdl-bot Download videos from YouTube and other platforms through a Telegram Bot Usage: https://t.me/benny_ytdlbot Send link from YouTube directly to

Telegram Bot Collection 289 Jan 03, 2023
Documentation and Samples for the Official HN API

Hacker News API Overview In partnership with Firebase, we're making the public Hacker News data available in near real time. Firebase enables easy acc

Y Combinator Hacker News 9.6k Jan 03, 2023
A simple and easy to use musicbot in python and it uses lavalink.

Lavalink-MusicBot A simple and easy to use musicbot in python and it uses lavalink. ✨ Features plays music in your discord server well thats it i gues

Afnan 1 Nov 29, 2021
Asynchronous and also synchronous non-official QvaPay client for asyncio and Python language.

Asynchronous and also synchronous non-official QvaPay client for asyncio and Python language. This library is still under development, the interface could be changed.

Leynier Gutiérrez González 8 Sep 18, 2021
MusicBot is the original Discord music bot written for Python 3.5+, using the discord.py library

The original MusicBot for Discord (formerly SexualRhinoceros/MusicBot)

Just Some Bots 2.9k Jan 02, 2023
Python client for Toyota North America service API

toyota-na Python client for Toyota North America service API Install pip install toyota-na[qt] [qt] is required for generating authorization code. Us

Gavin Ni 18 Sep 06, 2022
A simple library for interacting with Amazon S3.

BucketStore is a very simple Amazon S3 client, written in Python. It aims to be much more straight-forward to use than boto3, and specializes only in

Jacobi Petrucciani 219 Oct 03, 2022
Automatically Forward files from groups to channel & FSub

Backup & ForceSub Automatically Forward files from groups to channel & Do force sub on members Variables API_ID : Get from my.telegram.org API_HASH :

Arunkumar Shibu 7 Nov 06, 2022
A powerfull Telegram Leech Bot

owner of this repo :- Abijthkutty contact me :- Abijth Telegram Torrent and Direct links Leecher Dont Abuse The Repo ... this is intented to run in Sm

αвιנтн 9 Jun 11, 2022
NFTs Upload to OpenSea CuseEdition

NFTs-Upload-to-OpenSea-CuseEdition YOUTUBE VIDEO - Soon... Download Python and

Lil Cuse 2 Jan 04, 2022
Twitter bot to know the number of dislikes of a YouTube video

YT_dislikes is a twitter bot that allows you to know the number of dislikes (and likes) of a YouTube video. Now it is not possible to see the number o

1 Jan 08, 2022
Droplink URL Shortener Bot, deployable to Heroku and Railway.

Droplink-bot Make short link by using Droplink API key. Made by @dakshy. Installation The Easy Way Required Variables BOT_TOKEN: Create a bot using @B

ToonsHub 5 Jun 25, 2022
Pythonic event-processing library based on decorators

Process Events In Style This library aims to simplify the common pattern of event processing. It simplifies the process of filtering, dispatching and

Nicolas Marier 3 Sep 01, 2022
NiceHash Python Library and Command Line Rest API

NiceHash Python Library and Command Line Rest API Requirements / Modules pip install requests Required data and where to get it Following data is nee

Ashlin Darius Govindasamy 2 Jan 02, 2022