ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged

Chegg-Answer-Bot You can share your Chegg account for answers using this bot with your friends without getting your account blocked/flagged Reuirement

Ammey Saini 27 Dec 24, 2022
ShadowMusic - A Telegram Music Bot with proper functions written in Python with Pyrogram and Py-Tgcalls.

⭐️ Shadow Music ⭐️ A Telegram Music Bot written in Python using Pyrogram and Py-Tgcalls Ready to use method A Support Group, Updates Channel and ready

TeamShadow 8 Aug 17, 2022
Programa capaz de gerar QR Code a partir do link inserido.

QrCodePy Programa capaz de gerar QR Code, a partir do link inserido, em forma de imagem e salvar localmente. Exemplo de saída: Requisitos Pure Python

Jonas Carvalho 4 Sep 09, 2021
Hostapd-mac-monitor - Setup a hostapd AP to conntrol the connections of specific MACs

A brief explanation This script provides way to setup a monitoring service of sp

2 Feb 03, 2022
Deepl - DeepL Free API For Python

DeepL DeepL Free API Notice Since I don't want to make my AuthKey public, if you

Vincent Young 4 Apr 11, 2022
A simple bot which using an API , detects reported discord scams and kicks the user if possible while deleting the message

A simple bot which using an API , detects reported discord scams and kicks the user if possible while deleting the message

Vioshim 3 Nov 16, 2022
Código python para automatizar a junção de arquivos CSV's e salva-los em uma pasta final de destino.

merge_csv Código python para automatizar a junção de arquivos CSV's e salva-los em uma pasta final de destino. Esse projeto é usado pra unir alguns ar

Welder Fariles 1 Jan 12, 2022
Info & tools for reverse engineering the M6 smart fitness band

m6-reveng This repo contains information and tools for reverse engineering the $7 M6 smart fitness band. Hardware The SoC (system-on-a-chip) is a Teli

41 Dec 26, 2022
This is a repository for the Duke University Cloud Computing course project on Serveless Data Engineering Pipeline. For this project, I recreated the below pipeline.

AWS Data Engineering Pipeline This is a repository for the Duke University Cloud Computing course project on Serverless Data Engineering Pipeline. For

15 Jul 28, 2021
Easily report Instagram pages and close the page

Program Features - 📌 Delete target post on Instagram. - 📌 Delete Media Target post on Instagram - 📌 Complete deletion of the target account on Inst

hack4lx 11 Nov 25, 2022
A Really Simple and Advanced Google Colab NoteBook to Setup and start using Rclone on Google Colab easily.

Rclone on Google Colab (Advanced!) 🔥 1.Setup and Start using Rclone on Google Colab and Create/Edit/View and delete your Rclone config file and keep

Dr.Caduceus 14 May 24, 2022
YouTube playlist Files downloaded by FDM are not organized according to the original order on YouTube

Youtube-Playlist-File-Organizer YouTube playlist Files downloaded by Free Download Manager are not organized according to the original order on YouTub

David Mainoo 3 Dec 27, 2021
Azure free vpn for students only! (Self hosted/No sketchy services/Fast and free)

Azpn-Azure-Free-VPN Azure free vpn for students only! (Self hosted/No sketchy services/Fast and free) This is an alternative secure way of accessing f

Harishankar Kumar 6 Mar 19, 2022
A bot written in Python to automate attending classes on MyClass (Codetantra).

codetantrabot This is python program to attend class on myclass(codetantra) Prerequisites You should have Python3 and Pip installed on your system Run

Aniket Kumar 1 Feb 08, 2022
Rhythm bot clone for discord written in Python and uses YouTube to get media files.

Tunebot About Rhythm bot clone for discord written in Python and uses YouTube to get media files. Usage You need a .env file within the same directory

1 Oct 21, 2021
Dynamic Twitter banner, to show off your spotify status. Banner updated every 5 minutes.

Spotify Twitter Banner Dynamic Twitter banner, to show off your spotify status. Banner updated every 5 minutes. Installation and Usage Install the dep

Sunrit Jana 23 Jan 05, 2023
✨ 🐍 Python SDK for StarkNet.

✨ 🐍 starknet.py StarkNet SDK for Python 📘 Documentation Installation Quickstart Guide API Installation To install this package run pip install stark

Software Mansion 158 Jan 04, 2023
Python script to Funge NFTs.

Python script to Funge NFTs. It scrapes OpenSea for a given list of NFT collections and downloads a certain number of NFTs from each collection or the entire collections.

3 Apr 28, 2022
The Discord bot framework for Python

Pycordia ⚠️ Note! As of now, this package is under early development so functionalities are bound to change drastically. We don't recommend you curren

Ángel Carias 24 Jan 01, 2023
Telegram PHub Bot using ARQ Api and Pyrogram. This Bot can Download and Send PHub HQ videos in Telegram using ARQ API.

Tg_PHub_Bot Telegram PHub Bot using ARQ Api and Pyrogram. This Bot can Download and Send PHub HQ videos in Telegram using ARQ API. OS Support All linu

TheProgrammerCat 13 Oct 21, 2022