Python lib for Embedly

Overview

embedly-python

Python library for interacting with Embedly's API. To get started sign up for a key at embed.ly/signup.

Install

Install with Pip (recommended):

pip install embedly

Or easy_install:

easy_install Embedly

Or setuptools:

git clone git://github.com/embedly/embedly-python.git
python setup.py

Setup requires Setuptools 0.7+ or Distribute 0.6.2+ in order to take advantage of the 2to3 option. Setup will still run on earlier versions but you'll see a warning and 2to3 won't happen. Read more in the Setuptools docs

Getting Started

This library is meant to be a dead simple way to interact with the Embedly API. There are only 2 main objects, the Embedly client and the Url response model. Here is a simple example and then we will go into the objects:

>>> from embedly import Embedly
>>> client = Embedly(:key)
>>> obj = client.oembed('http://instagram.com/p/BL7ti/')
>>> obj['type']
u'photo'
>>> obj['url']
u'http://images.ak.instagram.com/media/2011/01/24/cdd759a319184cb79793506607ff5746_7.jpg'

>>> obj = client.oembed('http://instagram.com/error/error/')
>>> obj['error']
True

Embedly Client

The Embedly client is a object that takes in a key and optional User Agent and timeout parameters then handles all the interactions and HTTP requests to Embedly. To initialize the object, you'll need the key that you got when you signed up for Embedly.

>>> from embedly import Embedly
>>> client = Embedly('key')
>>> client2 = Embedly('key', 'Mozilla/5.0 (compatible; example-org;)')
>>> client3 = Embedly('key', 'Mozilla/5.0 (compatible; example-org;)', 30)
>>> client4 = Embedly('key', timeout=10, user_agent='Mozilla/5.0 (compatible; example-org;)')

The client object now has a bunch of different methods that you can use.

oembed

Corresponds to the oEmbed endpoint. Passes back an object that allows you to retrieve a title, thumbnail, description and the embed html:

>>> client.oembed('http://vimeo.com/18150336')
<embedly.models.Url at 0x10223d950>
extract

Corresponds to the Extract endpoint. Passes back an object that allows you to retrieve a title, description, content, html and a list of images.:

>>> client.extract('http://vimeo.com/18150336')
<embedly.models.Url at 0x10223d950>
preview

Preview is no longer available to new users and has been replaced by extract.

Corresponds to the Preview endpoint. Passes back a simple object that allows you to retrieve a title, description, content, html and a list of images.:

>>> client.preview('http://vimeo.com/18150336')
<embedly.models.Url at 0x10223d950>
objectify

Objectify is no longer available to new users and has been replaced by extract.

Corresponds to the Objectify endpoint. Passes back a simple object that allows you to retrieve pretty much everything that Embedly knows about a URL.:

>>> client.objectify('http://vimeo.com/18150336')
<embedly.models.Url at 0x10223d950>

The above functions all take the same arguements, a URL or a list of URLs and keyword arguments that correspond to Embedly's query arguments. Here is an example:

>>> client.oembed(['http://vimeo.com/18150336',
  'http://www.youtube.com/watch?v=hD7ydlyhvKs'], maxwidth=500, words=20)

There are some supporting functions that allow you to limit URLs before sending them to Embedly. Embedly can return metadata for any URL, these just allow a developer to only pass a subset of Embedly providers. Note that URL shorteners like bit.ly or t.co are not supported through these regexes.

regex

If you would like to only send URLs that returns embed HTML via Embedly you can match the URL to the regex before making the call. The matching providers are listed at embed.ly/providers:

>>> url = 'http://vimeo.com/18150336'
>>> client.regex.match(url)

<_sre.SRE_Match at 0x1017ba718>

is_supported

This is a simplified version of regex:

>>> url = 'http://vimeo.com/18150336'
>>> client.is_supported(url)

True

Url Object

The Url object is basically a response dictionary returned from one of the Embedly API endpoints.

>>> response = client.oembed('http://vimeo.com/18150336', words=10)

Depending on the method you are using, the response will have different attributes. We will go through a few, but you should read the documentation to get the full list of data that is passed back.

>>> response['type']
u'video'
>>> response['title']
u'Wingsuit Basejumping - The Need 4 Speed: The Art of Flight'
>>> response['provider_name']
u'Vimeo'
>>> response['width']
1280

As you can see the Url object works like a dictionary, but it's slightly enhanced. It will always have method and original_url attributes, which represent the Embedly request type and the URL requested.

>>> response.method
'oembed'
>>> response.original_url
'http://vimeo.com/18150336'

# useful because the response data itself may not have a URL
# (or it could have a redirected link, querystring params, etc)
>>> response['url']
...
KeyError: 'url'

For the Preview and Objectify endpoints the sub-objects can also be accessed in the same manner.

>>> obj = client.preview('http://vimeo.com/18150336', words=10)
>>> obj['object']['type']
u'video'
>>> obj['images'][0]['url']
u'http://b.vimeocdn.com/ts/117/311/117311910_1280.jpg'

Error Handling

If there was an error processing the request, the Url object will contain an error. For example if we use an invalid key, we will get a 401 response back

>>> client = Embedly('notakey')
>>> obj = client.preview('http://vimeo.com/18150336')
>>> obj['error']
True
>>> obj['error_code']
401

Copyright

Copyright (c) 2013 Embed.ly, Inc. See LICENSE for details.

Comments
  • Additional serialization tests

    Additional serialization tests

    Not sure if there are more edge cases I missed, but this change got the existing failing test to pass. It also fixes the problem I'm facing of serialization with cPickle.

    I also added some cases to the old failing test just to make sure serializing lots of data types worked.

    opened by se3000 8
  • v0.5.0 prep - READY FOR MERGE

    v0.5.0 prep - READY FOR MERGE

    Updates and fixes necessary for an official release.

    A previous PR (one of mine) broke testing in Python 2.6 and the Readme needed a refresh. This also officially drops Python 3.1 support.

    Other new things

    • setup.py test is available (and used by TravisCI)
    • setup.py uses 2to3 conversion during the build process for Python 3
    • test coverage at 100% (for all tested versions of Python)
    • mock is used in tests
    • minor fixes and improvements
    opened by joncotton 7
  • 0.4.5 release on pypi would be awesome

    0.4.5 release on pypi would be awesome

    The good work of @joncotton and @dokipen is hard to use until pypi gets updated. In the mean while folks can add -e git+git://github.com/embedly/embedly-python.git#egg=embedly to their requirements files or run it as a argument to pip install to install the current master.

    opened by Jbonnett 6
  • is_supported should follow rediects

    is_supported should follow rediects

    I just tried to embed a couple of bit.ly links on your site, and they worked fine, resolving the redirects and returning an embed code for the resulting url. I was confused when is_supported didn't work that way, and instead returned that the link was not supported. So this is my feature request, make is_supported support redirects.

    opened by EmilStenstrom 5
  • Small feature: Service Matcher on Embed.ly Client

    Small feature: Service Matcher on Embed.ly Client

    Added a service matcher to the python client - allows you to determine if a URL is serviced by embed.ly or not. I think this is a similar approach to what reddit uses, except theirs is hardcoded.

    In my own implementation I've also added a caching layer to get_services, which I think will be pretty important for speed, but isn't very implementation-agnostic just yet.

    Not sure if this uses the same sort of internal lingo you guys might use at embed.ly - feel free to edit.

    opened by umbrae 3
  • Error running example code.

    Error running example code.

    I have been using the API for about a month now with good results, but today everything I run returns a 401. Is the API down?

    from embedly import Embedly
    import settings
    client = Embedly('my-key')
    obj = client.extract('http://www.propublica.org/article/how-nonprofit-hospitals-are-seizing-patients-wages')
    print obj['type']
    >>> 'error'
    
    opened by abelsonlive 1
  • Fix version import

    Fix version import

    setup.py now accesses the version information without importing the module, which caused problems. __init__.py is the canonical location now, which is better I think. client.py avoids circular imports by only importing the version when it needs it.

    Follow up to changes in 563de9def0930ecc3a7ab19bf4a67e3e78a50180

    opened by joncotton 1
  • Install fails if you don't already have httplib2 installed

    Install fails if you don't already have httplib2 installed

    Install fails with the message below. Shouldn't it install the dependency?

    Obtaining embedly from git+https://github.com/embedly/[email protected]#egg=embedly-dev (from -r requirements.txt (line 2)) Updating ./venv/src/embedly clone (to 40be69dbfea07836f3e6e1a899ef7eab6c00cca4) Could not find a tag or branch '40be69dbfea07836f3e6e1a899ef7eab6c00cca4', assuming commit. Running setup.py egg_info for package embedly Traceback (most recent call last): File "", line 16, in File "/home/ubuntu/muckrack/venv/src/embedly/setup.py", line 13, in version = import('embedly').version File "embedly/init.py", line 2, in from .client import Embedly, version File "embedly/client.py", line 9, in import httplib2 ImportError: No module named httplib2 Complete output from command python setup.py egg_info: Traceback (most recent call last):

    File "", line 16, in

    File "/home/ubuntu/muckrack/venv/src/embedly/setup.py", line 13, in

    version = __import__('embedly').__version__
    

    File "embedly/init.py", line 2, in

    from .client import Embedly, __version__
    

    File "embedly/client.py", line 9, in

    import httplib2
    

    ImportError: No module named httplib2

    opened by lsemel 1
  • Support setting timeout of connection

    Support setting timeout of connection

    As embedly can be quite slow at times, we need to be able to set a timeout on the requests. This is a simple change to support that. If you want you can change the default to None, but I think 60 seconds is a better default than no timeout.

    opened by holm 1
  • a suggestion more than a pull request

    a suggestion more than a pull request

    not sure you'd be open to this, so I didn't put much time into it...

    I wanted to be able to cache the embedly 'services' data:

    • I don't want to make a blocking request on every appserver initialization or object instance
    • I don't want a user-generated action to require multiple API calls

    in order to handle this, the simplest way i thought of was to:

    • split get_services into -- get_services pulls the API data -- set_services actually does the setting & regex compiling
    • add a cache_services_data argument to initialization -- if set to True -- or previously set during a get_ call -- will store the raw response data text

    with these changes you can pull out the API info from client.cache_services_data and cache it, then create new objects with the same data by calling client2.set_services()

    opened by jvanasco 1
  • Embedly + rod.recipe.appengine

    Embedly + rod.recipe.appengine

    I was trying to integrate the Embedly egg into our App Engine project but kept getting an error. I setup a minimal project that reproduces what I'm seeing here.

    I'm definitely no Python pro so I don't know if it's a problem with my poor buildout skills, rod.recipe.appengine or the embedly package. Any chance you could sanity check?

    Here's the stack trace:

    An internal error occurred due to a bug in either zc.buildout or in a
    recipe being used:
    Traceback (most recent call last):
      File "/home/ubuntu/jcook793-embedly-appengine/eggs/zc.buildout-1.5.2-py2.7.egg/zc/buildout/buildout.py", line 1805, in main
        getattr(buildout, command)(args)
      File "/home/ubuntu/jcook793-embedly-appengine/eggs/zc.buildout-1.5.2-py2.7.egg/zc/buildout/buildout.py", line 584, in install
        installed_files = self[part]._call(recipe.install)
      File "/home/ubuntu/jcook793-embedly-appengine/eggs/zc.buildout-1.5.2-py2.7.egg/zc/buildout/buildout.py", line 1297, in _call
        return f()
      File "/home/ubuntu/jcook793-embedly-appengine/eggs/rod.recipe.appengine-2.0.0-py2.7.egg/rod/recipe/appengine/__init__.py", line 327, in install
        self.copy_packages(ws, temp_dir)
      File "/home/ubuntu/jcook793-embedly-appengine/eggs/rod.recipe.appengine-2.0.0-py2.7.egg/rod/recipe/appengine/__init__.py", line 250, in copy_packages
        top = open(top_level, 'r')
    IOError: [Errno 20] Not a directory: '/home/ubuntu/jcook793-embedly-appengine/eggs/Embedly-0.4.3-py2.7.egg/EGG-INFO/top_level.txt'
    
    opened by jcook793 1
  • Replacement library: embedly2

    Replacement library: embedly2

    Unfortunately this library appears to have been abandoned. I just released embedly2 0.6.0 (https://pypi.org/project/embedly2/), a fork of this library that drops Python 2 support and fixes #31. Feel free to use it and report bugs if you need to use embedly with a recent toolchain.

    If the original maintainers return to this project, I'm happy to archive my fork in favor of this package. Also, if anyone would like to contribute to improving embedly2, please let me know.

    opened by JelleZijlstra 0
  • Problems installing library with setuptools > 58.0.0

    Problems installing library with setuptools > 58.0.0

    I was installing your package recently and it came to my attention that installation fails.

    A simple test to replicate:

    Adams-MBP:programming claim$ mkdir testing_embedly_installation
    Adams-MBP:programming claim$ cd testing_embedly_installation/
    Adams-MBP:testing_embedly_installation claim$ python3 -m venv venv
    Adams-MBP:testing_embedly_installation claim$ source venv/bin/activate
    (venv) Adams-MBP:testing_embedly_installation claim$ pip install -U setuptools
    Collecting setuptools
      Using cached https://files.pythonhosted.org/packages/11/b9/adac241e2c4aca7ae4ddd86d3c18227667665b6e7eac550695bfc50c7e3d/setuptools-60.6.0-py3-none-any.whl
    Installing collected packages: setuptools
      Found existing installation: setuptools 41.2.0
        Uninstalling setuptools-41.2.0:
          Successfully uninstalled setuptools-41.2.0
    Successfully installed setuptools-60.6.0
    (venv) Adams-MBP:testing_embedly_installation claim$ python
    Python 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34)
    [Clang 6.0 (clang-600.0.57)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import setuptools
    >>> setuptools.__version__
    '60.6.0'
    >>> ^D
    (venv) Adams-MBP:testing_embedly_installation claim$ pip install embedly
    Collecting embedly
      Using cached https://files.pythonhosted.org/packages/d7/94/2e387186617fe450fd21da3fd08c4ea1c68914c21622e939a892755620bc/Embedly-0.5.0.tar.gz
        ERROR: Command errored out with exit status 1:
         command: /Users/claim/programming/testing_embedly_installation/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/z9/p68f7pds4nz6lg602cxzhm040000gn/T/pip-install-5c2mwtoc/embedly/setup.py'"'"'; __file__='"'"'/private/var/folders/z9/p68f7pds4nz6lg602cxzhm040000gn/T/pip-install-5c2mwtoc/embedly/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
             cwd: /private/var/folders/z9/p68f7pds4nz6lg602cxzhm040000gn/T/pip-install-5c2mwtoc/embedly/
        Complete output (2 lines):
        Warning: 'classifiers' should be a list, got type 'tuple'
        error in Embedly setup command: use_2to3 is invalid.
        ----------------------------------------
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    

    there are 2 issues that need to be addressed here:

    1. classifiers are a tuple (should be a list)
    2. use_2to3 is no longer available in setuptools (since version 58.0.0 - more on that here: https://setuptools.pypa.io/en/latest/history.html#v58-0-0)

    For now, I have created a fork that (I think?) addresses the problems as I'm not sure if this lib is still supported (but as my changes remove support for python2 completely, it would be a bit naughty to create a PR to this branch from it :D).

    opened by ajakubo1 1
  • use https?

    use https?

    I noticed that the URLs are 'http' and not 'https'.

    I created this pull request:

    https://github.com/embedly/embedly-python/pull/28

    And added 'https' to the URLs I saw.

    Alex

    opened by alex-money 0
  • Tests fail

    Tests fail

    I suppose something has changed on the service…

    py27 runtests: commands[0] | python embedly/tests.py
    ...FF..FF..........
    ======================================================================
    FAIL: test_get_services_retrieves_data_and_builds_regex (__main__.EmbedlyTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "embedly/tests.py", line 233, in test_get_services_retrieves_data_and_builds_regex
        self.assertTrue(client.regex.match('http://yfrog.com/h22eu4j'))
    AssertionError: None is not true
    
    ======================================================================
    FAIL: test_multi_errors (__main__.EmbedlyTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "embedly/tests.py", line 164, in test_multi_errors
        self.assertEqual(objs[1]['type'], 'error')
    AssertionError: u'link' != u'error'
    - link
    + error
    
    
    ======================================================================
    FAIL: test_provider (__main__.EmbedlyTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "embedly/tests.py", line 117, in test_provider
        self.assertEqual(obj['provider_url'], 'http://www.scribd.com/')
    AssertionError: u'https://www.scribd.com/' != u'http://www.scribd.com/'
    - https://www.scribd.com/
    ?     -
    + http://www.scribd.com/
    
    
    ======================================================================
    FAIL: test_providers (__main__.EmbedlyTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "embedly/tests.py", line 134, in test_providers
        self.assertEqual(objs[0]['provider_url'], 'http://www.scribd.com/')
    AssertionError: u'https://www.scribd.com/' != u'http://www.scribd.com/'
    - https://www.scribd.com/
    ?     -
    + http://www.scribd.com/
    
    
    ----------------------------------------------------------------------
    Ran 19 tests in 4.021s
    
    FAILED (failures=4)
    
    opened by mgorny 0
  • Properly handle non-200 responses on multi-url requests

    Properly handle non-200 responses on multi-url requests

    Currently, multi-url requests throw a ValueError when constructing the return list if the api requests returns anything other than a 200 response. The error happens specifically in this code:

    return map(lambda url, data: Url(data, method, url),
                            url_or_urls, data)
    

    When the status code isn't 200 (and it's a multi-url request), the data value is assigned to a dictionary (as opposed the list/json array you'd expect from a 200 response). When passed to map(lambda url, data: Url(data, method, url), url_or_urls, data), map iterates through data and url_or_urls in parallel. Under successful 200 responses, data and url_or_urls are lists of equal length. In any other case, data is a dictionary, so map iterates through the keys of the dictionary. This causes exceptions because the data arg passed to the lambda func is now a string, not a dict.

    This fix works by simply copying the error data len(url_or_urls) times in a list, mimicking the list format expected in the downstream map function.

    opened by dougkeen 0
Releases(v0.5.0)
  • v0.5.0(Nov 23, 2013)

    Quite a few changes, mostly behind the scenes. However, backwards compatibility is broken for the Url class. It is now a dictionary, behaving like dictionary. It still has the method and original_url attributes but no longer has dot access to the response data.

    Why? Treating those Url dictionary keys as class attributes wasn't pythonic; it was a JavaScripty approach that just doesn't fit the expectation of a Python dict. Before you could make a request, get a response and access the data in two ways: response.title and response['title']. Now you can only use response['title']. The first way will throw an error. See #15 for more info.

    New--

    • setup.py test is available
    • setup.py uses 2to3 conversion during the build process for Python 3
    • test coverage at 100% (for all tested versions of Python)
    • response data can be serialized (e.g. data is received as JSON and can be turned back into JSON)

    Fixed--

    • str vs unicode in Python 3 for the Url class
    • HTTP connections are properly closed
    • properly decoding response data in Embedly.get_services()
    • cleaner version reporting

    Removed--

    • dot/attribute access for Url dictionary data
    • Python 3.1 support (which may not have worked anyway)
    Source code(tar.gz)
    Source code(zip)
Send OpenWeatherMap alerts (One Call API) to telegram users.

OpenWeatherMap Telegram Alert Send OpenWeatherMap alerts (One Call API) to telegram users. Installation Requirements: $ apt install python3-yaml pytho

Michael Hacker 1 Jun 04, 2022
Collection of script to manage WLED devices

Collection of script to manage WLED devices

Daniel Poelzleithner 4 Sep 26, 2022
Due to changes to the discord API and discord.py being discontinued

Talia Due to changes to the discord API and discord.py being discontinued, Talia development has been halted permanently A customizable economy discor

2 Mar 08, 2022
A telegram string extractor bot

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.com/FayasNoushad/String-Extract-Bot/blob/main/LIC

Fayas Noushad 12 Jul 19, 2022
Trading bot - A Trading bot With Python

Trading_bot Trading bot intended for 1) Tracking current prices of tokens 2) Set

Tymur Kotkov 29 Dec 01, 2022
Discord Token Generator based on HTTPX, makes unverified tokens and automatically joins your server! this is used for memberboosting

Discord Token Generator | 2021 Features: (1) hCaptcha Bypasser, latest hfuck.py Updated by me (2) Free Proxy Support/Scrapper (3) Custom Realistic Dat

2 Nov 30, 2021
Michelle is a Discord Bot coded in Python with Discord.py by Mudit07.

Michelle is a Discord Bot coded in Python with Discord.py by Mudit07.

Michelle 3 Oct 09, 2021
A listener for RF >= 4.0 that prints a Stack Trace to console to faster find the code section where the failure appears.

robotframework-stacktrace A listener for RF = 4.0 that prints a Stack Trace to console to faster find the code section where the failure appears. Ins

marketsquare 16 Nov 24, 2022
Бот для мини-игры "Рабы" ("Рабство") ВКонтакте.

vk-slaves-bot Бот для мини-игры "Рабы" ("Рабство") ВК Группа в ВК, в ней публикуются новости и другая полезная информация. У группы есть беседа, в кот

Almaz 80 Dec 17, 2022
A python client for the Software-Challenge Germany.

sc-client-python A python client for the Software-Challenge Germany. Creating a new project (Optional) Install virtualenv virtualenv is a tool that cr

rpkak 3 Jan 22, 2022
Python gets the friend's articles from hexo's friend-links

你是否经常烦恼于友链过多但没有时间浏览?那么友链朋友圈将解决这一痛点。你可以随时获取友链网站的更新内容,并了解友链的活跃情况。

129 Dec 28, 2022
A secure and customizable bot for controlling cross-server announcements and interactions within Discord

DiscordBot A secure and customizable bot for controlling cross-server announcements and interactions within Discord. Within the code of the bot, you c

Jacob Dorfmeister 1 Jan 22, 2022
Read manga from your favourites websites on telegram.

tg-manga-bot Read manga from your favourites websites on telegram. Current Development Bot @idkpythonbot Telegram Channel tg_manga_bot Commands start

Daniel Rivero 41 Dec 22, 2022
Anti Spam/NSFW Telegram Bot Written In Python With Pyrogram.

Anti Spam/NSFW Telegram Bot Written In Python With Pyrogram.

Wahyusaputra 2 Dec 29, 2021
Url-shortener - A url shortener made in python using the API's from the pyshorteners lib

URL Shortener Um encurtador de link feito em python usando as API's da lib pysho

Spyware 3 Jan 07, 2022
A Python wrapper around the OpenWeatherMap web API

PyOWM A Python wrapper around OpenWeatherMap web APIs What is it? PyOWM is a client Python wrapper library for OpenWeatherMap (OWM) web APIs. It allow

Claudio Sparpaglione 740 Dec 18, 2022
Build a better understanding of your data in PostgreSQL.

Data Fluent for PostgreSQL Build a better understanding of your data in PostgreSQL. The following shows an example report generated by this tool. It g

Mark Litwintschik 28 Aug 30, 2022
Python library to download market data via Bloomberg, Eikon, Quandl, Yahoo etc.

findatapy findatapy creates an easy to use Python API to download market data from many sources including Quandl, Bloomberg, Yahoo, Google etc. using

Cuemacro 1.3k Jan 04, 2023
Checks instagram names to see if they're available

How to install You must have python 3.7.6 installed and make sure you click the 'ADD TO PATH' option when installing Open cmd and type pip install aio

2 Oct 20, 2021
An information scroller Twitter trends, news, weather for raspberry pi and Pimoroni Unicorn Hat Mini and Scroll Phat HD.

uticker An information scroller Twitter trends, news, weather for raspberry pi and Pimoroni Unicorn Hat Mini and Scroll Phat HD. Features include: Twi

kottuora 5 Oct 31, 2022