utils Package¶

`bittorrent` Module¶

Torrenting utils, mostly for handling bencoding and torrent files.

class flexget.utils.bittorrent.Torrent(content)¶

Bases: object

Represents a torrent

KEY_TYPE¶: alias of str

add_multitracker(tracker)¶: Appends multi-tracker to this torrent

comment¶

encode()¶

classmethod from_file(filename)¶: Create torrent from file on disk.

get_filelist()¶: Return array containing fileinfo dictionaries (name, length, path)

info_hash¶: Return Torrent info hash

private¶

remove_multitracker(tracker)¶: Removes passed multi-tracker from this torrent

size¶: Return total size of the torrent

trackers¶

Returns:	List of trackers, supports single-tracker and multi-tracker implementations

flexget.utils.bittorrent.bdecode(text)¶

flexget.utils.bittorrent.bencode(data)¶

flexget.utils.bittorrent.clean_meta(meta, including_info=False, logger=None)¶

Clean meta dict. Optionally log changes using the given logger.

@param logger: If given, a callable accepting a string message. @return: Set of keys removed from C{meta}.

flexget.utils.bittorrent.decode_item(next, token)¶

flexget.utils.bittorrent.encode_dictionary(data)¶

flexget.utils.bittorrent.encode_integer(data)¶

flexget.utils.bittorrent.encode_list(data)¶

flexget.utils.bittorrent.encode_string(data)¶

flexget.utils.bittorrent.encode_unicode(data)¶

flexget.utils.bittorrent.is_torrent_file(metafilepath)¶

Check whether a file looks like a metafile by peeking into its content.

Note that this doesn’t ensure that the file is a complete and valid torrent, it just allows fast filtering of candidate files.

@param metafilepath: Path to the file to check, must have read permissions for it. @return: True if there is a high probability this is a metafile.

flexget.utils.bittorrent.tokenize(text, match=<built-in method match of _sre.SRE_Pattern object>)¶

`cached_input` Module¶

class flexget.utils.cached_input.InputCache(**kwargs)¶

Bases: sqlalchemy.ext.declarative.api.Base

added¶

entries¶

hash¶

id¶

name¶

class flexget.utils.cached_input.InputCacheEntry(**kwargs)¶

Bases: sqlalchemy.ext.declarative.api.Base

cache_id¶

entry¶

id¶

class flexget.utils.cached_input.cached(name, persist=None)¶

Bases: object

Implements transparent caching decorator @cached for inputs.

Decorator has two parameters:

name in which the configuration is present in tasks configuration.
key in which the configuration has the cached resource identifier (ie. url). If the key is not given or present in the configuration :name: is expected to be a cache name (ie. url)

Note

Configuration assumptions may make this unusable in some (future) inputs

cache = TimedDict({})¶

flexget.utils.cached_input.config_hash(config)¶

Parameters:	config (dict) – Configuration
Returns:	MD5 hash for config

flexget.utils.cached_input.db_cleanup(manager, session)¶: Removes old input caches from plugins that are no longer configured.

`database` Module¶

class flexget.utils.database.CaseInsensitiveWord(word)¶

Bases: sqlalchemy.ext.hybrid.Comparator

Hybrid value representing a string that compares case insensitively.

lower()¶

operate(op, other)¶

flexget.utils.database.ignore_case_property(text_attr)¶

flexget.utils.database.pipe_list_synonym(name)¶: Converts pipe separated text into a list

flexget.utils.database.quality_property(text_attr)¶

flexget.utils.database.quality_requirement_property(text_attr)¶

flexget.utils.database.safe_pickle_synonym(name)¶

Used to store Entry instances into a PickleType column in the database.

In order to ensure everything can be loaded after code changes, makes sure no custom python classes are pickled.

flexget.utils.database.text_date_synonym(name)¶: Converts Y-M-D date strings into datetime objects

flexget.utils.database.with_session(*args, **kwargs)¶

” A decorator which creates a new session if one was not passed via keyword argument to the function.

Automatically commits and closes the session if one was created, caller is responsible for commit if passed in.

If arguments are given when used as a decorator, they will automatically be passed to the created Session when one is not supplied.

flexget.utils.database.year_property(date_attr)¶

`imdb` Module¶

class flexget.utils.imdb.ImdbParser¶

Bases: object

Quick-hack to parse relevant imdb details

parse(imdb_id)¶

class flexget.utils.imdb.ImdbSearch¶

Bases: object

best_match(name, year=None)¶: Return single movie that best matches name criteria or None

ireplace(text, old, new, count=0)¶: Case insensitive string replace

search(name)¶: Return array of movie details (dict)

smart_match(raw_name)¶: Accepts messy name, cleans it and uses information available to make smartest and best match

flexget.utils.imdb.extract_id(url)¶: Return IMDb ID of the given URL. Return None if not valid or if URL is not a string.

flexget.utils.imdb.is_imdb_url(url)¶: Tests the url to see if it’s for imdb.com.

flexget.utils.imdb.make_url(imdb_id)¶: Return IMDb URL of the given ID

`log` Module¶

Logging utilities

class flexget.utils.log.LogMessage(md5sum)¶

Bases: sqlalchemy.ext.declarative.api.Base

Declarative

added¶

id¶

md5sum¶

flexget.utils.log.purge(manager, session)¶: Purge old messages from database

`qualities` Module¶

class flexget.utils.qualities.Quality(text=u'')¶

Bases: object

Parses and stores the quality of an entry in the four component categories.

components¶

name¶

parse(text)¶

Parses a string to determine the quality in the four component categories.

Parameters:	text – The string to parse

class flexget.utils.qualities.QualityComponent(type, value, name, regexp=None, modifier=None, defaults=None)¶

Bases: object

matches(text)¶

Test if quality matches to text.

Parameters:	text (string) – data te be tested against
Returns:	tuple (matches, remaining text without quality data)

class flexget.utils.qualities.RequirementComponent(type)¶

Bases: object

Represents requirements for a given component type. Can evaluate whether a given QualityComponent meets those requirements.

add_requirement(text)¶

allows(comp, loose=False)¶

reset()¶

class flexget.utils.qualities.Requirements(req=u'')¶

Bases: object

Represents requirements for allowable qualities. Can determine whether a given Quality passes requirements.

allows(qual, loose=False)¶

Determine whether this set of requirements allows a given quality.

Parameters:	qual (Quality) – The quality to evaluate. loose (bool) – If True, only ! (not) requirements will be enforced.
Return type:	bool
Returns:	True if given quality passes all component requirements.

components¶

parse_requirements(text)¶

Parses a requirements string.

Parameters:	text – The string containing quality requirements.

flexget.utils.qualities.all_components()¶

flexget.utils.qualities.get(quality_name)¶: Returns a quality object based on canonical quality name.

`requests` Module¶

class flexget.utils.requests.Session(timeout=30, max_retries=1)¶

Bases: requests.sessions.Session

Subclass of requests Session class which defines some of our own defaults, records unresponsive sites, and raises errors by default.

add_cookiejar(cookiejar)¶

Merges cookies from cookiejar into cookiejar for this session.

Parameters:	cookiejar – CookieJar instance to add to the session.

request(method, url, *args, **kwargs)¶

Does a request, but raises Timeout immediately if site is known to timeout, and records sites that timeout. Also raises errors getting the content by default.

Parameters:	raise_status (bool) – If True, non-success status code responses will be raised as errors (True by default)

set_domain_delay(domain, delay)¶

Registers a minimum interval between requests to domain

Parameters:	domain – The domain to set the interval on delay – The amount of time between requests, can be a timedelta or string like ‘3 seconds’

flexget.utils.requests.get(url, **kwargs)¶

Sends a GET request. Returns Response object.

Parameters:	url – URL for the new `Request` object. kwargs – Optional arguments that `request` takes.

flexget.utils.requests.is_unresponsive(url)¶

Checks if host of given url has timed out within WAIT_TIME

Parameters:	url – The url to check
Returns:	True if the host has timed out within WAIT_TIME
Return type:	bool

flexget.utils.requests.post(url, data=None, **kwargs)¶

Sends a POST request. Returns Response object.

Parameters:	url – URL for the new `Request` object. data – (optional) Dictionary or bytes to send in the body of the `Request`. kwargs – Optional arguments that `request` takes.

flexget.utils.requests.request(method, url, **kwargs)¶

flexget.utils.requests.set_unresponsive(url)¶

Marks the host of a given url as unresponsive

Parameters:	url – The url that timed out

flexget.utils.requests.wait_for_domain(url, delay_dict)¶

`search` Module¶

Common tools used by plugins implementing search plugin api

flexget.utils.search.clean_symbols(text)¶: Replaces common symbols with spaces. Also normalize unicode strings in decomposed form.

flexget.utils.search.clean_title(title)¶: Removes common codec, sound keywords, and special characters info from titles to facilitate loose title comparison.

flexget.utils.search.normalize_unicode(text)¶

flexget.utils.search.torrent_availability(seeds, leeches)¶

Returns a rating based on seeds and leeches for a given torrent.

Parameters:	seeds – Number of seeds on the torrent leeches – Number of leeches on the torrent
Returns:	A numeric rating

`simple_persistence` Module¶

NOTE:

Avoid using this module on your own or in plugins, this was originally made for 0.9 -> 1.0 transition.

You can safely use task.simple_persistence and manager.persist, if we implement something better we can replace underlying mechanism in single point (and provide transparent switch).

class flexget.utils.simple_persistence.SimpleKeyValue(task, plugin, key, value)¶

Bases: sqlalchemy.ext.declarative.api.Base

added¶

id¶

key¶

plugin¶

task¶

value¶

class flexget.utils.simple_persistence.SimplePersistence(plugin=None)¶

Bases: _abcoll.MutableMapping

Store simple values that need to be persisted between FlexGet runs. Interface is like a dict.

This should only be used if a plugin needs to store a few values, otherwise it should create a full table in the database.

class_store = defaultdict(<function <lambda> at 0x7f7a0acc15f0>, {})¶

classmethod flush(task=None)¶: Flush all in memory key/values to database.

classmethod load(task=None)¶: Load all key/values from task into memory from database.

store¶

class flexget.utils.simple_persistence.SimpleTaskPersistence(task)¶

Bases: flexget.utils.simple_persistence.SimplePersistence

plugin¶

flexget.utils.simple_persistence.db_cleanup(manager, session)¶: Clean up values in the db from tasks which no longer exist.

flexget.utils.simple_persistence.flush_task(task)¶: Stores all in memory key/value pairs to database when a task has completed.

flexget.utils.simple_persistence.flush_taskless(manager)¶

flexget.utils.simple_persistence.load_task(task)¶: Loads all key/value pairs into memory before a task starts.

flexget.utils.simple_persistence.load_taskless(manager)¶: Loads all key/value pairs into memory which aren’t associated with a specific task.

`soup` Module¶

flexget.utils.soup.get_soup(obj, parser=u'html5lib')¶

`sqlalchemy_utils` Module¶

Miscellaneous SQLAlchemy helpers.

class flexget.utils.sqlalchemy_utils.ContextSession(bind=None, autoflush=True, expire_on_commit=True, _enable_transaction_accounting=True, autocommit=False, twophase=False, weak_identity_map=True, binds=None, extension=None, info=None, query_cls=<class 'sqlalchemy.orm.query.Query'>)¶

Bases: sqlalchemy.orm.session.Session

sqlalchemy.orm.Session which can be used as context manager

flexget.utils.sqlalchemy_utils.create_index(table_name, session, *column_names)¶

Creates an index on specified columns in table_name

Parameters:	table_name – Name of table to create the index on. session – Session object which should be used column_names – The names of the columns that should belong to this index.

flexget.utils.sqlalchemy_utils.drop_tables(names, session)¶: Takes a list of table names and drops them from the database if they exist.

flexget.utils.sqlalchemy_utils.get_index_by_name(table, name)¶

Find declaratively defined index from table by name

Parameters:	table – Table object name (string) – Name of the index to get
Returns:	Index object

flexget.utils.sqlalchemy_utils.table_add_column(table, name, col_type, session, default=None)¶

Adds a column to a table

Warning

Uses raw statements, probably needs to be changed in order to work on other databases besides SQLite

Parameters:	table (string) – Table to add column to (can be name or schema) name (string) – Name of new column to add col_type – The sqlalchemy column type to add session (Session) – SQLAlchemy Session to do the alteration default – Default value for the created column (optional)

flexget.utils.sqlalchemy_utils.table_columns(table, session)¶

Parameters:	table (string) – Name of table or table schema session (Session) – SQLAlchemy Session
Returns:	List of column names in the table or empty list

flexget.utils.sqlalchemy_utils.table_exists(name, session)¶

Use SQLAlchemy reflect to check table existences.

Parameters:	name (string) – Table name to check session (Session) – Session to use
Returns:	True if table exists, False otherwise
Return type:	bool

flexget.utils.sqlalchemy_utils.table_schema(name, session)¶

Returns:	Table schema using SQLAlchemy reflect as it currently exists in the db
Return type:	Table

`template` Module¶

class flexget.utils.template.FlexGetTemplate¶

Bases: jinja2.environment.Template

Adds lazy lookup support when rendering templates.

new_context(vars=None, shared=False, locals=None)¶

exception flexget.utils.template.RenderError¶

Bases: exceptions.Exception

Error raised when there is a problem with jinja rendering.

flexget.utils.template.filter_d(value, default_value=u'', boolean=False)¶

flexget.utils.template.filter_date_suffix(date)¶

flexget.utils.template.filter_default(value, default_value=u'', boolean=False)¶

flexget.utils.template.filter_format_number(val, places=None, grouping=True)¶: Formats a number according to the user’s locale.

flexget.utils.template.filter_formatdate(val, format)¶: Returns a string representation of a datetime object according to format string.

flexget.utils.template.filter_pad(val, width, fillchar=u'0')¶: Pads a number or string with fillchar to the specified width.

flexget.utils.template.filter_parsedate(val)¶: Attempts to parse a date according to the rules in RFC 2822

flexget.utils.template.filter_pathbase(val)¶: Base name of a path.

flexget.utils.template.filter_pathdir(val)¶: Directory containing the given path.

flexget.utils.template.filter_pathext(val)¶: Extension of a path (including the ‘.’).

flexget.utils.template.filter_pathname(val)¶: Base name of a path, without its extension.

flexget.utils.template.filter_pathscrub(val, os_mode=None)¶: Replace problematic characters in a path.

flexget.utils.template.filter_re_replace(val, pattern, repl)¶: Perform a regexp replacement on the given string.

flexget.utils.template.filter_re_search(val, pattern)¶: Perform a search for given regexp pattern, return the matching portion of the text.

flexget.utils.template.filter_to_date(date_time_val)¶

flexget.utils.template.get_template(templatename, pluginname=None)¶: Loads a template from disk. Looks in both included plugins and users custom plugin dir.

flexget.utils.template.make_environment(manager)¶: Create our environment and add our custom filters

flexget.utils.template.now()¶

flexget.utils.template.render(template, context)¶

Renders a Template with context as its context.

Parameters:	template – Template or template string to render. context – Context to render the template from.
Returns:	The rendered template text.

flexget.utils.template.render_from_entry(template_string, entry)¶: Renders a Template or template string with an Entry as its context.

flexget.utils.template.render_from_task(template, task)¶

Renders a Template with a task as its context.

Parameters:	template – Template or template string to render. task – Task to render the template from.
Returns:	The rendered template text.

`tools` Module¶

Contains miscellaneous helpers

class flexget.utils.tools.BufferQueue(maxsize=0)¶

Bases: Queue.Queue

Used in place of a file-like object to capture text and access it safely from another thread.

exception Empty¶

Bases: exceptions.Exception

Exception raised by Queue.get(block=0)/get_nowait().

BufferQueue.write(line)¶

exception flexget.utils.tools.MergeException(value)¶: Bases: exceptions.Exception

class flexget.utils.tools.ReList(*args, **kwargs)¶

Bases: list

A list that stores regexps.

You can add compiled or uncompiled regexps to the list. It will always return the compiled version. It will compile the text regexps on demand when first accessed.

flags = 34¶

class flexget.utils.tools.SmartRedirectHandler¶

Bases: urllib2.HTTPRedirectHandler

http_error_301(req, fp, code, msg, headers)¶

http_error_302(req, fp, code, msg, headers)¶

class flexget.utils.tools.TimedDict(cache_time=u'5 minutes')¶

Bases: _abcoll.MutableMapping

Acts like a normal dict, but keys will only remain in the dictionary for a specified time span.

flexget.utils.tools.arithmeticEval(s)¶

A safe eval supporting basic arithmetic operations.

Parameters:	s – expression to evaluate
Returns:	value

flexget.utils.tools.console(text)¶: Print to console safely.

flexget.utils.tools.convert_bytes(bytes)¶: Returns given bytes as prettified string.

flexget.utils.tools.decode_html(value)¶

Parameters:	value (string) – String to be html-decoded
Returns:	Html decoded string

flexget.utils.tools.encode_html(unicode_data, encoding=u'ascii')¶: Encode unicode_data for use as XML or HTML, with characters outside of the encoding converted to XML numeric character references.

flexget.utils.tools.merge_dict_from_to(d1, d2)¶: Merges dictionary d1 into dictionary d2. d1 will remain in original form.

flexget.utils.tools.multiply_timedelta(interval, number)¶: timedeltas can not normally be multiplied by floating points. This does that.

flexget.utils.tools.parse_timedelta(value)¶: Parse a string like ‘5 days’ into a timedelta object. Also allows timedeltas to pass through.

flexget.utils.tools.pid_exists(pid)¶: Check whether pid exists in the current process table.

flexget.utils.tools.singleton(cls)¶

flexget.utils.tools.str_to_boolean(string)¶

flexget.utils.tools.str_to_int(string)¶

flexget.utils.tools.strip_html(text)¶: Tries to strip all HTML tags from text. If unsuccessful returns original text.

flexget.utils.tools.urlopener(url_or_request, log, **kwargs)¶

Utility function for pulling back a url, with a retry of 3 times, increasing the timeout, etc. Re-raises any errors as URLError.

Warning

This is being replaced by requests library. flexget.utils.requests should be used going forward.

Parameters:	url_or_request (str) – URL or Request object to get. log – Logger to log debug info and errors to kwargs – Keyword arguments to be passed to urlopen
Returns:	The file-like object returned by urlopen

Subpackages¶

titles Package

utils Package¶

bittorrent Module¶

cached_input Module¶

database Module¶

imdb Module¶

log Module¶

qualities Module¶

requests Module¶

search Module¶

simple_persistence Module¶

soup Module¶

sqlalchemy_utils Module¶

template Module¶

tools Module¶

Subpackages¶

`bittorrent` Module¶

`cached_input` Module¶

`database` Module¶

`imdb` Module¶

`log` Module¶

`qualities` Module¶

`requests` Module¶

`search` Module¶

`simple_persistence` Module¶

`soup` Module¶

`sqlalchemy_utils` Module¶

`template` Module¶

`tools` Module¶