Navigation

Table Of Contents

Previous topic

Examples

Next topic

Changelog

This Page

Reference

class urlfetch.Response(r, **kwargs)[source]

A Response object.

>>> import urlfetch
>>> response = urlfetch.get("http://docs.python.org/")
>>> response.total_time
0.033042049407959
>>> response.status, response.reason, response.version
(200, 'OK', 10)
>>> type(response.body), len(response.body)
(<type 'str'>, 8719)
>>> type(response.text), len(response.text)
(<type 'unicode'>, 8719)
>>> response.getheader('server')
'Apache/2.2.16 (Debian)'
>>> response.getheaders()
[
    ('content-length', '8719'),
    ('x-cache', 'MISS from localhost'),
    ('accept-ranges', 'bytes'),
    ('vary', 'Accept-Encoding'),
    ('server', 'Apache/2.2.16 (Debian)'),
    ('last-modified', 'Tue, 26 Jun 2012 19:23:18 GMT'),
    ('connection', 'close'),
    ('etag', '"13cc5e4-220f-4c36507ded580"'),
    ('date', 'Wed, 27 Jun 2012 06:50:30 GMT'),
    ('content-type', 'text/html'),
    ('x-cache-lookup', 'MISS from localhost:8080')
]
>>> response.headers
{
    'content-length': '8719',
    'x-cache': 'MISS from localhost',
    'accept-ranges': 'bytes',
    'vary': 'Accept-Encoding',
    'server': 'Apache/2.2.16 (Debian)',
    'last-modified': 'Tue, 26 Jun 2012 19:23:18 GMT',
    'connection': 'close',
    'etag': '"13cc5e4-220f-4c36507ded580"',
    'date': 'Wed, 27 Jun 2012 06:50:30 GMT',
    'content-type': 'text/html',
    'x-cache-lookup': 'MISS from localhost:8080'
}
body[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

close()[source]

Close the connection

content[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

cookies[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

cookiestring[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

classmethod from_httplib(connection, **kwargs)[source]

Generate a Response object from a httplib response object.

headers[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

json[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

next()
raw_header[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

raw_response[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

read(chunk_size=8192)[source]

read content (for streaming and large files)

chunk_size: size of chunk, default: 8192

reason = None

Reason phrase returned by server.

status = None

Status code returned by server.

status_code = None

An alias of status.

text[source]

A property that is only computed once per instance and then replaces itself with an ordinary attribute. Deleting the attribute resets the property.

total_time = None

total time

version = None

HTTP protocol version used by server. 10 for HTTP/1.0, 11 for HTTP/1.1.

class urlfetch.Session(headers={}, cookies={}, auth=None)[source]

A session object.

urlfetch.Session can hold common headers and cookies. Every request issued by a urlfetch.Session object will bring u these headers and cookies.

urlfetch.Session plays a role in handling cookies, just like a cookiejar.

Parameters:
  • headers (dict, optional) – init headers
  • cookies (dict, optional) – init cookies
  • auth (tuple, optional) – (username, password) for basic authentication
cookies[source]
cookiestring[source]
delete(*args, **kwargs)[source]

Issue a delete request

fetch(*args, **kwargs)[source]

Fetch an URL

get(*args, **kwargs)[source]

Issue a get request

head(*args, **kwargs)[source]

Issue a head request

headers[source]
options(*args, **kwargs)[source]

Issue a options request

patch(*args, **kwargs)[source]

Issue a patch request

popcookie(key)[source]

Remove an cookie from default cookies

popheader(header)[source]

Remove an header from default headers

post(*args, **kwargs)[source]

Issue a post request

put(*args, **kwargs)[source]

Issue a put request

putcookie(key, value='')[source]

Add an cookie to default cookies

putheader(header, value)[source]

Add an header to default headers

request(*args, **kwargs)[source]

Issue a request

snapshot()[source]
trace(*args, **kwargs)[source]

Issue a trace request

urlfetch.request(url, method='GET', params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)[source]

request an URL

Parameters:
  • url – URL to be fetched.
  • method – (optional) HTTP method, one of GET, DELETE, HEAD, OPTIONS, PUT, POST, TRACE, PATCH. GET by default.
  • params – (optional) dict or string to attach to url as querystring.
  • headers – (optional) HTTP request headers in dict
  • timeout – (optional) timeout in seconds
  • files – (optional) files to be sended
  • randua – (optional) if True or path string, use a random user-agent in headers, instead of 'urlfetch/' + __version__
  • auth – (optional) (username, password) for basic authentication
  • length_limit – (optional) if None, no limits on content length, if the limit reached raised exception ‘Content length is more than ...’
  • proxies – (optional) HTTP proxy, like {‘http’: ‘127.0.0.1:8888’, ‘https’: ‘127.0.0.1:563’}
  • trust_env – (optional) If True, urlfetch will get infomations from env, such as HTTP_PROXY, HTTPS_PROXY
  • max_redirects – (integer, optional) Max redirects allowed within a request. Default is 0, which means redirects are not allowed.
Return type:

A Response object
urlfetch.fetch(*args, **kwargs)[source]

fetch an URL.

fetch() is a wrapper of request(). It calls get() by default. If one of parameter data or parameter files is supplied, post() is called.

urlfetch.get(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a GET request

urlfetch.post(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a POST request

urlfetch.head(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a HEAD request

urlfetch.put(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a PUT request

urlfetch.delete(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a DELETE request

urlfetch.options(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a OPTIONS request

urlfetch.trace(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a TRACE request

urlfetch.patch(url, params=None, data=None, headers={}, timeout=None, files={}, randua=False, auth=None, length_limit=None, proxies=None, trust_env=True, max_redirects=0, **kwargs)

Issue a PATCH request

helpers

urlfetch.decode_gzip(data)[source]

decode gzipped content

urlfetch.decode_deflate(data)[source]

decode deflate content

urlfetch.parse_url(url)[source]

returns dictionary of parsed url: scheme, netloc, path, params, query, fragment, uri, username, password, host and port

urlfetch.get_proxies_from_environ()[source]

get proxies from os.environ

urlfetch.mb_code(s, coding=None, errors='replace')[source]

encoding/decoding helper

urlfetch.sc2cs(sc)[source]

Convert Set-Cookie header to cookie string.

Set-Cookie can be retrieved from a Response instance:

sc = response.getheader('Set-Cookie')
Parameters:sc – (string) Set-Cookie
Return type:cookie string, which is name=value pairs joined by ;.
urlfetch.random_useragent(filename=None, *filenames)[source]

Returns a User-Agent string randomly from file.

>>> ua = random_useragent('file1')
>>> ua = random_useragent('file1', 'file2')
>>> ua = random_useragent(['file1', 'file2'])
>>> ua = random_useragent(['file1', 'file2'], 'file3')
Parameters:filename (string, optional) – path to the file from which a random useragent is generated
urlfetch.import_object(name)[source]

Imports an object by name.

import_object(‘x.y.z’) is equivalent to ‘from x.y import z’.

>>> import_object('os.path') is os.path
True
>>> import_object('os.path.dirname') is os.path.dirname
True
urlfetch.url_concat(url, args, keep_existing=True)[source]

Concatenate url and argument dictionary

>>> url_concat("http://example.com/foo?a=b", dict(c="d"))
'http://example.com/foo?a=b&c=d'
Parameters:
  • url – (string) url being concat to.
  • args – (dict) args being concat.
  • keep_existing – (bool, optional) Whether to keep the args which are alreay in url, default is True.
urlfetch.choose_boundary()[source]

Generate a multipart boundry.

Return type:string
urlfetch.encode_multipart(data, files)[source]

Encode multipart.

Parameters:
  • data – (dict) data to be encoded
  • files – (dict) files to be encoded
Return type:

encoded binary string

Navigation

© Copyright 2011-2013, Yue Du <ifduyue@gmail.com>. Created using Sphinx 1.2.