~brenthuisman/flickrsimplesync

d99efa3e5e8caaf483b6fa5466891cd9f3b4c349 — Brent Huisman 4 years ago bd53edb master
v3
18 files changed, 528 insertions(+), 2175 deletions(-)

M .gitignore
A LICENSE
D MANIFEST.in
R README.rst => README.md
D flickrapi/__init__.py
D flickrapi/cache.py
D flickrapi/exceptions.py
D flickrapi/multipart.py
D flickrapi/reportinghttp.py
D flickrapi/shorturl.py
D flickrapi/tokencache.py
D flickrapi/xmlnode.py
R flickrsmartsync/{__init__.py => /__init__.py}
A flickrsimplesync/__main__.py
D flickrsmartsync/__main__.py
D import
D setup.cfg
M setup.py
M .gitignore => .gitignore +10 -8
@@ 1,8 1,10 @@

.DS_Store
.svn
.idea
build
dist
*.pyc
*.egg-info
\ No newline at end of file
*
!.gitignore
!README.md
!*.py
!LICENSE
!*/
build/*
dist/*
flickrsimplesync.egg-info/*
__pycache__/*
\ No newline at end of file

A LICENSE => LICENSE +165 -0
@@ 0,0 1,165 @@
                   GNU LESSER GENERAL PUBLIC LICENSE
                       Version 3, 29 June 2007

 Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.


  This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.

  0. Additional Definitions.

  As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.

  "The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.

  An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.

  A "Combined Work" is a work produced by combining or linking an
Application with the Library.  The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".

  The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.

  The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.

  1. Exception to Section 3 of the GNU GPL.

  You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.

  2. Conveying Modified Versions.

  If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:

   a) under this License, provided that you make a good faith effort to
   ensure that, in the event an Application does not supply the
   function or data, the facility still operates, and performs
   whatever part of its purpose remains meaningful, or

   b) under the GNU GPL, with none of the additional permissions of
   this License applicable to that copy.

  3. Object Code Incorporating Material from Library Header Files.

  The object code form of an Application may incorporate material from
a header file that is part of the Library.  You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:

   a) Give prominent notice with each copy of the object code that the
   Library is used in it and that the Library and its use are
   covered by this License.

   b) Accompany the object code with a copy of the GNU GPL and this license
   document.

  4. Combined Works.

  You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:

   a) Give prominent notice with each copy of the Combined Work that
   the Library is used in it and that the Library and its use are
   covered by this License.

   b) Accompany the Combined Work with a copy of the GNU GPL and this license
   document.

   c) For a Combined Work that displays copyright notices during
   execution, include the copyright notice for the Library among
   these notices, as well as a reference directing the user to the
   copies of the GNU GPL and this license document.

   d) Do one of the following:

       0) Convey the Minimal Corresponding Source under the terms of this
       License, and the Corresponding Application Code in a form
       suitable for, and under terms that permit, the user to
       recombine or relink the Application with a modified version of
       the Linked Version to produce a modified Combined Work, in the
       manner specified by section 6 of the GNU GPL for conveying
       Corresponding Source.

       1) Use a suitable shared library mechanism for linking with the
       Library.  A suitable mechanism is one that (a) uses at run time
       a copy of the Library already present on the user's computer
       system, and (b) will operate properly with a modified version
       of the Library that is interface-compatible with the Linked
       Version.

   e) Provide Installation Information, but only if you would otherwise
   be required to provide such information under section 6 of the
   GNU GPL, and only to the extent that such information is
   necessary to install and execute a modified version of the
   Combined Work produced by recombining or relinking the
   Application with a modified version of the Linked Version. (If
   you use option 4d0, the Installation Information must accompany
   the Minimal Corresponding Source and Corresponding Application
   Code. If you use option 4d1, you must provide the Installation
   Information in the manner specified by section 6 of the GNU GPL
   for conveying Corresponding Source.)

  5. Combined Libraries.

  You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:

   a) Accompany the combined library with a copy of the same work based
   on the Library, uncombined with any other library facilities,
   conveyed under the terms of this License.

   b) Give prominent notice with the combined library that part of it
   is a work based on the Library, and explaining where to find the
   accompanying uncombined form of the same work.

  6. Revised Versions of the GNU Lesser General Public License.

  The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.

  Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.

  If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

D MANIFEST.in => MANIFEST.in +0 -2
@@ 1,2 0,0 @@
recursive-include flickrapi *
include README.md
\ No newline at end of file

R README.rst => README.md +25 -32
@@ 1,45 1,50 @@
flickrsmartsync - Sync/backup your photos to flickr easily
**********************************************************
## flickrsimplesync - Sync/backup your photos to flickr easily

flickrsmartsync is a tool you can use to easily sync up or down your
flickrsimplesync is a tool you can use to easily sync up or down your
photos in a drive/folder to flickr since now it has a free 1TB storage
you can probably sync all your photo collection.


Install
=======
### Install

Simply run the following::
Simply run the following:

    $ python setup.py install

or `PyPi`_::
Or run directly with:

    $ pip install flickrsmartsync
    $ python lastfmtagupdater

### Example Usage

Example Usage
==============

Both run from source and command line have same parameters::
Both run from source and command line have same parameters:

    start uploading all photos/videos under that folder
    $ flickrsmartsync
    $ flickrsimplesync

    ignore videos for others use --help
    $ flickrsmartsync --ignore-videos
    $ flickrsimplesync --ignore-videos

    start downloading all photos on flickr to that folder
    $ flickrsmartsync --download .
    $ flickrsimplesync --download .
    
    start downloading all paths starting with that path
    $ flickrsmartsync --download 2008/2008-01-01
    $ flickrsimplesync --download 2008/2008-01-01


    for direct python access
    $ python flickrsmartsync
### Changelog

3.0 (2016-03-21)
 * Ported to Python v3, FlickrApi v2
 * Renamed to flickrsimplesync
 * Removed dependency on IPTCInfo, tag filtering option removed
 * Removed file system monitoring
 * Removed logging module, using print() and sys.stderr.write()

Change log
==========
0.1.17-2 (2014-09-06)
 * Use Flickrs privacy preferences as a default setting for flicksmartsync
 * Pull request refused: https://github.com/faisalraja/flickrsmartsync/pull/28
 * Forked from https://github.com/faisalraja/flickrsmartsync

0.1.17 (2014-08-12)
 * allow filtering files to upload by IPTC keyword (thanks ricardokirkner)


@@ 93,15 98,3 @@ Change log
 * added run from source

0.1 (2013-06-13)


Links
=====
* `github.com`_ - source code
* `altlimit.com`_ - website
* `blog post`_ - blog post

.. _github.com: https://github.com/faisalraja/flickrsmartsync
.. _PyPi: https://pypi.python.org/pypi/flickrsmartsync
.. _altlimit.com: http://www.altlimit.com
.. _blog post: http://blog.altlimit.com/2013/05/backupsync-your-photos-to-flickr-script.html
\ No newline at end of file

D flickrapi/__init__.py => flickrapi/__init__.py +0 -909
@@ 1,909 0,0 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-

'''A FlickrAPI interface.

The main functionality can be found in the `flickrapi.FlickrAPI`
class.

See `the FlickrAPI homepage`_ for more info.

.. _`the FlickrAPI homepage`: http://stuvel.eu/projects/flickrapi
'''

__version__ = '1.4.4'
__all__ = ('FlickrAPI', 'IllegalArgumentException', 'FlickrError',
           'CancelUpload', 'XMLNode', 'set_log_level', '__version__')
__author__ = u'Sybren St\u00fcvel'.encode('utf-8')

# Copyright (c) 2007 by the respective coders, see
# http://www.stuvel.eu/projects/flickrapi
#
# This code is subject to the Python licence, as can be read on
# http://www.python.org/download/releases/2.5.2/license/
#
# For those without an internet connection, here is a summary. When this
# summary clashes with the Python licence, the latter will be applied.
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

import urllib
import urllib2
import os.path
import logging
import webbrowser

# Smartly import hashlib and fall back on md5
try:
    from hashlib import md5
except ImportError:
    from md5 import md5

from flickrapi.tokencache import (TokenCache, SimpleTokenCache,
                                  LockingTokenCache)
from flickrapi.xmlnode import XMLNode
from flickrapi.multipart import Part, Multipart, FilePart
from flickrapi.exceptions import (IllegalArgumentException, FlickrError,
                                  CancelUpload)
from flickrapi.cache import SimpleCache
from flickrapi import reportinghttp

logging.basicConfig()
LOG = logging.getLogger(__name__)
LOG.setLevel(logging.INFO)


def make_utf8(dictionary):
    '''Encodes all Unicode strings in the dictionary to UTF-8. Converts
    all other objects to regular strings.

    Returns a copy of the dictionary, doesn't touch the original.
    '''

    result = {}

    for (key, value) in dictionary.iteritems():
        if isinstance(value, unicode):
            value = value.encode('utf-8')
        else:
            value = str(value)
        result[key] = value

    return result


def debug(method):
    '''Method decorator for debugging method calls.

    Using this automatically sets the log level to DEBUG.
    '''

    LOG.setLevel(logging.DEBUG)

    def debugged(*args, **kwargs):
        LOG.debug("Call: %s(%s, %s)" % (method.__name__, args,
                  kwargs))
        result = method(*args, **kwargs)
        LOG.debug("\tResult: %s" % result)
        return result

    return debugged


# REST parsers, {format: parser_method, ...}. Fill by using the
# @rest_parser(format) function decorator
rest_parsers = {}


def rest_parser(format):
    '''Method decorator, use this to mark a function as the parser for
    REST as returned by Flickr.
    '''

    def decorate_parser(method):
        rest_parsers[format] = method
        return method

    return decorate_parser


def require_format(required_format):
    '''Method decorator, raises a ValueError when the decorated method
    is called if the default format is not set to ``required_format``.
    '''

    def decorator(method):
        def decorated(self, *args, **kwargs):
            # If everything is okay, call the method
            if self.default_format == required_format:
                return method(self, *args, **kwargs)

            # Otherwise raise an exception
            msg = 'Function %s requires that you use ' \
                  'ElementTree ("etree") as the communication format, ' \
                  'while the current format is set to "%s".'
            raise ValueError(msg % (method.func_name, self.default_format))

        return decorated
    return decorator


class FlickrAPI(object):
    """Encapsulates Flickr functionality.

    Example usage::

      flickr = flickrapi.FlickrAPI(api_key)
      photos = flickr.photos_search(user_id='73509078@N00', per_page='10')
      sets = flickr.photosets_getList(user_id='73509078@N00')
    """

    flickr_host = "api.flickr.com"
    flickr_rest_form = "/services/rest/"
    flickr_auth_form = "/services/auth/"
    flickr_upload_form = "/services/upload/"
    flickr_replace_form = "/services/replace/"

    def __init__(self, api_key, secret=None, username=None,
                 token=None, format='etree', store_token=True,
                 cache=False):
        """Construct a new FlickrAPI instance for a given API key
        and secret.

        api_key
            The API key as obtained from Flickr.

        secret
            The secret belonging to the API key.

        username
            Used to identify the appropriate authentication token for a
            certain user.

        token
            If you already have an authentication token, you can give
            it here. It won't be stored on disk by the FlickrAPI instance.

        format
            The response format. Use either "xmlnode" or "etree" to get a
            parsed response, or use any response format supported by Flickr
            to get an unparsed response from method calls. It's also possible
            to pass the ``format`` parameter on individual calls.

        store_token
            Disables the on-disk token cache if set to False (default is True).
            Use this to ensure that tokens aren't read nor written to disk, for
            example in web applications that store tokens in cookies.

        cache
            Enables in-memory caching of FlickrAPI calls - set to ``True`` to
            use. If you don't want to use the default settings, you can
            instantiate a cache yourself too:

            >>> f = FlickrAPI(api_key='123')
            >>> f.cache = SimpleCache(timeout=5, max_entries=100)
        """

        self.api_key = api_key
        self.secret = secret
        self.default_format = format

        self.__handler_cache = {}

        if token:
            # Use a memory-only token cache
            self.token_cache = SimpleTokenCache()
            self.token_cache.token = token
        elif not store_token:
            # Use an empty memory-only token cache
            self.token_cache = SimpleTokenCache()
        else:
            # Use a real token cache
            self.token_cache = TokenCache(api_key, username)

        if cache:
            self.cache = SimpleCache()
        else:
            self.cache = None

    def __repr__(self):
        '''Returns a string representation of this object.'''

        return '[FlickrAPI for key "%s"]' % self.api_key
    __str__ = __repr__

    def trait_names(self):
        '''Returns a list of method names as supported by the Flickr
        API. Used for tab completion in IPython.
        '''

        try:
            rsp = self.reflection_getMethods(format='etree')
        except FlickrError:
            return None

        def tr(name):
            '''Translates Flickr names to something that can be called
            here.

            >>> tr(u'flickr.photos.getInfo')
            u'photos_getInfo'
            '''

            return name[7:].replace('.', '_')

        return [tr(m.text) for m in rsp.getiterator('method')]

    @rest_parser('xmlnode')
    def parse_xmlnode(self, rest_xml):
        '''Parses a REST XML response from Flickr into an XMLNode object.'''

        rsp = XMLNode.parse(rest_xml, store_xml=True)
        if rsp['stat'] == 'ok':
            return rsp

        err = rsp.err[0]
        raise FlickrError(u'Error: %(code)s: %(msg)s' % err)

    @rest_parser('etree')
    def parse_etree(self, rest_xml):
        '''Parses a REST XML response from Flickr into
           an ElementTree object.'''

        try:
            import xml.etree.ElementTree as ElementTree
        except ImportError:
            # For Python 2.4 compatibility:
            try:
                import elementtree.ElementTree as ElementTree
            except ImportError:
                raise ImportError("You need to install "
                                  "ElementTree for using the etree format")

        rsp = ElementTree.fromstring(rest_xml)
        if rsp.attrib['stat'] == 'ok':
            return rsp

        err = rsp.find('err')
        raise FlickrError(u'Error: %(code)s: %(msg)s' % err.attrib)

    def sign(self, dictionary):
        """Calculate the flickr signature for a set of params.

        data
            a hash of all the params and values to be hashed, e.g.
            ``{"api_key":"AAAA", "auth_token":"TTTT", "key":
            u"value".encode('utf-8')}``

        """

        data = [self.secret]
        for key in sorted(dictionary.keys()):
            data.append(key)
            datum = dictionary[key]
            if isinstance(datum, unicode):
                raise IllegalArgumentException("No Unicode allowed, "
                                               "argument %s (%r) should have "
                                               "been UTF-8 by now"
                                               % (key, datum))
            data.append(datum)
        md5_hash = md5(''.join(data))
        return md5_hash.hexdigest()

    def encode_and_sign(self, dictionary):
        '''URL encodes the data in the dictionary, and signs it using the
        given secret, if a secret was given.
        '''

        dictionary = make_utf8(dictionary)
        if self.secret:
            dictionary['api_sig'] = self.sign(dictionary)
        return urllib.urlencode(dictionary)

    def __getattr__(self, attrib):
        """Handle all the regular Flickr API calls.

        Example::

            flickr.auth_getFrob(api_key="AAAAAA")
            etree = flickr.photos_getInfo(photo_id='1234')
            etree = flickr.photos_getInfo(photo_id='1234', format='etree')
            xmlnode = flickr.photos_getInfo(photo_id='1234', format='xmlnode')
            json = flickr.photos_getInfo(photo_id='1234', format='json')
        """

        # Refuse to act as a proxy for unimplemented special methods
        if attrib.startswith('_'):
            raise AttributeError("No such attribute '%s'" % attrib)

        # Construct the method name and see if it's cached
        method = "flickr." + attrib.replace("_", ".")
        if method in self.__handler_cache:
            return self.__handler_cache[method]

        def handler(**args):
            '''Dynamically created handler for a Flickr API call'''

            if self.token_cache.token and not self.secret:
                raise ValueError("Auth tokens cannot be used without "
                                 "API secret")

            # Set some defaults
            defaults = {'method': method,
                        'auth_token': self.token_cache.token,
                        'api_key': self.api_key,
                        'format': self.default_format}

            args = self.__supply_defaults(args, defaults)

            return self.__wrap_in_parser(self.__flickr_call,
                                         parse_format=args['format'], **args)

        handler.method = method
        self.__handler_cache[method] = handler
        return handler

    def __supply_defaults(self, args, defaults):
        '''Returns a new dictionary containing ``args``, augmented with defaults
        from ``defaults``.

        Defaults can be overridden, or completely removed by setting the
        appropriate value in ``args`` to ``None``.

        >>> f = FlickrAPI('123')
        >>> f._FlickrAPI__supply_defaults(
        ...  {'foo': 'bar', 'baz': None, 'token': None},
        ...  {'baz': 'foobar', 'room': 'door'})
        {'foo': 'bar', 'room': 'door'}
        '''

        result = args.copy()
        for key, default_value in defaults.iteritems():
            # Set the default if the parameter wasn't passed
            if key not in args:
                result[key] = default_value

        for key, value in result.copy().iteritems():
            # You are able to remove a default by assigning None, and we can't
            # pass None to Flickr anyway.
            if result[key] is None:
                del result[key]

        return result

    def __flickr_call(self, **kwargs):
        '''Performs a Flickr API call with the given arguments. The method name
        itself should be passed as the 'method' parameter.

        Returns the unparsed data from Flickr::

            data = self.__flickr_call(method='flickr.photos.getInfo',
                photo_id='123', format='rest')
        '''

        LOG.debug("Calling %s" % kwargs)

        post_data = self.encode_and_sign(kwargs)

        # Return value from cache if available
        if self.cache and self.cache.get(post_data):
            return self.cache.get(post_data)

        url = "https://" + self.flickr_host + self.flickr_rest_form
        flicksocket = urllib2.urlopen(url, post_data)
        reply = flicksocket.read()
        flicksocket.close()

        # Store in cache, if we have one
        if self.cache is not None:
            self.cache.set(post_data, reply)

        return reply

    def __wrap_in_parser(self, wrapped_method, parse_format, *args, **kwargs):
        '''Wraps a method call in a parser.

        The parser will be looked up by the ``parse_format`` specifier. If
        there is a parser and ``kwargs['format']`` is set, it's set to
        ``rest``, and the response of the method is parsed before it's
        returned.
        '''

        # Find the parser, and set the format to rest if we're supposed to
        # parse it.
        if parse_format in rest_parsers and 'format' in kwargs:
            kwargs['format'] = 'rest'

        LOG.debug('Wrapping call %s(self, %s, %s)' % (wrapped_method, args,
                  kwargs))
        data = wrapped_method(*args, **kwargs)

        # Just return if we have no parser
        if parse_format not in rest_parsers:
            return data

        # Return the parsed data
        parser = rest_parsers[parse_format]
        return parser(self, data)

    def auth_url(self, perms, frob):
        """Return the authorization URL to get a token.

        This is the URL the app will launch a browser toward if it
        needs a new token.

        perms
            "read", "write", or "delete"
        frob
            picked up from an earlier call to FlickrAPI.auth_getFrob()

        """

        encoded = self.encode_and_sign({"api_key": self.api_key,
                                        "frob": frob,
                                        "perms": perms})

        return "https://%s%s?%s" % (self.flickr_host,
                                    self.flickr_auth_form, encoded)

    def web_login_url(self, perms):
        '''Returns the web login URL to forward web users to.

        perms
            "read", "write", or "delete"
        '''

        encoded = self.encode_and_sign({"api_key": self.api_key,
                                        "perms": perms})

        return "https://%s%s?%s" % (self.flickr_host,
                                    self.flickr_auth_form, encoded)

    def __extract_upload_response_format(self, kwargs):
        '''Returns the response format given in kwargs['format'], or
        the default format if there is no such key.

        If kwargs contains 'format', it is removed from kwargs.

        If the format isn't compatible with Flickr's upload response
        type, a FlickrError exception is raised.
        '''

        # Figure out the response format
        format = kwargs.get('format', self.default_format)
        if format not in rest_parsers and format != 'rest':
            raise FlickrError('Format %s not supported for uploading '
                              'photos' % format)

        # The format shouldn't be used in the request to Flickr.
        if 'format' in kwargs:
            del kwargs['format']

        return format

    def upload(self, filename, callback=None, **kwargs):
        """Upload a file to flickr.

        Be extra careful you spell the parameters correctly, or you will
        get a rather cryptic "Invalid Signature" error on the upload!

        Supported parameters:

        filename
            name of a file to upload
        callback
            method that gets progress reports
        title
            title of the photo
        description
            description a.k.a. caption of the photo
        tags
            space-delimited list of tags, ``'''tag1 tag2 "long
            tag"'''``
        is_public
            "1" or "0" for a public resp. private photo
        is_friend
            "1" or "0" whether friends can see the photo while it's
            marked as private
        is_family
            "1" or "0" whether family can see the photo while it's
            marked as private
        content_type
            Set to "1" for Photo, "2" for Screenshot, or "3" for Other.
        hidden
            Set to "1" to keep the photo in global search results, "2"
            to hide from public searches.
        format
            The response format. You can only choose between the
            parsed responses or 'rest' for plain REST.

        The callback method should take two parameters:
        ``def callback(progress, done)``

        Progress is a number between 0 and 100, and done is a boolean
        that's true only when the upload is done.
        """

        return self.__upload_to_form(self.flickr_upload_form,
                                     filename, callback, **kwargs)

    def replace(self, filename, photo_id, callback=None, **kwargs):
        """Replace an existing photo.

        Supported parameters:

        filename
            name of a file to upload
        photo_id
            the ID of the photo to replace
        callback
            method that gets progress reports
        format
            The response format. You can only choose between the
            parsed responses or 'rest' for plain REST. Defaults to the
            format passed to the constructor.

        The callback parameter has the same semantics as described in the
        ``upload`` function.
        """

        if not photo_id:
            raise IllegalArgumentException("photo_id must be specified")

        kwargs['photo_id'] = photo_id
        return self.__upload_to_form(self.flickr_replace_form,
                                     filename, callback, **kwargs)

    def __upload_to_form(self, form_url, filename, callback, **kwargs):
        '''Uploads a photo - can be used to either upload a new photo
        or replace an existing one.

        form_url must be either ``FlickrAPI.flickr_replace_form`` or
        ``FlickrAPI.flickr_upload_form``.
        '''

        if not filename:
            raise IllegalArgumentException("filename must be specified")
        if not self.token_cache.token:
            raise IllegalArgumentException("Authentication is required")

        # Figure out the response format
        format = self.__extract_upload_response_format(kwargs)

        # Update the arguments with the ones the user won't have to supply
        arguments = {'auth_token': self.token_cache.token,
                     'api_key': self.api_key}
        arguments.update(kwargs)

        # Convert to UTF-8 if an argument is an Unicode string
        kwargs = make_utf8(arguments)

        if self.secret:
            kwargs["api_sig"] = self.sign(kwargs)
        url = "https://%s%s" % (self.flickr_host, form_url)

        # construct POST data
        body = Multipart()

        for arg, value in kwargs.iteritems():
            part = Part({'name': arg}, value)
            body.attach(part)

        filepart = FilePart({'name': 'photo'}, filename, 'image/jpeg')
        body.attach(filepart)

        return self.__wrap_in_parser(self.__send_multipart, format,
                                     url, body, callback)

    def __send_multipart(self, url, body, progress_callback=None):
        '''Sends a Multipart object to an URL.

        Returns the resulting unparsed XML from Flickr.
        '''

        LOG.debug("Uploading to %s" % url)
        request = urllib2.Request(url)
        request.add_data(str(body))

        (header, value) = body.header()
        request.add_header(header, value)

        if not progress_callback:
            # Just use urllib2 if there is no progress callback
            # function
            response = urllib2.urlopen(request)
            return response.read()

        def __upload_callback(percentage, done, seen_header=[False]):
            '''Filters out the progress report on the HTTP header'''

            # Call the user's progress callback when we've filtered
            # out the HTTP header
            if seen_header[0]:
                return progress_callback(percentage, done)

            # Remember the first time we hit 'done'.
            if done:
                seen_header[0] = True

        response = reportinghttp.urlopen(request, __upload_callback)
        return response.read()

    def validate_frob(self, frob, perms):
        '''Lets the user validate the frob by launching a browser to
        the Flickr website.
        '''

        auth_url = self.auth_url(perms, frob)
        try:
            browser = webbrowser.get()
        except webbrowser.Error:
            if 'BROWSER' not in os.environ:
                print 'Please authorize: %s' % auth_url
                return
            browser = webbrowser.GenericBrowser(os.environ['BROWSER'])

        browser.open(auth_url, True, True)

    def get_token_part_one(self, perms="read", auth_callback=None):
        """Get a token either from the cache, or make a new one from
        the frob.

        This first attempts to find a token in the user's token cache
        on disk. If that token is present and valid, it is returned by
        the method.

        If that fails (or if the token is no longer valid based on
        flickr.auth.checkToken) a new frob is acquired. If an auth_callback
        method has been specified it will be called. Otherwise the frob is
        validated by having the user log into flickr (with a browser).

        To get a proper token, follow these steps:
            - Store the result value of this method call
            - Give the user a way to signal the program that he/she
              has authorized it, for example show a button that can be
              pressed.
            - Wait for the user to signal the program that the
              authorization was performed, but only if there was no
              cached token.
            - Call flickrapi.get_token_part_two(...) and pass it the
              result value you stored.

        The newly minted token is then cached locally for the next
        run.

        perms
            "read", "write", or "delete"
        auth_callback
            method to be called if authorization is needed. When not
            passed, ``self.validate_frob(...)`` is called. You can
            call this method yourself from the callback method too.

            If authorization should be blocked, pass
            ``auth_callback=False``.

            The auth_callback method should take ``(frob, perms)`` as
            parameters.

        An example::

            (token, frob) = flickr.get_token_part_one(perms='write')
            if not token:
                raw_input("Press ENTER after you authorized this program")
            flickr.get_token_part_two((token, frob))

        Also take a look at ``authenticate_console(perms)``.
        """

        # Check our auth_callback parameter for correctness before we
        # do anything
        authenticate = self.validate_frob
        if auth_callback is not None:
            if hasattr(auth_callback, '__call__'):
                # use the provided callback function
                authenticate = auth_callback
            elif auth_callback is False:
                authenticate = None
            else:
                # Any non-callable non-False value is invalid
                raise ValueError('Invalid value for auth_callback: %s'
                                 % auth_callback)

        # see if we have a saved token
        token = self.token_cache.token
        frob = None

        # see if it's valid
        if token:
            LOG.debug("Trying cached token '%s'" % token)
            try:
                rsp = self.auth_checkToken(auth_token=token, format='xmlnode')

                # see if we have enough permissions
                tokenPerms = rsp.auth[0].perms[0].text
                if tokenPerms == "read" and perms != "read":
                    token = None
                elif tokenPerms == "write" and perms == "delete":
                    token = None
            except FlickrError:
                LOG.debug("Cached token invalid")
                self.token_cache.forget()
                token = None

        # get a new token if we need one
        if not token:
            # If we can't authenticate, it's all over.
            if not authenticate:
                raise FlickrError('Authentication required but '
                                  'blocked using auth_callback=False')

            # get the frob
            LOG.debug("Getting frob for new token")
            rsp = self.auth_getFrob(auth_token=None, format='xmlnode')

            frob = rsp.frob[0].text
            authenticate(frob, perms)

        return (token, frob)

    def get_token_part_two(self, (token, frob)):
        """Part two of getting a token,
           see ``get_token_part_one(...)`` for details."""

        # If a valid token was obtained in the past, we're done
        if token:
            LOG.debug("get_token_part_two: no need, token already there")
            self.token_cache.token = token
            return token

        LOG.debug("get_token_part_two: "
                  "getting a new token for frob '%s'" % frob)

        return self.get_token(frob)

    def get_token(self, frob):
        '''Gets the token given a certain frob. Used by ``get_token_part_two`` and
        by the web authentication method.
        '''

        # get a token
        rsp = self.auth_getToken(frob=frob, auth_token=None, format='xmlnode')

        token = rsp.auth[0].token[0].text
        LOG.debug("get_token: new token '%s'" % token)

        # store the auth info for next time
        self.token_cache.token = token

        return token

    def authenticate_console(self, perms='read', auth_callback=None):
        '''Performs the authentication, assuming a console program.

        Gets the token, if needed starts the browser and waits for the user to
        press ENTER before continuing.

        See ``get_token_part_one(...)`` for an explanation of the
        parameters.
        '''

        (token, frob) = self.get_token_part_one(perms, auth_callback)
        if not token:
            raw_input("Press ENTER after you authorized this program")

        self.get_token_part_two((token, frob))

    @require_format('etree')
    def __data_walker(self, method, **params):
        '''Calls 'method' with page=0, page=1 etc. until the total
        number of pages has been visited. Yields the photos
        returned.

        Assumes that ``method(page=n, **params).findall('*/photos')``
        results in a list of photos, and that the toplevel element of
        the result contains a 'pages' attribute with the total number
        of pages.
        '''

        page = 1
        total = 1  # We don't know that yet, update when needed
        while page <= total:
            # Fetch a single page of photos
            LOG.debug('Calling %s(page=%i of %i, %s)' %
                      (method.func_name, page, total, params))
            rsp = method(page=page, **params)

            photoset = rsp.getchildren()[0]
            total = int(photoset.get('pages'))

            photos = rsp.findall('*/photo')

            # Yield each photo
            for photo in photos:
                yield photo

            # Ready to get the next page
            page += 1

    @require_format('etree')
    def walk_set(self, photoset_id, per_page=50, **kwargs):
        '''walk_set(self, photoset_id, per_page=50, ...) -> \
                generator, yields each photo in a single set.

        :Parameters:
            photoset_id
                the photoset ID
            per_page
                the number of photos that are fetched in one call to
                Flickr.

        Other arguments can be passed, as documented in the
        flickr.photosets.getPhotos_ API call in the Flickr API
        documentation, except for ``page`` because all pages will be
        returned eventually.

        .. _flickr.photosets.getPhotos:
            http://www.flickr.com/services/api/flickr.photosets.getPhotos.html

        Uses the ElementTree format, incompatible with other formats.
        '''

        return self.__data_walker(self.photosets_getPhotos,
                                  photoset_id=photoset_id,
                                  per_page=per_page, **kwargs)

    @require_format('etree')
    def walk(self, per_page=50, **kwargs):
        '''walk(self, user_id=..., tags=..., ...) -> generator, \
                yields each photo in a search query result

        Accepts the same parameters as flickr.photos.search_ API call,
        except for ``page`` because all pages will be returned
        eventually.

        .. _flickr.photos.search:
            http://www.flickr.com/services/api/flickr.photos.search.html

        Also see `walk_set`.
        '''

        return self.__data_walker(self.photos_search,
                                  per_page=per_page, **kwargs)


def set_log_level(level):
    '''Sets the log level of the logger used by the FlickrAPI module.

    >>> import flickrapi
    >>> import logging
    >>> flickrapi.set_log_level(logging.INFO)
    '''

    import flickrapi.tokencache

    LOG.setLevel(level)
    flickrapi.tokencache.LOG.setLevel(level)


if __name__ == "__main__":
    print "Running doctests"
    import doctest
    doctest.testmod()
    print "Tests OK"

D flickrapi/cache.py => flickrapi/cache.py +0 -107
@@ 1,107 0,0 @@
# -*- encoding: utf-8 -*-

'''Call result cache.

Designed to have the same interface as the `Django low-level cache API`_.
Heavily inspired (read: mostly copied-and-pasted) from the Django framework -
thanks to those guys for designing a simple and effective cache!

.. _`Django low-level cache API`: http://www.djangoproject.com/documentation/cache/#the-low-level-cache-api
'''

import threading
import time


class SimpleCache(object):
    '''Simple response cache for FlickrAPI calls.

    This stores max 50 entries, timing them out after 120 seconds:
    >>> cache = SimpleCache(timeout=120, max_entries=50)
    '''

    def __init__(self, timeout=300, max_entries=200):
        self.storage = {}
        self.expire_info = {}
        self.lock = threading.RLock()
        self.default_timeout = timeout
        self.max_entries = max_entries
        self.cull_frequency = 3

    def locking(method):
        '''Method decorator, ensures the method call is locked'''

        def locked(self, *args, **kwargs):
            self.lock.acquire()
            try:
                return method(self, *args, **kwargs)
            finally:
                self.lock.release()

        return locked

    @locking
    def get(self, key, default=None):
        '''Fetch a given key from the cache. If the key does not exist, return
        default, which itself defaults to None.
        '''

        now = time.time()
        exp = self.expire_info.get(key)
        if exp is None:
            return default
        elif exp < now:
            self.delete(key)
            return default

        return self.storage[key]

    @locking
    def set(self, key, value, timeout=None):
        '''Set a value in the cache. If timeout is given, that timeout will be
        used for the key; otherwise the default cache timeout will be used.
        '''

        if len(self.storage) >= self.max_entries:
            self.cull()
        if timeout is None:
            timeout = self.default_timeout
        self.storage[key] = value
        self.expire_info[key] = time.time() + timeout

    @locking
    def delete(self, key):
        '''Deletes a key from the cache,
           failing silently if it doesn't exist.'''

        if key in self.storage:
            del self.storage[key]
        if key in self.expire_info:
            del self.expire_info[key]

    @locking
    def has_key(self, key):
        '''Returns True if the key is in the cache and has not expired.'''
        return self.get(key) is not None

    @locking
    def __contains__(self, key):
        '''Returns True if the key is in the cache and has not expired.'''
        return self.has_key(key)

    @locking
    def cull(self):
        '''Reduces the number of cached items'''

        doomed = [k for (i, k) in enumerate(self.storage)
                  if i % self.cull_frequency == 0]
        for k in doomed:
            self.delete(k)

    @locking
    def __len__(self):
        '''Returns the number of cached items -- they might be expired
        though.
        '''

        return len(self.storage)

D flickrapi/exceptions.py => flickrapi/exceptions.py +0 -30
@@ 1,30 0,0 @@
'''Exceptions used by the FlickrAPI module.'''


class IllegalArgumentException(ValueError):
    '''Raised when a method is passed an illegal argument.

    More specific details will be included in the exception message
    when thrown.
    '''


class FlickrError(Exception):
    '''Raised when a Flickr method fails.

    More specific details will be included in the exception message
    when thrown.
    '''


class CancelUpload(Exception):
    '''Raise this exception in an upload/replace callback function to
    abort the upload.
    '''


class LockingError(Exception):
    '''Raised when TokenCache cannot acquire a lock within the timeout
    period, or when a lock release is attempted when the lock does not
    belong to this process.
    '''

D flickrapi/multipart.py => flickrapi/multipart.py +0 -106
@@ 1,106 0,0 @@
# -*- encoding: utf-8 -*-

'''Module for encoding data as form-data/multipart'''

import os
import base64


class Part(object):

    '''A single part of the multipart data.

    >>> Part({'name': 'headline'}, 'Nice Photo')
    ... # doctest: +ELLIPSIS
    <flickrapi.multipart.Part object at 0x...>

    >>> image = open('tests/photo.jpg')
    >>> Part({'name': 'photo', 'filename': image}, image.read(), 'image/jpeg')
    ... # doctest: +ELLIPSIS
    <flickrapi.multipart.Part object at 0x...>
    '''

    def __init__(self, parameters, payload, content_type=None):
        self.content_type = content_type
        self.parameters = parameters
        self.payload = payload

    def render(self):
        '''Renders this part -> List of Strings'''

        parameters = ['%s="%s"' % (k, v)
                      for k, v in self.parameters.iteritems()]

        lines = ['Content-Disposition: form-data; %s' % '; '.join(parameters)]

        if self.content_type:
            lines.append("Content-Type: %s" % self.content_type)

        lines.append('')

        if isinstance(self.payload, unicode):
            lines.append(self.payload.encode('utf-8'))
        else:
            lines.append(self.payload)

        return lines


class FilePart(Part):
    '''A single part with a file as the payload

    This example has the same semantics as the second Part example:

    >>> FilePart({'name': 'photo'}, 'tests/photo.jpg', 'image/jpeg')
    ... #doctest: +ELLIPSIS
    <flickrapi.multipart.FilePart object at 0x...>
    '''

    def __init__(self, parameters, filename, content_type):
        parameters['filename'] = filename

        imagefile = open(filename, 'rb')
        payload = imagefile.read()
        imagefile.close()

        Part.__init__(self, parameters, payload, content_type)


def boundary():
    """Generate a random boundary, a bit like Python 2.5's uuid module."""

    bytes = os.urandom(16)
    return base64.b64encode(bytes, 'ab').strip('=')


class Multipart(object):
    '''Container for multipart data'''

    def __init__(self):
        '''Creates a new Multipart.'''

        self.parts = []
        self.content_type = 'form-data/multipart'
        self.boundary = boundary()

    def attach(self, part):
        '''Attaches a part'''

        self.parts.append(part)

    def __str__(self):
        '''Renders the Multipart'''

        lines = []
        for part in self.parts:
            lines += ['--' + self.boundary]
            lines += part.render()
        lines += ['--' + self.boundary + "--"]

        return '\r\n'.join(lines)

    def header(self):
        '''Returns the top-level HTTP header of this multipart'''

        return ("Content-Type",
                "multipart/form-data; boundary=%s" % self.boundary)

D flickrapi/reportinghttp.py => flickrapi/reportinghttp.py +0 -106
@@ 1,106 0,0 @@
# -*- encoding: utf-8 -*-

'''HTTPHandler that supports a callback method for progress reports.
'''

import urllib2
import httplib
import logging

__all__ = ['urlopen']

logging.basicConfig()
LOG = logging.getLogger(__name__)

progress_callback = None


class ReportingSocket(object):
    '''Wrapper around a socket. Gives progress report through a
    callback function.
    '''

    min_chunksize = 10240

    def __init__(self, socket):
        self.socket = socket

    def sendall(self, bits):
        '''Sends all data, calling the callback function for every
        sent chunk.
        '''

        LOG.debug("SENDING: %s..." % bits[0:30])
        total = len(bits)
        sent = 0
        chunksize = max(self.min_chunksize, total // 100)

        while len(bits) > 0:
            send = bits[0:chunksize]
            self.socket.sendall(send)
            sent += len(send)
            if progress_callback:
                progress = float(sent) / total * 100
                progress_callback(progress, sent == total)

            bits = bits[chunksize:]

    def makefile(self, mode, bufsize):
        '''Returns a file-like object for the socket.'''

        return self.socket.makefile(mode, bufsize)

    def close(self):
        '''Closes the socket.'''

        return self.socket.close()


class ProgressHTTPConnection(httplib.HTTPConnection):
    '''HTTPConnection that gives regular progress reports during
    sending of data.
    '''

    def connect(self):
        '''Connects to a HTTP server.'''

        httplib.HTTPConnection.connect(self)
        self.sock = ReportingSocket(self.sock)


class ProgressHTTPHandler(urllib2.HTTPHandler):
    '''HTTPHandler that gives regular progress reports during sending
    of data.
    '''
    def http_open(self, req):
        return self.do_open(ProgressHTTPConnection, req)


def set_callback(method):
    '''Sets the callback function to use for progress reports.'''

    global progress_callback  # IGNORE:W0603

    if not hasattr(method, '__call__'):
        raise ValueError('Callback method must be callable')

    progress_callback = method


def urlopen(url_or_request, callback, body=None):
    '''Opens an URL using the ProgressHTTPHandler.'''

    set_callback(callback)
    opener = urllib2.build_opener(ProgressHTTPHandler)
    return opener.open(url_or_request, body)

if __name__ == '__main__':
    def upload(progress, finished):
        '''Upload progress demo'''

        LOG.info("%3.0f - %s" % (progress, finished))

    conn = urlopen("http://www.flickr.com/", 'x' * 10245, upload)
    data = conn.read()
    LOG.info("Read data")
    print data[:100].split('\n')[0]

D flickrapi/shorturl.py => flickrapi/shorturl.py +0 -74
@@ 1,74 0,0 @@
# -*- coding: utf-8 -*-

'''Helper functions for the short http://fli.kr/p/... URL notation.

Photo IDs can be converted to and from Base58 short IDs, and a short
URL can be generated from a photo ID.

The implementation of the encoding and decoding functions is based on
the posts by stevefaeembra and Kohichi on
http://www.flickr.com/groups/api/discuss/72157616713786392/

'''

__all__ = ['encode', 'decode', 'url', 'SHORT_URL']

ALPHABET = u'123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ'
ALPHALEN = len(ALPHABET)
SHORT_URL = u'http://flic.kr/p/%s'


def encode(photo_id):
    '''encode(photo_id) -> short id

    >>> encode(u'4325695128')
    u'7Afjsu'
    >>> encode(u'2811466321')
    u'5hruZg'
    '''

    photo_id = int(photo_id)

    encoded = u''
    while photo_id >= ALPHALEN:
        div, mod = divmod(photo_id, ALPHALEN)
        encoded = ALPHABET[mod] + encoded
        photo_id = int(div)

    encoded = ALPHABET[photo_id] + encoded

    return encoded


def decode(short_id):
    '''decode(short id) -> photo id

    >>> decode(u'7Afjsu')
    u'4325695128'
    >>> decode(u'5hruZg')
    u'2811466321'
    '''

    decoded = 0
    multi = 1

    for i in xrange(len(short_id)-1, -1, -1):
        char = short_id[i]
        index = ALPHABET.index(char)
        decoded = decoded + multi * index
        multi = multi * len(ALPHABET)

    return unicode(decoded)


def url(photo_id):
    '''url(photo id) -> short url

    >>> url(u'4325695128')
    u'http://flic.kr/p/7Afjsu'
    >>> url(u'2811466321')
    u'http://flic.kr/p/5hruZg'
    '''

    short_id = encode(photo_id)
    return SHORT_URL % short_id

D flickrapi/tokencache.py => flickrapi/tokencache.py +0 -267
@@ 1,267 0,0 @@

'''Persistent token cache management for the Flickr API'''

import os.path
import logging
import time

from flickrapi.exceptions import LockingError

logging.basicConfig()
LOG = logging.getLogger(__name__)
LOG.setLevel(logging.INFO)

__all__ = ('TokenCache', 'SimpleTokenCache')


class SimpleTokenCache(object):
    '''In-memory token cache.'''

    def __init__(self):
        self.token = None

    def forget(self):
        '''Removes the cached token'''

        self.token = None


class TokenCache(object):
    '''On-disk persistent token cache for a single application.

    The application is identified by the API key used. Per
    application multiple users are supported, with a single
    token per user.
    '''

    def __init__(self, api_key, username=None):
        '''Creates a new token cache instance'''

        self.api_key = api_key
        self.username = username
        self.memory = {}
        self.path = os.path.join("~", ".flickr")

    def get_cached_token_path(self):
        """Return the directory holding the app data."""
        return os.path.expanduser(os.path.join(self.path, self.api_key))

    def get_cached_token_filename(self):
        """Return the full pathname of the cached token file."""

        if self.username:
            filename = 'auth-%s.token' % self.username
        else:
            filename = 'auth.token'

        return os.path.join(self.get_cached_token_path(), filename)

    def get_cached_token(self):
        """Read and return a cached token, or None if not found.

        The token is read from the cached token file.
        """

        # Only read the token once
        if self.username in self.memory:
            return self.memory[self.username]

        try:
            f = open(self.get_cached_token_filename(), "r")
            token = f.read()
            f.close()

            return token.strip()
        except IOError:
            return None

    def set_cached_token(self, token):
        """Cache a token for later use."""

        # Remember for later use
        self.memory[self.username] = token

        path = self.get_cached_token_path()
        if not os.path.exists(path):
            os.makedirs(path)

        f = open(self.get_cached_token_filename(), "w")
        f.write(token)
        f.close()

    def forget(self):
        '''Removes the cached token'''

        if self.username in self.memory:
            del self.memory[self.username]
        filename = self.get_cached_token_filename()
        if os.path.exists(filename):
            os.unlink(filename)

    token = property(get_cached_token, set_cached_token,
                     forget, "The cached token")


class LockingTokenCache(TokenCache):
    '''Locks the token cache when reading or updating it, so that
    multiple processes can safely use the same API key.
    '''

    def get_lock_name(self):
        '''Returns the filename of the lock.'''

        token_name = self.get_cached_token_filename()
        return '%s-lock' % token_name
    lock = property(get_lock_name)

    def get_pidfile_name(self):
        '''Returns the name of the pidfile in the lock directory.'''

        return os.path.join(self.lock, 'pid')
    pidfile_name = property(get_pidfile_name)

    def get_lock_pid(self):
        '''Returns the PID that is stored in the lock directory, or
        None if there is no such file.
        '''

        filename = self.pidfile_name
        if not os.path.exists(filename):
            return None

        pidfile = open(filename)
        try:
            pid = pidfile.read()
            if pid:
                return int(pid)
        finally:
            pidfile.close()

        return None

    def acquire(self, timeout=60):
        '''Locks the token cache for this key and username.

        If the token cache is already locked, waits until it is
        released. Throws an exception when the lock cannot be acquired
        after ``timeout`` seconds.
        '''

        # Check whether there is a PID file already with our PID in
        # it.
        lockpid = self.get_lock_pid()
        if lockpid == os.getpid():
            LOG.debug('The lock is ours, continuing')
            return

        # Figure out the lock filename
        lock = self.get_lock_name()
        LOG.debug('Acquiring lock %s' % lock)

        # Try to obtain the lock
        start_time = time.time()
        while True:
            try:
                os.makedirs(lock)
                break
            except OSError:
                # If the path doesn't exist, the error isn't that it
                # can't be created because someone else has got the
                # lock. Just bail out then.
                if not os.path.exists(lock):
                    LOG.error('Unable to acquire lock %s, aborting' %
                              lock)
                    raise

                if time.time() - start_time >= timeout:
                    # Timeout has passed, bail out
                    raise LockingError('Unable to acquire lock ' +
                                       '%s, aborting' % lock)

                # Wait for a bit, then try again
                LOG.debug('Unable to acquire lock, waiting')
                time.sleep(0.1)

        # Write the PID file
        LOG.debug('Lock acquired, writing our PID')
        pidfile = open(self.pidfile_name, 'w')
        try:
            pidfile.write('%s' % os.getpid())
        finally:
            pidfile.close()

    def release(self):
        '''Unlocks the token cache for this key.'''

        # Figure out the lock filename
        lock = self.get_lock_name()
        if not os.path.exists(lock):
            LOG.warn('Trying to release non-existing lock %s' % lock)
            return

        # If the PID file isn't ours, abort.
        lockpid = self.get_lock_pid()
        if lockpid and lockpid != os.getpid():
            raise LockingError(('Lock %s is NOT ours, but belongs ' +
                                'to PID %i, unable to release.') % (lock,
                                                                    lockpid))

        LOG.debug('Releasing lock %s' % lock)

        # Remove the PID file and the lock directory
        pidfile = self.pidfile_name
        if os.path.exists(pidfile):
            os.remove(pidfile)
        os.removedirs(lock)

    def __del__(self):
        '''Cleans up any existing lock.'''

        # Figure out the lock filename
        lock = self.get_lock_name()
        if not os.path.exists(lock):
            return

        # If the PID file isn't ours, we're done
        lockpid = self.get_lock_pid()
        if lockpid and lockpid != os.getpid():
            return

        # Release the lock
        self.release()

    def locked(method):
        '''Decorator, ensures the method runs in a locked cache.'''

        def locker(self, *args, **kwargs):
            self.acquire()
            try:
                return method(self, *args, **kwargs)
            finally:
                self.release()

        return locker

    @locked
    def get_cached_token(self):
        """Read and return a cached token, or None if not found.

        The token is read from the cached token file.
        """

        return TokenCache.get_cached_token(self)

    @locked
    def set_cached_token(self, token):
        """Cache a token for later use."""

        TokenCache.set_cached_token(self, token)

    @locked
    def forget(self):
        '''Removes the cached token'''

        TokenCache.forget(self)

    token = property(get_cached_token, set_cached_token,
                     forget, "The cached token")

D flickrapi/xmlnode.py => flickrapi/xmlnode.py +0 -104
@@ 1,104 0,0 @@

'''FlickrAPI uses its own in-memory XML representation, to be able to easily
use the info returned from Flickr.

There is no need to use this module directly, you'll get XMLNode instances
from the FlickrAPI method calls.
'''

import xml.dom.minidom

__all__ = ('XMLNode', )


class XMLNode:
    """XMLNode -- generic class for holding an XML node

    >>> xml_str = '''<xml foo="32">
    ... <taggy bar="10">Name0</taggy>
    ... <taggy bar="11" baz="12">Name1</taggy>
    ... </xml>'''
    >>> f = XMLNode.parse(xml_str)
    >>> f.name
    u'xml'
    >>> f['foo']
    u'32'
    >>> f.taggy[0].name
    u'taggy'
    >>> f.taggy[0]["bar"]
    u'10'
    >>> f.taggy[0].text
    u'Name0'
    >>> f.taggy[1].name
    u'taggy'
    >>> f.taggy[1]["bar"]
    u'11'
    >>> f.taggy[1]["baz"]
    u'12'

    """

    def __init__(self):
        """Construct an empty XML node."""
        self.name = ""
        self.text = ""
        self.attrib = {}
        self.xml = None

    def __setitem__(self, key, item):
        """Store a node's attribute in the attrib hash."""
        self.attrib[key] = item

    def __getitem__(self, key):
        """Retrieve a node's attribute from the attrib hash."""
        return self.attrib[key]

    @classmethod
    def __parse_element(cls, element, this_node):
        """Recursive call to process this XMLNode."""

        this_node.name = element.nodeName

        # add element attributes as attributes to this node
        for i in range(element.attributes.length):
            an = element.attributes.item(i)
            this_node[an.name] = an.nodeValue

        for a in element.childNodes:
            if a.nodeType == xml.dom.Node.ELEMENT_NODE:

                child = XMLNode()
                # Ugly fix for an ugly bug. If an XML element <name />
                # exists, it now overwrites the 'name' attribute
                # storing the XML element name.
                if not hasattr(this_node, a.nodeName) or a.nodeName == 'name':
                    setattr(this_node, a.nodeName, [])

                # add the child node as an attrib to this node
                children = getattr(this_node, a.nodeName)
                children.append(child)

                cls.__parse_element(a, child)

            elif a.nodeType == xml.dom.Node.TEXT_NODE:
                this_node.text += a.nodeValue

        return this_node

    @classmethod
    def parse(cls, xml_str, store_xml=False):
        """Convert an XML string into a nice instance tree of XMLNodes.

        xml_str -- the XML to parse
        store_xml -- if True, stores the XML string in the root XMLNode.xml

        """

        dom = xml.dom.minidom.parseString(xml_str)

        # get the root
        root_node = XMLNode()
        if store_xml:
            root_node.xml = xml_str

        return cls.__parse_element(dom.firstChild, root_node)

R flickrsmartsync/__init__.py => flickrsimplesync/__init__.py +304 -398
@@ 1,398 1,304 @@
#
# -*- coding: utf-8 -*-
import HTMLParser
import json
import os
import re
import urllib
import argparse
import time
import flickrapi
import logging
from logging.handlers import SysLogHandler
from iptcinfo import IPTCInfo
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler


logger = logging.getLogger(__name__)
hdlr = SysLogHandler()
formatter = logging.Formatter('flickrsmartsync %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logger.setLevel(logging.INFO)

__author__ = 'faisal'

EXT_IMAGE = ('jpg', 'png', 'jpeg', 'gif', 'bmp')
EXT_VIDEO = ('avi', 'wmv', 'mov', 'mp4', '3gp', 'ogg', 'ogv', 'mts')

#  flickr api keys
KEY = 'f7da21662566bc773c7c750ddf7030f7'
SECRET = 'c329cdaf44c6d3f3'


def start_sync(sync_path, cmd_args, specific_path=None):
    is_windows = os.name == 'nt'
    is_download = cmd_args.download
    keywords = set(cmd_args.keyword) if cmd_args .keyword else ()

    if not os.path.exists(sync_path):
        logger.error('Sync path does not exists')
        exit(0)

    # Common arguments
    args = {'format': 'json', 'nojsoncallback': 1}
    api = flickrapi.FlickrAPI(KEY, SECRET, cmd_args.username)  # pass username argument to api
    # api.token.path = 'flickr.token.txt'

    # Ask for permission
    (token, frob) = api.get_token_part_one(perms='write')

    if not token:
        raw_input("Please authorized this app then hit enter:")

    try:
        token = api.get_token_part_two((token, frob))
    except:
        logger.error('Please authorized to use')
        exit(0)

    # Get default user privacy preferences
    try:
        #Currently not handled in the flickrapi library
        privacypref = api.prefs_getPrivacy().getchildren()[0].attrib['privacy']
        # https://www.flickr.com/services/api/flickr.prefs.getPrivacy.html
    except:
        logger.error('Failed to get Flickr privacy preferences.')
        exit(0)

    #Default to private
    args.update({
        'is_public': 0,
        'is_friend': 0,
        'is_family': 0
    })

    #Update if so defined in preferences. Note that the privacy number is a string!
    if privacypref == '1':
        args.update({'is_public': 1})
    elif privacypref == '2':
        args.update({'is_friend': 1})
    elif privacypref == '3':
        args.update({'is_family': 1})
    elif privacypref == '4':
        args.update({'is_friend': 1,'is_family': 1})

    args.update({'auth_token': token})

    # Build your local photo sets
    photo_sets = {}
    skips_root = []
    for r, dirs, files in os.walk(sync_path if not specific_path else os.path.dirname(specific_path)):

        if cmd_args.starts_with and not r.startswith('{}{}'.format(sync_path, cmd_args.starts_with)):
            continue

        files = [f for f in files if not f.startswith('.')]
        dirs[:] = [d for d in dirs if not d.startswith('.')]

        for file in files:
            if not file.startswith('.'):
                ext = file.lower().split('.').pop()
                if ext in EXT_IMAGE or \
                   ext in EXT_VIDEO:

                    if r == sync_path:
                        skips_root.append(file)
                    else:
                        # filter by keywords
                        if keywords:
                            file_path = os.path.join(r, file)
                            info = IPTCInfo(file_path, force=True)
                            matches = keywords.intersection(info.keywords)
                            if not matches:
                                # no matching keyword(s) found, skip file
                                logger.info('Skipped [%s] does not match any keyword %s' % (file, list(keywords)))
                                continue

                        photo_sets.setdefault(r, [])
                        photo_sets[r].append(file)

    if skips_root:
        logger.warn('To avoid disorganization on flickr sets root photos are not synced, skipped these photos: %s' % skips_root)
        logger.warn('Try to sync at top most level of your photos directory')

    # custom set builder
    def get_custom_set_title(path):
        title = path.split('/').pop()

        if cmd_args.custom_set:
            m = re.match(cmd_args.custom_set, path)
            if m:
                if not cmd_args.custom_set_builder:
                    title = '-'.join(m.groups())
                elif m.groupdict():
                    title = cmd_args.custom_set_builder.format(**m.groupdict())
                else:
                    title = cmd_args.custom_set_builder.format(*m.groups())
        return title

    # Get your photosets online and map it to your local
    html_parser = HTMLParser.HTMLParser()
    photosets_args = args.copy()
    page = 1
    photo_sets_map = {}

    # Show 3 possibilities
    if cmd_args.custom_set:
        for photo_set in photo_sets:
            logger.info('Set Title: [%s]  Path: [%s]' % (get_custom_set_title(photo_set), photo_set))

        if raw_input('Is this your expected custom set titles (y/n):') != 'y':
            exit(0)

    while True:
        logger.info('Getting photosets page %s' % page)
        photosets_args.update({'page': page, 'per_page': 500})
        sets = json.loads(api.photosets_getList(**photosets_args))
        page += 1
        if not sets['photosets']['photoset']:
            break

        for current_set in sets['photosets']['photoset']:
            # Make sure it's the one from backup format
            desc = html_parser.unescape(current_set['description']['_content'])
            desc = desc.encode('utf-8') if isinstance(desc, unicode) else desc
            if desc:
                photo_sets_map[desc] = current_set['id']
                title = get_custom_set_title(sync_path + desc)
                if cmd_args.update_custom_set and desc in photo_set and title != current_set['title']['_content']:
                    update_args = args.copy()
                    update_args.update({
                        'photoset_id': current_set['id'],
                        'title': title,
                        'description': desc
                    })
                    logger.info('Updating custom title [%s]...' % title)
                    json.loads(api.photosets_editMeta(**update_args))
                    logger.info('done')

    logger.info('Found %s photo sets' % len(photo_sets_map))

    # For adding photo to set
    def add_to_photo_set(photo_id, folder):
        # If photoset not found in online map create it else add photo to it
        # Always upload unix style
        if is_windows:
            folder = folder.replace(os.sep, '/')

        if folder not in photo_sets_map:
            photosets_args = args.copy()
            custom_title = get_custom_set_title(sync_path + folder)
            photosets_args.update({'primary_photo_id': photo_id,
                                   'title': custom_title,
                                   'description': folder})
            photo_set = json.loads(api.photosets_create(**photosets_args))
            photo_sets_map[folder] = photo_set['photoset']['id']
            logger.info('Created set [%s] and added photo' % custom_title)
        else:
            photosets_args = args.copy()
            photosets_args.update({'photoset_id': photo_sets_map.get(folder), 'photo_id': photo_id})
            result = json.loads(api.photosets_addPhoto(**photosets_args))
            if result.get('stat') == 'ok':
                logger.info('Success')
            else:
                logger.error(result)

    # Get photos in a set
    def get_photos_in_set(folder):
        # bug on non utf8 machines dups
        folder = folder.encode('utf-8') if isinstance(folder, unicode) else folder

        photos = {}
        # Always upload unix style
        if is_windows:
            folder = folder.replace(os.sep, '/')

        if folder in photo_sets_map:
            photoset_args = args.copy()
            page = 1
            while True:
                photoset_args.update({'photoset_id': photo_sets_map[folder], 'page': page})
                if is_download:
                    photoset_args['extras'] = 'url_o,media'
                page += 1
                photos_in_set = json.loads(api.photosets_getPhotos(**photoset_args))
                if photos_in_set['stat'] != 'ok':
                    break

                for photo in photos_in_set['photoset']['photo']:

                    if is_download and photo.get('media') == 'video':
                        # photo_args = args.copy()
                        # photo_args['photo_id'] = photo['id']
                        # sizes = json.loads(api.photos_getSizes(**photo_args))
                        # if sizes['stat'] != 'ok':
                        #     continue
                        #
                        # original = filter(lambda s: s['label'].startswith('Site') and s['media'] == 'video', sizes['sizes']['size'])
                        # if original:
                        #     photos[photo['title']] = original.pop()['source'].replace('/site/', '/orig/')
                        #     print photos
                        # Skipts download video for now since it doesn't work
                        continue
                    else:
                        photos[photo['title'].encode('utf-8')] = photo['url_o'] if is_download else photo['id']

        return photos

    # If download mode lets skip upload but you can also modify this to your needs
    if is_download:
        # Download to corresponding paths
        os.chdir(sync_path)

        for photo_set in photo_sets_map:
            if photo_set and is_download == '.' or is_download != '.' and photo_set.startswith(is_download):
                folder = photo_set.replace(sync_path, '')
                logger.info('Getting photos in set [%s]' % folder)
                photos = get_photos_in_set(folder)
                # If Uploaded on unix and downloading on windows & vice versa
                if is_windows:
                    folder = folder.replace('/', os.sep)

                if not os.path.isdir(folder):
                    os.makedirs(folder)

                for photo in photos:
                    # Adds skips
                    if cmd_args.ignore_images and photo.split('.').pop().lower() in EXT_IMAGE:
                        continue
                    elif cmd_args.ignore_videos and photo.split('.').pop().lower() in EXT_VIDEO:
                        continue

                    path = os.path.join(folder, photo)
                    if os.path.exists(path):
                        logger.info('Skipped [%s] already downloaded' % path)
                    else:
                        logger.info('Downloading photo [%s]' % path)
                        urllib.urlretrieve(photos[photo], os.path.join(sync_path, path))
    else:
        # Loop through all local photo set map and
        # upload photos that does not exists in online map
        for photo_set in sorted(photo_sets):
            folder = photo_set.replace(sync_path, '')
            display_title = get_custom_set_title(photo_set)
            logger.info('Getting photos in set [%s]' % display_title)
            photos = get_photos_in_set(folder)
            logger.info('Found %s photos' % len(photos))

            for photo in sorted(photo_sets[photo_set]):
                # Adds skips
                if cmd_args.ignore_images and photo.split('.').pop().lower() in EXT_IMAGE:
                    continue
                elif cmd_args.ignore_videos and photo.split('.').pop().lower() in EXT_VIDEO:
                    continue

                if photo in photos or is_windows and photo.replace(os.sep, '/') in photos:
                    logger.info('Skipped [%s] already exists in set [%s]' % (photo, display_title))
                else:
                    logger.info('Uploading [%s] to set [%s]' % (photo, display_title))
                    upload_args = {
                        'auth_token': token,
                        # (Optional) The title of the photo.
                        'title': photo,
                        # (Optional) A description of the photo. May contain some limited HTML.
                        'description': folder,
                        # (Optional) Set to 0 for no, 1 for yes. Specifies who can view the photo.
                        'is_public': args['is_public'],
                        'is_friend': args['is_friend'],
                        'is_family': args['is_family'],
                        # (Optional) Set to 1 for Safe, 2 for Moderate, or 3 for Restricted.
                        'safety_level': 1,
                        # (Optional) Set to 1 for Photo, 2 for Screenshot, or 3 for Other.
                        'content_type': 1,
                        # (Optional) Set to 1 to keep the photo in global search results, 2 to hide from public searches.
                        'hidden': 2
                    }

                    file_path = os.path.join(photo_set, photo)
                    file_stat = os.stat(file_path)

                    if file_stat.st_size >= 1073741824:
                        logger.error('Skipped [%s] over size limit' % photo)
                        continue

                    try:
                        upload = api.upload(file_path, None, **upload_args)
                        photo_id = upload.find('photoid').text
                        add_to_photo_set(photo_id, folder)
                        photos[photo] = photo_id
                    except flickrapi.FlickrError as e:
                        logger.error(e.message)
                    except:
                        # todo add tracking to show later which ones failed
                        pass

    logger.info('All Synced')


class WatchEventHandler(FileSystemEventHandler):

    args = None
    sync_path = None

    def __init__(self, args):
        self.args = args
        self.sync_path = self.args.sync_path.rstrip(os.sep)

    def on_created(self, event):
        super(WatchEventHandler, self).on_created(event)

        if not event.is_directory:
            start_sync(self.sync_path + os.sep, self.args, event.src_path)

    def on_moved(self, event):
        super(WatchEventHandler, self).on_moved(event)

        if not event.is_directory and os.path.dirname(event.dest_path).replace(self.sync_path, ''):
            start_sync(self.sync_path + os.sep, self.args, event.dest_path)


def main():
    parser = argparse.ArgumentParser(description='Sync current folder to your flickr account.')
    parser.add_argument('--monitor', action='store_true', help='starts a daemon after sync for monitoring')
    parser.add_argument('--starts-with', type=str, help='only sync that path that starts with')
    parser.add_argument('--download', type=str, help='download the photos from flickr specify a path or . for all')
    parser.add_argument('--ignore-videos', action='store_true', help='ignore video files')
    parser.add_argument('--ignore-images', action='store_true', help='ignore image files')
    parser.add_argument('--version', action='store_true', help='output current version')
    parser.add_argument('--sync-path', type=str, default=os.getcwd(),
                        help='specify the sync folder (default is current dir)')
    parser.add_argument('--custom-set', type=str, help='customize your set name from path with regex')
    parser.add_argument('--custom-set-builder', type=str, help='build your custom set title (default just merge groups)')
    parser.add_argument('--update-custom-set', action='store_true', help='updates your set title from custom set')
    parser.add_argument('--username', type=str, help='token username') #token username argument for api
    parser.add_argument('--keyword', action='append', type=str, help='only upload files matching this keyword')

    args = parser.parse_args()

    if args.version:
        # todo get from setup.cfg
        logger.info('v0.1.17')
        exit()

    start_sync(args.sync_path.rstrip(os.sep) + os.sep, args)

    if args.monitor:
        logger.info('Monitoring [{}]'.format(args.sync_path))
        event_handler = WatchEventHandler(args)
        observer = Observer()
        observer.schedule(event_handler, args.sync_path, recursive=True)
        observer.start()
        try:
            while True:
                time.sleep(1)
        except KeyboardInterrupt:
            observer.stop()
        observer.join()
import sys,os,argparse,json,re,urllib.request,html.parser,flickrapi

EXT_IMAGE = ('jpg', 'png', 'jpeg', 'gif', 'bmp')
EXT_VIDEO = ('avi', 'wmv', 'mov', 'mp4', '3gp', 'ogg', 'ogv', 'mts')

#  flickr api keys
KEY = 'f7da21662566bc773c7c750ddf7030f7'
SECRET = 'c329cdaf44c6d3f3'

def start_sync(sync_path, cmd_args, specific_path=None):
    is_windows = os.name == 'nt'
    is_download = cmd_args.download

    if not os.path.exists(sync_path):
        sys.stderr.write('Sync path does not exists')
        exit(0)

    # Common arguments
    args = {'format': 'json', 'nojsoncallback': 1}
    api = flickrapi.FlickrAPI(KEY, SECRET, cmd_args.username)  # pass username argument to api

    # Ask for permission
    token = api.authenticate_console(perms='write')
    args.update({'auth_token': token})

    # Get default user privacy preferences
    try:
        #Currently not handled in the flickrapi library
        privacypref = api.prefs_getPrivacy().getchildren()[0].attrib['privacy']
        # https://www.flickr.com/services/api/flickr.prefs.getPrivacy.html
    except:
        sys.stderr.write('Failed to get Flickr privacy preferences.')
        exit(1)

    #Default to private
    args.update({
        'is_public': 0,
        'is_friend': 0,
        'is_family': 0
    })

    #Update if so defined in preferences. Note that the privacy number is a string!
    if privacypref == '1':
        args.update({'is_public': 1})
    elif privacypref == '2':
        args.update({'is_friend': 1})
    elif privacypref == '3':
        args.update({'is_family': 1})
    elif privacypref == '4':
        args.update({'is_friend': 1,'is_family': 1})

    # Build your local photo sets
    photo_sets = {}
    skips_root = []
    for r, dirs, files in os.walk(sync_path if not specific_path else os.path.dirname(specific_path)):

        if cmd_args.starts_with and not r.startswith('{}{}'.format(sync_path, cmd_args.starts_with)):
            continue

        files = [f for f in files if not f.startswith('.')]
        dirs[:] = [d for d in dirs if not d.startswith('.')]

        for file in files:
            if not file.startswith('.'):
                ext = file.lower().split('.').pop()
                if ext in EXT_IMAGE or \
                   ext in EXT_VIDEO:

                    if r == sync_path:
                        skips_root.append(file)
                    else:
                        photo_sets.setdefault(r, [])
                        photo_sets[r].append(file)

    if skips_root:
        print('To avoid disorganization on flickr sets root photos are not synced, skipped these photos: %s' % skips_root)
        print('Try to sync at top most level of your photos directory')

    # custom set builder
    def get_custom_set_title(path):
        title = path.split('/').pop()

        if cmd_args.custom_set:
            m = re.match(cmd_args.custom_set, path)
            if m:
                if not cmd_args.custom_set_builder:
                    title = '-'.join(m.groups())
                elif m.groupdict():
                    title = cmd_args.custom_set_builder.format(**m.groupdict())
                else:
                    title = cmd_args.custom_set_builder.format(*m.groups())
        return title

    # Get your photosets online and map it to your local
    html_parser = html.parser.HTMLParser()
    photosets_args = args.copy()
    page = 1
    photo_sets_map = {}

    # Show 3 possibilities
    if cmd_args.custom_set:
        for photo_set in photo_sets:
            print('Set Title: [%s]  Path: [%s]' % (get_custom_set_title(photo_set), photo_set))

        if input('Is this your expected custom set titles (y/n):') != 'y':
            exit(0)

    while True:
        print('Getting photosets page %s' % page)
        photosets_args.update({'page': page, 'per_page': 500})
        sets = json.loads(api.photosets_getList(**photosets_args).decode())
        page += 1
        if not sets['photosets']['photoset']:
            break

        for current_set in sets['photosets']['photoset']:
            # Make sure it's the one from backup format
            desc = str(html_parser.unescape(current_set['description']['_content']))
            if desc:
                photo_sets_map[desc] = current_set['id']
                title = get_custom_set_title(sync_path + desc)
                if cmd_args.update_custom_set and desc in photo_set and title != current_set['title']['_content']:
                    update_args = args.copy()
                    update_args.update({
                        'photoset_id': current_set['id'],
                        'title': title,
                        'description': desc
                    })
                    print('Updating custom title [%s]...' % title)
                    json.loads(api.photosets_editMeta(**update_args).decode())
                    print('done')

    print('Found %s photo sets' % len(photo_sets_map))

    # For adding photo to set
    def add_to_photo_set(photo_id, folder):
        # If photoset not found in online map create it else add photo to it
        # Always upload unix style
        if is_windows:
            folder = folder.replace(os.sep, '/')

        if folder not in photo_sets_map:
            photosets_args = args.copy()
            custom_title = get_custom_set_title(sync_path + folder)
            photosets_args.update({'primary_photo_id': photo_id,
                                   'title': custom_title,
                                   'description': folder})
            photo_set = json.loads(api.photosets_create(**photosets_args).decode())
            photo_sets_map[folder] = photo_set['photoset']['id']
            print('Created set [%s] and added photo' % custom_title)
        else:
            photosets_args = args.copy()
            photosets_args.update({'photoset_id': photo_sets_map.get(folder), 'photo_id': photo_id})
            result = json.loads(api.photosets_addPhoto(**photosets_args).decode())
            if result.get('stat') == 'ok':
                print('Success')
            else:
                sys.stderr.write(result)

    # Get photos in a set
    def get_photos_in_set(folder):
        photos = {}
        # Always upload unix style
        if is_windows:
            folder = folder.replace(os.sep, '/')

        if folder in photo_sets_map:
            photoset_args = args.copy()
            page = 1
            while True:
                photoset_args.update({'photoset_id': photo_sets_map[folder], 'page': page})
                if is_download:
                    photoset_args['extras'] = 'url_o,media'
                page += 1
                photos_in_set = json.loads(api.photosets_getPhotos(**photoset_args).decode())
                if photos_in_set['stat'] != 'ok':
                    break

                for photo in photos_in_set['photoset']['photo']:

                    if is_download and photo.get('media') == 'video':
                        # photo_args = args.copy()
                        # photo_args['photo_id'] = photo['id']
                        # sizes = json.loads(api.photos_getSizes(**photo_args))
                        # if sizes['stat'] != 'ok':
                        #     continue
                        #
                        # original = filter(lambda s: s['label'].startswith('Site') and s['media'] == 'video', sizes['sizes']['size'])
                        # if original:
                        #     photos[photo['title']] = original.pop()['source'].replace('/site/', '/orig/')
                        #     print photos
                        # Skipts download video for now since it doesn't work
                        continue
                    else:
                        photos[photo['title']] = photo['url_o'] if is_download else photo['id']

        return photos

    # If download mode lets skip upload but you can also modify this to your needs
    if is_download:
        # Download to corresponding paths
        os.chdir(sync_path)

        for photo_set in photo_sets_map:
            if photo_set and is_download == '.' or is_download != '.' and photo_set.startswith(is_download):
                folder = photo_set.replace(sync_path, '')
                print('Getting photos in set [%s]' % folder)
                photos = get_photos_in_set(folder)
                # If Uploaded on unix and downloading on windows & vice versa
                if is_windows:
                    folder = folder.replace('/', os.sep)

                if not os.path.isdir(folder):
                    os.makedirs(folder)

                for photo in photos:
                    # Adds skips
                    if cmd_args.ignore_images and photo.split('.').pop().lower() in EXT_IMAGE:
                        continue
                    elif cmd_args.ignore_videos and photo.split('.').pop().lower() in EXT_VIDEO:
                        continue

                    path = os.path.join(folder, photo)
                    if os.path.exists(path):
                        print('Skipped [%s] already downloaded' % path)
                    else:
                        print('Downloading photo [%s]' % path)
                        urllib.request.urlretrieve(photos[photo], os.path.join(sync_path, path))
    else:
        # Loop through all local photo set map and
        # upload photos that does not exists in online map
        for photo_set in sorted(photo_sets):
            folder = photo_set.replace(sync_path, '')
            display_title = get_custom_set_title(photo_set)
            #print(folder,photo_set,display_title)
            print('Getting photos in set [%s]' % display_title)
            photos = get_photos_in_set(folder)
            print('Found %s photos' % len(photos))

            for photo in sorted(photo_sets[photo_set]):
                # Adds skips
                if cmd_args.ignore_images and photo.split('.').pop().lower() in EXT_IMAGE:
                    continue
                elif cmd_args.ignore_videos and photo.split('.').pop().lower() in EXT_VIDEO:
                    continue

                if photo in photos or is_windows and photo.replace(os.sep, '/') in photos:
                    print('Skipped [%s] already exists in set [%s]' % (photo, display_title))
                else:
                    print('Uploading [%s] to set [%s]' % (photo, display_title))
                    upload_args = {
                        'auth_token': token,
                        # (Optional) The title of the photo.
                        'title': photo,
                        # (Optional) A description of the photo. May contain some limited HTML.
                        'description': folder,
                        # (Optional) Set to 0 for no, 1 for yes. Specifies who can view the photo.
                        'is_public': args['is_public'],
                        'is_friend': args['is_friend'],
                        'is_family': args['is_family'],
                        # (Optional) Set to 1 for Safe, 2 for Moderate, or 3 for Restricted.
                        'safety_level': 1,
                        # (Optional) Set to 1 for Photo, 2 for Screenshot, or 3 for Other.
                        'content_type': 1,
                        # (Optional) Set to 1 to keep the photo in global search results, 2 to hide from public searches.
                        'hidden': 2
                    }

                    file_path = os.path.join(photo_set, photo)
                    file_stat = os.stat(file_path)

                    if file_stat.st_size >= 1073741824:
                        sys.stderr.write('Skipped [%s] over size limit' % photo)
                        continue

                    try:
                        upload = api.upload(file_path, None, **upload_args)
                        photo_id = upload.find('photoid').text
                        add_to_photo_set(photo_id, folder)
                        photos[photo] = photo_id
                    except flickrapi.FlickrError as e:
                        sys.stderr.write(e)
                    except:
                        # todo add tracking to show later which ones failed
                        pass

    print('All Synced')

def main():
    parser = argparse.ArgumentParser(description='Sync current folder to your flickr account.')
    parser.add_argument('--starts-with', type=str, help='only sync that path that starts with')
    parser.add_argument('--download', type=str, help='download the photos from flickr specify a path or . for all')
    parser.add_argument('--ignore-videos', action='store_true', help='ignore video files')
    parser.add_argument('--ignore-images', action='store_true', help='ignore image files')
    parser.add_argument('-dir','--directory', type=str, default=os.getcwd(),
                        help='specify the sync folder (default is current dir)')
    parser.add_argument('--custom-set', type=str, help='customize your set name from path with regex')
    parser.add_argument('--custom-set-builder', type=str, help='build your custom set title (default just merge groups)')
    parser.add_argument('--update-custom-set', action='store_true', help='updates your set title from custom set')
    parser.add_argument('--username', type=str, help='token username') #token username argument for api

    args = parser.parse_args()

    start_sync(args.directory.rstrip(os.sep) + os.sep, args)

A flickrsimplesync/__main__.py => flickrsimplesync/__main__.py +4 -0
@@ 0,0 1,4 @@
#!/usr/bin/env python
import __init__

__init__.main()
\ No newline at end of file

D flickrsmartsync/__main__.py => flickrsmartsync/__main__.py +0 -12
@@ 1,12 0,0 @@
import os
import sys

__author__ = 'faisal'


if __name__ == "__main__":
    # Access from source
    sys.path.insert(0, os.sep.join(os.path.dirname(__file__).split(os.sep)[:-1]))

    import flickrsmartsync
    flickrsmartsync.main()
\ No newline at end of file

D import => import +0 -0

D setup.cfg => setup.cfg +0 -3
@@ 1,3 0,0 @@
[egg_info]
tag_build =
tag_svn_revision = true

M setup.py => setup.py +20 -17
@@ 1,34 1,37 @@
#! /usr/bin/env python
from setuptools import setup

from setuptools import setup, find_packages
import sys
import os

VERSION = '0.1.17'
VERSION = '3.0'

with open("README.md", "rb") as f:
    long_descr = f.read()

def main():
    setup(name='flickrsmartsync',
    setup(name='flickrsimplesync',
          version=VERSION,
          description="Sync/backup your photos to flickr easily",
          long_description=open('README.rst').read(),
          long_description=open('README.md').read(),
          classifiers=[
              'Development Status :: 3 - Alpha',
              'Development Status :: 4 - Beta',
              'Environment :: Console',
              'Programming Language :: Python',
              'License :: OSI Approved :: MIT License'
              'Programming Language :: Python :: 3',
              'Programming Language :: Python :: 3.5',
              'License :: OSI Approved :: LGPL License',
              'Topic :: Utilities',
              'Topic :: Multimedia :: Graphics',
              'Operating System :: OS Independent'
          ],
          keywords='flickr backup photo sync',
          author='Faisal Raja',
          author_email='support@altlimit.com',
          url='https://github.com/faisalraja/flickrsmartsync',
          license='MIT',
          packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),
          author='Brent Huisman',
          author_email='mail@brenthuisman.net',
          url='https://github.com/brenthuisman/flickrsimplesync',
          license='LGPL',
          include_package_data=True,
          zip_safe=False,
          install_requires=['watchdog', 'IPTCInfo'],
          install_requires=['flickrapi'],
          packages=['flickrsimplesync'],
          entry_points={
              "console_scripts": ['flickrsmartsync = flickrsmartsync:main'],
              "console_scripts": ['flickrsimplesync = flickrsimplesync:main'],
          },
          )