Python script to bring all files from subfolders to main folder

Posted January 3, 2012 by TechMyWay
Categories: Python, Windows

Tags: , , , , ,

A usual plight with me is bringing all photos I transfer from my phone to one folder. My phone transfer creates a subfolder for each date it has a pic for.

This was till I discovered the module shutil. (No, I knew about it for around _4 months, but I was too lazy to actually write the code 😛 )

This code I have written is meant to be cross-platform. In case of any discrepancies, please do tell.

import shutil
import os

# copy all the files in the subfolders to main folder

# The current working directory
dest_dir = os.getcwd()
# The generator that walks over the folder tree
walker = os.walk(dest_dir)

# the first walk would be the same main directory
# which if processed, is
# redundant
# and raises shutil.Error
# as the file already exists

rem_dirs = walker.next()[1]

for data in walker:
for files in data[2]:
try:
shutil.move(data[0] + os.sep + files, dest_dir)
except shutil.Error:
# still to be on the safe side
continue

# clearing the directories
# from whom we have just removed the files
for dirs in rem_dirs:
shutil.rmtree(dest_dir + os.sep + dirs)


 

Since the code is all documented, I would skip the explanation here.

Please comment, share or like the post. (Good for my writing spirit 🙂 )

Advertisements

Some tricks of Python

Posted December 31, 2011 by TechMyWay
Categories: Python

Tags: , , ,

Zen of Python

First of all, see the Zen of Python 😉

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

List Comprehensions

These are one of the most smart things to happen in python. They build up a list for you pretty simply and in ONE line.

>>> x
[]
>>> for i in range(20):
x.append(10 * i) ## append elements to list b

>>> print x ## use the list
[0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190]

Now this can be done (pretty easily) by:


>>> print [10 * i for i in range(20)]
[0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190]

A better example would be


>>> [i**2 for i in xrange(10) if i%2 == 0]
[0, 4, 16, 36, 64]
## this gives a list containing squares of all even numbers in range of [0, 10)
## means including 0 and not including 10

>>> [ord(i) for i in raw_input()]
1wed342k
[49, 119, 101, 100, 51, 52, 50, 107]

Now, a more Memory efficient solution to above exists with generators:


>>> for j in (i**2 for i in xrange(10) if i%2 == 0):
print j

0
4
16
36
64

Generator Expression

The generator expression do not calculate all the values at the same time (like it was happening in list comprehensions). It calculates values as and when required.
A generator expression is very similar to list comprehensions (just a change in brackets [] -> () )

In python 2.7

dict and set comprehensions

are also provided:


>>> {i: chr(i) for i in range(48, 58)}  ## a dict formed
{48: '0', 49: '1', 50: '2', 51: '3', 52: '4', 53: '5', 54: '6', 55: '7', 56: '8', 57: '9'}
>>> {chr(i) for i in range(48, 58)}     ## a set formed
set(['1', '0', '3', '2', '5', '4', '7', '6', '9', '8'])

P.S. the above code snippets have been pasted from IDLE python shell. If similar statements are used in scripts the results are not printed.

zip function:


>>> print zip.__doc__
zip(seq1 [, seq2 [...]]) -> [(seq1[0], seq2[0] ...), (...)]
Return a list of tuples, where each tuple contains the i-th element
 from each of the argument sequences. The returned list is truncated
 in length to the length of the shortest argument sequence.

The above function can be used to transpose a matrix.!


>>> a = [[1, 2], [4, 3], [5, 6]]
>>> zip(*a)
[(1, 4, 5), (2, 3, 6)]

The (*a) means unpack the list/tuple named a i.e. this operator is valid on both tuples and lists.
It can be used only in function call arguments.
This * is also called the

Splat Operator

😀 .
Dictionaries respond differently and have got an extra operator (**)


>>> a = {'a': 1, 'b': 2, 'c': 3}
>>> def splat(*args):
print args

>>> splat(a)
({'a': 1, 'c': 3, 'b': 2},)
>>> splat(*a)
('a', 'c', 'b')
>>> splat(**a)
Traceback (most recent call last):
File "<pyshell#75>", line 1, in <module>
splat(**a)
TypeError: splat() got an unexpected keyword argument 'a'

>>> def splat(a, b, c):
print a, b, c

>>> splat(a)
Traceback (most recent call last):
File "<pyshell#79>", line 1, in <module>
splat(a)
TypeError: splat() takes exactly 3 arguments (1 given)

>>> splat(*a)
a c b
>>> splat(**a)
1 2 3

Ok, this is something I was mistaking from long.
When you have a list named say my_list and you need its index when looping it, you usually do


>>> my_list = [1, 4, 5, 2, 3, 6]
>>> for i in range(len(my_list)):
print i, my_list[i]

0 1
1 4
2 5
3 2
4 3
5 6

Now, this is something they say “unpythonic” (though i like it 😛 ,  even if its dirty, at least it works 😉 ).

Enumerate

But there is a better way to it:


>>> for index, val in enumerate(my_list):
print index, val

0 1
1 4
2 5
3 2
4 3
5 6

(Ok, now I also agree, it was “unpythonic” 😀 )

Reversing

Now, remember one thing, every slicable type (list and strings to be precise) can be simply reversed by


>>> a
[1, 4, 5, 2, 3, 6]
>>> a[::-1]
[6, 3, 2, 5, 4, 1]
>>> b = "this will be reversed"
>>> b[::-1]
'desrever eb lliw siht'

Though reversed(sequence) is better 😛


>>> import timeit
>>> a = timeit.Timer('a[::-1]', 'a = [2, 4, 1, 6]')
>>> a.timeit()
0.7627712499545058
>>> a = timeit.Timer('reversed(a)', 'a = [2, 4, 1, 6]')
>>> a.timeit()
0.5815445977713054

itertools module

And this is for some testing purposes (my favorite module)
These ones are most used by me to generate tests. You can see other functions in the modules.


>>> import itertools as it
>>> print list(it.combinations('asd', 2))         ## all the combinations
[('a', 's'), ('a', 'd'), ('s', 'd')]
>>> print list(it.combinations_with_replacement('asd', 2)) ## combinations + repeated values
[('a', 'a'), ('a', 's'), ('a', 'd'), ('s', 's'), ('s', 'd'), ('d', 'd')]
>>> print list(it.permutations('asd', 2))         ## permutations
[('a', 's'), ('a', 'd'), ('s', 'a'), ('s', 'd'), ('d', 'a'), ('d', 's')]
>>> print list(it.product('asd', 'def'))          ## product of sequences
[('a', 'd'), ('a', 'e'), ('a', 'f'), ('s', 'd'), ('s', 'e'), ('s', 'f'), ('d', 'd'), ('d', 'e'), ('d', 'f')]

P.S. The above functions actually return an iterator (the one with next() function), but tho show the use, I have put out all the values in the form of a list.

 

This was one of my best finds on internet:


>>> import antigravity

Just give it a try 😉

 

And at last, something to laugh.. 😀


>>> from __future__ import braces
SyntaxError: not a chance (<pyshell#106>, line 1)

 

Do remember to comment, like or share the post. 🙂

N – QUEEN Backtracking problem solved

Posted November 9, 2011 by TechMyWay
Categories: C++

Tags: , , ,

This all started when I started doubting my coding and downloaded the code from geeksforgeeks for this N-Queen problem. I told one of my friends that I am gonna read it and understand. Then he told me that the problem is quite easy to code, just see the wiki (actually he and me were interested more in the animation on that page 😉

Anyhow the wiki pages : Eight Queen , Backtracking were not of much use as they were not giving good pseudo code. But this here, see it and try to think, what all can I need for the N – Queen problem..

Enough of waiting now, here is the C++ code for the problem I wrote.

#include<iostream>
using namespace std;

/** this is the size of chess board */
#define N 8

/** Given a completed board, this prints it to the stdout */
void print_solution(int state[N][N])
{
int i,j;
for(i=0; i<N; ++i)
{
for(j=0; j<N; ++j)
cout<<state[i][j]<<" ";
cout<<endl;
}
cout<<endl;
}

/** return true if placing a queen on [row][col] is acceptable
else return false */
bool accept(int state[N][N], int row, int col)
{
int i,j;

/* check the column */
for (i=0; i<N; ++i)
{
if (state[row][i])
return false;
}

/* check the row */
for (i=0; i<N; ++i)
{
if (state[i][col])
return false;
}

/* check the upper left diagnol */
for (i = row, j = col; i >= 0 && j >= 0; i--, j--)
{
if (state[i][j])
return false;
}

/* check the lower left diagnol */
for (i = row, j = col; i < N && j >= 0; ++i, --j)
{
if (state[i][j])
return false;
}

/* check the upper right diagnol */
for (i = row, j = col; i >= 0 && j < N; i--, ++j)
{
if (state[i][j])
return false;
}

/* check the lower right diagnol */
for (i = row, j = col; i < N && j < N; ++i, ++j)
{
if (state[i][j])
return false;
}

/* return true if all tests passed */
return true;
}

void solve_state(int state[N][N], int row)
{
int i;

/* if all queens have been placed at non conflicting positions */
if (row == N)
{
print_solution(state);
return;
}

/* Place queens on all positions in a given row and
check if the state is acceptable
Continue the process till all possibilities found */
for(i=0; i<N; ++i)
{
if(accept(state, row, i))
{
state[row][i] = 1;
solve_state(state, row+1);
}
state[row][i] = 0;
}
}

int main()
{
/* initialise the board */
int state[N][N] = {0};

solve_state (state, 0);

return 0;
}

This code can be easily converted to C, just make amendments to bool values. Rest of the program deals with the board matrix, so C won’t have problems.

Result: Backtracking is pretty easy if you get the idea.

The new dropbox API

Posted October 22, 2011 by TechMyWay
Categories: Internet, Python

Tags: , , ,

Just going through the feeds I found out that Dropbox had given out new APIs. Luckily, they had better support for python this time.

Going along, I have written a script (or an introduction you can say) to these APIs in Python.

I have tried to document the code as much as possible.

do remember to install oauth, setuptools, simplejson as these are not included by default in Python 2.7 installation


## author: Ayush Goel
## Python 2.7 used
## get the Dropbox new API from https://www.dropbox.com/developers/
## mail: ayushgoel111@gmail.com

## do remember to install oauth, setuptools, simplejson
## not included by default in Python 2.7 installation
## http://pypi.python.org/pypi/setuptools
## http://pypi.python.org/pypi/oauth/1.0.1
## http://pypi.python.org/pypi/simplejson/

# Include the Dropbox SDK libraries
from dropbox import client, rest, session

# Get your app key and secret from the Dropbox developer website
APP_KEY =    'INSERT_APP_KEY_HERE'
APP_SECRET =  'INSERT_SECRET_HERE'

# ACCESS_TYPE should be 'dropbox' or 'app_folder' as configured for your app
ACCESS_TYPE = "app_folder" #'INSERT_ACCESS_TYPE_HERE'
sess = session.DropboxSession(APP_KEY, APP_SECRET, ACCESS_TYPE)

request_token = sess.obtain_request_token()

while True:
  url = sess.build_authorize_url(request_token)
  print "url:", url
  import webbrowser
  webbrowser.open_new_tab(url)
  print "You have been redirected to the authorization page."
  print "Please do the authorization within 5 minutes else the URL would expire."
  print "Press ENTER here once you are done. To create the url again, enter any character"
  #print "Please visit this website and press the 'Allow' button, then hit 'Enter' here."
  s=raw_input()
  if s=='':
    break

# This will fail if the user didn't visit the above URL and hit 'Allow'
access_token = sess.obtain_access_token(request_token)

client = client.DropboxClient(sess)
print "linked account:", client.account_info()

## The client object is what is required for the whole
## app buildup by anyone

## anyways, lets have a look at some things of interest of
## all those tokens we just saw

## the url we produced above
print  url
#'https://www.dropbox.com/1/oauth/authorize?oauth_token=n8yyjthf92hv1g5'

print sess.API_CONTENT_HOST
# 'api-content.dropbox.com'
print sess.API_HOST
# 'api.dropbox.com'
print sess.API_VERSION
# 1
print  sess.WEB_HOST
# 'www.dropbox.com'
print  sess.is_linked()
# True
print  sess.locale
# nothing None
print  sess.root
# 'sandbox'
print  sess.signature_method.get_name()
# 'PLAINTEXT'

## Very important to NOTICE: every token we generated
## has two unique identifiers (key, secret)

print  request_token.key
# 'n8yyjthdgff92hvasafd1g5'
print  request_token.secret
# 'qu1dfozfgafeg1hmwrijwum'
print  request_token.verifier
# None
print  access_token.key
# 'd2rdsfjjfgzgd8hwc3j9kiiof'
print  access_token.secret
# 'wfsdfjwglhho0odek2jqby44'

print  access_token.callback_confirmed
# None
print  access_token.get_callback_url()
# None

## file_create_folder() creates a folder in the app folder given to you.
## The folder name is passed as an argument
## a dict is returned giving details of the folder
## errors are raised otherwie (see the documentation)

print client.file_create_folder("folder1")
##        {
##        u'size': u'0 bytes',
##        u'rev': u'104721f34',
##        u'thumb_exists': False,
##        u'bytes': 0,
##        u'modified': u'Fri, 21 Oct 2011 19:48:25 +0000',
##        u'path': u'/folder1',
##        u'is_dir': True,             # memory management
##        u'icon': u'folder',
##        u'root': u'app_folder',      # authotization
##        u'revision': 1
##        }

## client.put_file("C:\Users\Ayush\Desktop\dropify.txt","folder1/")
##
##Traceback (most recent call last):
##  File "<pyshell#31>", line 1, in <module>
##    client.put_file("C:\Users\Ayush\Desktop\dropify.txt","folder1/")
##  File "C:\Users\Ayush\Applications\python_pkg\dropbox-python-sdk-1.2\dropbox-1.2\dropbox\client.py", line 147, in put_file
##    return RESTClient.PUT(url, file_obj, headers)
##  File "C:\Users\Ayush\Applications\python_pkg\dropbox-python-sdk-1.2\dropbox-1.2\dropbox\rest.py", line 142, in PUT
##    return cls.request("PUT", url, body=body, headers=headers, raw_response=raw_response)
##  File "C:\Users\Ayush\Applications\python_pkg\dropbox-python-sdk-1.2\dropbox-1.2\dropbox\rest.py", line 109, in request
##    raise ErrorResponse(r)
##ErrorResponse: [400] {u'path': u"Path 'C:\\Users\\Ayush\\Desktop\\dropify.txt' can't contain \\"}

## correct way of uploading a file
f=open("C:\Users\Ayush\Desktop\dropify.txt")
print client.put_file("folder1/dropify.txt",f)

##        {
##        u'size': u'34 bytes',
##        u'rev': u'204721f34',
##        u'humb_exists': False,
##        u'bytes': 34,
##        u'modified': u'Fri, 21 Oct 2011 19:57:39 +0000',
##        u'path': u'/folder1 (1)',
##        u'is_dir': False,
##        u'icon': u'page_white',
##        u'root': u'app_folder',
##        u'mime_type': u'application/octet-stream',
##        u'revision': 2
##        }

##read a file from dropbox
## we are actually reading the same file we just
## uploaded. It's a text document
a=client.get_file("folder1/dropify.txt")

print a.fileno()
# 588

## get headers of the file we are reading
## a list of tuples is returned
## difference between tuples and lists will be covered later

print a.getheaders()
##        [
##        ('content-length', '34'),
##        ('accept-ranges', 'bytes'),
##        ('server', 'dbws'),
##        ('connection', 'keep-alive'),
##        ('etag', '2n'),
##        ('pragma', 'public'),
##        ('cache-control', 'max-age=0'),
##        ('date', 'Fri, 21 Oct 2011 19:59:28 GMT'),
##        ('content-type', 'text/plain; charset=ascii')
##        ]

print a.read()
## the contents
## please don't do this on your "big files"
## may slow down or clog your app as memory requirements would go very high

## some additional data about the file
print a.reason
# 'OK'
print a.status
# 200
print  a.strict
# 0
print  a.version
# 11
print  a.chunk_left
# 'UNKNOWN'
print  a.chunked
# 0
print  a.begin()
# None
print  a.read()
# ''

## another list of headers
s=a.getheaders()

# I am actually printing the headers beautifully 🙂
for i in s:
print "%15s%s%20s"%(i[0]," : ", i[1])

## content-length :                   34
##  accept-ranges :                bytes
##         server :                 dbws
##     connection :           keep-alive
##           etag :                   2n
##         pragma :               public
##  cache-control :            max-age=0
##           date : Fri, 21 Oct 2011 19:59:28 GMT
##   content-type : text/plain; charset=ascii

print  client.metadata('/')
##{
##        u'hash': u'00d3e63a8e91467dddaf18d04b206e57',
##        u'thumb_exists': False,
##        u'bytes': 0,
##        u'path': u'/',
##        u'is_dir': True,
##        u'icon': u'folder',
##        u'root': u'app_folder', u
##        'contents': [
##                {
##                        u'size': u'0 bytes',
##                        u'rev': u'104721f34',
##                        u'thumb_exists': False,
##                        u'bytes': 0,
##                        u'modified': u'Fri, 21 Oct 2011 19:48:25 +0000',
##                        u'path': u'/folder1',
##                        u'is_dir': True,
##                        u'icon': u'folder',
##                        u'root': u'dropbox',
##                        u'revision': 1
##                },
##                {
##                        u'size': u'34 bytes',
##                        u'rev': u'204721f34',
##                        u'thumb_exists': False,
##                        u'bytes': 34,
##                        u'modified': u'Fri, 21 Oct 2011 19:57:39 +0000',
##                        u'path': u'/folder1',
##                        u'is_dir': False,
##                        u'icon': u'page_white',
##                        u'root': u'dropbox',
##                        u'mime_type': u'application/octet-stream',
##                        u'revision': 2
##                }
##        ],
##        u'size': u'0 bytes'
##}

And yes, I have tested this on my machine, so I am sure it is working..

So, put your coding caps on and go get your own keys from dropbox.

And yes, don’t worry about the .key and .secret , they are scrambled and tempered with.. 😉

Retrieving files from URLs

Posted October 20, 2011 by TechMyWay
Categories: Internet, Python, Windows

Tags: , , , ,

This script was writeen by me way long back. I documented it a little so that it’s easy to understand what it’s trying to do.

## python 3.x compliant
## author: Ayush Goel

import urllib.request as ur

file_url=input('Enter the file URL you want to be downloaded: ')
file_name=input('Enter the path where you want the file to be saved(/enter): ')

if file_name=='':
## if no location provided, we get ourselves a default one

## change the location of download as suited for you
## this one worked on my Win7 machine
file_name='C:\\Users\\Ayush\\Downloads'+file_url.split('/')[-1]

try:
## try to retrieve the file using the URL
ur.urlretrieve(file_url,filename=file_name)

except ur.URLError:
## urls like : "edoc.ub.uni-muenchen.de/7505/1/Fischer_Johannes.pdf"
## headers like http:// https:// are missing
print ("The URL is parsed to be incorrect.. please provide with the complete url, including the protocol name (http,https..)")

except IOError:
## the url given ain't to a file.
## It might be a forwarding URL, we would need the actual file url
print("We are facing issues with the url you provided")

I have included some error issues. If you find any, comment here or PM me.

Hootsuite is amazing..

Posted August 27, 2011 by TechMyWay
Categories: Uncategorized


Hootsuite is amazing..

Python script to get all the distinct substrings from a string

Posted August 11, 2011 by TechMyWay
Categories: Python

Tags: , , ,

Recently I came across an interview question:

Print out all the substrings of a given string.?

Now, the catch in the question is for strings like “ababc”. If you are blindly adding the substrings to a storage you have, you will end up with copies of “ab”, which is not what we want. Now, what is the best method to avoid duplicates..??
Ans: hash tables.. 😉

Thus, using the power of Python (i.e. the sets available in python) I have written the following script which takes the string as an input and returns the set of the substrings.

def substr(string):
    j=1
    a=set()
    while True:
        for i in range(len(string)-j+1):
            a.add(string[i:i+j])
        if j==len(string):
            break
        j+=1
        #string=string[1:]
    return a

Also, to print the set sequentially, we can easily convert the returned set to an ordered list. For example, here I have sorted the list based on the length of the substrings.

def prnt(seta):
    l=list(seta)
    l.sort(key=lambda(x):len(x))
    print l
    ## return l # if you want to return this object too

The normal execution on the commandline looks like this:

>>> x
 set(['a', 'ayus', 'h', 'yus', 'ayush', 'us', 'ush', 'ay', 's', 'sh', 'u', 'yush', 'y', 'yu', 'ayu'])
>>> prnt(x)
 ['a', 'h', 's', 'u', 'y', 'us', 'ay', 'sh', 'yu', 'yus', 'ush', 'ayu', 'ayus', 'yush', 'ayush']

Complexities:
time O(n**2) ## worst but this method provides us with the substrings also
memory O(2**n) ## worst when all characters in the string are different


%d bloggers like this: