In certain situations particularly when dealing with networks and distributed systems you'd want your program to retry certain a particular operation certain number of times in case it fails before giving up. For instance your program is trying to connect to a mail server which is due to network connectivity temporarily giving socket error.

The simple way is to put a loop, try certain number of times and if the result still fails give up. I found this pattern normal so I wrote a decorator in Python which can be generically used.

def attempt_retries(func, retries=3, delay=3, IgnoreException=Exception):
    """
    Decorator for ignoring certain exception for certain number times and retrying with certain delay
    e.g.   func = @retry(connect_imap, 10, 5, SocketError) # Tries ten times
    e.g.2. retry(connect_imap)(username, password)
    """
    def dec(*args, **kwargs):
        for i in range(retries-1):
            try:
                return func(*args, **kwargs)
            except IgnoreException:
                sleep(delay)
        func(*args, **kwargs)
    return dec

Usage:

attempt_retries( server_connect, 10, 20, SocketError ) ( host, user, pass)

or

r_server_connect = attempt_retries( server_connect, 10, 50, SocketError )
r_server_connect( host, user, pass)

Sometimes you want your File Transfer tool to map certain folders on your dev machine to remote folders on your different remote machines. Wouldn't it be great if you could choose a particular file in your project, click it and it automatically uploads in appropriate folder on desired server?

For instance I have an XP dev machine with a project in folder "D:\workspace\saima" and another in "D:\workspace\ismspk". I would like all files in any sub-directory in "saima" to upload on my server "saima" in the appropriate sub-directory in "/home/saima/workspace/". Same goes for rest of my projects and servers. Of course I'd like to do it with a simple click rather than choosing the sessions and folders in my FTP client every time I make a change in a file.

I tried such an option with FileZilla and WinSCP but couldn't find any such option (maybe there exists one). So I thought about writing my own. Since I'm running XP with RSA keys setup with pageant, pscp (Putty SCP Client) was a good choice. Though other command-line utilities such as rsync etc. may do the job pretty as well.

I wrote a small Python script which I linked to my "Send To" menu. To add something in your Send-To menu, goto run, type "sendto" and create a new link here. I created a link to my Python Script and named it "Upload".

UPDATE: For Windows 7, type "shell:sendto" in your explorer bar to make a Send-To shortcut

Here is how it looks like

Here is the Python script I wrote

"""
Author: Sharjeel Ahmed Qureshi
Description: Script for uploading files via pscp
You need to setup putty sessions and your RSA keys first.
Make sure that pageant is running and pscp is in your PATH variable.
Works for windows only
You are free to use this script anyway you like
"""
 
from os import path, popen
import sys
 
import logging
logger = logging
logger.basicConfig(level=logging.DEBUG)
 
FILES_TO_UPLOAD = sys.argv[1:]
 
# PATH_RULES is a config variable which is list of path rules
# Each path rule is a list of three members:
#    ['Local Drive + Directory', 'putty session name', 'remote directory']
# e.g.
# PATH_RULES = [
#    ['d:/workspace/saima/', 'saima', '/home/saima/workspace/'],
#    ['d:/workspace/smsfriends/', 'facebooksms', '/home/fbsms/public_html/www/smsfriends/'], ]
 
try:
    PATH_RULES = [
        # ['localdir', 'puttysession', 'remote_dir'],
        ]
 
    for i in PATH_RULES: i[0] = path.abspath(i[0]).lower()
    if not PATH_RULES: raise Exception("No rules defined!")
except Exception, e:
    logging.exception("There was an loading the config. Check your rules: %s" % e.message)
 
def err(msg):
    logging.error(msg)
    sys.stderr.write(msg + '\n')
 
def match_server(filename):
    """ Gets matching server for a given filename including fullpath """
    global PATH_RULES
    filename = path.abspath(filename).lower()
    for r in PATH_RULES:
        if path.dirname( filename ).startswith( r[0] ):
            psession = r[1]
            r_path = r[2] + filename.rsplit(r[0],1)[1].replace('\\', '/')
            return (psession, r_path)
    return None
 
def upload(filename):
 
    if path.isdir(filename):
        err("Directory Upload is currently not supported")
 
    server = match_server(filename)
    if not server:
        err("No rule available about uploading this file")
        return
    psession, r_path = server
    cmd = "pscp %s %s:%s" % (filename, psession, r_path)
    # logger.debug("Executing command %s" % cmd)
    print cmd
    print popen(cmd).read()
 
for f in FILES_TO_UPLOAD:
    try:
        upload(f)
    except:
        logging.exception("Error while uploading %s" % f)
 
raw_input("Press ENTER to Continue")

I use this small utility function for debugging certain Python scripts, especially those for processing data, e.g. moving some bits of information to Excel. The idea is pretty simple. Just pass it some information and it'll be available in your clipboard.

import sys
import win32clipboard as wc
import win32con
 
def copy_to_clipboard(msg):
   if sys.platform == 'win32':
      wc.OpenClipboard()
      wc.EmptyClipboard()
      wc.SetClipboardData(win32con.CF_TEXT, msg)
      wc.CloseClipboard()

It works on Win32 and you need CTypes installed.

Many great hackers and programmers do not consider Java programming language a good choice. The safety features of Java such as pure OO, strong typing etc. force its users to laboriously write lots of classes, declare and typecast variables, call extraneous methods to fetch simple data and puts under a lot of restrictions. Such techniques might be good to prevent a mediocre coder from making blunders and protecting him from shooting himself in the foot. However for great hackers these things only take away the freedom to freely convert their thought process into code and make effective use of their time. The extra safeties are considered distractions which force the users to focus on implementation rather than the logic. However it is a fact that many good programmers do use Java as their main tool. Some of them use it because they’ve never been exposed to great stuff such as Lisp, Ruby or Python. Others are those who have been forced to use it by their Pointy-Haired-Bosses or unavailability of a good library. I want to help out both of these types of good programmers. Recently I went through a similar situation where I could not find a library except in Java. I thought being forced to Java is going to be a nightmare for me but I found a fantastic work around which lets you use power of great languages while staying in the Java platform. I wanted to use a good library for interfacing with my GSM Modem. The only reasonable library I could find was SMSLib. Bad day: it was written in Java. I tried to play around with it for a while by the standard hit & trial methods of writing small code snippets with many printf’s and compiling and running to see the result. But I was soon lost in memorizing dozens of classes, pointless getters and setters and other irrelevant stuff I did not need at all to implement my logic. Before banging my head against the wall and rewriting the whole library in C with interfaces to Python and Lisp, I decided to give a try to one of the scripting languages for Java platform. I’d heard about Jython, Rhino, Grooby and JRuby. Since my favorite language is Python, I chose Jython for the experiment. I downloaded Jython from Jython.org. Its installation was a charm. Double clicking the downloaded JAR did everything for me including writing a batch file for setting up environment variables. Running the batch files gives Jython’s shell. The shell is exactly like Python’s shell except that it runs in Java and can run Python code as well as uses Java’s libraries and classes. First thing to do for any programming experiment: Print Hello World. How do I do that? What? Don’t I have to make a separate class with a static void type main method which instantiates printer factory, inherited by abstract factory to take a message as argument and forward it to system.out.println and use that factory to print “Hello World”? No! Because you are still using Java’s platform, but on top of it instead of Java language you are using one of the most powerful languages. Before moving on I just wanted to make sure if I still had the power of Python. So I tried writing a MD5 reverser in three lines which takes a file containing list of commonly used passwords. Beautiful! Lets try importing Java classes. Cool! I can now use all classes of Java with Python’s syntax. Let’s try SMSLib with Jython. All I had to do was put SMSLib’s Jar in CLASSPATH. You can introspect methods while you are experimenting giving commands. This effectively means that you can debug, code and execute all at the same time. In the interactive mode the state of execution is maintained as you give command so you can try out small things without regenerating the whole scenario! By trying out small things I was able to easily figure out how to use different stuff of SmsLib. Afterwards I copy pasted all the relevant commands I executed in a separate .jy file and made small changes to carve out my required code. Jython made my day! Not only just Python, but you can also use Ruby and Javascript over Java platform using JRuby and Rhino. There is also Groovy, another dynamic language specifically designed for Java. These languages make programming with Java platform much more powerful. By power, I mean the ability to code conveniently and timely without getting frustrated. So the next time your Pointy-Haired-Boss tells you that the next project is to be done in Java, don’t argue; just include an extra JAR of your favorite powerful language in your Java project. PS: Python and Ruby are also available for .net but I haven’t tried them out.