Archive
Tags
android (3)
ant (2)
beautifulsoup (1)
debian (1)
decorators (1)
django (9)
dovecot (1)
encryption (1)
fix (4)
gotcha (2)
hobo (1)
htmlparser (1)
imaplib (2)
java (1)
json (2)
kerberos (2)
linux (7)
lxml (5)
markdown (4)
mechanize (6)
multiprocessing (1)
mysql (2)
nagios (2)
new_features (3)
open_source (5)
optparse (2)
parsing (1)
perl (2)
postgres (1)
preseed (1)
pxe (4)
pyqt4 (1)
python (41)
raid (1)
rails (1)
red_hat (1)
reportlab (4)
request_tracker (2)
rt (2)
ruby (1)
scala (1)
screen_scraping (7)
shell_scripting (8)
soap (1)
solaris (3)
sql (2)
sqlalchemy (2)
tips_and_tricks (1)
twitter (2)
ubuntu (1)
vmware (2)
windows (1)
zimbra (2)

I have been writing an IMAP archival utility and one of the obstacles was getting PLAIN authentication to work. I know it does not sound complicated. PLAIN authentication allows for multiple identities, which allows a master user to access the mailbox of a regular user using the master user's password. This is vital to my utility as I do not want to keep track of the passwords of each user in the system. This was only a challenge because imaplib has close to no documentation. After sifting through the IMAP specifications and reading the code of imapsync, I have a working example!

1
2
3
4
5
6
7
def login_plain(imap, user, password, authuser=None):
    def plain_callback(response):
        if authuser is None:
            return "%s\x00%s\x00%s" % (user, user, password)
        else:
            return "%s\x00%s\x00%s" % (user, authuser, password)
    return imap.authenticate('PLAIN', plain_callback)

The imaplib.IMAP4 authenticate method has a weird way of working. It take two arguments. The first is a string declaring the method to use, PLAIN in this case. The second is a callable that returns a string or None. What should that string contain though? It is different for everything and you have to read specifications to know what is needed. PLAIN takes a null, \x00, delimited string of three fields. The first field is the authorization identity whose mailbox will be accessed. The second is the authentication identity whose credentials will be used. The third is the password. Pretty easy after that!!! Here is an example of how to use this:

import imaplib
imapconn = imaplib.IMAP4_SSL(host1, 993) # Non-SSL works too
# Login as tlesmann using adminuser's credentials    
login_plain(imapconn, 'tlesmann', 'adminuser', 'adminuserspassword')

This is how imaplib should work and now it will for you.

Posted by Tyler Lesmann on April 15, 2009 at 15:33
Tagged as: imaplib python

I had a recent problem at work. We needed to copy a folder in one IMAP mailbox to another. It wasn't as easy as it should have been. The folder in question is just below 15GB. The obvious route is to copy it using a mail client. The first one I tried was Evolution 2.24. Evolution does copy folders, but it only copies the mail out of folders you've opened. This is a problem with 3500+ folders. I'll probably file a bug with GNOME about this. The next client was Kmail.
Kmail 3.5.10 does the copying perfectly...until is crashes. Kmail is pathetic in the realm of stability. The KDE team needs to make Kmail more resilient. It should display an error instead of dying. I didn't have a Windows machine available to try Outlook Express on or I would have tried that too.

Since everything seems to have trouble with folders this size, I decided to try my hand at the task with Python. I ended up with this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
#!/usr/bin/env python

import getpass
import imaplib
import sys

def cpimap(user1, host1, pass1, user2, host2, pass2, target='/'):
    m1 = imaplib.IMAP4_SSL(host1, 993)
    m2 = imaplib.IMAP4_SSL(host2, 993)
    m1.login(user1,pass1)
    m2.login(user2,pass2)

    folders = [folder.split(' "/" ')[1][1:-1] for folder in m1.list(target)[1]]

    folders.insert(0, target) # Copy messages in the root of the target too

    print 'Copying', len(folders), 'folders...'

    for f in folders:
        if '\\' in f:
            print 'Skipping', f
            # imaplib does not support backslashes in mailbox names!
            continue
        print 'Copying', f
        m2.create(f)
        m1.select(f)
        print 'Fetching messages...'
        typ, data = m1.search(None, 'ALL')

        msgs = data[0].split()

        sys.stdout.write(" ".join(['Copying', str(len(msgs)), 'messages']))

        for num in msgs:
            typ, data = m1.fetch(num, '(RFC822)')
            sys.stdout.write('.')
            m2.append(f, None, None, data[0][1])
        sys.stdout.write('\n')

If you look through the code, you may notice that imaplib is a messy module to work with. At the very least, the user of imaplib doesn't need to know exactly how the IMAP protocol works. There is a lot more output that what is needed. With a few tutorials, like this one, you can get a usable piece of code in a few minutes.

One limitation of imaplib is operations on folders containing backslashes. The module does not properly escape them. If you try to do anything with a folder named back\slash, then it will try to apply the procedure on back\\slash.This is a bug in imaplib.

Posted by Tyler Lesmann on October 10, 2008 at 12:39
Tagged as: imaplib python