This is the documentation for getmail version 4. Version 4 includes numerous changes from version 3.x; if you are using getmail version 3, please refer to the documentation included with that version of the software.
getmail is Copyright © 1998-2004 Charles Cazabon.
getmail is licensed under the GNU General Public License version 2 (only). If you wish to obtain a license to distribute getmail under other terms, please contact me directly.
The following questions about getmail are answered more-or-less frequently. Please also read the unexpected behaviour section of the troubleshooting document.
getmail is a mail retriever with support for POP3, POP3-over-SSL, IMAP4, IMAP4-over-SSL, and SDPS mail accounts. It supports normal single-user mail accounts and multidrop (domain) mailboxes. getmail is written in Python, and licensed under the GNU General Public License version 2.
getmail runs on basically any platform. It's designed to, and written in a language that helps to maintain cross-platform compatibility. getmail is known to run on the following platforms:
But getmail will also run on other, less common platforms. The only real requirement is that Python run on that platform, and porting Python is generally very easy.
getmail is Free Software. As such, it comes with no warranty. However, I will do my best to support getmail on a voluntary basis through the getmail mailing list.
If you are using getmail in a commercial or other environment where problems cost money, consider contacting me privately for commercial support, by emailing <getmail-support @ discworld.dyndns.org>
If you have questions about getmail, the first step is to read the documentation, and the remainder of the Frequently Asked Questions. If your question isn't answered there, search the getmail mailing list archives.
If you still haven't found an answer to your question, please subscribe to the getmail users' mailing list by sending a blank email to <getmail-subscribe @ discworld.dnsalias.org>. If you post your question there, I will see it. As an additional bonus, your question may be answered by another member of the list.
First, make sure that you are running the latest version. You can always
find what is the latest version by checking this page at the original
web site:
http://www.qcc.ca/~charlesc/software/getmail-4/.
If you running an older version of the software, chances are whatever bug
you may have found has already been fixed.
Ideally, you should join the mailing list and send your bug report there. You should include the following information:
If you absolutely cannot sign up for the mailing list, send the report to me directly at <getmail-bugs @ discworld.dyndns.org>. I may not be able to respond to all reports privately, but I will try to address any bugs I find out about this way.
Yes. You can use getmail message filtering options to do this with an external virus scanning program, or invoke your virus scanning program during delivery with getmail's support for external MDAs.
Also see the FAQ about using getmail with the ClamAV program.
Yes. You can use getmail message filtering options to do this with an external spam filtering program, or invoke your spam filtering program during delivery with getmail's support for external MDAs.
Also see the FAQ about using getmail with the SpamAssassin program.
Yes. getmail has built in support for POP3-over-SSL and IMAP4-over-SSL.
No. Rewriting message header fields is bad for many reasons; the biggest problem is that it causes a loss of critical technical information necessary to track down many mail problems. getmail will add a new Received: header field and a new Delivered-To: header field, but does not rewrite existing headers. You can disable the creation of these header fields.
Yes. getmail version 4 uses exactly the same oldmail- naming convention for its oldmail files. The only difference is that version 4 escapes a couple of additional characters in this string so that it is truly cross-platform compatible. If you upgrade from version 3 to version 4, getmail will still remember which messages you've already retrieved. - -
To upgrade, do the following:
That's it.
Short answer: … well, the short answer is mostly unprintable. The long answer is … well, long:
I do not like some of the design choices which were made with fetchmail. getmail does things a little differently, and for my purposes, better. In addition, most people find getmail easier to configure and use than fetchmail. Perhaps most importantly, getmail goes to great lengths to ensure that mail is never lost, while fetchmail (in its default configuration) frequently loses mail, causes mail loops, bounces legitimate messages, and causes many other problems.
When people have pointed out problems in fetchmail's design and implementation, it's maintainer has frequently ignored them, or (worse yet) gone in the completely wrong direction in the name of "fixing" the problems. For instance, fetchmail's configuration file syntax has been criticized as being needlessly difficult to write; instead of cleaning up the syntax, the maintainer instead included a GUI configuration-file-writing program, leading to comments like:
The punchline is that fetchmail sucks, even if it does have giddily-engineered whizbang configurator apps.
The maintainer also ignored dozens of complaints about fetchmail's behaviour, stating (by fiat) that fetchmail was bug-free and had entered "maintenance mode", allowing him to ignore further bug reports.
fetchmail's default configuration values frequently cause lost or misdirected mail, and seem to be chosen to cause maximum pain and inconvenience. From fetchmail's to-do file (emphasis mine):
Maybe refuse multidrop configuration unless "envelope" is _explicitly_ configured … This would prevent a significant class of shoot-self-in-foot problems.
perhaps treat a delivery as "temporarily failed" … This is so you don't lose mail if you configure the wrong envelope header.
fetchmail is famous for mangling messages it retrieves, rather than leaving them alone as a mail-handling program should. getmail will add trace information to messages (so you can see what happened, and when), but will otherwise leave message content alone.
In addition, fetchmail has a long history of security problems:
As of July, 2004, there may be at least 2 unfixed denial-of-service attacks, 2 unfixed remote-code-execution, 2 unfixed remote-user-access, and 3 unfixed remote-shell attacks against fetchmail. See http://www.mail-archive.com/euglug@euglug.org/msg00971.html for details
I've given up even trying to stay abreast of the various security holes in fetchmail.
But don't just take my word for it; see http://docs.freebsd.org/cgi/mid.cgi?200102172349.QAA11724 and http://esr.1accesshost.com/.
getmail users have not had to worry about any of these security holes or design and implementation errors.
A domain (or multidrop) mailbox is a POP3 mailbox which receives mail for all users in a given domain. Normal mailboxes contain mail for a single user (like jason@myisp.co.uk); some Internet Service Providers which provide webhosting or other services will provide a POP3 mailbox which receives mail for all addresses in a given domain (i.e. mail for service@smallcompany.net, sales@smallcompany.net, and indeed anything @smallcompany.net ends up in the same POP3 mailbox).
getmail provides a method of retrieving mail from a domain mailbox and distributing it among the various users automatically. The retriever classes MultidropPOP3Retriever, MultidropPOP3SSLRetriever, MultidropSDPSRetriever, MultidropIMAPRetriever, and MultidropIMAPSSLRetriever provide this capability.
No. getmail delivers directly to maildirs, mboxrd files, or via arbitrary MDAs, and never injects mail via SMTP, so no MTA is necessary.
No. When run as the root user on a Unix-like system, getmail drops privileges (switches to an unprivileged group and user id) before delivering to maildirs or mboxrd files. You can specify the user explicitly, or let getmail use the owner of the maildir or mboxrd file.
A maildir is a mail storage format invented by D. J. Bernstein (author of qmail) that requires no file locking to deliver to safely and reliably, even over NFS. getmail natively supports delivery to maildirs.
See http://qmail.org/man/man5/maildir.html and http://cr.yp.to/proto/maildir.html for details.
There are various sub-types of the mbox mail storage format. mboxrd is the most reliable of them, though (like all mbox types) it still relies on file locking and is therefore more easily corrupted than maildir format. In particular, using mbox files with multiple writers over NFS can be problematic.
For details on the differences between the various mbox sub-types, see http://qmail.org/man/man5/mbox.html.
The "envelope" of an email message is "message metadata"; that is, the message is information, and the envelope is information about the message (information about other information). Knowing this is critical to understanding what a domain or multidrop mailbox is, how it works, and what getmail can do for you.
Others have tried to explain this with varying degrees of success. I'll use the standard analogy of normal postal (i.e. non-electronic) mail:
When you receive a letter (a reply from the customer-disservice department of your telephone company, say) it arrives in an envelope. You tear it open, remove the letter, and read it. At the top of the letter is the telephone company's return address, followed by the date the letter was written. Your name and mailing address follow that, and then the remainder of the letter.
The important thing to keep in mind is that the contents of the letter (including the addresses just discussed) are never looked at by the post office. If they can't deliver the letter (your mailing address on the envelope got smudged in the rain), they'll return it to the address listed in the top-left corner of the envelope. They don't check to make sure that the address listed there is the same as the one listed at the top of the letter. Similarly, when they can successfully deliver it, they don't check to make sure that the recipient name and address on the envelope matches the one listed on the letter between the date and the salutation.
The message header fields From: and Resent-from: are equivalent to the block of address information at the top of the letter; it usually contains the name and address of the sender of the message, but it is never actually used in the delivery of the message. Similarly, the To:, cc:, Resent-to:, and Resent-cc: header fields are the equivalent of the block of address information between the date and the salutation on the letter; they usually contain the names and addresses of the intended recipients of the message, but they too are not used in the delivery of the message.
You might open an envelope addressed to you and find that the letter inside makes no mention of your name. Your name and address don't appear anywhere in the letter, but it was still successfully delivered to you based on the envelope information. There's nothing strange about this. If someone else opens your mail for you, discards the envelopes, and places the contents in your in-basket, you might wonder how some of it ended up there, because there's nothing to connect you with the message contents.
Email is exactly like this. Each message has two parts, the message contents, and the message envelope. The message contents include the message header, and the message body. The message envelope is made up of exactly one envelope sender address (which can be empty) and one or more envelope recipient addresses. If the message cannot be delivered for any reason, and the envelope sender address is not empty, the message must be returned to the envelope sender address by the mail transfer agent (MTA) which last accepted responsibility for delivering the message. These notifications are known as "bounce messages" or sometimes as "non-delivery notifications". Bounce messages are sent using the empty envelope return path, to prevent mail loops from occurring when a bounce message itself cannot be delivered.
Confusion often arises among novice users about the difference between the message header and the message envelope; they seem to believe that they are not independant. This appears to be an artifact of their use of simple-minded GUI mail user agents (MUAs) that do not allow them to set the envelopes of their messages explicitly, but instead simply use the contents of the From: header field as the envelope sender address, and any addresses found in To:, cc:, and bcc: header fields as the envelope recipient addresses. While these are sensible as default values, more powerful MUAs allow the user to override this choice.
The last MTA to receive a message (usually the one running on the POP or IMAP server where you retrieve your mail from) essentially acts as your correspondence secretary, accepting your mail from the postman, opening it, and placing it into your in-basket. Note that this would normally destroy the important information contained in the message envelope. To prevent this loss of information, this MTA is supposed to copy the information from the envelope into new fields in the header of the message content, as if your secretrary copied the sender and recipient addresses onto the back of your letters in felt pen. Unfortunately, some MTAs do not always do this properly, and envelope information can then be lost. When this happens, it makes dealing with certain types of mail messages problematic:
MTAs are supposed to record the envelope sender address by placing it into a new Return-Path: header field at the top of the message. They should then record the envelope recipient address(es) in another new header field; sometimes this header field is named Delivered-To:, but it can also be Envelope-To: or one of a few other names.
A domain or multidrop mailbox is one which receives mail for multiple email addresses (commonly all addresses in a given domain). If you do not want all of this mail to go to one person, you need to know who the messages were originally addressed to after retrieving them from the POP/IMAP multidrop mailbox. You cannot do this by looking at the To:, cc:, or other informational message header fields, because they do not actually reflect the message envelope at the time of delivery. Instead, you have to reconstruct the envelope information from the message header fields which the MTA on the server used to record it at the time of delivery.
If the final MTA does not record the message envelope (the envelope sender, and all envelope recipient addresses in the domain mailbox the message was sent to), then mail will be lost or misdirected regardless of which software you use to access the mailbox. The mailbox cannot actually be said to be a domain mailbox in this case; the defining characteristic of a domain mailbox is that it records the envelope correctly. The configuration of the MTA running on the server needs to be fixed so that the envelope is properly recorded for every message it receives.
The configuration file format is actually very simple; you don't need to worry about most of it if you're not interested in using those features. The simplest and most common getmail rc file configuration will be for users who want to retrieve all mail from a single-user POP3 mailbox, deliver those messages to a maildir or mbox file, and delete the mail from the server. For maildir, that configuration is:
[options] delete = True [retriever] type = SimplePOP3Retriever server = my-pop3-servername username = my-pop3-username password = my-pop3-password [destination] type = Maildir path = ~/Maildir/
For an mbox file, that configuration is:
[options] delete = True [retriever] type = SimplePOP3Retriever server = my-pop3-servername username = my-pop3-username password = my-pop3-password [destination] type = Mboxrd path = ~/inbox
Create a separate getmail rc file for each account, and run getmail with multiple --rcfile options.
Use the delivered_to [options] parameter.
Use the received [options] parameter.
You don't need to. getmail can deliver to maildirs, mboxrd files, or through arbitrary external MDAs.
If you still think you need to, you can use getmail's external MDA support to do so.
mh clients (and nmh, or "new mh" clients) include a command for delivering a message into your mh folder. In nmh, this command is called rcvstore. You use it as an external message delivery agent (MDA) with getmail's MDA_external destination. Ensure your $HOME/.mh_profile file is configured properly; getmail user Frankye Fattarelli suggests a line like the following is necessary to indicate the path to your mh mail root:
Path: Mail
Then use MDA_external like this (which, after adjusting the path of the command to reflect your mh/nmh installation, should work with either mh or nmh):
[destination] type = MDA_external path = /usr/local/libexec/nmh/rcvstore arguments = ("+inbox", )
Thanks to Frankye Fattarelli for contributing this answer.
Use your system's cron utility to run getmail periodically if you wish to have mail retrieved automatically at intervals. This is precisely what cron is designed to do; there's no need to add special code to getmail to do this.
With a reasonably standard system cron utility, a crontab(5) entry like the following will make getmail retrieve mail every hour:
0 * * * * /usr/local/bin/getmail --quiet
Well, you could write a retriever that speaks Hotmail's proprietary, undocumented, and unsupported access protocol (or pay me to write one), or simply set up the POP3 proxy from the httpmail package, and have getmail retrieve mail from that POP3 proxy.
getmail user Frankye Fattarelli contributed to the following questions about integrating getmail with SpamAssassin and ClamAV.
SpamAssassin can be run in standalone mode or in a client/server configuration. In both configurations, SpamAssassin accepts a wide variety of arguments; please refer to SpamAssassin's manual pages or online documentation for details.
To filter messages through SpamAssassin in a client/server configuration (i.e. with the spamd daemon), use a configuration like this:
[filter] type = Filter_external path = /usr/local/bin/spamc arguments = ("-s 10000", )
The value supplied to the -s option is the maximum message size accepted (in bytes). The default is 250k.
A similar configuration without the spamd daemon would be:
[filter] type = Filter_external path = /usr/local/bin/spamassassin arguments = ("--report", )
The --report option sends the message to the various spam-blocker databases and tags it as spam in your bayesian database.
Note that if you are using Bayesian (learning) filtering, and you've put your SpamAssassin filter after any getmail Filter_classifier, you may have a problem with your learning filter learning getmail's header fields. That is, the headers added by the other filters may get learned, and affect your database. To prevent this, ensure that SpamAssassin ignores these fields by adding the following to your SpamAssassin configuration:
bayes_ignore_header X-getmail-filter-classifier
ClamAV, like SpamAssassin, can by used in standalone or client/server configurations. In either case, you need to add the StreamSaveToDisk option to your clamav.conf file to enable scanning from stdin.
To use ClamAV without the clamd daemon, use a filter configuration like this:
[filter] type = Filter_classifier path = /usr/local/bin/clamscan arguments = ("--stdout", "--no-summary", "--mbox", "--infected", "-") exitcodes_drop = (1,)
The above assumes you do not want the infected emails to be delivered. If you do want them delivered, you would use a slightly different configuration:
[filter] type = Filter_classifier path = /usr/local/bin/clamscan arguments = ("--stdout", "--no-summary", "--mbox", "--infected", "-") exitcodes_keep = (0,1)
To use ClamAV with the clamd daemon, use a filter configuration like this:
[filter] type = Filter_classifier path = /usr/local/bin/clamdscan arguments = ("--stdout", "--disable-summary", "-") exitcodes_drop = (1, )
As with Clamscan (above), if you do want the infected messages delivered instead of dropped, you should modify your configuration as follows:
[filter] type = Filter_classifier path = /usr/local/bin/clamdscan arguments = ("--stdout", "--disable-summary", "-") exitcodes_keep = (0,1)
You may find it necessary to specify the paths of some decompression utilities used by ClamAV with additional arguments like:
arguments = ( …, "--unzip=/usr/local/bin/unzip", "--unrar=/usr/local/bin/unrar", "--unarj=/usr/local/bin/unarj", "--lha=/usr/local/bin/lha", "--jar=/usr/local/bin/unzip", "--tar=/usr/bin/tar", "--tgz=/usr/bin/tar"
Note: if you want to use the daemonized (client/server) version of ClamAV, ensure that your clamav.conf file contains:
ScanMail
The paths to the various decompression utilities must be specified in this file as well.
See the following mailing list message from Frankye Fattarelli for additional notes on using ClamAV with getmail: http://marc.theaimsgroup.com/?l=getmail&m=109128345509273&w=2
Using getmail's Filter_classifier, the output of your filtering program (in this case ClamAV) is placed into a X-getmail-filter-classifier: header field in the message. This can make auditing the actions of filters difficult if you use multiple filters and cannot tell which filter added which line.
To correct this, you can use an additional filter to change the name of the added filter header lines immediately after each filter is run. For example, reformail, from the maildrop package (which is in turn part of the Courier MTA ) can be used in this fashion to rename the added header fields (say, to "X-mypersonalmailscan") with a filter configuration like this:
type = Filter_external path = /usr/local/bin/reformail arguments = ("-R", "X-getmail-filter-classifier:", "X-mypersonalmailscan:")
Simply ensure ClamAV is invoked as the first filter, and this is invoked as the second filter (or immediately after the ClamAV filter, if it is the second, third, etc. filter).
Simply invoke procmail as an external MDA. procmail requires that the message begin with a Unix "From " line (the mbox message delimiter), so you'll need to either set the MDA_external parameter unixfrom to True, or supply arguments that tell procmail to recreate this line. One of the following would be fine:
[destination] type = MDA_external path = /path/to/procmail arguments = ("-f", "%(sender)")
Or:
[destination] type = MDA_external path = /path/to/procmail unixfrom = True
Simply invoke maildrop as an external MDA. maildrop requires that the message begin with a Unix "From " line (the mbox message delimiter), so you'll need to either set the MDA_external parameter unixfrom to True, or supply arguments that tell maildrop to recreate this line. One of the following would be fine:
[destination] type = MDA_external path = /path/to/maildrop arguments = ("-f", "%(sender)")
Or:
[destination] type = MDA_external path = /path/to/maildrop unixfrom = True
If you want to specify a maildrop rc file as one of its arguments, that would be something like:
[destination] type = MDA_external path = /path/to/maildrop arguments = ("-f", "%(sender)", "~/.maildroprc")
Simply use the Filter_TMDA module as a message filter:
[filter-X] type = Filter_TMDA
See the documentation for details on optional parameters to the Filter_TMDA module.