migrating email is a pain in the ass (notes on the long, tedious process)
April 10, 2016
Email is one of those technologies that people love to hate. But I don’t think email is actually a problem we need to solve and as one of the older communication methods on the internet its also one of the best. Why? Because it remains one of the few non-walled, open protocols that people still regularly use. IRC is too fringe to count and federated chat via XMPP has died. Everything else is walled gardens (imessage, facebook messenger, google hangouts, whatsapp, the list goes on and on).
Email remains as one of the few commonly used technologies that doesn’t depend on a particular corporation. You can communicate with anyone who has an email address, doesn’t matter who is their provider. These days, there are very few of ‘social’ technologies that don’t lock you into a specific website or provider.1 Of course, some companies are trying to change this (eg slack) but… email has proved pretty resilient. And I hope it remains that way.
But as privacy and other concerns become a pressing issue for a lot of people, I know more than a few who want to migrate their email to a more private/secure email provider (than gmail which is pretty much what a lot of ppl use these days). I’ve migrated my email like a bajillion times over the past five or so years (as I keep trying – and failing – to escape google’s clutches). What follows are my thoughts and suggestions to consider before you start the process.
1) This is a painful, slow, tedious, and time consuming process.
The two most painful parts of the process are migrating the emails themselves and then updating all of your accounts with the new email.
Because of gmail and their generous storage (which changed the game, btw) a lot of people are in the habit of basically keeping every single email they get (besides spam and such). And why not? You have the space and gmail’s search is really quick and powerful.
Of course… the more emails you have, the longer the migration process will take. And, unless the new provider has a automagic migration button (like fastmail’s import from gmail feature), you really do have to manually migrate every single message. This is especially going to be a stopping point for a lot of people since the popularity of email clients has faded and a lot of people don’t already have one configured with imap.
(Which is basically how you transfer the emails… you get the two email accounts set up in a client and, depending on the client, drag and drop the emails from one account folder to another. Then you get to wait for all the messages to transfer from one server to another. It can be slow and prone to failure (eg, thunderbird doesn’t handle transfering huge amounts of emails gracefully and will often fail so you have to do it in smaller batches.)
After you get to this point, then you suddenly realize that every single account you’ve ever opened online uses your email for identification or communication. And so you must go to every single one and update your email address. I recommend doing this even if you have forwarding set up since you may not want to keep the other email account open indefinitely and because you want to make sure that account recovery stuff is going directly to you.
These two steps are the most time consuming and tedious. And you can’t really get around them. For the data migration, there are some ‘easier’ ways but they are still slow (I usually use a CLI client like mutt or alpine since they can handle transferring large numbers of emails without crapping out like thunderbird).
2) Data, my precioussss data.
If you do decide that migration is worth the effort above, this is actually the best time to take care of some data housekeeping. Doing this can reduce the number of emails you transfer between servers.
Whenever you do data migration it is always recommended that you back up the data. For emails, you can combine backing up your data with proper digital preservation techniques to ensure that you have redundant data that will also be accessible for a long, long time.
This process can also be time consuming, but its more ‘rewarding’ than the above since it involves actually deleting unnecessary messages (and even this isn’t strictly necessary). In any case, it is comforting to know that your data is backed up and properly preserved.
In the case of email, you want to export/save/archive/whatever your email as ‘mbox’ files (which are essentially giant plain text files that preserve all the emails with their metadata – such that if you open the file in an email client it will look like just another email folder/inbox). As with all backing up and preservation, this is something that you should do regularly. And, of course, you should make sure that multiple copies exist in different places (the LOCKSS principle: lots of copies keeps stuff safe).
This step will be essential if you are moving to a free but privacy conscious email provider like riseup. Because of what they do (which is great btw), their free email accounts have a tiny amount of storage compared to gmail (like we are talking a couple hundred megabytes at the most).
And, of course, this means that if there are any flubbs during your migration, you won’t lose any data. So. Yeah. Do this even if you aren’t going to migrate your email. Your future self will thank you (I know I thanked myself for preserving my old emails from every account I’ve ever had when I needed to find something someone sent to my yahoo account like 12 years ago).
3) To custom or not?
Because of how tedious updating all of your accounts can be, migrating your email is a great opportunity to disentagle your email address from a specific provider by using a custom domain (eg, going from firstname.lastname@example.org to email@example.com). Doing this ensures that if you ever have to migrate accounts again, you can keep the same email address and not have to repeat updating at every single website you have an account.
I’d actually recommend doing this regardless of whether or not you decide to migrate. It will make life easier in the future and, well, means that you don’t have to be constant advertisement for some corporate brand.
The unfortunate thing with this option is that it will cost money. The cheapest domains are around $10/year. So not a huge amount of money but its still an amount of money.
It will also cost in terms of time and effort. Using your own domain for email means that you’ll have to learn something about mx records and such. Most email providers that allow custom domains will have instructions somewhere for how to do this. But… since there are so many domain registrars and they don’t all use the exact same UI, you may not be able to find a tutorial that exactly matches yours.
4) Deciding where to go…
Funny how this is the fourth consideration. But honestly? If you aren’t prepared to spend time and effort, you shouldn’t bother unless you’re okay with losing data (and its okay if you are – I lose data all the time and rarely miss it or even remember I had it in the first place).
Where you decide to go will depend on a few factors:
- how secure do you want your email to be?
- how private do you want it to be?
- how much storage space do you want?
- are you using a custom domain?
- how much are you willing to pay?
- what additional features do you want?
- quality of service
Security, these days, usually implies some level of encryption. The most ‘secure’ providers will have end-to-end encryption. As far as I’m concerned, this is a non-factor. End-to-end encryption means that everyone you communicate with will either have to have the same provider or knowledge of encryption stuff (ie, have a pgp key or some such somewhere). For encrypted emails to accounts like gmail, they aren’t encrypted. Because the recipient won’t be able to read them. You can send people what amounts to a link to a webpage that they access with a password… but this is kind of a hassle and I know a lot of people who just wouldn’t bother.
Encryption for email (or IM) is only meaningful if most or all people are using it. They aren’t and most aren’t about to start unless the technology becomes significantly more accessible than it currently is. I mean… it is still a challenge to get people to use what is considered a ‘good’ password. The tech is useless unless humans actually use it the way we ‘should’.
The only security feature which has any value is whether or not your emails are encrypted while stored on the providers’ servers. As in, can they read your emails? Can they give your data to a government and have it be easy to use? Etc.
This is a bigger concern and one that can be a big factor. And I think it is the primary (along with security) reason why a lot of people these days consider migrating their email. Things to watch out for: do the email headers expose your IP address, where are the servers located and what data privacy laws are in place, does the provider keep user logs, and so on.
Gmail has basically spoiled us for how much space we expect for free. But say if you want to go to protonmail and use their free account… you get 500mb for free. If you want/need more space, you’ll have to pay. Or you can use riseup which only has free accounts (but you need to apply for an account or get an invite)… they only have a small amount of storage for email.
This is true for basically all of the ‘good’ alternatives to the Big Name email providers (google, microsoft, etc).
Not all of the services will support a custom domain, so if you want to use one, this is something to watch for. Riseup doesn’t support this but something like fastmail or protonmail will (for a price). If you are going this way, I’d also spend a moment taking a peek at the available documentation (unless you are already a Web Master). Good documentation is super mega important.
How much will you pay?
This is one area where the old internet proverb applies: if its free, you’re the product. For using their ‘free’ email, google scans your mail and places ads all over your inbox (if you use the web UI). So if you want a comparable experience to what you’re used to from one of the Big Tech Corporations, then you’ll need to pay money.
Near as I can tell, most of the good alternatives will cost you around $5/month for a decent amount of storage and using a custom domain (and don’t forget that having a domain also costs money). The thing to remember about the internet these days is that privacy and security are for the privileged. Look over the above and note how doing this requires time, money, and skills that most people I know don’t have and aren’t likely to get any time soon.
Ultimately, its this that drove me back to using gmail. I have the skills and time but not the money. So.
What additional features do you want?
One of the hardest parts about migrating email from a major tech corp is that you quickly realize how enmeshed you are in their ecosystem. Gmail, for example, ties really nice with contacts and your calendar. Are you also planning to move away from that? Do you want this bundled with your email? Do you want to be able to sync this between all your devices?
Fastmail will do contacts and calendar, but if you want full syncing you need to buy the $40/year plan. Something like riseup doesn’t do contacts or calendar (and it doesn’t look like proton does either). So you’ll have to find some sort of solution for this.
Ultimately, this is one of the hardest things about migrating email. These days, its rarely just email anymore.
Quality of service
I almost left this out… But given how vital email is in a lot of our lives, how dependable the service is matters a great deal. By ‘reliable’ I mean a few different things that I’m not going to get into; the main point is that when people send you messages you receive them and when you send messages they are received.
A reliable email provider will do this almost all of the time (no service has 100% uptime not even gmail). For example, I actually like riseup a lot. Their overall values are somewhat appealing. But I had to stop using their email after I consistently got a bunch of people telling me that their messages were bouncing (and my inbox was not full).
Ok. I think I’m done. This is a long enough post already. But… these are all things that come into play when making the decision to migrate.
As I’ve learned from my own experiences, I honestly don’t recommend it unless you really do have the time, money, and skills necessary (and you will need some combination of all three). Given our current tech and political climate, there really isn’t much point.
At no future point will things like encryption and whatever become commonplace technologies. Not unless the entire tech paradigm changes in massive, fundamental ways. The web runs on advertising. All of the major tech corps have a vested interest in ensuring that some third party (usually themselves) is able to access your data.
Either that or you have to have enough money to pay for ‘secure’ everything. And it adds up. This post only talks about email. But getting to a reasonable place will cost at least $50/year. Not a lot of money for the techbr0s who have the least to worry about re: government surveillance but it can be a lot of money for vulnerable, marginalized people who are more frequently targets of government suirveillance. And if you are looking for security online, email is only one piece of the puzzle. All the other things cost money too.