Review of Mailbox.org: Email privacy with a thick German accent

, I migrated my email from FastMail, an Australian email provider hosted in the United States, to Mailbox.org, a privacy-focused email service provider hosted and operated out of Germany. I’m using my own domain names for email so changing providers isn’t supposed to be a big deal as my email address won’t change even though I’m changing service provider.

Mailbox.org offers webmail, calendar, and an online office suite starting at just 1 EUR/month for 2 GB of email storage (thousands of messages) and 100 MB for file storage including the use of custom domains at no extra cost (many providers charge extra for this). They have some unique privacy related features like PGP-encryption of all messages at rest (including messages that arrived unencrypted.)

This will not be an extensive review of everything that Mailbox.org offers but rather focus on my initial impressions after using their email, calendar, and address book services for only two days. I’ll focus on the web tools as there isn’t much to say about the support for standard protocols like IMAP and SMTP (for email), CalDAV (calendars), and CardDAV (contacts).

Data migration

I was quite surprised to learn that Mailbox.org didn’t offer an IMAP migration tool. IMAP is a standard used for synchronizing email folders, and many commercial email service providers offer migration tools that rely on IMAP to copy over messages from another email service. I had to manually setup an email client with my old email account and my new Mailbox.org account, and then match up and copy over messages from one to the other.

Calendars and contacts involved a similar manual process of exporting data from my old provider and importing files in Mailbox.org. I’m glad we have common standard file formats so that this sort of data migration is actually possible, but it still took me some two hours to copy all my data across.

Paragraph two of the General Data Protection Regulation (GDPR), Article 20 Right to data portability also clearly states that data transfers should happen directly between controllers (from company to company without involving a third-party) where technically feasible. For the domain of email, calendars, and contacts there are well-established and widely supported protocols (IMAP, CalDAV, and CardDAV; or Exchange) for synchronizing data and these protocols are well suited for data transfers from all the major email providers. I’d expect there to at least be an IMAP migration tool for emails as a bare minimum.

Mailbox.org have partnered with a third-party company to offer a data migration service that only work with some email providers. This feature costs extra and you also have to agree to share your account passwords and all your data with another company. This isn’t ideal from a security and privacy perspective, doesn’t comply with the words or spirit of the GDPR, and charging extra for making it convenient for someone to become a customer of your company can’t make any business sense.

Web interface, mobile, and accessibility

On the face of things, at least on large screens, Mailbox.org looks and behaves like most webmail clients that you may already be familiar with. You can also configure the interface to look and behave more like other popular email providers if you don’t like the defaults.

Support for Firefox (Gecko) and Safari (WebKit) leaves a lot to be desired. When you delete or move a message, or mark it as spam in these web browsers the message will disappear but will often pop back in the folder after a few seconds. This isn’t an issue I’ve had in any Chromium (Blink) based web browsers, but you should be aware that web compatibility and testing may be an issue if you use any but the current market leading web browser engine.

The mobile website on the other hand is painful to use in a web browser on Android. The Back button, a key user interaction point on Android and in web browsers in particular, don’t work within Mailbox.org’s webapp. When you open a message you have to use their tiny X button instead of the Back button to go back to the parent folder. On larger screens you see the folder and message at the same time so you don’t really notice the issue as much. Menus pop-up weirdly disjointed from where you tap and the Compose message button is all alone on its own toolbar wasting vertical screen space at the bottom of the screen.

The message action buttons (delete, reply, etc.) are uncomfortably small (50 % smaller than in Yahoo Mail and Google Mail), unlabeled, and has a low 2,36:1 contrast ratio in a light green color. For reference, a contrast ratio of 7:1 is what is required to pass established web accessibility testing requirements. The same low contrast ratio is used throughout the interface for link texts and actionable interface elements. The small icon and contrast issue is kind of baffling as there is evidence of some attempts at making the web interface work with accessibility tools. The low contrast ratio is caused by a misguided effort to use the Mailbox.org brand color everywhere even for purposes its clearly not suited for.

You can work-around the contrast issue by going into Settings and changing from the Default theme to the Default theme (sic). This change changes most elements from the low contrast sickly green color to a much more comfortable blue. The Default theme really should be the default theme instead of the Default theme to let the service accessible by default. Each theme should also be given a more descriptive and unique name, but vague and confusing option names is sort of the theme for the Settings page.

Difficult settings and poor translation

The settings page clearly shows that it was designed to fit the underlying technical infrastructure and not around how customers would use it or what goals they may want to accomplish (task-oriented design). I appreciate the level of customization available, but the available settings makes no sense to anyone who isn’t extremely well versed in email server software and related standards.

The product is clearly design by engineers that want to expose an configuration option while having existing knowledge of how it works and what the change does; and not by a interaction or product designer who’d try to design a solution for the end user. Options and their descriptions are weirdly phrased overly technical and the documentation mostly consists of noting that the option exists and not what effect it has.

This problem is made much worse by most texts clearly having been translated verbatim from German to English and losing their meaning (and anything resembling natural sentence structure) in the process. I’m not an English native speaker and I’d rather not call out anyone else’s poor English levels but I’ll still come out and say that Mailbox.org’s translations are really bad.

Not everything is translated either. An American customer is expected to know to set their Country to Vereinigte Staaten von America from a long list of country names during registration rather than United States of America. Frankly, I think that is expecting too much German knowledge of potential customers.

The selection of which settings are enabled is inconsistent. You can configure messages in your Trash folder to be automatically deleted after 30 days, but not messages in your Spam folder. You can set the first day in calendar week-views to be either Sundays or Mondays: but the first day of the week in monthly calendar views is always locale dependent regardless of the setting.

Custom domain and identify management

One of the things I found the most difficult to configure with Mailbox.org was how to setup email delivery handling for my own domain names. There is no obvious place to set this up on the account Settings page.

The process involves first adding your domain name as a recipient-less email address in the “E-mail Aliases” section once, receiving a confirmation code that needs to be added to the domain’s DNS configuration, and then going through the same process again. Mailbox.org doesn’t remember your domain name after you submitted it the first time so you have to submit it a second time on the same screen. After verifying domain ownership you need to change the email exchange (MX) records in your DNS before you can receive any emails.

At this point you don’t have any way to send outgoing emails from your own domain through the web interface. To setup that you also have to go to a screen titled “Alternative Sender”, type in the desired email address you want to send emails from, send yourself a confirmation email that is sent and delivered to and from Mailbox.org, and click on a link to confirm that you want Mailbox.org to be able to send emails through this address. This is a baffling setup procedure considering that I’ve already verified ownership of the entire domain and directed all emails sent to that domain to my Mailbox.org account.

Any emails sent from this email address will be sent without a sender name. To assign a sender name to the email I have to leave the Settings page and go to Compose a new email message. From that page, I can click on the From field and choose “Edit names” which opens a new page that lists all my email addresses and lets me assign a name to each address. As far as I’ve been able to work out, you cannot assign specific signatures to specific sender addresses (e.g. a signature with your company name for your work email address or a personal signature for your private address).

Competing webmail providers and email clients have for the most part settled on a systems where you have an Identity manager dialog where you add domains, email addresses, and assign names and email signatures to your different sending profiles. Mailbox.org could really need a unified sending profile or identity manager because their current approach is just way too complicated and confusing without offering any benefits to the customer. The available documentation is helpful in regard to configuring custom domains and addresses, but good documentation in this area is a tiny patch on a bigger interaction design problem.

I was disappointed to learn that Mailbox.org currently doesn’t offer DomainKeys Identified Mail (DKIM) configuration for custom domain names. DKIM is used to cryptographically sign and verify emails as having being sent from a specific domain name. This requires the generation and management of cryptographic keys by the email service provider as well as changes to the domain configuration by the customer. Mailbox.org has stated that they’ll add this capability in the future.

Message identified as spam still delivered to my inbox

By default, Mailbox.org will reject all messages that have been identified as spam. However, I’d like to review these messages in case something important is accidentally marked as spam. This is especially important as I’ve just changed email provider and their spam filters may have had the same training data, and thus behavior, as my old email provider. Therefor I changed the default settings to deliver spam messages to the spam folder instead; which was one of the available configuration presets.

To my surprise (and annoyance), all spam messages ended up being delivered to my inbox instead. I reviewed every setting and read up on the available documentation. As far as I can tell, there was no reason why spam messages would end up in my inbox. I had set up some custom filter rules that would direct messages to dedicated folder based on the recipient address. These filters weren’t applied to spam messages either and spam messages that otherwise should have triggered the custom filtering rules also ended up in my inbox.

After reviewing the email headers I was able to confirm that all the spam messages were indeed correctly identified as spam. The option to deliver them to a dedicated folder just didn’t work. I ended up setting up another custom filter that moved all emails with the X-Spam-Flag: YES header to the spam folder.

I also added a second filter to reject emails with a score so high they could only have come from a Nigerian princes claiming to to have hacked my computer and taken photos of me while watching niche adult content and demanding payments in Bitcoin to sell me enlargement pills from a Canadian pharmacy. I’d expect these two filters to have been part of the default configuration for all customers, though.

I contacted Mailbox.org support about the issue, and they were entirely unhelpful with regards to resolving this issue. I’ve looked through their user forums and noticed other reports of the same problem so I’m no the only customer who’ve experienced this problem. As for first impressions for an email service provider goes, this is a pretty devastating issue.

On a more positive note, Mailbox.org’s spam filters have caught every spam message I’ve received over the last two days except some Norwegian-language ones. I’m guessing that Mailbox.org may not have many customers in Norway so they may not have adequate training data yet. I’m expecting this issue to sort itself out with in a little while a bit of manual spam-reporting from my side.

Conclusions

I’m not impressed by Mailbox.org’s web interface. Their prices and the overall service offerings are very competitive and I believe they have all the pieces necessary to make a good product. I think it’s positive that they focus on customer privacy without setting their prices unreasonably high as other privacy-focused service providers in the market tend to do.

The only feature I miss at Mailbox.org compared to my previous provider, FastMail, is app-specific passwords. App-specific passwords generates a unique password for use with client software (like an email client) that grant these programs access to specific protocols (like IMAP for accessing email) without granting them access to the web interface and the entire user account. I must use the main account password to access my account via client software, which means the password must be stored in multiple locations and quite possibly make it more susceptible to be compromised.

I’ve been able to find workarounds for the issues I’ve encountered and found ways to make Mailbox.org do what I want it to do. However, it was a bumpy road and took much longer than I had expected. Mailbox.org requires quite intimate knowledge of how email software and standards works. This may be acceptable to some customers, but this probably does needlessly limit their potential customer base.

Mailbox.org could really need to hire an interaction experience designer to look over their product and help them iron out some of the rough edges. Involving a microcopy editor and a translation service would also help boost their product quality considerably. They’re so close to having an impressive product, but the poor language and ssome of the jarring design details ruins the user experience.

I would not recommend Mailbox.org to anyone that doesn’t have the time required to learn about how the service works. Some familiarity with German may be helpful too. I believe any determined customer could work out how to make Mailbox.org work for them, it may be worth considering another provider first. In a market where most customers get the services you offer for no upfront cost, you’re by definition a premium service when you charge a fee. Mailbox.org isn’t a premium experience, however. I would definitely not recommend Mailbox.org to anyone who doesn’t have good or average eyesight or regularly use screens with poor color accuracy.

You can get a full refund of account credit in the first two weeks. After that, you can donate remaining account credit to one of three charities instead of leaving it in the company’s coffers. Transaction fees for refunds doesn’t make it worthwhile to refund customers directly but giving customers the option to donate it to a charity is a commendable solution.