Unexpected Changes

E-mail is one of those things that you take for granted. Once you have set-up an address and a client you can reliably receive mail for many years until something changes.

Over the past 2 months something changed and I almost didn’t notice. There are certain sites that mail me a few offers, I subscribe to a few mailing list items, github updates me on a few projects that I keep an eye on. I don’t get a huge amount of email but there is normally something being downloaded when I open the client. There were a couple of days when I didn’t get as much as I expected. Not a problem as perhaps there wasn’t anything being sent. When this persisted for a few days I began my investigations.

I was aware of my web host updating the mail-server back-end. I had received an email stating that this would be occurring and to expect a little downtime. But apart from that I wasn’t expecting any significant changes. Significant changes had happened.

Having a domain name had allowed me to use a catch-all email address. Any email that would be sent to the domain after being spam filtered would be received. This made it incredibly easy to create a new address on the fly when creating an account for a new service. I had a few actual mail box addresses in addition to the catch-all. What I had noticed was email only being received from actual mailboxes and not the catch-all addresses.

This was a problem as I had a few accounts where I really needed to be receiving correspondence. I needed to get a solution and quickly. I was able to contact my host and get the catch-all re-enabled. This workaround allowed me a little more time to arrange a permanent solution.

The permanent solution would be to create actual mailboxes rather than rely on a catch-all system. How many unique email addresses had I been using over the years? I stopped counting at 114. Creating mailboxes for all these would be too much effort since most wouldn’t see a large volume of traffic.

Looking through the long list of addresses I was quickly able to discard a large number of them. I had a few that I had used for stores and services that have since closed down. I shouldn’t be getting any future mail from these so I have just made a note of them in my spam filter and ignore them. To reduce the remaining addresses I decided it was best to create a single mailbox and then update various accounts to use it. I can then discard the original address. This was a long and time consuming task.

Currently I have reduced my 114+ addresses down to a more manageable 25. In an ideal world I would have been able to reduce it down even more. A stumbling block was not being able to change my email address on some accounts. For the ones that matter it was easier to create a mailbox, for the others it was easier to unsubscribe and resubscribe.

In future I am only going to use a more limited selection of addresses to hopefully reduce the 25 even more. Whilst this process has been an annoyance it has been necessary to tackle it. I think it has been about 5 years since I last tackled a big email change. All being well I shouldn’t have to do anything as significant for at least another 5.

Email Settings

I checked on the connection settings I am using to log into the mailboxes. Technical information from another email provider provides a good overview of the available settings.

RFC8314 (Jan 2018) outlines the current thinking over connection security for email.

Simply put implicit security using SSL/TLS is preferred as it encrypts the connection by default. STARTTLS is discouraged as it begins as a plain text connection which can upgraded to an encrypted connection. It is worth checking your settings with your own provider to ensure you are as secure as you can be.

Tidying Up

Starting off a new year always seems a good time to get a few things in order for the months ahead. Today I have tacked my email inbox. I have been building upon my list of filters for years such that most of the mailing list items, special offers, online order confirmations etc. all get sorted automatically. The filters never remain perfect as a company might change the email address they are sending from or retitle a newsletter. The result has been a spam folder filled with things I want and an inbox full of clutter.

I have spent 30 minutes today updating my filters and sorting out the mess. Since putting the majority of the filters in place a few years ago the annual maintenance is now minimal. I will of course go along and bulk delete old news letters in future but having the vast majority auto sorted mean even this task takes a few moments per folder.

Email wasn't the only clutter I found myself with. My browser downloads folder and my home directory have accumulated a great number of files. Going through and having a good tidy up means not only can I find things again but I don't end up downloading files multiple times. Tidying files is never as easy as email as the filters aren't available. Downloaded files especially PDFs seems to have obscure file names which have no resemblance to the content, meaning each one has to be opened, assessed and renamed. Perhaps a little more regular maintenance on this front is needed.

System Email

On my Fedora system I collect the system emails to my user account. This results in a selection of daily emails outlining statuses which I am prompted to deal with whenever I launch a terminal. In the past I have used the "mail" command to read and delete the messages. It is not the easiest mail reader to deal with however. The fallback has been to use Evolution which is part of Gnome. I have never been a fan of Evolution, instead preferring Thunderbird. Thunderbird does not support the maildir format used by the system email and as such has been ruled out. I am going to be trying out Mutt, it is a command line reader that I found already installed on my system. The interface is structured unlike "mail" and even lists the shortcuts for commonly used commands. For my needs I don't need to configure Mutt to do anything. I did find a couple of links that may prove useful in making Mutt your primary email client.



The Legacy Email Problem

Over the years, like many other people, I have obtained numerous email accounts/addresses. Trying to manage all these used to be a problem so a few years back I began to condense them down into the few I actually use.

I maintain access to the older accounts to receive any incoming mail from websites that I need to amend my contact details for. One of these accounts is subscribed to a mailing list and continues to get substantial incoming mail. So it's time to unsubscribe.

Unsubscribing can take a number of forms. On some sites you login uncheck a box and away you go, no more mail. Alternatively changing your email address to something random so you never get the email. In the past I have clicked a link and been taken to a page to enter the subscribed email address and later clicked another link in an unsubscribe confirmation email. These methods have always worked because the mailing list/site doesn't care who or where you are. I needed to send an email from the subscribed account to the unsubscribing email address of the mailing list.

The problem I have today is related to an issue with email accounts and spam prevention in general. Email is usually accessed in 2 ways via webmail/app or desktop client. I am using a Mozilla Thunderbird on the desktop to read my emails. To send email I need to use an SMTP server, every email account specifies which one to use and you need to pass along your login details to use it. Even though I have access to a lot of SMTP servers via a lot of accounts they are slowly locking themselves down. Most will now only send an email with an address they provide.

Smtp.example.com only sends emails from bob@example.com and not bob@somewhere.com.

This can stop spam and reduce traffic on an SMTP server as it isn't processing emails for people who aren't paying for their services.

Back to my unsubscribing problem. I had the correct SMTP server settings for the account and I was sending the email from the subscribed account in question. It failed. The reason was due to the fact that the email address is tied to an Internet Service Provider (I have switched since). It was detected that I was attempting to send an email through their SMTP server from outside their network. This for reasons of reducing spam and server load had been blocked.

So how do I send the email? The easiest option is to find an open SMTP server which would be happy to send the email for me. Google is able to do this, however the caveat is that rewrites the email address to that of the Gmail account used to send it. You can specify an alternative reply address in the account settings which will result in any replies to the email going back to the original account address and not the Gmail address. This is ok for most cases but I need the email to come direct from the subscribed account.

The solution, the webmail interface. Logging into the email account through the providers website allows me to send email from within their network. It means my unsubscribe email is sent from the subscribed account, through an SMTP server that is happy to send it without changes. I still had to confirm an unsubscribe email but luckily this was sent within a few seconds of the initial email. It has meant jumping through a few hoops but the mailing list is no longer sending me any more mail.

Refactoring Email

I have been using email for many years now. Over that time I have changed between mail clients, addresses, operating systems and spam filters. Whilst I have been able to keep on top of filtering mail into folders it finally reached the point today where the the mess of filtering and folders needed a clean up.

First of all were the mailing lists and subscriptions which seemed a good idea at the time but never get read these days. Due to the traffic on these mailing lists I had them all automatically filtered into individual folders. I had folders in folders sometimes 3-4 levels deep. Unsubscribing means I can delete the levels of folders and the extensive filtering rules for them.

The next stage was to have a think about the marketing emails I get from the likes of Amazon, Screwfix etc. I still like to receive them as they sometimes contain offers I am interested in. For each of the stores that I had ordered from there was a sub-folder with the past ordering history. Order emails I sort by hand as they don't justify the need for their own filter. So the orders I want to keep but the general marketing is a read and discard affair. So why keep an individual folder for each marketing email? It makes it obvious where any new mail has come from but adds extra folders. So the decision was to group marketing mail into categories like 'Electronics Components'. All the marketing mail from related suppliers goes into a single folder. Any order emails can be moved by hand into an order folder for storage whenever they arrive. Suddenly a huge number of extra folders disappear.

The next stage was to rename and move the remaining folders into a more structured arrangement. The result is a more easily navigable tree of folders.

The final stage was to redefine and cleanup the mail filters. Having scrapped a lot of folders many rules were not needed. Others could now be grouped together to put mail in a single folder rather than many.

The result is that I now have a much cleaner and easier to use email experience. It took a few hours to get it how I wanted but it has been worth the effort. If nothing else, when I need to add another folder or filtering rule its going to be easier to do so. I have also removed the stuff I never read any more, its less stuff to deal with when checking email. It might be another year or two before I do something like this again, but a periodic review of the setup can only help to keep it clean an manageable.