Securing email server in a Nutshell

Email stands out as one of the most used means of communications we owe in this digital world. When information is involved in such digital communications, security should be considered as the highest priority and this article helps you to understand how an email security mechanism works and how it is implemented.

Before diving into the security of email servers let's take a brief look on the protocols involved in building mail servers. Basically, there are three types of protocols which played a key role in introducing the E-mail system to the world.

• POP(110)
• IMAP(143)
• SMTP(Port 25)

POP stands for Post Office Protocol in which all the mails stored on a mail server gets deleted as soon as they are delivered to the client device. The backup is not available in this scenario if the mails get deleted on the client's device also.

‍IMAP stands for Internet Message Access Protocol. In this protocol there is a feature to view, organize and delete your emails from the server. Until you decide to delete the mails they will be there.

‍SMTP stands for Simple Mail Transfer Protocol. Here, we can store all the mails in the server and can sync them to any of the devices we want. SMTP works along with POP and IMAP.

All these protocols generally run in unsecure connection but when they are configured and changed to secure mode then the default port that they will run on will also change to secure mode automatically.

SMTP , POP and IMAP are the core protocols on which email will work but there are many other protocols launched by vendors like MAPI(which is API), Messaging Application Program Interface by Microsoft which is used by Outlook and EAS stands for Exchange Active Sync which is used to sync mails, contacts, events, calendars to connected devices.

Architecture of Email server.

Email server architecture is simple to understand and easy to implement any changes. In Mail server architecture, either POP or IMAP will be working within the server and SMTP will be working along the server.

SMTP is just a management protocol, which helps to manage the email protocols. It is not involved in sending and receiving emails. All the work will be done by POP/IMAP as per configured. SMTP just manages the emails they receive and send.

When it comes to security, everything we communicate, manage, handle has to be concealed and protected.

Email Server Security Checklist:

• Closing ports
• Malware scan of attachments
• Spam control
• File types filtering
• Disable Relaying
• Enable SPF, DKIM, DMARC
• Securing DNS
• Enable reverse DNS

Closing ports:

‍Usually every protocol runs on a port that is specified to and every device has 65535 ports, for an email server we might be using two ports for secured communication and unsecure communication.

Tit is always advisable to open only the ports you are meant to open rather than opening all the ports available. Always filter the traffic using firewall.

‍Malware Scan of attachments:

‍There are many advanced techniques to hide some data in some file formats and one of the basic techniques is steganography. Steganography is used to hide data in the images like jpeg, png. We can hide any data we desire and extract the data using various tools. Similarly, we have to scan compressed files and should observe for any abnormal behavior of files.

‍Spam control: Spam mails are one of the most challenging problems internet is facing today. Spamming is always ahead in its way whereas the Counter spam mechanisms are lagging behind.

‍Uncontrolled spam results DoS attack. The memory will be occupied to the most when there are more spam mails bouncing into your inbox. So, we can expect a situation where the server runs out of space and that effects availability of the server which is nothing but termed as Denial of Service(DoS). 

File types filtering: There are so many file types out there like a heap but most of the communications in the mails include word, excel sheet exchanging and some basic formats which can be counted by finger tips. Limiting file types to share in the emails will help to tighten security.

There might be users that will be looking for sharing executable files or some binary files without any intention to harm. It is better to stay away from such file sharings and there are many platforms out there to share the files. We can use integrations in mail service to make things handy.

Another way is to add some extension like salt to password for all the attachments before saving in the server and removing that extension while serving to user. This helps a lot in improving the security because an attacker knows he uploaded executable file, when we change the extension, attacker might not know where the file is and as the extension is changed he cannot filter it even if he is authenticated.

‍Disable Relaying:

‍Relaying is a process where anyone can connect to any mail server configured with open relay and send email. This will help spammer to show their true potential.

Throttling limit:

‍There is a possibility to set limit for number of emails sent by individual in an interval of time which will helps to find out if someone is spamming around and can block that user.

‍Enable SPF, DKIM, DMARC:

SPF:

‍SPF stands for Sender Policy Framework in which a receiver server only accepts the emails from the domain that are allowed.

1. An SPF enabled email server receives an email from somebody@example.com
2. The email server looks up example.com and reads the SPF TXT record in DNS.
3. If the originating server of the email matches one of the allowed servers in the SPF record, the message is accepted.

DKIM:

‍Domain Keys Identified Mail (DKIM) adds an encrypted signature on every message that can be validated by a remote server against a DNS TXT record.

DMARC:

‍The Domain-based Message Authentication, Reporting and Conformance (DMARC) protocol builds on SPF and DKIM to handle verification of sender domains. DMARC also provides reporting feature which grants the organizations to access to their email policy. Additionally, DMARC specifies what to do with a message if the SPF and DKIM authentication mechanisms fail. The combination of SPF, DKIM and DMARC creates a trustworthy email environment. All three rely on DNS TXT records to work.

Securing DNS:

‍DNS is where the web browser picks IP address of the website from. If DNS is tampered, then our entire online activity like what we watch and what we communicate will be tracked. So, it is highly recommended to use some trusted DNS or build your own.

‍Enable reverse DNS:

‍When reverse DNS is activated, SMTP verifies that the senders IP address matches both the host and domain names that were submitted by the SMTP client in the EHLO/HELO command. This is helpful in blocking the messages which fail in address matching test.

These are all the different techniques, which can help you to secure your email server without any hassles. Please let us know in the comment section if we miss any.

‍Credit: Manindra - Security Researcher