Why Using Dummy Email for Testing Just Doesn’t Work

On July 25, 2019
8min read
Andriy Zapisotskyi Growth Manager @Mailtrap

Testing email workflows might not be the most fun part of a QA professional’s life. It usually requires a lot of manual work setting up test email accounts. It needs to be done rather quickly to enable focus on more important aspects of the user’s journey. And, in the end, something usually goes wrong.

Or, they get locked out of their newly created account and can’t reset the password due to a malfunctioning link.

To avoid such issues, QA analysts and developers create dozens of dummy emails and try to test their workflows before a product is shipped. It usually brings decent enough results so they repeat it for every new project. Is there a better way to go about it, though? Let’s see.

How do dummy emails work?

The concept of temporary email accounts is simple. You create them with one of the dedicated services or by setting things up on your own. You get a unique email address that can receive (and sometimes also send) emails and can be used for any purpose. Due to their temporary nature, most addresses stop working after a specific time. Frequently, the lifespan of such an account ranges from 10 minutes (hence the commonly used term ‘10minutemail’) to a few hours. 

Disposable email addresses work in a similar fashion. They’re usually meant to be used only once, after which they either get disposed of or are simply replaced by their newer “colleagues”. There’s also the concept of dynamic aliases which are variations of legitimate email addresses. They don’t disappear unless the main account gets deactivated. We’ll cover them in more detail below.

People create temporary emails mostly to stay anonymous on the internet. Email addresses are frequently required to use an app, take part in a promotion, or ask a question on a public forum. If you were to leave your private email, you would likely get tons of spam later on or your address could be used for more malicious purposes.

The other idea behind using temporary emails is to test email workflows. QAs or developers set them up and then prompt their application to perform a specific action that will result in an email being sent to a disposable account. If an email is received, they check if links work as expected, if dynamic content is pasted properly, or if the correct person received the correct email in the first place.

If everything checks out, they move on to another account, testing similar workflows. On complex platforms with multiple email channels, this can take quite a while, especially if tests return numerous errors and everything needs to be fixed and tested again (and again).

In this article, we’ll focus strictly on using dummy emails for testing.

Where to create test email accounts?

Now, let’s go over the most popular services for creating disposable emails for testing.

Mailinator

Mailinator is one of the most popular tools and is focused strictly on workflow testing. Its free plan allows for using multiple inboxes with the @mailinator.com test email domain, which can then be utilized for testing incoming emails. Mailinator doesn’t allow attachments to be added to emails and each message is deleted after a few hours. All inboxes and their content are publicly visible.

The premium plan is a much better option for testing as it comes with a private domain, email inboxes that only you can access, and API access for automating the testing process. An enterprise plan is also available. 

We cover the free and premium features in more detail in our Mailinator guide.

Price: $159 monthly when billed annually. The very limited free personal plan also available.

Guerilla Mail

Guerilla Mail is a free service offering temporary email addresses without registration. It allows you to both send and receive emails and all incoming messages are displayed directly at the homepage. The page’s UI is rather rough, but you don’t create test emails for aesthetics in any case. With Guerilla Mail, you’ll need to create addresses manually in multiple tabs but for limited testing it might do the job.

Price: free (relies on ads)

Maildrop

Maildrop is an open-source project for generating email addresses in an instant. It comes with a much more pleasant UI than for example Guerilla Mail but the features remain virtually the same. You can pick from one of the suggestions (jealouswhale@maildrop.cc being an example) or type in your own address, for example, “pleaseworkthistimeplease@maildrop.cc”. It works with plain text and HTML but no attachments. Maildrop also features a spam filter, preventing unwanted messages from entering your inbox.

Price: free (tiny ads might appear)

Tempmail

Tempmail is another tool for generating disposable emails. It allows you to claim random email addresses and checking their inbox without even refreshing the page. As is the case, with all the preceding entries on our list, Tempmail addresses get killed a few hours after creation, giving you space to add completely new emails to your QA process.

Price: free (but heavy on ads)

There are many many other services for creating fake email addresses and with the very same functionalities but it wouldn’t make sense to cover them all here. Bear with us though, as chances are you won’t even need any of these tools.

How to create a fake email address with an existing account?

If omnipresent ads or high pricing discourage you from all these tools, there’s a simple trick you can use. Chances are you have a Gmail account that you don’t really use. If not, it only takes a few minutes to create one.

With Gmail, you can create unlimited email addresses from within your account. You don’t need to go through any configuration and won’t get any addresses like fnfh3443hj3hgt@gmail.com (unless you request that!). With Gmail, you can simply add a ‘+’ to the username and follow it with any word or combination of alphanumeric characters you can think of. So, for example, for the email address mailtrapsupertesting@gmail.com, we could create:

And so on, the sky’s the limit. Because Gmail treats all these addresses as one, each message sent to these emails will be delivered as if though it was sent from the base mailtrapsupertesting@gmail.com address. Simply add those addresses to your QA procedure and start prompting different processes that will result in an email being sent. If everything is configured properly, each message will end up in mailtrapsupertesting@gmail.com’s mailbox, causing quite a mess but giving you clear information on whether a message was delivered and how it looks for the end-user.

To make the data more readable, people usually set up filters. The simplest idea is to add a “signup” label to all the emails arriving to mailtrapsupertesting+signup@gmail.com and do the same for every different address you’re monitoring. It will give you very clear information on what worked and what didn’t.

This method likely also works with some other email providers but we haven’t tested it yet. If you have a chance, drop us an email (to the real address!) and we’ll be happy to update the article.

Why using temporary emails might not be the best approach?

As you might have guessed from the title of this post, we don’t believe temporary email addresses really solve the problem. They’re perfectly fine if you need to test a very simple web application. By that, we mean a platform where email workflows are only limited to password reset and registration confirmation, without any focus on user permissions, media, formatting of emails and others. Set up a few temporary addresses and you’ll do all the testing within minutes.

The real problem, in our opinion, comes with more complex applications that include but are not limited to:

  • Different user permissions and levels
  • Mobile and web apps
  • Paid and free plans
  • Email onboarding and contextual messages
  • Transactional emails built into the platform etc.

Such workflows require dozens if not hundreds of test email accounts to be set up. Not only before the product launch, but for every single change in the application that could impact email communication. Unless you have a dedicated team at your disposal that can be used solely for setting up and tracking throwaway accounts, this immediately disqualifies disposable email services. Typically, they vanish within up to a few hours. If you added 100s of coherenthamster@maildrop.cc type addresses to your codebase, they would be nothing but useless by your next testing session.

Another drawback of such solutions is that they have very limited design testing capabilities. Sure, most tools can preview plain text and HTML. This should let you spot the most notorious errors and safely test, for example, password reset emails. But, if you’re going to send emails to thousands of users who will open them on hundreds of devices, you will want to put in a bit more effort. Ultimately, you’ll need to test your emails with a separate tool before they’re ready to be delivered.

Inspect&Debug Your Emails

Mailinator is a bit of a different story as the email addresses are already created and they don’t disappear after your initial tests. The free tier raises security concerns though. In theory, anyone could read your confidential emails and click on links to your users’ profiles, without even registering.

A paid ($159/month) plan is a better option. This way, you can set up hundreds of email accounts and reuse them freely for your tests. API access will let you automate the process and you’ll be able to see all incoming test emails in a Super-Inbox without having to load each account. 

The issue, however, for both Mailinator and throwaway emails is sending thousands of emails to test accounts in the first place. These domains are not really known for a high sender reputation. Test emails, by definition, also have poor engagement. Both of these factors can seriously affect the deliverability of your legitimate emails.

And finally, there is still a risk of spamming real users with your test messages. Even the most proficient QAs and developers can miss a line of code here and there that handles some email workflow. If the reference to a real database is not replaced with a dummy email address for testing, real users will likely get your “hello world” message. Some might say “hi back!” but most will be rather disappointed with your lack of professionalism.

Can I set things up on my own?

For someone who’s had some DevOps experience in the past, the obvious approach might be to set up the test environment on their own, without relying on 3rd party tools. This would mean:

  1. Setting up an infrastructure to handle hundreds of email addresses on your domain
  2. Generating names and settings for all accounts
  3. Setting up rules for tracking if emails are received
  4. Adding these newly created addresses to your application
  5. Setting up an SMTP server to handle the sending of test emails
  6. Executing the QA tests on your new farm of dummy emails

The biggest hurdle here might be with setting up the SMTP infrastructure, especially if you want to do it on a budget. There are even free, ad-powered SMTP servers available to be used but, very likely, such emails won’t even make it through a basic spam firewall. As a result, you might end up investigating failed deliveries when everything, in fact, is set up properly. To get a reliable SMTP server to set things up, you’ll likely need to spend a few dozen bucks on a monthly basis (+ the regular server expanses).

Even if you do, you will still be facing some of the obstacles mentioned above. If emails are formatting-heavy, you’ll likely need to check each delivery individually. Even if the messages look fine on Gmail, you won’t be sure if they have the same look on Thunderbird or in Outlook, not to mention less popular browsers. If you were to check this manually, this would amount to quite a lot of work you likely can’t afford. Sure, you could integrate your testing environment with other tools to preview emails or check them for spam “eligibility” but this would mean extra time and likely additional subscriptions.

Last but not least, you would still be at risk of sending emails accidentally to real users for the same reasons outlined in the last chapter. 

Is there an alternative to dummy email testing?

Now, if you’re wondering if there’s an easier way to go about all of this, we think there is. Mailtrap was created precisely to overcome such obstacles and make the testing process a breeze rather than a nightmare.

Mailtrap is a software for testing your email workflows without the risk of accidentally reaching your users. It’s a fake SMTP server that can be integrated with your application to perform the needed tests. Since the server is fake, the emails sent don’t reach any real or temporary mailbox and don’t affect your deliverability in any way.

Try Mailtrap for Free

They stay on the Mailtrap platform, giving you clear information on what was delivered and how it looked like for end-users, regardless of the browser or app they used. Each email can be also previewed, tested for spam and forwarded to real addresses of your teammates. And all of these can be set up within an hour or so and is likely even more cost-effective than your own infrastructure built for the same purposes.

Learn why The Software House replaced other solutions with Mailtrap and streamlined their email testing processes from our case study.

Article by Andriy Zapisotskyi Growth Manager @Mailtrap