[Case Study] The Software House + Mailtrap

The Software House is a popular software development company based in Poland. They also are frequent users of Mailtrap, our email testing solution for dev teams. Interestingly, they found and implemented not one, but two distinct use cases for Mailtrap – one for their own needs, and another for their open-source product, Kakunin. 

The Software House – what do they do?

The Software House teams build software products and solutions for various types of customers, from startups to enterprises. 

They deliver a wide range of services, such as business analysis, web and mobile development, UI/UX design, DevOps, and quality assurance. 

The portfolio of the successfully implemented projects includes real estate and marketplaces platforms, an employee benefits system, a ticket sales CMS, language learning and business accelerator platforms, as well as a magazine subscription system.

Use Case I – Mailtrap as an SMTP testing environment

Most of the solutions that The Software House develop rely on email communication with end users. Account registration and management, password recovery, various notifications, such as product updates, weekly reports, or payment reminders are just some of the examples. What connects them all is that each needs to be thoroughly tested before it’s deployed.

Previously, The Software House set up their own dedicated SMTP servers to send emails, check and confirm activation links, and run other email tests in a testing environment. In addition, they used disposable email addresses and services. 

However, configuring and managing their own infrastructure was a time-consuming and expensive process. It took several hours just to set things up. On top of that, the team needed to regularly commit time to maintaining the infrastructure.

In addition, TSH (short for ‘The Software House’) opted to set up numerous disposable email addresses. This solution, however, couldn’t provide the appropriate level of security as such inboxes are publicly available. 

And, in general, the more emails were sent, the harder it was to find a particular message and share it with colleagues. There was also no easy way to re-channel emails, e.g. when they had deployed a project to another server and wanted to receive emails in a separate place.

Solution

Eventually, The Software House switched to Mailtrap as a dedicated email testing solution. They set it up to capture all emails going out from development and testing stages. The email use cases included:

• checking whether emails were sent to the appropriate users.
• verifying there were no duplicates. 
• reviewing how messages were displayed.

As they told us, they were impressed with the speed of setup. In 10 minutes, they had a ready testing inbox configured and integrated with their app. As the test emails kept flowing, they could view any of them, without the fear that the old messages would be lost.

TSH also opted to send emails from different environments and projects into separate Mailtrap inboxes, rather than pushing all emails into one folder. This made it easy to navigate around and quickly find the right messages.

As all the teams were involved in the testing process, this setup also greatly benefited their cooperation. A joint testing environment allowed them to freely access the right emails, quickly run new iterations, and forward emails when needed.

How does it work?

For web development, The Software House uses Node.js, PHP and ReactJS. As Mailtrap provides quick implementation samples for each of them, it was easy to integrate it with the first project.

Mailtrap is technology agnostic and compatible with any solution or framework that supports SMTP authentication. It’s integrated as a usual SMTP server, but it doesn’t deliver emails to the real recipients. Instead, it captures all the traffic before it’s passed to the SMTP servers.

As the results were promising, TSH soon expanded it onto the other projects. 

They have also integrated Mailtrap with the user acceptance testing servers, where emails are forwarded to the regular inboxes, but are still available in the Mailtrap inboxes. This way, all the messages are kept in one place, which is highly appreciated by testing teams.

Orchestrating so many environments is not an easy task, but The Software House is managing it on a great level. Mailtrap helps them with it by providing the possibility to easily switch inboxes between stages/servers. Usually, a separate Mailtrap inbox is used for each server – for example UAT, staging, or local.

Use Case II – Mailtrap as Kakunin’s default testing solution

End-to-end testing is one of the major areas of The Software House’s expertise, and they have built their own automated, open-source testing framework. Its name comes from the Japanese word for “confirm” and also “verify, certify, and validate”. Adam Polak, Head of the Node.js team, one of the main creators of the tool, is fond of Japanese culture, which is also reflected in the Kakunin web page design. 

Kakunin is a Protractor extension that enables easy writing of complex testing scenarios with minimum coding skills. Its goal is to empower people with little to no technical background to create automated tests from day one and eliminate the steep learning curve. Kakunin is based on the human-readable Gherkin language and JavaScript. 

As is often the case, testing scenarios also require a reliable solution for testing emails. Building their own infrastructure again wasn’t much of an option though. It would have meant a significant time investment in setting up and maintaining it, without a major benefit for their customers.

What was also important was the customer-facing part of the framework, responsible for emails. Users, often not very tech-savvy, would need an easy way to access such an environment and quickly check the results of their tests.

Finally, using the common email services for this purpose would inevitably lead to some false negatives. You can’t control all the aspects of a third-party inbox, and can’t access it via API. Automating access to the contents of an inbox would imply using web UI and DOM – not very reliable solutions and sensitive to changes.

Solution

As The Software House was already using Mailtrap for their own projects, it seemed natural to at least try it out for this use case.

Accessing Mailtrap via its UI isn’t an option in CI and automated test environments. Under such circumstances, you can’t rely on any manual activities and need to automate every test. For that reason, TSH opted to integrate Mailtrap via its API and run email checks automatically as one of the numerous features available within Kakunin.

We created it in this way so that a user creates a Mailtrap account on their own, then copies the inbox ID and the API key, and pastes them to their configuration in Kakunin. We added another layer of abstraction that allows a user to handle the majority of Mailtrap functionality without coding, and then those adaptors are used in the Gherkin steps. The integration with Mailtrap was quick and effortless!

Tomasz Górski Senior QA Engineer

TSH built a Mailtrap adaptor in the Kakunin framework. It allows testing email-related user scenarios in a safe environment and keeps test emails in one place. 

In addition, they added methods to get and filter emails without the need to go deeper into the Mailtrap documentation. Kakunin allows users to handle most of the Mailtrap functionality without coding. Depending on the project scale and complexity, such out-of-the-box implementation can save from a few hours to a week of testing time. 

How does it work?

TSH added the Mailtrap adaptor as the default email testing option, along with a set of generic steps that users can write in Gherkin. Without much actual technical expertise, Kakunin users can test any email feature of their application. 

The first step is to insert several vital details of your Mailtrap account and let Kakunin handle the rest.

Then, decide on the pages and elements to be included in your testing suite, for example:

const { BasePage } = require('kakunin');

class HomePage extends BasePage {
    constructor() {
        super();

        //URL of the homepage
        this.url = '/';

        //signup button in the header
        this.SignupButton = element(by.css('.button .button--secondary .header__button'));
        this.LoginButton = element(by.linkText('Log in'));
    }
}

module.exports = HomePage;

const { BasePage } = require('kakunin');

class SigninPage extends BasePage {
    constructor() {
        super();

        // URL of the login page
        this.url = 'signin';

        // signin form
        this.SigninForm = element(by.id('new_user'));

        // input fields
        this.EmailInput = element(by.id('user_email'));
        this.PasswordInput = element(by.id('user_password'));


        // submit button
        this.SubmitButton = element(by.name('commit'));

    }
}

module.exports = SigninPage;

const { BasePage } = require('kakunin');

class ProjectPage extends BasePage {
    constructor() {
        super();

        // URL of the project page
        this.url = 'companies/:projectId';

        // invite form
        this.InviteForm = element(by.css('.project_invite_form'));
        this.EmailInput = element(by.name('email'));
        this.SendInviteButton = element(by.buttonText('Send Invite'));

    }
}

module.exports = ProjectPage;

And describe the scenario you wish to test:

Create the test scenario:
Feature: invite user
@demo
Scenario: invite user to project

Given I visit the "home" page
And the "LoginButton" element is visible
And I click the "LoginButton" element
Then the "signin" page is displayed
And the "EmailInput" element is visible
And the "PasswordInput" element is visible
When I fill the "SigninForm" form with:
    | EmailInput | mailtrap+business@railsware.com |
    | PasswordInput | password |
And I click the "SubmitButton" element
Then the "inboxes" page is displayed

Given I visit the "project" page with parameters:
    | projectId | 817967 |
And the "project" page is displayed
And the "InviteForm" element is visible
And the "EmailInput" element is visible
And the "SendInviteButton" element is visible
When I fill the "InviteForm" form with:
    | EmailInput | invite-16a7f1@inbox.mailtrap.io |
And I click the "SendInviteButton" element

Then the email has been sent and contains:
    | subject   | t:Invitation to join |
    | html_body | t:your email address has been invited  |

In our example, Kakunin will launch a browser, log in to a Mailtrap account, and invite a new user to a project. It will then check automatically whether an invitation email was sent.

As you can see, it was very quick and painless. Kakunin is available for free under the MIT license. If you’d like to learn more about how it works, its documentation will be an invaluable resource.

Mailtrap can be used as a stand-alone tool and, as you can see, works seamlessly with many third-party solutions too. If you haven’t already done so, make sure you sign up for a free Mailtrap account today.