Automated testing for responsive layout using Galen Framework

As browsing is increasingly done on mobile devices with varying screensizes – its important that our website is also ‘responsive’ – ie the layout adapts to the current screensize and everything still looks good and works.

At Marktplaats we are working on getting our website to adapt to mobile screens and become responsive.

Our website has quite a lot of pages so its extremely complicated and time consuming to test the layout for each page on all mobile and desktop browsers.   At the same time we have built a stable continuous integration environment where different levels of tests: unit, Selenium tests, services tests run on every code push.

As we don’t like to do everything manually we decided to automate the testing of layout for our responsive pages. To perform this task one of our engineers wrote the Galen Framework – an open-source layout testing tool based on Selenium. This framework has an expressive language for defining the layout of a page and it verifies locations of elements on the page relative to each other.

So how does it work?

We are all fans of TDD and we tried to follow the test driven approach for frontend redesign. Though layout testing is a bit different compared to acceptance or unit testing so we had to adapt TDD approach a bit. The way it works for us is like this: either a designer or frontend developer comes up with sketches of the page on different devices

Screen Shot 2014-05-27 at 22.56.40 Screen Shot 2014-05-27 at 22.55.20

These sketches are used later by frontend developers and QA engineers to write layout tests in Galen Framework. Here is an example of Galen test that checks the layout on 3 devices: mobile, tablet and desktop:

==============================================
form-header     css #content h1
form-box        css .box-content
form-label-*    css .edit-profile label.form-label
form-input-*    css .edit-profile input
form-hint-*     css .edit-profile .field-hint

button-save     css .buttonbar button
button-cancel   css .buttonbar #cancel-profile
===================================================

@ Edit Form | all
----------------------------------------------
form-header
    text is: Contactgegevens
    height: 35 to 55px

form-box
    below: form-header 0 to 10px

@ ^ | desktop, tablet
-----------------------------------------------
form-header
    color scheme: >70% #FAD6AC, < 10% #333333
    centered horizontally inside: screen 1px form-box
    centered horizontally inside: screen 1px button-save
    width: ~ 98px
    height: ~ 46px 
    near: button-cancel 10 to 15px left 
    inside: form-box ~ 20px left bottom 
    aligned horizontally all: button-cancel 1px 

button-cancel
    width: ~ 106px 
    height: ~ 46px 
    inside: form-box ~ 20px bottom 

@ ^ | mobile
----------------------------------------------- 
form-header 
    color scheme: > 60% white, < 10% #333333
    inside: screen 10px left right

form-box
    inside: screen 10px left right

button-save
    inside: form-box ~ 0px left right
    height: ~ 46px
    aligned vertically all: button-cancel 1px

button-cancel
    inside: form-box ~ 0px left right
    height: ~ 46px
    below: button-save ~ 10px

Reports

After the tests are written and the frontend is ready we can run the tests. In the end of test run we get reports like this:

Screen Shot 2014-05-27 at 14.30.29
A more detailed report for a specific test:
screenshot_1
An example of error feedback
Screen Shot 2014-05-27 at 14.33.24
A screenshot of the website with highlighted misbehaving elements
Screen Shot 2014-05-27 at 14.33.44

As you can see in the last screenshot the two buttons “Opslaan” and “Annuleren” are misaligned. That is what being spotted by our check “aligned vertically all: button-cancel 1px“.

Recap

There are a lot of pages currently on Marktplaats and the codebase is quite large. Of course we cannot completely rely on Galen Framework for layout testing and often we need to use manual testing to check the page layouts. But with current setup Galen Framework allows us to get a quick feedback if something goes wrong on the page and that saves time for our developers.

Sources

Galen Framework
Blog with Galen Framework tutorials