Replacing our legacy search engine with elasticsearch

We’re actually hoping you didn’t notice, but recently Marktplaats replaced its legacy search engine (called Voyager) with Elasticsearch.

Search lies at the core of our user experience and over the years our users have become accustomed to a certain behavior when they interact with the platform. At the same time features have been introduced that make use of search in various ways. The introduction of a new technology that has completely different possibilities for such an essential component poses a massive risk of disrupting all of this.

In this article I’m going to address the key decisions that we believe made this a smooth and rather uneventful process. We completed this migration with zero downtime and a small team (2-3 developers depending on how you count) in 3 months.

Why change?

Voyager was developed internally by eBay in the USA and was introduced at Marktplaats 6-7 years ago. It is very fast and scales quite well (just add hardware), but over the years the limited feature set has become more and more of a hindrance in bringing the best search experience to our users. Ebay itself has moved on from Voyager several years ago and thus development has stopped, while the number of people with in-depth knowledge about the system has dropped to a point that it has almost become a black box for us.

So Marktplaats and other eBay Classifieds Group (eCG) companies facing similar problems have been working together to look at other technologies to replace Voyager and other legacy systems used for search. We decided that Elasticsearch was the best option, where the rich feature set and the availability of support from both an active online community and commercial parties were key to us.

Keep it simple

Our approach to replacing Voyager can be summarized as “keep it simple”.  Elasticsearch offers some very exciting options for our search experience, and we will definitely try them out in the (near) future, but for the migration we decided to replicate the things we were already doing with Voyager as much as possible. This would reduce development time and allow us to compare the old and new platforms better. We believed that with relatively little effort we could get Elasticsearch to produce the same results as Voyager and thus keep any disturbances to a minimum.

We were helped in this by our service oriented architecture. We have several services that do things like manage users, send email, index advertisements or perform search. These services communicate with each other using Apache Thrift and asynchronous messaging via ActiveMQ or Kafka. By introducing new services that interact with Elasticsearch while keeping the same interfaces as their legacy equivalent, we were confident that we could keep the changes we had to make to the rest of the platform to an absolute minimum. This way we could also use the several hundred integration and acceptance tests we have been building up over the years. Any failures could then be easily attributed to our new search platform.

Build what you need first

Secondly, we decided to take things slowly by first covering the bare essentials before expanding our work to include all edge cases. For example, we first build the mechanisms to just get advertisements in an Elasticsearch index, without bothering too much yet about what fields should be indexed or how they should be analyzed. This allowed us to quickly explore what would be needed to set up an index from scratch and we did the first basic (re)index on production 3 weeks after we started development.

Test both old and new

Once it was established we had a good foundation, we started to include more functionality by just looking at the old code and replicating its functionality in the new services.

Once we believed we had covered everything, we ran all our existing integration and acceptance tests against them. We ran them on the same integration environment as the rest of the platform and by just flipping a switch we could direct all search traffic to  our new services and Elasticsearch or back to Voyager again.

This proved to be quite a shock at first. We made a friendly wager among ourselves and lets just say the most pessimistic (or experienced?) of us won, as failure was abundant. After we fixed these issues we repeated the process a couple of times, reducing the number of failures every time. In the end it turned out a lot of failures were caused by all kinds of hidden functionality in other parts of the system that we couldn’t detect in the code of the legacy services.

After we made our own test suites pass, we diverted our attention to the real test: our users. We had frequent debates on how we would ensure that our users would get the same results from the old and the new system.

A solution would be running live queries against both systems in parallel on production and recording the differences. This would give us accurate user behavior, but it also makes it very hard to reproduce any issue, since the contents of the index has more than likely changed between the original request and the time we would be able to have a look at it.

We settled on a solution where we would record actual user queries and run them against indices made from a copy of the production database. We also did not run all those queries at once, but in small random batches every 5 minutes. This way we would be able to quickly get an overview of potential issues and not be flooded by a large set of differences at once.

Switch gradually

Two and a half months after we started development we put 1% of our production traffic on Elasticsearch on a Friday. We kept it at that during the weekend and increased traffic gradually every day the next week until we were running at 50% at the end of the week.

All the time we were monitoring the performance of the new cluster as well as user metrics like page views and complaints received by customer support. Two minor bugs were identified this way and fixed quickly. We then flipped the switch to go to 100% while keeping the old platform running.

During the next couple of weeks we experienced very few issues; the most serious was a single node that started to eject items from the field data cache. We fixed it by increasing the size of the cache and a rolling restart. Users were not impacted as far as we could determine. At no point did we consider going back to the legacy system.

Conclusion

All in all this has been a very smooth process for us. If your planning to replace your legacy search engine with something different and are anxious about any impact it may have on your overall metrics, our approach might help you. But as always, your mileage may vary.

For us the next steps are exploring the possibilities we now have with Elasticsearch.

Girlsday 2015 @ Marktplaats

logo-girlsday

IMG_0498

‘GIRLSDAY’ is a European initiative: a day in which technical companies, research  and educational institutes open their doors to young girls aged 10-15 years in order to awaken their interest in science, technology and IT, and consider them as potential fields of work.

The second edition of the Marktplaats Girlsday took place on April 23rd, 2015 in our office. We hosted a wonderful group of 14 elementary school girls, mostly aged between 9 and 10 years old.  IMG_0504

The introduction was given by Alessandro Coppo – the General Manager of the Ebay Classifieds Group, with fun facts about Marktplaats and insights on how technology empowers people in our daily life: Alessandro made it very interactive and fun for the young audience.

 

paper_rembrandt_towerThe young students showed an incredible interest and curiosity when faced with the big numbers: for example, how many advertisements are currently live on Marktplaats?How many users visit the website? How many new advertisements per second are published on Marktplaats?
If one tried to print all the pages viewed on Marktplaats on a second, it would be as printing an encyclopedia!

If you would keep printing, the pile of paper would become taller than the Amsterdam Rembrandt tower after just one hour.

When the volunteers met before the Girls Day we asked what got us first interested in technology.  A common theme was that we liked the instant feedback of giving a command and seeing the result.

 

We felt strongly that the take away for the girls should be that you do not have to just be a consumer of technology but also help build what is to come next.

We had pre-installed the following on our macbooks

brew install cowsay

gem install lolcat

and at the start of the workshop introduced them to the concept that programmers tell a computer exactly what to do when they receive a certain command.

On the terminal they played with various commands to show what happened when they changed input, the resulting output and then how to pass the output  from one program to another as input.

echo “girls day”

cowsay girls day

cowsay girls day | lolcat

say howdy

Screen Shot 2015-05-26 at 11.29.57 AM

Once they got comfortable with changing commands, hitting return and getting a result we moved onto explaining how animation worked.  Using a physical flipbook of images with small changes, we showed that this is all the computer is doing whenever they are watching an animated film like Frozen.

Our project is available https://github.com/svandeven/girlsdayIMG_4737

which contains an example programme ./animation.py and some predefined ascii images of various variations

By deciding what order to show the images in – they could build an animation.  They then could chose to move the image up, down or change colour by adding the commands to the python script.

Next, the girls stated editing the images so that they could add their own ideas to the animation.

Finally the browsed for ascii images that they wanted to animated – we ended up with a team animating Spongebob eating ice cream.
IMG_4736

 

When we were done with the animation – the girls went online and played with http://lightbot.com/hocflash.html which also proved to be a huge hit.

The tour of the office was fun and it gave the girls a chance to think about the spaces which facilitate collaboration: they saw how the designers and testers work together and how the office space is set up to support and encourage a sharing of ideas in a comfortable work environment. The feedback was extremely positive from the girls, from the parents and the teacher. The organizers were are also very pleased with the interest the participants showed and the energy they have.

The result? The young girls were amazingly interested and grasped the concepts of simple coding and running a program very quickly.IMG_4738

 

We think the key to the success was having so many helpers available for them, and one computer for every 2 girls, so they could really try out things and type code, which they loved to do.  There was a lot of energy in the room and we had several different activities planned as we weren’t sure how long they would want to continue doing each activity.  This was very useful and necessary as it enabled is to adapt –  if the girls got a bit bored or started flagging we could switch to the next cool thing.

 

What about next year?

We were really happy with how the day went.  What could we maybe do to make it even better? One comment was that if we could turn the animation into making a game it would be a more finished program for the girls to be proud of.

Perhaps we could also have let them add some secret code to our website so they could show at home their names ‘hidden’ in the html to their friends and family.  Its good that we have a year to think how we can make it even better….

IMG_4743

Mobile app test automation

App Logo
Our iOS and Android apps are amongst the most popular apps in the Netherlands.  Our apps were launched in 2011, in 2012 we had 3 releases a year per platform.   This has now grown to releasing every 2-3 weeks.   The way we developed and tested the apps had to grow from a 2 person team to a cross functional team to cover iOS, Android , API development, testing, design and UX.

Regular Releases

We review the comments made in the App and Play stores and try to include fixes or new features that users have requested in every release.  This has led to a continual rise in our app store reviews and ratings.  To help increase the speed in which we can release, we have automated as much as we can.

In general, we release to Android first, with a small percentage of our users getting an update, we monitor for crashes or other issues and then increase the percentage of users that get the new app.  To become a Beta user and get early access to our new releases – go here.

Pipeline

When new app code is pushed to the develop branch – a test run is started in a remote device cloud, where we can choose various screen sizes, phone and tablet, different rotations, OS versions and device make and models.   If all of the regression test runs are green, then we will build a release candidate which is available to everyone in the teams to download and test with.

iPhone Device Cloud

We use the open-source test framework Calabash for our automated regression acceptance tests enabling us to write the same tests for Android and IOS despite different UI patterns on the different platforms.  The tests are run on a mixture of local devices and the Xamarin device cloud

Reporting of Tests

Nico

We use Jenkins for our CI environment, and all jobs are shown like this on monitors around the office

– Green background means the last run passed without failures,

– Red background means it had failures with the final failure count is shown.

– The dial icon means the tests are in progress

– Orange background with dial means it failed last run but no failures yet this run

– Red background with dial means its already failing and shows current failed test count.

This is really useful because at a glance we can see how ‘red’ a build is. We can already look at failures while a test run is ongoing.

By clicking on the box we can look at the failing tests

nico-2

We have the Test Feature name, Scenario name, screenshot and error message

On the right hand side there is an indication of the status of the last 30 runs.

You can see that this is not a flaky test but is actually failing because of a push 5 runs ago.

How does it work?

In Cucumber, we can use the ‘After’ hook to execute code after every test is completed.  We call our internal test reporting api with the details of the scenario, name, status (PASS|FAIL) and if failed a screenshot and stack trace of the error.

As a future improvements – we would like to know the memory usage and the CPU usage so we could show trends and highlight if we are suddenly using more memory than normal

Rerunning Failed Tests

Due to the fact that we are running against a test environment also used for backend testing, which can have services restarted or broken at any point, we have added a retry mechanism for failing tests.

In Jenkins, within the same job, when the first test run is completed the status is checked, if there are failed tests, then the second run is started for only the failed tests, when this is completed a third and final run is started if there are still failing tests.

Example from our jenkins console log

2015-03-17 18:49:56 +0100 Status: Testing: 1 running, 0 enqueued, 0 complete…

2015-03-17 18:50:06 +0100 Status: Finished Done!

Total scenarios: 21

20 passed

1 failed

Total steps: 63

Test Report: https://testcloud.xamarin.com/test/marktplaats-cal_8e943114-77b3-4811-bac3-a1517d6d0b32/

Should retry

2015-03-17 18:50:33 +0100 Status: Testing: 0 running, 1 enqueued, 0 complete…

2015-03-17 18:54:42 +0100 Status: Finished Done!

Total scenarios: 1

1 passed

0 failed

Total steps: 4

Test Report: https://testcloud.xamarin.com/test/marktplaats-cal_9b97440d-5091-4017-a309-de3df71584ff/

Should not retry

How does it work?

Cucumber allows us to use the ‘Around’ hook to determine if the test scenario should run or not.

Around do | scenario, block|

if should_run_scenario?(scenario)

block.call

end

end

We call our reporting api and get a json response of the scenarios in the previous attempts within the same Jenkins run and whether they passed or failed

Should_run_scenario? Returns true if it’s the first attempt or the test failed in the previous attempt within the same Jenkins run.

Finally

Mobile Automation is still quite early in its development but we have seen vast improvements in the stability and reliability of the tests we are running.   Its not at the same level of sophistication as browser testing with Selenium but its getting better and having the ability to run on devices rather than simulators or emulators has increased our stability and coverage.

Links

Cucumber – https://github.com/cucumber

Calabash – http://calaba.sh/

Xamarin – http://xamarin.com/test-cloud

Be the Customer – 5 things your teams can do to be more user-focused

User-centered design practices are incredibly useful for helping us solve problems for the people using the software and services we create.

Usercentered design (UCD) is a process (not restricted to interfaces or technologies) in which the needs, wants, and limitations of end users of a product, service or process are given extensive attention at each stage of the design process. (Wikipedia)

Thinking of UCD as only a concern of a UX and design department can undermine its effectiveness, to get the best results from UCD it should form a part of all the activities that can effect our customer’s perception of our products and services.

Of course practice will vary across the functions of the business, marketing, customer service, product and development will need different processes in place but some of the things we can do are universal and if your team does not do these any of things in some form already this is a great place to start.

1. EAT YOUR OWN DOG FOOD.

It seems really obvious but I still find most us do not use on a regular basis the product or service we create. For our teams at marktplaats.nl this means as a minimum buying and selling products on marktplaats.nl on a regular basis.

Ideally we should also be using our products in other personas. How do new users see our product, how does a business user experience our pro seller features.

2. FLY ON THE WALL

Take some time out of your job to follow some of the people that use your product or service for half a day. This will help you understand how your product fit into their daily life.   Even if you only manage to follow you mum you’ll have gained a valuable insight into the context in which your product or service is consumed.

So often when we think about a website or an app we concentrate on what the user sees on the screen. Shadowing show you what else effects the usability of your product, it could be user cannot read the screen because of daylight glare, they might get interrupted by the kids or avoid using your product in a public place. Contextualization helps us see how our product fits into the real world and enables teams to develop empathy for the people they are solving problems for.

 

3. TAILORS DUMMY

hobbyist[1]

My mum had a tailors dummy in the spare room next to her sowing machine. She adjusts it to the size of the person she was making clothes for.

Develop a consensus in your team about who your customers are, create and visualize your own personas and stick them on the wall in your team space. Try also to understand how your customers feel about your product, thinking about emotion (not just data) make your personas more human. Their presence will be great for discussion, when you talking about ideas you can walk over to your customers to see if they fit.

4. WHAT’S THE PROBLEM

A project often starts with a description of the thing you are going to make (A Solution). Without an understanding of the problem you are solving and the people you are solving it for its very difficult to understand and validate if a solution will work.

Start projects with a definition of the problem you are trying to solve and a description of the people you are solving it for and how your solution will make them feel.

This puts the user in the middle of the story.

5. DONE IS WHEN IT WORKS

We get very used to ending a project when we ship a product or feature, when a campaign goes live or a design goes off to the printer.  Any changes we make after a project is done often flow into a new work stream and have to compete with work on the next project.

Think of the project a done only you have validated what you have made works for the customer. Build validation into you project lifecycle and be prepared to iterate if you get it wrong the first time.

Why the site went down (or how to not send messages using camel)

Recently, we went down because of an issue that was hard to find.

The last release was the day before, so we didn’t directly suspect it to be release-related.

However, it was.
This is some new code that was in the release:

class UserActionNotifier(producerTemplate: org.apache.camel.ProducerTemplate, serialization: Serialization) {
 private[this] val userActionsEndpointUri: String = "activemq:topic:VirtualTopic.UserActions"

 def notifyOfUserAction(userId: Int, userActionType: String): Unit = {
 val ua = new UserAction(s"u$userId", 1, userActionType, new DateTime().getMillis)
 producerTemplate.sendBody(userActionsQueueEndpointUri, serialization.serialize(ua))
 }
}

Looks pretty innocuous, right?

Wrong.

What happens here is that we’re sending a message to a Virtual Topic in ActiveMQ.
When sending a message via a camel producerTemplate and passing in a String for the Endpoint, camel will do a lookup to get the actual endpoint to send to:

    protected Endpoint addEndpointToRegistry(String uri, Endpoint endpoint) {
        ObjectHelper.notEmpty(uri, "uri");
        ObjectHelper.notNull(endpoint, "endpoint");

        // if there is endpoint strategies, then use the endpoints they return
        // as this allows to intercept endpoints etc.
        for (EndpointStrategy strategy : endpointStrategies) {
            endpoint = strategy.registerEndpoint(uri, endpoint);
        }
        endpoints.put(getEndpointKey(uri, endpoint), endpoint);
        return endpoint;
    }

    protected EndpointKey getEndpointKey(String uri, Endpoint endpoint) {
        if (endpoint != null && !endpoint.isSingleton()) {
            int counter = endpointKeyCounter.incrementAndGet();
            return new EndpointKey(uri + ":" + counter);
        } else {
            return new EndpointKey(uri);
        }
    }

Source here and here (we are using Apache Camel 2.9.2).
As you can see, on line 15, it checks whether the Endpoint is a singleton. A Virtual Topic in ActiveMQ is not configured as a singleton, since its intent is to have multiple consumers.
However, when you’re sending messages, this goes wrong.

For each UserAction message that the service sends, a lookup is done to get the Endpoint and then a new EndpointKey is returned (with an incremented counter), which is added to the endpoints map (line 10).This leads to an ever-increasing map, which leads to the service blowing up and the site going down.

The following screenshots from Eclipse Memory Analyzer show the problem clearly:

Geert1

Geert2

In order to prevent this in the future, send messages using the actual Endpoint (instead of the endpoint name), like so:

class UserActionNotifier(producerTemplate: org.apache.camel.ProducerTemplate, context: org.apache.camel.CamelContext, serialization: Serialization) {

  private[this] val userActionsEndpoint: Endpoint = context.getEndpoint("activemq:topic:VirtualTopic.UserActions")

  def notifyOfUserAction(userId: Int, userActionType: String): Unit = {
    val ua = new UserAction(s"u$userId", 1, userActionType, new DateTime().getMillis)
    producerTemplate.sendBody(userActionsEndpoint, serialization.serialize(ua))
  }

}

Automated testing for responsive layout using Galen Framework

As browsing is increasingly done on mobile devices with varying screensizes – its important that our website is also ‘responsive’ – ie the layout adapts to the current screensize and everything still looks good and works.

At Marktplaats we are working on getting our website to adapt to mobile screens and become responsive.

Our website has quite a lot of pages so its extremely complicated and time consuming to test the layout for each page on all mobile and desktop browsers.   At the same time we have built a stable continuous integration environment where different levels of tests: unit, Selenium tests, services tests run on every code push.

As we don’t like to do everything manually we decided to automate the testing of layout for our responsive pages. To perform this task one of our engineers wrote the Galen Framework – an open-source layout testing tool based on Selenium. This framework has an expressive language for defining the layout of a page and it verifies locations of elements on the page relative to each other.

So how does it work?

We are all fans of TDD and we tried to follow the test driven approach for frontend redesign. Though layout testing is a bit different compared to acceptance or unit testing so we had to adapt TDD approach a bit. The way it works for us is like this: either a designer or frontend developer comes up with sketches of the page on different devices

Screen Shot 2014-05-27 at 22.56.40 Screen Shot 2014-05-27 at 22.55.20

These sketches are used later by frontend developers and QA engineers to write layout tests in Galen Framework. Here is an example of Galen test that checks the layout on 3 devices: mobile, tablet and desktop:

==============================================
form-header     css #content h1
form-box        css .box-content
form-label-*    css .edit-profile label.form-label
form-input-*    css .edit-profile input
form-hint-*     css .edit-profile .field-hint

button-save     css .buttonbar button
button-cancel   css .buttonbar #cancel-profile
===================================================

@ Edit Form | all
----------------------------------------------
form-header
    text is: Contactgegevens
    height: 35 to 55px

form-box
    below: form-header 0 to 10px

@ ^ | desktop, tablet
-----------------------------------------------
form-header
    color scheme: >70% #FAD6AC, < 10% #333333
    centered horizontally inside: screen 1px form-box
    centered horizontally inside: screen 1px button-save
    width: ~ 98px
    height: ~ 46px 
    near: button-cancel 10 to 15px left 
    inside: form-box ~ 20px left bottom 
    aligned horizontally all: button-cancel 1px 

button-cancel
    width: ~ 106px 
    height: ~ 46px 
    inside: form-box ~ 20px bottom 

@ ^ | mobile
----------------------------------------------- 
form-header 
    color scheme: > 60% white, < 10% #333333
    inside: screen 10px left right

form-box
    inside: screen 10px left right

button-save
    inside: form-box ~ 0px left right
    height: ~ 46px
    aligned vertically all: button-cancel 1px

button-cancel
    inside: form-box ~ 0px left right
    height: ~ 46px
    below: button-save ~ 10px

Reports

After the tests are written and the frontend is ready we can run the tests. In the end of test run we get reports like this:

Screen Shot 2014-05-27 at 14.30.29
A more detailed report for a specific test:
screenshot_1
An example of error feedback
Screen Shot 2014-05-27 at 14.33.24
A screenshot of the website with highlighted misbehaving elements
Screen Shot 2014-05-27 at 14.33.44

As you can see in the last screenshot the two buttons “Opslaan” and “Annuleren” are misaligned. That is what being spotted by our check “aligned vertically all: button-cancel 1px“.

Recap

There are a lot of pages currently on Marktplaats and the codebase is quite large. Of course we cannot completely rely on Galen Framework for layout testing and often we need to use manual testing to check the page layouts. But with current setup Galen Framework allows us to get a quick feedback if something goes wrong on the page and that saves time for our developers.

Sources

Galen Framework
Blog with Galen Framework tutorials

DevOps at Marktplaats

The journey

How did we get from this:

old-marktplaats

 

….to this?

new-marktplaats

 

TL;DR

We’re deploying to production multiple times per week, running tests after every commit, pushing only to master, not working on branches, you should do it too, it’s awesome.

pipeline-dashboard-edited

 


 

The DevOps story

When we started on the big Migration project, there was not a lot of communication between developers and system administrators (SiteOps). It kind of looked like this:

wall-of-confusion

 

It didn’t work, lots of confusion and miscommunication.

 

The next step was to put system administrators in the development teams. The developers really liked it, they could ask (stupid) questions whenever they wanted.

How did the SiteOps guys like it, you ask?

Not so much.

 

Where we are now:

We have a Sysadmin Support team which does releases and picks up the smaller tasks like environment configuration changes.

Next to this, there is a Sysadmin Solutions team for longer running, more architectural issues (e.g. configuring a Hadoop cluster).

 

As developers, we are pretty happy with the current situation, since the tickets we create are being picked up quickly.

 


How to release?

A big question in the DevOps movement is: how can we make it easier to do a release?

 

Here’s how we solved it:

  1. We have a service-oriented architecture.
  2. Developers push code to different services, to the master branch.
  3. The different services each have their own git repo, which is polled by Jenkins.
  4. Jenkins compiles, runs tests and deploys to the test environment (called ‘integration).
  5. Acceptance tests (e.g. Selenium tests) run against the full test environment.
  6. When these tests are green, a deployable tarball is created (with a specific version)*.
  7. The tarball is being deployed to the next environment (called ‘demo’).
  8. On demo, the integration with other platforms is checked. This is also where QA manually checks the stories that have just been implemented.
  9. The (versioned) tarball is deployed to the Load & Performance test environment.
  10. Load & Performance tests are run against the L&P environment.
  11. If all is green, it can be deployed to the production environment (this is a manual step).

 

* This is how the tarball is created:

  1. All services deployed to the test environment are checked for their version.
  2. Such a version consists of a git commit hash and a timestamp.
  3. The services are then copied from the test environment and packaged into a tarball.
  4. The tarball is versioned using a timestamp.
  5. The tarball is archived for later use.

 

This is our overview screen, to easily see what version of the application is deployed on which environment:

pipeline

 


Doing DevOps as a developer

Doing DevOps will not work if developers don’t change. Luckily, we did change.

These are some things that we’re doing differently than before:

Devs are making puppet changes (reviewed through gerrit)

Our local development environment is managed by puppet and configured to be as much as Production as possible (using boxenvagrant and homebrew (to name a few))

 

 

#1 Tip

Tip #1 for a developer to become more DevOps-savvy: setup a new environment. Some benefits include, but are not limited too:

  • direct communication and tight integration with sysadmins
  • learning the conventions and consistencies in naming
  • improve your knowledge of the communication flows within the application (hosts, ports, using proxies, db access, access to messaging system, firewall, etc.)
  •  beer more recognition from your SiteOps colleagues

 

 

Sources

devopsweekly.com

“Release It!” by Michael Nygard

news.ycombinator.com

Awesome colleagues 😉

iOS Accessibility

Our iOS and Android apps are the most popular Dutch apps in the Apple and Play stores.  Millions of Dutch people use our app everyday, including those with visual impairments.

We thought that we had a good accessibility app as we used the Accessibility labels for our automated testing.  Everything had a label so we thought that was good enough.

However, one of the unintended side effects of our iOS 3.0 app release was that we had unknowingly broken a lot of the functionality of our app when VoiceOver was on.

A visually impaired user contacted our customer services to complain that the app had become unusable and we asked him to help us. We visited him and watched how he used the app and where he got stuck.

Here is what we have learned.

Getting Started

Switch on VoiceOver shortcut.

Go to Settings > General > Accessibility > Accessibility Shortcut off > VoiceOver

This enables you to use the switch VoiceOver on anytime by clicking the home button 3 times quickly.

IMG_2661

Learn the gestures.

The basic navigation is a swipe from left to right across the screen anywhere on the screen.  When a page is loaded the first element is highlighted and the text is read out – if its an icon there should be an accessibilityLabel that is read out instead.

The swipe is used to move onto the next element to be read out and if they want to ‘touch’ that item then they double click with one finger anywhere on screen

Copy the native apps 

For us the mail app was the closest to our own.  We based how our app should behave by comparing to the mail app.

Lists

ListingResultIMG_2654

Each ‘result’ of the list should be read out as one VoiceOver text.  The different elements making up the result should be read out in order of importance. The user want to be able to work out as quickly as possible if they want to keep listening or move onto the next.

Our app was reading each individual text element so of the user wanted to skip that result then they had to skip 4 or 5 times to get to the next result.  The user should be able to drag a finger down the list and each click is a result – so they can skip the first 3 results in the list by dragging from top to bottom for 3 clicks.

IMG_2656IMG_2658IMG_2657 IMG_2659

Next Improvements

We hadn’t realised that it was also important to know where you were when you returned to this list.  In our app, the result opens an advertisement for an item for sale.  When the user wants to go back to the list – the back gesture is a two-fingered backwards Z that starts from the bottom and goes to the top of the screen – then the same result should be still selected and VoiceOver should read out the contents again. This verifies to the user that they are back at the same point and can continue scanning the list.

Accessibility labels for icons

Our screen for a single advertisement has an action bar at the bottom.

Ad Screenshot

To our eternal shame – even the Contact and Place Bid (Plaats Bod) buttons, the two most important functions on the screen, were not available to visually impaired users.  All the actions on the bottom bar were obscured by one label that was read out as “VIP floating bar”.

We saw that the user had to switch off VoiceOver, and select Contact, then switch VoiceOver back on again to be able to contact the seller about the item for sale.

We fixed this so that the 4 actions are read out as individual elements, Contact, Place bid, Add ad to my favorites, and Share.

Images

Screen Shot 2014-03-21 at 10.37.49 PM

We had written our own custom image picker.  This had given us the ability to select more than one image from the library at the time. A handy feature for most of our users but we lost entirely all the accessibility features the native image library has that we were not even aware of.

It was not possible to select a photo at all when VoiceOver was on.  The user had to switch it off pick and image and switch it on again.  There is meta data for an image which should be read out – containing the time it was taken and some idea of the sharpness and lightness of the image.  Its also possible to add a label to a photo from the native image library.  Our app should allow that label to be read out too.

Our solution is to detect if VoiceOver is on and use the native image library instead of our own.  This way we get all the native accessibility features for free.

Adding a label to an image

IMG_2662

Go to the native Photos app, turn on VoiceOver, navigate till an image is highlighted. This will read out the metadata.  To add a label you need to tap twice with two fingers but hold down on the second tap till you hear three pings.  A popup should appear where you can add text.  This one takes practice, 2 taps on its own just starts your Music playing.

Screen Shot 2014-03-21 at 10.37.31 PM

Next Improvements

Our user had a simple request for this screen.  When a photo has been added, read out the label and meta data – on the empty tiles say ‘add photo’.  We had everything saying tile 1, tile 2, tile 3, no matter if an image had been added already or not.

Conclusion

Try it for yourself or even better watch a real user using your app with VoiceOver. For visually impaired people the technology in their iPhone makes life easier for them.  Make sure your app does too.

 

Further reading

This is a great blog by Matt Gemmell about Accessibility on iOS apps.

http://mattgemmell.com/accessibility-for-iphone-and-ipad-apps/