Testing Fast and Slow - Accessibility & Fiori

Jocelyn_Dart · ‎04-14-2016

Image courtesy of Danilo Rizzuti at freedigitalphotos.net

Latest Update: Adding my latest find... another automated readability checking site!

Accessibility testing checks your app doesn’t just look good. Accessible apps maximize the productivity of all your users. That’s a very important part of creating a truly great user experience.

Gain some tips, tricks, and tools for testing accessibility. Get a heads-up on what you need to do to plan for accessibility testing. HINT: Don’t leave it until the end of the project! Learn some fast quick-and-dirty test techniques that can be used by anyone, and the slow and thorough test techniques to use to ensure you are meeting legal and audit compliance requirements.

This is Phuong Hoang’s story.

Phuong was hired as a Tester on a SAP Fiori project. He’s never worked with SAP before let alone Fiori. Day 1 he starts his new job and told he will be testing accessibility – and that accessibility is a critical requirement for the project. Worse – accessibility compliance is needed to the highest possible standards – WCAG 2.0 AAA. Day 1 is the first time Phoung’s even heard the word accessibility!

So – deep breath – and he starts doing a little research into accessibility guidelines, and trying out a few things.

I first met Phuong 8 months later. By then Phuong was a critical member of the project, not only diagnosing accessibility issues but working out the appropriate fix and feeding that information back to the developers. In the process he has amassed a heap of knowledge about: accessibility testing tools; interpreting accessibility guidelines; and how to do realistic testing - as if you yourself had a disability.

In this blog I'll introduce some tips, tricks, and tools for testing accessibility in Fiori apps. Most of these are also good for testing accessibility in any web app. Along the way Phuong will give us some of his favourite tips and tricks to help identify and resolve accessibility issue. And I’ll briefly point out a few things you need to know when planning accessibility testing in your project.

Traditionally accessibility testing has been the domain of specialists. One of the great things about working with commonly available web development technologies is that there are lots of tools out there that anyone can use do quick accessibility sanity check on your Fiori app.

Of course there are some tools that do more thorough checks and need a little more practice. And there is some cool tech you can use to make the process a little more fun… but just as effective.

We’ll cover tools that help you check:

General accessibility through automated testing
Readability
Colour and contrast in your themes
Keyboard navigation
Zooming
Screen Reader provision

But before we get into that… let’s start with how you plan for accessibility testing in your project.

Planning Accessibility Tests

There are 5 main considerations:

Establish clear success criteria
Establish a test tracking mechanism
Make accessibility testing business-as-usual
Plan for team enablement
Plan for parallel tests to minimize test time

Establish Accessibility Success Criteria

Decide on your goal for accessibility

If you have formal legal requirement you may need to plan for a formal audit by accessibility professionals before go-live. For that you might want to contact the International Association of Accessibility Professionals (IAAP ) to find local experts. If it is a socially responsible best efforts approach, consider how you will motivate your team.

Decide on your compliance level

Which accessibility guidelines will you meet? There are many. Some of the most well-known include:

International standard WCAG 2.0 (which has 3 levels A, AA, and AAA)
USA standard section US508
German standard BITV 2.0

What are the agreed interpretations of those guidelines for your organization?

When do you apply each guideline?

Decide which assistive technologies and devices you will use for testing

Remember that these usually involve a combination of assistive tech software, web browsers, and device operating systems. Making sure you have the correct combinations and versions minimizes the issues you will encounter during testing.

Establish your test tracking mechanism

How will you track issues and resolutions?

Pass/Fail is not enough. How does Phuong track issues?

He uses a spreadsheet with rows listing each and every guideline.

Each app is given its own column with a Pass, Fail, or Not Applicable rating.

The final columns record the issues and resolutions.

This way it is easy to identify if there are common problems with certain guidelines. Resolutions can then be discussed centrally to ensure consistency in approach across all apps.

He also tracks pass/fail across different assistive technologies. Remember that assistive technology is software too. Tracking issues per assistive tech is particularly useful if you suspect that there is a bug in the assistive tech software itself.

Make accessibility testing business as usual from the beginning

Considering accessibility should be business as usual for both designers and developers.

Designers should create Personas, use cases, story boards and points of view that speak to persons with disabilities. User validation of these use cases is important – e.g. what happens if you assume that blind people only use the desktop, and then find that they mostly use tablets? Just like design saves time and costs, so does early accessibility testing.

Make it business as usual for developers to include automated accessibility tests, and some fast sanity checks in their own unit testing. This reduces testing effort for the dedicated accessibility testers doing slow tests.

Assign dedicated accessibility testers – like Phuong - who test each iteration of the design and development. These are the people who are responsible for knowing the fine detail of the guidelines and how they are applied. Make sure your accessibility testers have the information they need for testing Fiori apps: such as which keyboard shortcuts are used in keyboard navigation.

Plan for team enablement

Even if you are lucky enough to have experienced accessibility testers, your testers are likely to be new to Fiori. They may need an introduction to Fiori – e.g. how to launch apps, how to navigate, what appears in the header or footer, standard icons, etc.

For accessibility testing they will specifically need an understanding of how keyboard navigation and logical reading order typically work in Fiori transactions.

If your testers are new to accessibility testing, they will need a little time to familiarise themselves with the tools and assistive technology. Most are easy to use, however Screen Readers take more practice.

Plan for parallel testing as much as possible

Accessibility testing can be done in parallel to other app testing. Even better most accessibility tests do not depend on each other: Automated tests, Readability, Colour/Contrast, Zoom, Keyboard Navigation, and Screen Reader test on tablet/phone can be tested in parallel.

The exception is Screen reader testing on desktop which usually depends on keyboard navigation to be mostly working.

Allow extra time for testing. Testers need to run the tests on multiple types of assistive technology, and multiple devices. Testing on certain types of assistive technology is typically slower - yet another reason why you want to start accessibility testing as early as possible.

Planning your accessibility testing maximizes success by: making accessibility business-as-usual for your project; enabling your test team, and ensuring you are testing efficiently and consistently across the project.

Automated Tests as a First Step

Automated tests are a good to knock out a bunch of obvious issues … and a great first step for sanity checking your app before you get into the serious testing.

When it comes to accessibility testing, the most frequently asked question has to be “can I automate the testing?”. Well… Yes and No. Yes some testing can be automated, and that’s a great as a first pass to knock out obvious problems. But they are no substitute for human judgement and intuition.

Phuong describes the ratio as 30% automated, 70% manual. No need to just take Phuong’s word for it that automated testing is not enough. We have seen this same ratio stated by accessibility professionals we know and trust. There are just so many areas where deciding whether accessibility has been applied correctly or not requires human judgement.

“Most automated tools only test the technical things – that’s about 30% of the job.” http://www.accessibilityoz.com/ozart/#!

Many of the tools listed on the Web Accessibility Initiative site also reinforce the need for manual testing in addition to automated testing in their description or disclaimers. There are many automated test tools that test different aspects of accessibility. Here are a few:

Browser tools such as:

Device tools such as:

iOS – Accessibility Inspector in the iOS simulator
Google Talkback so far mostly just gives advice

Open Source alternatives such as:

Don’t get too concerned about finding the perfect tool. Any automated tool can only highlight potential problems. Many aspects of accessibility are use case specific and therefore human judgement is still required. Select one that helps test highlight basic issues and use it consistently.

What does Phuong use at the moment?

Automated tests typically check basics such as well-formed HTML document structure, correct use of header labelling, alt-text for images. Of course if you have followed the Fiori Design Guidelines and layouts correctly your app should already be a correctly structured HTML document.

Automated tests are a good start… but only a start. Let’s look at what else you need to add.

Testing Readability

Some guidelines require language to be readable to a certain level of schooling. This ensures your site is understandable by people with cognitive disabilities. It also minimizes misunderstandings and misreading of your site by other users. But how do you judge the reading level? Well there are some additional automated test tools Phuong likes such as:

Tip: You can set the readability statistics on in your Word preferences for proofing. These return a school grade level for each statement or question that can be checked against the compliance requirement for your guidelines.

Readability isn’t a matter of opinion – it can be easily measured and checked by automated testing. Now onto the manual tests.

Testing Theming – Colour and Contrast

There are some good simple tests for testing your theming changes fast. These are great for checking your theming changes.

Be warned – the more you adjust your custom theme, the more slow testing you will need to do. If you want to save time, avoid app-specific theming. You really don’t want to have to test each and every app as well as your standard theme.

Colour and contrast are some of the most important tests you can do to maximize the reach of your app to sighted users. They also impact some of the biggest number of hidden or undiagnosed disability users – such as those with colour blindness, ageing users, and other vision problems.

For instance, colour blindness (also known as Colour Vision Deficiency) affects 1 in 12 (8%) of males, 1 in 200 (0.5%) of females. So even without considering other disabilities good colour contrast improves the productivity of approx. 10% of your users. And of course good contrast helps all users read your app easily – even when the ambient lighting conditions are not ideal.

Fast testing:

Print screenshots in black and white
Apply greyscale theme

Slow testing:

Once you have identified potential problems with your fast testing, you want to check out how bad the problem is with a colour contrast checker. Phuong likes the Colour Contrast Analyser because it it has an eyedropper tool to pick up foreground/background colours straight from your screen and gives a clear pass/fail against the WCAG 2.0 guidelines at both AA and AAA levels.

You will also need to understand the large font rule. If you are using large fonts the colour contrast requirement is relaxed.

You can go the extra mile try testing your app using visual impairment filters to see your site as it will appear to people with different types of vision impairments. I really like SEE for this – another Chrome add-in.

What to watch out for when testing colour contrast:

Identify anything that’s difficult to see – those are things that you want to pass on to the slow testers

Can you see the focus halo fully when on every control? Inappropriate use of fixed height/width may crop the halo.

Quick tests for colour contrast can be easily done by designers and developers as part of sanity checking. These quick tests will focus your attention on resolving any issues via slow testing. Of all your accessibility testing, good contrast arguably provides the most immediate productivity boost to most of your people.

Testing Keyboard Navigation

Keyboard navigation is beloved by power users as much as by persons with disabilities.

Power users love keyboard navigation because it’s faster and more accurate to press a key than to move a mouse or even to swipe a touch screen. For persons with disabilities, they may not have the motor skills to use a mouse or even a touch screen, so for them, keyboard navigation is essential. And of course a lot of assistive tech including Voice Recognition software like Dragon Naturally Speaking use keyboard navigation as a proxy for accessing the screen.

VERY IMPORTANT: Make sure your testers know the standard SAPUI5 keyboard shortcuts to use first. Especially when to use [TAB] vs the arrow [] keys vs the [ESC] key. You can find more information on this in Keyboard Navigation enables the cool toys - Accessibility & Fiori

Fast testing test on a desktop device:

Put your mouse to one side. Can you run your app like a power user - using just the keyboard?

Slow testing on a desktop device:

Put your mouse to one side. Can you run your app using just the keyboard? Does the tab order make sense, i.e. does it follow the logical reading order? Are there any keyboard traps – e.g. perhaps you can enter a pop-up dialog with the keyboard but have to use the mouse to get out again.

Why don’t we have keyboard tests on other devices such as tablets or phones? Most of them use touch rather than keyboards.

By the way – don’t expect a keyboard attached to a tablet or smartphone to work in the same way as a desktop. The operating system of the device rarely supports keyboard attachments fully. It may not even support all your usual desktop keys.

Consider including a power user as one of your quick testers. They will be already motivated to make sure keyboard navigation works.

Testing Zoom

There are few things more annoying to the many of us with multiple devices than sites and apps that don’t zoom correctly.

The smaller the device the more important it is that zooming works. For your ageing workforce, zoom is a critical for productivity. Eyes naturally deteriorate with age. % of population with eye defects – shortsighted…. Remember thanks to contacts, you may not know how many people you are helping.

Fast testing:

On desktop, set your browser zoom to 200% and test your app.

On Tablet and Phone, use pinch zoom to check your app zooms correctly.

Slow testing:

Use screen magnification software such as ZoomText or MaGic by Freedom Scientific.

What to watch out for when testing zoom:

At 200% when you use dropdowns or other pop-up dialogs, can you still reach all the buttons and features of the popups? Does the scrollbar show appropriately if the data would otherwise go over the edge of the screen? Is the cursor focus halo cropped when on controls towards the edge of the screen?

When messages pop-up, do they disappear after a set time, or if the user accidentally clicks on another part of the screen.

Remember that users using screen magnification software may only be able to see a very small portion of the screen at any one time. If your message pop-ups out of view, they may never see it. Tip: Notification bars are often a better option than message toast for important messages.

Zooming needs to work so users can see the detail – no matter what device or the condition of their eyesight.

Testing Screen Reader Provision

When it comes to the content of labels there is one consideration that overrides all others – is the label meaningful to the end user.

Having a field read something is easy. Making it read something useful is the real test. OF course making sure the label is truly meaningful is the job of the designer. But the developer needs to ensure the label is correctly associated with the relevant field.

Fast testing:

Turn on your screen reader. On desktop we like to use the free source NVDA. On tablet, you can use iOS Voiceover or Google Talkback.

Focus your cursor (on desktop) or touch (on tablet/phone) on each editable control, button, and image. Does the screen reader read the text you expected?

Preparing for Slow Testing:

It’s important to do keyboard navigation testing first. This must be working in order to do screen reader testing on desktops.

Remember that each Screen Reader is a software program itself. Desktop Screen Readers may depend on web browser capabilities to expose what they use. So even once you are sure that your Fiori, SAPUI5 or OPENUI5 app is setting the correct ARIA properties, a particular Screen Reader may or may not choose to use that information. And of course, as they are themselves software they are not immune from version dependencies and software bugs.

This means you need to test with each Screen Reader you plan to use, including both desktop and mobile devices.

IMPORTANT: Do not assume vision impaired people do not use devices – actually often a mobile device is far more convenient.

Slow testing:

Use a screen reader just like a blind person would by turning off the monitor (desktop) or turning on the screen curtain (tablet/phone) so that you only have your ears to hear.

Make sure you learn the Screen Reader functionality first

Each Screen Reader has hotkeys so the user can tell it to read the whole screen, or just the current field, or optional descriptive text. You need to know these hotkeys to work the screen reader correctly

Make sure you test the whole app functionality with the Screen Reader turned on

Occasionally we have found issues where Screen Reader software negatively impact the coding or behaviour of specific screen controls, so you make sure all the app features work as expected with the Screen Reader on

Testing like a blind person is not for everyone. You will need someone who’s used to working with a screen reader to do that - in other words a dedicated Accessibility Tester like Phuong. HINT: You may want to supply your testers with headphones.

What to watch out for when testing:

Does the screen reader announce the control correctly when the cursor is positioned on the control?
Does the announcement make sense even when the control is blank?
Does every control work when the screen reader software is turned on?
For mandatory fields, does the screen reader announce that the field is required?
If the field is empty, does it read out the placeholder?

Screen reader testing is the most time consuming of all accessibility testing. It helps the most vulnerable of your users – blind people in an increasingly visual world. Don’t forget screen reader capability also provides some benefits for other users. Placeholders minimize confusion over what to enter into an empty field. And having the option of using a Screen Reader is great for situations where you want to work hands-free. These days with Screen Readers built into the operating system of most devices, working hands-free is an increasingly popular choice.

Some Final Thoughts

A lot of accessibility testing can be done by anyone with a little knowledge and access to a few readily available tools. Better yet, fast testing not only rapidly identifies major problems, it minimizes slow testing by identifying where testers should focus their attention.

So how should accessibility testing impact your Fiori/SAPUI5/OPENUI5 project

Make accessibility a part of everyone’s responsibility. There’s no good excuse for designers or developers or anyone else to pretend accessibility is someone else’s problem. Accessibility makes a better, more robust, UX for everyone. Accessible apps improve productivity – because they make it easier for all sorts of people, software, and devices to do their job more easily.

Expect that you will need dedicated accessibility testers for testing with specialized assistive technology such as Screen Reader.

If you have a legal requirement to comply with accessibility we recommend considering an external audit by accessibility professionals.

There’s one last thing you should really consider doing if you are serious about accessibility...

Include some people who actually have disabilities. If possible have them on your testing team.

Tip: You might only involve them towards the end of your testing when you think you have solved the worst of the problems. If that’s not an option, at least make sure your accessibility testers liaise with them to find out how they really use assistive tech, and what features or options work best for them.

After all these are your real end users – the people who will actually be using your app. Because if you want effective user experiences, there’s no substitute for validating your app with the real end users.