WAI-Tools Documentation of WCAG Interpretation and Test Rule

As a part of the WAI-Tools project, the Norwegian Digitalisation Agency has developed documentation which contains details on WCAG interpretation, test rules developed in the WAI-Tools project, and mapping between both of them. In addition, the agency has performed and documented a gap analysis between WCAG success criteria and test rules developed in the project.

Published: 15. Oct 2020, Last modified: 23. Oct 2020

Preface

As a part of the WAI-Tools project, the Norwegian Digitalisation Agency (previously the Agency for Public Management and eGovernment – Difi) constitute the documentation of their internal interpretation of the Web Content Accessibility Guidelines (WCAG) and Accessibility Conformance Testing (ACT) Rules. This documentation is the part of deliverable “D2.4 Working Instance of Digdir Observatory” under Work Package 2 in the WAI-Tools project.

The purpose is to document the Agency’s interpretation of the WCAG success criteria and coverage of these success criteria by ACT Rules. In addition, the gap analysis between WCAG success criteria and ACT Rules has also been documented, such as aspects of the success criteria that are not covered by ACT Rules in WAI-Tools and aspects of the ACT Rules that go beyond the scope of the success criteria. This documentation is subdivided into one chapter per WCAG success criteria. Each chapter defines:

  • specific details relevant to WCAG success criteria,
  • purpose of success criteria,
  • user accessibility needs (functional performance statements (fps)) (based on EN 301 549),
  • interpretation and specification of the success criteria (Norwegian Digitalisation Agency),
  • coverage of Success Criteria by ACT rules developed in WAI-Tools,
  • gap analysis, and ACT Rules related to each WCAG success criterion.

The Norwegian Digitalisation Agency’s existing interpretation and specification of WCAG success criteria are limited to WCAG 2.0 with a brief overview of associated test procedures. The Norwegian Digitalisation Agency also has a fundamental interpretation and specification of the new WCAG 2.1 success criteria but does not have associated test procedures. Our contributions to the test rules development were based on our existing interpretation of the WCAG 2.0 and 2.1 success criteria.

Each chapter of this documentation has been written using a template specially designed for this documentation. This template contains two modules, one for WCAG success criteria and the other one for ACT Rule. To understand how these two modules have been filled out by the authors, the interested readers are referred to the template which can be found in Chapter 55 at the end of this documentation.

Background

This documentation has been written between 01 July 2020 and 31 October 2020 as one of the main results of deliverables in a European Commission (EC) co-funded project named Web Accessibility Initiative - Advanced Decision Support Tools for Scalable Web Accessibility Assessments (WAI-Tools) Project under Horizon 2020 Program (780057).

WAI-Tools Project

WAI-Tools is an Innovation Action project that has been active for 39 months between 1 November 2017 and 31 January 2021.

In WAI-Tools Work Package 1 the objective is “to build, implement, and validate a set of open test rules, developed through the international consensus process of W3C”. These test rules are developed by project staff through the W3C consensus process, implemented in open source engines developed and maintained by Deque, FCID (University of Lisbon), and Siteimprove.

The Norwegian Digitalisation Agency (previously the Agency for Public Management and eGovernment – Difi) is responsible for deliverable “Documentation of Digdir’s WCAG interpretation and ACT Rules” under deliverable “D2. Deployment of Test Rules” under Work Package 2. In this documentation, interpretation, and mapping of test rules developed under this project to WCAG 2.0/2.1 have been documented.

Web Accessibility Directive (WAD)

The Web Accessibility Directive (WAD), Directive (EU) 2016/2102, in force since 22 December 2016, aims to provide people with disabilities with better access to the public services websites and mobile applications. The Directive covers websites and applications of the public sector with a limited number of exceptions e.g. broadcasters, live streaming.

It is also required in WAD and its corresponding, that the member states are required to monitor the conformity of websites and mobile applications of public sector bodies with the accessibility requirements provided for in Article 4 of Directive (EU) 2016/2102 using a simplified monitoring method to detect non-compliance and an in-depth monitoring method to verify compliance.

Article 6 of Directive (EU) 2016/2102 refers to European standard EN 301 549. The content of websites and mobile applications meets harmonised standards and shall ensure at least a level of accessibility equivalent to that ensured by European standard EN 301 549, which in turn refers to WCAG 2.1 success criteria. In other words, it has been statutory throughout Europe for public sector bodies websites to be designed in accordance with WCAG 2.1.

In addition, we know from our experience and with reference to paragraph 56 of Directive (EU) 2016/2102 that there are different interpretations of WCAG success criteria across member states which requires harmonisation to monitor conformity. WAI-Tools is an initiative of a harmonized interpretation of WCAG, with associated test rules. This interpretation of WCAG is implicit in the test rules developed in WAI-Tools.

The terms compliance and non-compliance are not defined in Directive (EU) 2016/2102. The directive refers to European standard EN 301 549, where the determination of compliance is defined in the following way:

“Compliance is achieved either when the pre-condition is true and the corresponding test (in Annex C in EN 301 549) is passed, or when the pre-condition is false (i.e. the pre-condition is not met or not valid).”

Footer inn her

For each of the requirements regarding web pages, the pre-conditions and tests are stated in the following way:

Type of assessmentInspection
Pre-conditions1. The ICT is a web page.
Procedure1. Check that the web page does not fail WCAG 2.1 Success Criteria X.Y.Z [Name of Success Criteria]
Result

Pass: Check 1 is true

Fail: Check 1 is false

Web Content Accessibility Guidelines (WCAG) 2.0/2.1)

The Web Content Accessibility Guidelines (WCAG) standard is developed through a W3C process in cooperation with individuals and organisations around the world. WCAG defines guidelines to make web content more accessible for people with disabilities. These accessibility guidelines cover a wide range of disabilities such as visual, auditory, physical, auditory, speech, cognitive, language, learning, and neurological disabilities.

WCAG 2.0 was published on 11 December 2008. WCAG 2.1 was published on 5 June 2018. There are 13 guidelines organised under 4 principles: perceivable, operable, understandable, and robust. For each guideline, there are testable success criteria, which are at three levels of conformance: A, AA, and AAA. All requirements (“success criteria”) from 2.0 are included in 2.1. The 2.0 success criteria are exactly the same (verbatim, word-for-word) in 2.1.

The accessibility requirements in WCAG 2.0/2.1 are mostly written in a technology-neutral language which requires interpretation of what exactly is required to comply with these requirements especially when it comes to specific technology.

Accessibility Conformance Testing (ACT) Rules

An ACT Rule is a plain language explanation of how to test a specific type of content for specific aspects of WCAG accessibility requirements (“success criteria”). An ACT Rule describes the expected results of accessibility testing tools and/or methodologies when running a conformance test against a specific part of the applicable test subject. In the context of WCAG, ACT Rules test for failures in satisfying Success Criteria. ACT rules are written for the testing process and usually more specific and limited to a single aspect of a WCAG accessibility requirement. Each ACT Rule contains a description of the type of content under test, the test to perform, and the expected results.

The ACT Rules were written and developed by the ACT Rules Community Group (ACT-R) using W3C ACT Rules Format. One of the objectives of this project was to establish an open community by bringing together key partners from industry, government, and research, who can develop authoritative resources on accessibility conformance testing (ACT).

The ACT Rules Community Group (ACT-R) is an open forum with the aim to document and harmonise the interpretation of W3C accessibility standards such as WCAG and WAI-ARIA. ACT-R is an initiative to keep working on the development of ACT Rules and harmonising the interpretation of W3C accessibility standards even though the WAI-Tools project has been ended. This long-term process of development of ACT Rules will continuously increase the coverage of WCAG success criteria requirements which later can be used not only in WAD simplified but in in-depth monitoring as well.

According to W3C ACT Rules Format, there are two types of ACT Rules:

  • Atomic rules describe how to test a specific type of solution such as website content. It contains a precise definition of what elements, nodes or other “parts” of a test subject are to be tested, and when those test targets are considered to pass or fail the rule. Atomic rules are to be kept small and specific. These rules test a single “condition” and do so without using the outcomes from other rules.
  • Composite rules describe how the outcomes of multiple atomic rules can be combined into a single outcome for each test target. In short, a composite rule can have multiple “conditions”, each of these tested in separate atomic rules.

The implementation of ACT Rules is based on test mode. The Test Mode of a rule describes how a test was carried out, in the case of ACT Rule, implementation can be automated, semi-automated, or manual. In an automated implementation, the test is carried out automatically means such as software tools and without any human intervention. In a manual implementation, the test is carried out by human evaluators, it not just includes actual test procedure carried out by the evaluators but also where the evaluators helped by instructions or guidance provided by software tools. An implementation is considered as semi-automated, where the test is carried out with the help of software tools, but human evaluation or judgment is still required to decide the outcome of the test.

The outcome of evaluation using ACT Rule on a test subject or one of its constituent test targets can be one of the three following types:

  • Inapplicable: No part of the test subject matches the applicability of the rule
  • Passed: A test target meets all expectations of the rule
  • Failed: A test target does not meet all expectations of the rule

The outcome of an ACT Rule “Passed” does not mean that a test subject is compliance with all the applicable requirements of WCAG success criteria but only compliance with a specific aspect of WCAG success criteria covered by that ACT Rule. There are a few manual ACT Rules that possibly fully satisfy some of the WCAG success criteria. However, in general, “Passed” and “Inapplicable” outcomes of ACT Rules require further testing or manual checks.

ACT Rules are open source and can be used by member states and other organisations to implement these rules in the development of manual testing procedures, semi-automatic and/or automatic testing tools.

Summary of our findings

The WCAG success criteria are generic, complex, and require a wide range of various aspects and/or situations to be compliant. It is hard to cover all the aspects and/or situations required by WCAG success criteria, ACT Rules developed in the WAI-Tools project are very specific and atomic to a specific aspect and/situation that cover a small portion of WCAG success criteria. There can be several ACT Rules to cover all applicable aspects of WCAG success criteria accessibility requirements, which means that there could still be a gap in compliance with all aspects of a WCAG success criterion.

Currently, the ACT Rules developed in WAI-Tools can be used in and limited to testing contents created using web technologies, such as Hypertext Markup Language (HTML), Cascading Style Sheets (CSS-2018), Accessible Rich Internet Applications (WAI-ARIA), Scalable Vector Graphics (SVG2).

There have been 07 Composite ACT Rules (uses outcomes from 26 Input Rules that are atomic ACT Rules) and 49 Atomic ACT Rules developed until October 2020 in the WAI-Tools project. In total including both composite and atomic, there are 56 ACT Rules that have been documented in this documentation and covers compliance and/or non-compliance of certain accessibility requirements of 30 WCAG success criteria (SC). The documentation includes 4 atomic ACT Rules that are not related to any WCAG success criteria conformance.

The WAI-Tools project is moving forward, and the goal is to develop 70 ACT Rules by the end of January 2021. The overview of ACT Rules that have been documented in this documentation along with WCAG success criteria covered by these rules are shown in Table 1 below where N/A stands for not applicable:

NOTE: We documented each ACT Rule in this documentation when the rule has been completed after the intensive development process established by the project. There have been many changes along the way until the end of the project period. We may not update all the changes that were made after the ACT Rules are documented in this documentation. Therefore, the dates when the ACT Rule was last updated (completed) and documented in this documentation are also added in each ACT Rule module.

Note: Links in the "ACT-R ID" column of the following Table 1 navigates to the external webpage (ACT Rules website) and links in the "ACT Rules Name" column of the following Table 1 navigates to the internal section of contents (ACT Rules) of this document.

Table 1. Overview of ACT Rules and WCAG success criteria
Rule No.ACT-R IDACT Rule NameWCAG SCType
15c01eaARIA state or property is permittedN/AAtomic
273f2c2autocomplete attribute has valid value1.3.5Atomic
397a4e1Button has non-empty accessible name4.1.2Atomic
4e086e5Form field has non-empty accessible name4.1.2Atomic
52779a5HTML page has non-empty title2.4.2Atomic
6b5c3f8HTML has lang attribute3.1.1Atomic
75b7ae0HTML page lang and xml:lang attributes have matching values3.1.1Atomic
823a2a8Image has non-empty accessible name1.1.1Atomic
9c487aeLink has non-empty accessible name4.1.2, 2.4.4, 2.4.9Atomic
10bc659ameta element has no refresh delay2.2.1, 2.2.4, 3.2.5Atomic
11674b10Role attribute has valid value4.1.2Atomic
124e8ab6Element with role attribute has required states and properties4.1.2Atomic
13de46e4Element with lang attribute has valid language tag3.1.2Atomic
14bf051aHTML page lang attribute has valid language tag3.1.1Atomic
156cfa84Element with aria-hidden has no focusable content1.3.1, 4.1.2Atomic
16cae760iframe element has non-empty accessible name4.1.2Atomic
183ea0c8id attribute value is unique4.1.1Atomic
195f99a7aria-* attribute is defined in WAI-ARIAN/AAtomic
206a7281ARIA state or property has valid valueN/AAtomic
21c3232fvideo element visual-only content has accessible alternative1.2.1Composite
fd26cfvideo element visual-only content is media alternative for textN/AAtomic
ee13b5video element visual-only content has transcriptN/AAtomic
ac7dc6video element visual-only content has description trackN/AAtomic
d7ba54video element visual-only content has audio track alternativeN/AAtomic
22e7aa44audio element content has text alternative1.2.1Composite
2eb176audio element content has transcriptN/AAtomic
afb423audio element content is media alternative for textN/AAtomic
23eac66bvideo element auditory content has accessible alternative1.2.2Composite
f51b46Video element auditory content has captionsN/AAtomic
ab4d13Video element content is media alternative for textN/AAtomic
24c5a4eavideo element visual content has accessible alternative1.2.3Composite
1a02b0Video element visual content has transcript1.2.8Atomic
1ea59cVideo element visual content has audio descriptionN/AAtomic
ab4d13Video element content is media alternative for textN/AAtomic
f196ceVideo element visual content has description trackN/AAtomic
251ec09bVideo element visual content has strict accessible alternative1.2.5Composite
1ea59cVideo element visual content has audio descriptionN/AAtomic
ab4d13Video element content is media alternative for textN/AAtomic
f196ceVideo element visual content has description trackN/AAtomic
2680af7bFocusable element has no keyboard trap2.1.2Composite
ebe86aFocusable element has no keyboard trap via non-standard navigationN/AAtomic
a1b64eFocusable element has no keyboard trap via standard navigationN/AAtomic
27e6952fAttribute is not duplicated4.1.1Atomic
28c4a8a4HTML page title is descriptive2.4.2Atomic
29cc0f0aForm control label is descriptive2.4.6Atomic
3159796fImage button has non-empty accessible name1.1.1, 4.1.2Atomic
32ff89c9Aria required context role1.3.1Atomic
33bc4a75Aria required owned element1.3.1Atomic
34ffd0e9Heading has non-empty accessible name1.3.1, 2.4.6Atomic
357d6734svg element with explicit role has non-empty accessible name1.1.1Atomic
369eb3f6Image filename is accessible name for image1.1.1Atomic
38b20e66Links with identical accessible names have equivalent purpose2.4.9Atomic
394b1c6ciframe elements with identical accessible names have equivalent purpose4.1.2Atomic
40e88epeImage not in the accessibility tree is decorative1.1.1Atomic
4180f0bfaudio or video avoids automatically playing audio1.4.2Composite
4c31dfaudio or video that plays automatically has a control mechanismN/AAtomic
aaa1bfaudio or video that plays automatically has no audio that lasts more than 3 secondsN/AAtomic
42b33effOrientation of the page is not restricted using CSS transform property1.3.4Atomic
44a25f45headers attribute specified on a cell refers to cells in the same table element1.3.1Atomic
45d0f69eTable header cell has assigned data cells1.3.1Atomic
47b4f0c3meta viewport allows for zoom1.4.4Atomic
4836b590Error message describes invalid form field value3.3.1Atomic
49fd3a94Links with identical accessible names and context serve equivalent purpose2.4.4Atomic
5059br37 Zoomed text node is not clipped with CSS overflow1.4.4Atomic
515effbbLink in context is descriptive2.4.4,
2.4.9
Atomic
53ffbc54No keyboard shortcut uses only printable characters2.1.4Atomic
54c249d5Device motion based changes to the content can be disabled2.5.4Atomic
55efbfc7Text content that changes automatically can be paused, stopped or hidden2.2.2Atomic
5746ca7fElement marked as decorative is not exposedN/AAtomic
580ssw9kScrollable element is keyboard accessible2.1.1Atomic
628fc3b6Object element rendering non-text content has non-empty accessible name1.1.1Atomic
65ucwvc8HTML page language subtag matches default language3.1.1Atomic
677677a9Device motion based changes to the content can also be created from the user interface2.5.4Atomic
68qt1vmoImage accessible name is descriptive1.1.1Atomic

Further use of this documentation

This section describes the value of this documentation and suggests how it can be used and further developed.  After WAD was introduced and adopted by the member states, the accessibility of websites, applications, and ICT services is becoming one of the legal requirements around the member states in Europe.

According to WAD and the corresponding implementing act, all member states are required to perform monitoring of websites and mobile applications. This monitoring requirement of WAD made it more important for the member states to have a harmonized testing methodology to perform consistent monitoring.

ACT Rules can be used to monitor the degree of compliance with WAD accessibility requirements, which will make it possible for the organisations to know how to fulfil accessibility requirements required by the directive in a consistent manner. Monitoring bodies in member states can use this documentation of ACT Rules and interpretation of accessibility requirements to provide guidelines to businesses and perform controls, audits, and monitoring in their countries.

The Norwegian Agency tasks also include to perform a pilot for simplified and in-depth monitoring in line with Directive (EU) 2016/2102 on a limited set of websites under Work Package 2, to establish a “demonstrator”. The testing of websites was performed using testing tools and/or open-source engines developed by Deque, FCID (University of Lisbon), and Siteimprove that have adopted the test rules developed under the project. Note that the pilot monitoring documentation which includes a description of the sampling approach, testing, analysis, data model, and reporting results is delivered as a separate deliverable in the project.

These efforts provide valuable input and documentation on how a monitoring body enforces the Web Accessibility Directive through the use of the ACT Rules and deliverables from the WAI-Tools project. The complete documentation of WCAG interpretation and test rule is documented in a PDF document. 

References

  •  W3C, Web Accessibility Initiative (2020). Web Accessibility Initiative - Advanced Decision Support Tools for Scalable Web Accessibility Assessments (WAI-Tools) Project. Retrieved 15.10.2020, from https://www.w3.org/WAI/about/projects/wai-tools/

  • European Commission (2017). Grant Agreement number 780057 – WAI-Tools.

  • How WAI Develops Accessibility Standards through the W3C Process: Milestones and Opportunities to Contribute. Retrieved 15.10.2020, from https://www.w3.org/WAI/standards-guidelines/w3c-process/

  • Interpretation and overview of test procedures for WCAG 2.0 A and AA. Retrieved 15.10.2020, from https://uu.difi.no/om-oss/english/interpretation-and-overview-test-procedures-wcag-20-and-aa

  • Accessibility Conformance Testing (ACT) Rules Format 1.0. Retrieved 15.10.2020, from https://www.w3.org/TR/act-rules-format/

  • ACT Rules Community (2020). Retrieved 15.10.2020, from https://act-rules.github.io/pages/about

  • ACT Rules Community (2020). Rules. Retrieved 15.10.2020, from https://act-rules.github.io/rules/

  • W3C, Web Accessibility Initiative (2020). Evaluation and Report Language (EARL) 1.0 Schema. Retrieved 15.10.2020, from https://www.w3.org/TR/EARL10-Schema/

  • European Commission (2016). Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies.

  • European Commission (2018). Commission Implementing Decision (EU) 2018/1523 of 11 October 2018 establishing a model accessibility statement in accordance with Directive (EU) 2016/2102 of the European Parliament and of the Council on the accessibility of the websites and mobile applications of public sector bodies.

  • European Commission (2018). Commission Implementing Decision (EU) 2018/1524 establishing a monitoring methodology and the arrangements for reporting by Member States in accordance with Directive (EU) 2016/2102 of the European Parliament and of the Council on the accessibility of the websites and mobile applications of public sector bodies.

  • European Commission (2018). Commission Implementing Decision (EU) 2018/2048 of 20 December 2018 on the harmonised standard for websites and mobile applications drafted in support of Directive (EU) 2016/2102 of the European Parliament and of the Council.

  • ETSI, CEN & CENELEC (2018). European standard EN 301 549 V2.V2.1.2 (2018-08) Accessibility requirements for ICT products and services.

Sharethis

Fant du det du lette etter?

Stjerne(*) er obligatoriske felter som du må fylle ut for å sende skjemaet. 

MERK: Du får ikke svar på denne meldingen. Har du spørsmål du ønsker svar på, send en e-post til uu@digdir.no.

Fant du det du lette etter?
*