A LIST Apart: For People Who Make Websites

No. 240

Discuss: Testability Costs Too Much

Pages

 <  1 2 3 4 >  Last »

11

Patrick raises some good points in his post on Accessify about this article. I tried to comment but seeing as I got an error I thought I would add the comments here instead.

Guideline 1.1.1
Patrick says: “I don’t quite follow why the form (All non-text content has a text alternative that presents equivalent information, except for the situations listed below) that’s now in the latest working draft is supposed to be bad.”

It’s actually the four sub-sections that are the problem – which didn’t make it into the article because it made the page too long! These four sub-sections are:

1. Controls-Input: If non-text content is a control or accepts user input, then it has a name that describes its purpose. (See also Guideline 4.1.)

2. Media, Test, Sensory: If non-text content is multimedia , live audio-only or live video-only content, a test or exercise that must be presented in non-text format , or primarily intended to create a specific sensory experience , then text alternatives at least identify the non-text content with a descriptive text label. (For multimedia, see also Guideline 1.2.)

3. CAPTCHA: If the purpose of non-text content is to confirm that content is being accessed by a person rather than a computer, then text alternatives that identify and describe the purpose of the non-text content are provided and alternative forms in different modalities are provided to accommodate different disabilities.

4. Decoration, Formatting, Invisible: If non-text content is pure decoration, or used only for visual formatting, or if it is not presented to users, then it is implemented such that it can be ignored by assistive technology.

Subjective words
With regards to your comment on the use of subjective words such as “inconsistent”, “predictable” etc in WCAG2 – it really comes down to the definition of “testability”. In my time on the Working Group it appeared to me that testability was applied quite strictly to certain checkpoints – such as the clear and simple language checkpoint – whereas it was more lax on other checkpoints – such as alt attributes. One could argue that eight out of ten people couldn’t agree on whether the result from activating a search is “predictable” – will it provide ten search results? twenty? Will it include summary information? etc. If we are going to allow subjective terms like “predictable” then what is the problem with other subjective terms like “clear” and “simple”?

The WCAG Samurai Errata
Another area of confusion with WCAG2 (once again due to testability) is that it increases the complexity of the success criteria. Because WCAG2 is technology-neutral, the guidelines have to be testable in a technology-neutral way. Therefore a simple checkpoint in WCAG1 like “Title each frame to facilitate frame identification and navigation” becomes “The destination of each programmatic reference to another delivery unit is identified through words or phrases that either occur in text or can be programmatically determined.” It’s not until you drill down to the techniques that it becomes clear that this success criterion is actually about frames at all. The WCAG Samurai Errata is certainly clearer when it comes to frames (it says not to use them!)

In my conclusion I wasn’t trying to say that the WCAG Samurai Errata are a better set of guidelines – I was attempting to say that if there are easier to use guidelines available then developers will turn to them, instead of WCAG2. I see that being a real problem in the accessibility community – developers choosing one set of guidelines while Government and management choose another.

Cognitive disabilities
I am really hoping that the WCAG Samurai address issues faced by people with cognitive disabilities. I see it as a way to really set the two guidelines apart. I believe the Working Group is wrong to say there isn’t much research in the area- a set of guidelines can be developed.

In conclusion
When it comes to testability I think it was a noble aim. I think it would be fantastic if every accessibility requirement could be defined in a testable way. However, for those requirements that can’t (eg. clear and simple language), the requirement should not be excluded from WCAG2. At the end of the day, if it helps people with disabilities use the web then it should be included in the accessibility guidelines – whether they be WCAG2 or the WCAG Samurai Errata.

posted at 03:29 pm on June 26, 2007 by Gian Sampson-Wild

12 Samurai vs WCAG 2

for instance the WCAG Samurai Errata are ten pages, whereas WCAG2 is close to the 500 page mark

that’s not really comparing like for like, though. the normative part of WCAG 2, minus the appendices, is just over 10 pages as well. you could say that, to understand WCAG 2, you need all the other material and techniques relating to each SC, but that is just as true for the Samurai – only that there is no techniques document for Samurai yet, so you have to track down the related best practices etc yourself.

also, Samurai only applies to HTML/CSS, with added bits specific to PDF and some things that could reasonably be applied, at least in principle, to other formats like Flash…while WCAG 2, as you noted of course, tries to be tech agnostic, thus making it far more relevant for areas such as e-learning, for instance…

posted at 03:41 pm on June 26, 2007 by patrick lauke

13 Thanks for clarifying...

…and sorry for the current commenting issues on Accessify – there seem to be some gremlins in the works there.

posted at 03:49 pm on June 26, 2007 by patrick lauke

14 In reply to Stephen Down

Relevant alt text, consistent layout, logical grouping of elements (esp form elements), even semantic code – these are all, to a greater or lesser extent, subjective. They can’t reliably be tested, but it is still vital that all authors do as much as they can to ensure that these guidelines are met.

if they’re subjective, and not testable, how can you ever hope to enforce them, say within an institution? and if the guidelines are taken as a basis for legislation, would you still want to wait for case-law to clarify each point (maybe having a jury of 10 people, or just a – non tech savvy – judge)?

i don’t think there was any need to rail on about the brains behind WCAG 2 and automated testing, as that’s not what the guidelines emphasise, quite clearly.

posted at 03:59 pm on June 26, 2007 by patrick lauke

15

you could say that, to understand WCAG 2, you need all the other material and techniques relating to each SC, but that is just as true for the Samurai – only that there is no techniques document for Samurai yet…

I don’t believe there ever will be techniques for the WCAG Samurai Errata (although I could be wrong) – simply because they aren’t required. The Errata provide enough strategic detail that further, background documents are unnecessary.

also, Samurai only applies to HTML/CSS [whereas] WCAG 2, as you noted of course, tries to be tech agnostic, thus making it far more relevant for areas such as e-learning, for instance…

I think this is the only way to write accessibility guidelines – make them technology specific. Create a set of guidelines that apply to the most common technologies, then create versions that cover other technologies.

If WCAG2 removed the testability requirement, addressed the concerns around people with cognitive disabilities, and developed a set of technology-specific guidelines, with addendums for alternative technologies, then it could be a very powerful document.

if they’re subjective, and not testable, how can you ever hope to enforce them, say within an institution?

WCAG1 was subjective and not testable and it was enforceable: we had a court case against the Sydney Operating Commission of the Olympic Games to prove it. But regardless – it is the Working Group’s responsibility to create guidelines that assist people with disabilities use web sites, not make sure those guidelines can be enshrined in law or that they are enforced within an institution. The W3C has said time and again that they do not write policy: they write guidelines. They don’t write laws. They don’t write rules. They write guidelines.

posted at 04:30 pm on June 26, 2007 by Gian Sampson-Wild

16 Too Far Entrenched in the Fortress of Academia

I am just beginning to scratch the surface of web standards and accessibility, and it takes real-world experience and personal discovery before I am capable of understanding why any of it is important at all. Sure, in school they wax poetic about the importance of the W3C and the WCAG in particular, but none have bothered to participate in the WCAG personally.

It sounds to me like the people pushing testability through are the ones who enjoy the political game or are encased in the impenetrable walls of academia, where time is in abundance and implementing their guidelines/lofty goals are within reach of no one else. Testability at the expense of accessibility? Perhaps this is why I get so frustrated and confused when attempting anything beyond the basics.

posted at 04:42 pm on June 26, 2007 by Thuy Nguyen

17 One Point Oh, here we come...

If WCAG 2.0 is too hard to understand, or not clear enough to be sure we can (and how we can) comply with it, I doubt that very many people on the job will be looking to the Samurai Errata for clarification, and fewer still managers (people who decide what the standards and the testing criteria will be) will use Samurai. How many web workers even KNOW about that?

Instead, I see WCAG 1.0 hanging around for a long, long while. We’re used to it. We can comply with it (when we really, really have to).

When 2.0 is doable, we’ll do 2.0.

posted at 06:34 pm on June 26, 2007 by John Dunagan

18 technology specific guidelines don't age well, tho

that’s been the problem with WCAG 1, and it could become a problem if WCAG 2 was rewritten to be technology-specific rather than agnostic + good techniques documents. also, new best practices emerge all the time (even just for HTML). enshrining the current practices into guidelines (which seem to take forever to be amended and updated) may result in the same issues that actually led to WCAG + Samurai in the first place.

as for SOCOG, weren’t the things that were wrong with the site technical (missing ALT text on images and image maps, tables that didn’t linearise properly) rather than subjective?

posted at 06:37 pm on June 26, 2007 by patrick lauke

19 oops, wrong formatting...

there shouldn’t have been an underline in my above post.

posted at 06:38 pm on June 26, 2007 by patrick lauke

20 Now, now, now, let’s not be impatient

The final version of the WCAG Samurai Errata aren’t out yet, and it’s disingenuous to blow up your favourite failing of the Errata into a claim that the whole thing is naff. Give us time, please.

posted at 07:39 pm on June 26, 2007 by Joe Clark

Pages

 <  1 2 3 4 >  Last »

Discussion Closed

New comments are not being accepted, but you are welcome to explore what people said before we closed the door.

Got something to say?

Discuss this article. We reserve the right to delete flames, trolls, and wood nymphs.

Create a new account or sign in below if you’d like to leave a comment.

Remember me

Forgot your password?

Subscribe to this article's comments: RSS (what’s this?)