A political history of Windows accessibility standards

I see around the Internet articles about accessibility APIs and standards like MSAA and IAccessible2, and I don’t think they reflect the history of the technology as much as the politics of the technology. This is a quick note of my memory and take on it from my experience working in screenreaders and AT.

Microsoft comes up with the first accessibility API in the late 90s, MSAA. It’s fine for buttons and textboxes and checkboxes, but richer user interfaces are too much for it – the document area in Word, or web pages.

So Microsoft comes up with UI Automation (UIA), which is a richer accessibility API that addresses the shortcomings of MSAA. This was in 2005.

But no-one picks it up properly: within Microsoft, Office doesn’t, I assume because of the usual Office versus Windows internal politics. Internet Explorer doesn’t, because Microsoft has stopped developing Internet Explorer. And outside Microsoft, all the screenreader guys have already written their code to “slurp” the contents of Internet Explorer and Office, so they don’t care that UIA doesn’t become the new standard. (And if UIA fails, so much more value for their products, which cost lots of money and so can support developers to write custom code for each and every program that fails to support accessibility standards…)

Outside Microsoft UIA adoption is even worse: Sun and IBM actually go to the trouble of creating a competing accessibility API, IAccessible2, and foist that on Java and OpenOffice to aid their competition with Microsoft and prevent Microsoft owning the accessibility API on Windows like Microsoft owns the document format standard (Word) and the web page standard (IE). (Disabled users have a history of being used to further other agendas, whether they know it or not…) Of course, nerds rally round and also foist it on Firefox, then the only Internet Explorer alternative, because hey everyone hates Micro$oft, right?

But, it’s important to understand, at this point in history – mid 00s – things are kind of OK. JAWS and NVDA and other screenreaders write special code (accessing some kind of application-specific API like the Word DOM) to talk to really really important applications, like Internet Explorer and Office, and use the MSAA accessibility API to talk to everything else. We all argue about the price of screenreaders and whether videos should autoplay but everything is stable. If you want to use more applications with your screenreader you pay more for a screenreader with more special code – JAWS is the standard, of course.

Then it all changes. The web and mobile explodes, by which we mean HTML5 and Chrome and iOS breaking the Internet Explorer/Windows dominance, and websites becoming applications in their own right like Google Docs or Facebook. Internet Explorer is dead, the mainstream declares, which means the sophisticated support screenreaders have developed to talk to its application-specific APIs is all useless: at the same time the websites are getting more and more complex. MSAA isn’t going to cut it, and there is no application-specific API for Chrome, the new dominant browser (there is a chrome.automation in development – how nice!) And here we are in 2018, and it’s much harder to use a web app than a desktop application. (Of course, you’ve probably jumped ship to iOS and an iPad with VoiceOver, or you are using a restricted set of desktop applications…)

Hey, maybe UI Automation’s time has come! Everyone knows VoiceOver is super-accessible and great and built into macOS, and so Microsoft needs a better AT story round Office and Windows, so the newly-invigorated Microsoft builds UIA into Office and Edge and here we go! At least until Edge is killed off too…

(The usual caveats and apologies about focusing mainly on blind people apply: showing my history in that area.)

BBC iPlayer Radio 10

After a good few weeks I have the BBC iPlayer Radio application working again! You should find that the previous version 9 force-updates to the new version 10 when you start it. If not you can use this link for the installer: BBC iPlayer Radio MSI Installer

As usual, it was broken by changes to the BBC system. I’ve had help from some kind BBC staff and I’ve got it working again: it now downloads without needed to convert as well, so it’s a little faster in operation. I’ve also improved the metadata in the files saved to disk for those that use them directly. “Version 10” indicates that I’m following the modern convention of updating the version number by a whole point when you do a release – like Google Chrome – not that there is anything very special and new. Sorry! Smile.

BBC iPlayer Radio 9.4

A small change the BBC iPlayer Radio program. It downloads BBC programmes and then converts them to MP3 to play them. Previously it’s used a moderate quality (bitrate) but some users got in touch and pointed out that many radio programmes would benefit from a better quality, especially Radio 3. So BBC iPlayer Radio now uses 192kbps MP3 files.

BBC iPlayer TV 8.0.0

If you’ve tried to use the BBC iPlayer TV 7 program recently you’ll know that it is completely broken. A few months ago all the channels went except for Audio Described, and more recently that disappeared too.

Over the years (fifteen, I think, though that may be Radio) I’ve taken many approaches to delivering BBC TV programmes to blind users, from RealPlayer days to now. In version 7 I received some help from the BBC: a nice, fast way to get programme listings. So I put that into the software, and all was well.

You would think I would learn! This handy mechanism disappeared this year, I think without warning, and the channel listings with it. However, Audio Described lasted a bit longer because it didn’t use the approved BBC mechanism, and on examination it only took a tweak to make it work again. I’ve therefore done the work to make the other channels use the Audio Described mechanism, so we have a new BBC iPlayer TV 8, and you once again have Audio Described, channels including One, Two, Four and News, and a new Signed channel listing too. The listings will all take longer to load, but that’s just how it has to work now.

Generally every couple of years there is a burst of enthusiasm at the BBC and someone will reach out to me with great plans and proposals: a year or two later they disappear with their technical help. Individuals have been kind and supportive, but the BBC faces great commercial and political pressure, so I can’t blame them. Anyway, for the geeks: we’re back to screen-scraping, which is slow but I can patch it up when the website changes and doesn’t rely on flaky BBC cooperation.

Finally, I observe that we now have more hoops to jump through before programmes play – like a BBC logon. You should only have to get this set up and working once, and it will work smoothly after that. But you might well need sighted help to get you through these steps: I have fitfully automate these steps, but it was just too much for me to keep up with changes, so I’m afraid you’re on your own. More reliable, though, which is nice.

New text-to-speech options in Amazon Kindle App

eBooks have always been a problem for blind screenreader users. Legal eBooks are kept heavily protected to stop you copying or sharing them, which means it’s hard for assistive technology to get into them and read them back. Governments have been lobbied to clamp down on file-sharing sites where you can get eBooks that read easily. And anything complex in layout, like a textbook, usually has shoddy text-to-speech support.

There are ways to do it, like using iBooks on iPad or getting books from Bookshare and other organisations. But that’s not every book, and that means an additional process that sighted users don’t have to follow.

Good news, then, that Amazon has updated its Kindle Apps. This lets you get text-to-speech to read out loads more books, and you can still change the content (colour, font size) to make it easier to read if you have some sight. This includes Amazon Kindle for Windows PC.

All taken from this great write-up at the CALL Scotland blog, which I recommend you go read: Giving your Kindle App a voice.

BBC iPlayer Radio 9.3.2

A small update to BBC iPlayer Radio. I’ve been having some reports that the program is having problems connecting to the Internet. I can’t replicate the problems here, sadly. But I’ve made the error message provide more information, so hopefully I’ll be able to get to the bottom of it!

A couple of other little bugfixes, including a fix for Windows 7 machines without .Net 4, and removing some Windows Firewall code that I now realise is redundant (and who knows, might be causing the connection problems!)

  • BBC iPlayer Radio 9.3.2 Installer

BBC iPlayer TV 7.5.0

I’ve updated BBC iPlayer TV to 7.5.0. For 7.3 and 7.4 I was using a JavaScript interface, but the new “Have you got a BBC TV Licence?” dialogue and other changes appears to have broken it (again!)

I’ve therefore gone to using the venerable Microsoft Active Accessibility (MSAA) interface to hunt down and press the correct buttons on the iPlayer page. This is a bit slow, and you’ve lost the mechanism for pause or skipping. I could put that in another time, but for now it works, and that’s the main thing. You’ve also sadly lost the switching interface, but I suspect no-one was using it!

Making your (Windows) computer easier to use with a sight problem

This is a set of suggestions and tips for helping you or someone you know with problems with sight to use their Windows computer. It’s probably most useful for a technically-minded person looking to help someone with sight loss but still some vision. Age-related sight loss, for example. If you are completely blind you will need a screenreader and more specialist training.

The focus is on changing Windows settings and free and low-cost software. It’s basically the things I would do if I was round your house trying to get your PC set up to be useful for you.

I’m assuming you’re on Windows 10, which has better accessibility features than previous versions. Windows 8 is just about as good, and Windows 7 is okay, but earlier than that and you really should upgrade.

I start, though, with a couple of hardware suggestions.

Get a big TV or monitor

If you have an HDMI output from your computer (most computers do nowadays) you can plug it into a big television or a big monitor to give you a bigger brighter display. A relatively inexpensive 22″ TV will work fine. You can try plugging your TV in to see how it works!

Get a high-visibility physical keyboard

You can buy keyboards with bright big yellow-on-black keys. This will help you find all the key combinations you’re going to be using. For example, from the RNIB Shop:

Black-on-yellow keys and big letters and numbers make this keyboard easier to see and use.

Now on to changing your Windows settings.

Change your display scaling and/or screen resolution

This is surprisingly tricky because there are two settings in Windows that interact.

You can change the screen scaling or DPI (dots per inch). This is in “Start”, “Settings”, “System”, “Display”, “Change the size of text, apps and other items”. Drag the slider left to make things smaller and right to make them bigger.

The Settings, Display dialog that lets you change text size. The slider makes text and anything else on the screen bigger and smaller.

You’ll have to sign out and sign in to see the full effects. You might not be able to drag the slider to make things bigger if you have a low-resolution screen: see “Get a big TV or monitor” above or read on.

Changing the screen scaling or DPI used to work very badly: lots of applications would not display correctly, with overlapping text and missing buttons. Nowadays things work much better. Note that older Windows machines (e.g. Windows 7) won’t let you scale much here so you’re better off with Windows 10.

The alternative to changing the screen scaling or DPI is to change the screen resolution. This is a cruder option, but means every application continues to display correctly. It won’t look pretty to people with good vision, but you probably don’t care about that. This is in “Start”, “Settings”, “System”, “Display”, scroll to the bottom and follow the “Advanced display settings” link, and change the value in the “Resolution” box. The smaller the resolution – the lower the numbers – the bigger everything will get. ScreenResolutionSettingsWindows10

Again, sign out and in to check the full effects.

Finally, things get trickier if you have multiple monitors – like a really big screen plugged into your laptop. You may find some resolutions or scaling levels are unavailable. You could try restricting Windows’ output to just the big monitor you want to use by pressing the Windows key and P together to bring up the Project options, then selecting “Second screen only”. There is more complexity over choosing monitors that I won’t go into here, but “Display” is where you’ll find settings if you look.

Whichever way you do it, some things won’t fit on the screen if you make everything really big. Not much you can do about that.

Make the mouse pointer bigger

“Start”, “Control Panel”, “Hardware and Sound”, and under “Devices and Printers” you’ll see a link for “Mouse”. Go to the “Pointers” tab and select the drop-down in the “Schema” section. There you will find several schemes labelled “extra-large” that make the mouse pointer bigger and easier to see.

Combobox "Scheme" lets you select a different mouse cursor size. Choose "Extra-large"

Switch to a High-Contrast Theme

These make Windows use more contrasting colours for things like window edges and title bars so you can see them more clearly. You can also make the colour scheme yellow or white on black, which might be easier on your eyes.

You can find the High-Contrast themes in “Start”, “Control Panel”, “Appearance and Personalisation”, “Personalisation”. Under “Change the visuals and sounds on your computer” you’ll find four themes labelled “High Contrast”. Pick one of those to try it.

Click/select one of the themes and you get the results instantly.

You will find that signing out and signing in again is needed for the full effect of the themes to be observed.

Make some text in Windows bigger and bolder

If you go to “Start”, “Control Panel”, “Hardware and Sound”, “Display”, you will find a “Change the text size only” section. This lets you make various bits of text in Windows bigger: not that many, sadly, but still some.

Use the comboboxes to change text (font) size for items in Windows.

“Icons” does the icons on the Desktop, which is good and bad: bigger text is easier to read, but there isn’t much space for icon text on the Desktop so you will quickly find it gets truncated. Go to the Desktop itself, right-click the mouse to bring up the context menu, select “View” and “Large Icons” to help.

You may find that you have to sign out and sign in to see the changes. You might also have to select a High Contrast theme (see above) to have the changes take effect.

Try using Windows Narrator (Windows 8 and later)

Windows Narrator is the Microsoft free, built-in screenreader. A screenreader is software that reads out the screen. It wasn’t very powerful before Windows 8, but it’s pretty good now, and the voices it uses for reading are better, and it even works in Microsoft Word. You can start Narrator from “Start” like any program, but you can also set it to start when you start the computer.

A particularly useful function for someone who still has some sight is that you can make it speak whatever you point the mouse at. In the Narrator settings, select “Navigation” and check “Read and interact with the screen using the mouse”.

Windows Narrator settings showing how to turn on "read whatever I point at with the mouse"

This will let you point the mouse at text and hear aloud what it says. Turn off the other speech options and that will stop it chattering when other things happen so you are not confused. Press the Control key at any time to stop Narrator speaking immediately.

You can also easily use Narrator to read a Word document out loud to you. Click in the Word document to choose where to start. Then press the Capslock key and the M key together. Narrator will read and scroll through the document. Again, Control key stops reading.

Try using Windows Magnifier (Windows 8 and later)

Magnifiers make part of the screen bigger, which sounds like a totally perfect solution until you try it and realise that you get completely lost on the screen really easily. You may find that this is therefore too hard to use: try changing the screen scaling or resolution instead (see above).

You can press the Windows key and tap the Plus key or Minus key to turn on and adjust the Magnifier any time you want, so you could use it for some quick magnification to check something out then zoom back out to 100% so you don’t get lost.

Magnifier does have one additional feature that might be useful: you can set magnification to 100% (i.e. no magnification) and turn on “Turn on colour inversion” to make your screen yellow-on-black instead of the normal black-on-white. This has the advantage, or disadvantage, of affecting absolutely everything on the screen. Selecting a High Contrast theme does not – things like videos in web pages and icons and some applications will retain their normal colours. You will almost certainly not need both High Contrast and colour inversion.

Try using NVDA

Non-Visual Desktop Access is a fully-featured free screenreader for completely blind people. It has loads of features and is completely free with a strong volunteer community. However, it’s a “proper” program for blind people, so it’s focused on being powerful but will require some work to learn it – notably the shortcut keys you use to get around.

Try using WindowEyes

This is not a free piece of software. However, Microsoft has done a deal with the makers of WindowEyes, Ai Squared, and if you have a copy of Microsoft Office you can get a free copy of WindowEyes.

WindowEyes is a fully-featured professional screenreader and magnifier, so is more powerful and have more features than Narrator or Magnifier:

I hope these suggestions are helpful! Let me know if I’ve missed anything obvious out.

Additional Notes

  • If changing the screen scaling or DPI works for you but you want things a bit bigger than the slider allows, there is a second place to set the screen scaling or DPI. “Start”, “Control Panel”, “Hardware and Sound”, “Display” has a “Change size of items” section. In there is a link to “set a custom scaling level”, which has a drop-down combo box with more scaling options than you get from the Settings option. This is more likely still to break older applications, but do try it out. On my machine, I can go up to 500% here but only 300% on the Settings option.
  • You can also check the box “Enable pointer shadow” in the Mouse settings to give your mouse pointer a tail as you move it, which again makes it easier to find on the screen: just wiggle it and spot its tail.
  • You can also buy stick-on large bright letters and numbers for your existing keyboard if you prefer not to buy a high-visibility one.

MSAA, UIA and WinForms in .Net

Another item for programmers, more on UIA and MSAA and developing accessible user interfaces, following on from using WPF/XAML with MSAA and UIA.

This is a super-quick guide to using WinForms to make UIs that work well with screenreaders like NVDA or Thunder or JAWS.

  • You don’t need to do anything to Button, CheckBox, MenuItem or MenuStrip controls – other than put some text on them! Image-only controls are no good. It’s fine if only the image is shown to the end user and the text is hidden.
  • You can correctly label content controls that don’t have a Text property, like TextBox, ListBox, ListView, ComboBox, ProgressBar and the like, in one of two ways:
    • Put a Label just before the control you want to label in the tab index order. The Text on the Label labels the control. Use the TabIndex property to make sure the order is correct.
    • Set the AccessibleName property on the content control itself. This labels the control for the screenreader.
  • Groupbox controls don’t label their contents, use one of the other techniques here.
  • You almost certainly want to avoid labelling Panel controls, since they are generally for layout, which is communicated to the screenreader user in other ways than labeling.
  • For WebBrowser controls you need to think about the content in terms of web accessibility – use label elements, ARIA, and other HTML accessibility techniques. You might also need to label the actual WebBrowser control on your form with a Label or AccessibleName, just like any other content control.

Unsurprisingly, these techniques are basically the same as writing forms using the Windows C++ API, since WinForms is a wrapper round that.

WebbIE Web Browser 4.5.1

Small but (I hope!) useful updates to WebbIE web browser today, version 4.5.1.

First, I’ve finally fixed a longstanding and annoying bug where Windows Narrator lost focus on the main text display area in WebbIE. You could observe this simply by navigating to google.com: when the JavaScript fired to put focus on the text input area, it took focus from WebbIE, and though I would put it back, and a sighted user could see the caret moving quite happily around the text area, as far as the MSAA/UIA screenreader interface was concerned it was still on the web page. Not Much Use.

Happily, though, WebbIE was on of my first significant .Net project three years ago, and I’ve learned much more since then – including how to fix this one! Focus is placed back in the text area correctly and Narrator works just fine.

Second, I’ve made the browser junk failing frames and external resources as it tries to load the page. This will probably break the display of some advertisements, and I’m afraid some page content too, but WebbIE will be much more responsive and useful. The is in line with a general approach I think is right for WebbIE now: it’s about convenience, and speed, and ease of use over trying to get every last HTML tag and JavaScript call to render in some kind of text format for the end user. WebbIE users are using it because they need it to be faster and simpler and better. If they want it to be more techie… they’re on, who knows, ChromeVox or EmacSpeaks!