Alasdair King's WebbIE Blog – Page 3 – Assistive technology software

MSAA, UIA and WinForms in .Net

Another item for programmers, more on UIA and MSAA and developing accessible user interfaces, following on from using WPF/XAML with MSAA and UIA.

This is a super-quick guide to using WinForms to make UIs that work well with screenreaders like NVDA or Thunder or JAWS.

You don’t need to do anything to Button, CheckBox, MenuItem or MenuStrip controls – other than put some text on them! Image-only controls are no good. It’s fine if only the image is shown to the end user and the text is hidden.
You can correctly label content controls that don’t have a Text property, like TextBox, ListBox, ListView, ComboBox, ProgressBar and the like, in one of two ways:
- Put a Label just before the control you want to label in the tab index order. The Text on the Label labels the control. Use the TabIndex property to make sure the order is correct.
- Set the AccessibleName property on the content control itself. This labels the control for the screenreader.
Groupbox controls don’t label their contents, use one of the other techniques here.
You almost certainly want to avoid labelling Panel controls, since they are generally for layout, which is communicated to the screenreader user in other ways than labeling.
For WebBrowser controls you need to think about the content in terms of web accessibility – use label elements, ARIA, and other HTML accessibility techniques. You might also need to label the actual WebBrowser control on your form with a Label or AccessibleName, just like any other content control.

Unsurprisingly, these techniques are basically the same as writing forms using the Windows C++ API, since WinForms is a wrapper round that.

WebbIE Web Browser 4.5.1

Small but (I hope!) useful updates to WebbIE web browser today, version 4.5.1.

First, I’ve finally fixed a longstanding and annoying bug where Windows Narrator lost focus on the main text display area in WebbIE. You could observe this simply by navigating to google.com: when the JavaScript fired to put focus on the text input area, it took focus from WebbIE, and though I would put it back, and a sighted user could see the caret moving quite happily around the text area, as far as the MSAA/UIA screenreader interface was concerned it was still on the web page. Not Much Use.

Happily, though, WebbIE was on of my first significant .Net project three years ago, and I’ve learned much more since then – including how to fix this one! Focus is placed back in the text area correctly and Narrator works just fine.

Second, I’ve made the browser junk failing frames and external resources as it tries to load the page. This will probably break the display of some advertisements, and I’m afraid some page content too, but WebbIE will be much more responsive and useful. The is in line with a general approach I think is right for WebbIE now: it’s about convenience, and speed, and ease of use over trying to get every last HTML tag and JavaScript call to render in some kind of text format for the end user. WebbIE users are using it because they need it to be faster and simpler and better. If they want it to be more techie… they’re on, who knows, ChromeVox or EmacSpeaks!

Updated BBC Live Radio

A simple update to BBC Live Radio today, to version 3.2.0, so three radio stations that haven’t worked for a while now work again:

BBC World Service
Classic FM
National Public Radio (NPR)

Of course, two of these aren’t BBC stations, but never mind! What NPR and Classic FM have in common is that the URLs change relatively frequently, which breaks my code – I’m not using a documented, supported API, I’m just finding the URL by investigating the website and coding it in. It’s a pain to maintain, since I don’t know they are broken until someone tells me, then I have to go and fix them. Still, because people tell me, I know people use it, so there we are.

The BBC World Service one is more interesting. It also exists in the official BBC API, but doesn’t seem to work correctly. So I’ve found the URL by searching and put in that working one. I know the World Service is a bit of a separate part of the BBC, and I know the Internet services provided by the BBC have been stitched together gradually over many years from my conversations with BBC engineers going back fifteen years. So I suppose this simply represents this internal division.

MSAA, UIA and WPF

A technical note today for programmers.

Screenreaders (blind users) interact with Windows programs through accessibility APIs. The most modern is UI Automation. But the first, and possibly most important still, is MSAA, Microsoft Active Accessibility. Windows will supply information from one interface to the other system if required, so most of the time you shouldn’t have to worry too much.

I’ve been doing some .Net programs using WPF or XAML (that is, not WinForms) and here’s a quick guide to what I’ve found:

Buttons and CheckBoxes work fine just as they are. In fact, don’t set any Automation properties or you’ll get odd-looking controls in MSAA.
ListBox, ComboBox, TextBox, ProgressBar work fine for their contents just as they are. But:
Labels don’t label Listbox/ComboBox/TextBox/ProgressBar unless you use LabeledBy.

In C++ applications you associate labels (e.g. “Enter username:”) with the control they are labelling (e.g. a textbox for the username) by using tab order. The label with tab index N labels the control with tab index N+1. So label with tab index 0 provides the name to the combobox with tab index 1.

In WPF you have to set this explicitly using AutomationProperties.LabeledBy, like this:


<Label Content="A ListBox Control:" x:Name="lblList"></Label>
<ListBox AutomationProperties.LabeledBy="{Binding ElementName=lblList}">
  <TextBlock>Elephant</TextBlock>
  <TextBlock>Koala</TextBlock>
</ListBox>

Note the funky “Binding” code – you can’t just use the x:Name of the label. Notice also that your Label must have an x:Name! You can also use a TextBlock instead of a label. Generally the pattern is:


<TextThing Content=”The label for the other control” x:Name=”textThingLabel” />
<OtherControl AutomationProperties.LabeledBy=”{Binding ElementName=textThingLabel}” />

Anyway, that suffices for the basics of MSAA/UIA support for WPF if you’re writing .Net Windows applications.

PDF Reader 3.2.0

Update to the PDF Reader 3 application. This is a nice simple program that lets you open PDF files and displays them as plain text to make them super-simple to read with your screenreader. It’s essentially a nice GUI front-end on the XPDF pdftotext.exe program from Foo Labs.

Anyway, PDF Reader only supported PDFs coded in Western European languages (e.g. English), but now supports any language (e.g. Polish, Japanese). Good for non-English users. Other minor bugfixes include not hanging while it opens the PDF file, which is nicer, especially for large PDFs.

PDF Reader 3

End of the Thunder Screenreader

I can now announce the end of Thunder, the first free Windows screenreader. Thunder will no longer be actively maintained, supported or developed, but you can still download installers from the WebbIE site.

Thunder was launched in April 2006 and was the first fully-functional free screenreader: Windows Narrator had launched in Windows 2000, but had never been given the features needed to make it a true alternative to the dominant commercial players. Thunder had all of:

Worked with Microsoft Word, Excel and Outlook, so you could use standard business applications.
An offscreen model, so more applications spoke, even when they didn’t comply with accessibility APIs (MSAA)
Scripting, so you could write application-specific support, including hotkeys and spoken messages and access to all the accessibility and offscreen model information provided by the screenreader.
A reading toolbar, magnifying the current text to display in large, high-contrast at the top of the screen.
Integration with UAC, so you could do admin functions on your machine and log in and out without sighted help.
Full support and translations for many languages, including Arabic, Bulgarian, Czech, German, Greek, English, Spanish, Estonian, French, Italian, Polish, Slovak and Turkish.

Technically, Thunder used a combination of the MSAA API (and UIA through the MSAA bridge), an offscreen model that hooked Windows messages, and specific COM APIs. At the time of its launch the accessibility APIs like MSAA and UIA were not as mature so the offscreen model and COM APIs were required. Other screenreaders still take this approach. Thunder also relied on WebbIE for web browsing, though we wrote in support for direct use of Internet Explorer at one point (using the MSHTML DOM).

Thunder was not open-source/free software, though it was zero cost. This allowed closed-source components to be used in Thunder, such as the powerful third-party offscreen model, and some code that could not be publicly distributed and was provided by application and operating system vendors.

Sounds great! So why end it?

Thunder was never a commercial proposition, though versions were sold with additional features or add-on voices. The aim was to make a real screenreader available to end users for nothing, compared to the very expensive commercial alternatives. But there are now three strong free (zero-cost) alternatives:

The great NVDA launched in 2007, and provides a free and open-source screenreader for download. It’s written by blind developers (unlike Thunder) and has attracted a loyal following (as it should).
Microsoft licensed WindowEyes for free with anyone with Microsoft Office. Which is pretty much anyone. So you can get a fully-fledged commercial solution.
Windows 8 finally improved Windows Narrator enormously, and the MSAA/UIA interfaces on which it depends have increased in utility hugely (for example, they are now available in Microsoft Office and Internet Explorer), so you can probably often get away with the onboard assistive technology much more than you could.

In parallel, we’ve seen fewer innovations in Windows GUI toolkits – XAML was nine years ago, for example, and most toolkits provide UIA and/or MSAA interfaces. So there is perhaps less need to update your copy of JAWS regularly than there was ten years ago.

And finally the huge change: blind users may not be on Windows at all any more. The iPhone and iPad with VoiceOver built in, touch interfaces, and rigorously-controlled walled-garden apps have provided a more accessible and productive environment for many blind people than the free-for-all mouse-and-keyboard world of Windows.

So new, free alternatives, and changes in the marketplace, have meant that the need for Thunder has simply reduced over time. All great news for blind users, which is the important thing. Thunder is dead, long live free screenreaders!

Microsoft Speech

Updated September 2020.

Microsoft Windows has had a number of speech systems over the years. Windows 10 has made more changes, and the MSDN documentation is fragmented and confusing. Here, then, is a description of Microsoft speech systems on desktop Windows as of November 2016, mainly for developers and enthusiasts.

The three systems are SAPI5, Microsoft Speech Server, and the new system for Windows 10, the variously-named Windows Runtime or Bing or Mobile system.

SAPI5, Microsoft Speech API version 5 (System.Speech.Synthesis)

From Windows XP onwards, this is the main Windows speech system for desktop applications, like screenreaders. It’s got a huge range of voices from third parties like Acapela, Nuance, Cereproc and Ivona.

The default Windows Voices have been SAPI5 since Windows XP. Microsoft Sam in Windows XP, Microsoft Anna in Windows Vista and 7, Microsoft Hazel in Windows 8 and 10.
You can find SAPI5 voices that are installed on your machine in the Control Panel. You are looking for the Text to Speech window, which is hidden away in the Speech Recognition settings in Control Panel. Some SAPI5 voices may be hidden (e.g. Acapela voices) but the ones you can see there can generally work in any program that uses SAPI5:

You can get lots and lots of SAPI5 voices from third parties, including the free eSpeak.
You can also get new SAPI5 voices from Microsoft by installing a new Language Pack from Control Panel. Language Packs are all free for Windows 8 and later (and mostly free for Windows Vista and 7 too) and some of them come with SAPI5 voices. For example, if you install the French language pack, you get a French SAPI5 voice that appears in Control Panel and can be used in software that supports SAPI5. List of free SAPI5 voices on Windows 8 and Windows 10

Voices can be either 32-bit or 64-bit, just like Windows. If you’re on a 64-bit Windows machine, 32-bit voices won’t show up in the Speech window in Control Panel, because it is a 64-bit version of the Speech window. You have to find and run C:\Windows\SysWOW64\Speech\SpeechUX\sapi.cpl to see 32-bit voices. Also, 64-bit programs won’t see or be able to use 32-bit-only SAPI5 voices.
You can find the installed voices in the registry, under HKEY_LOCAL_MACHINE > Software > Microsoft > Speech, or if you are on a 64-bit machine, both that key and in HKEY_LOCAL_MACHINE > Software > WOW6432Node > Microsoft > Speech for the 32-bit voices. You can more rarely find per-user installation of SAPI5 voices in HKEY_CURRENT_USER > Software > Microsoft > Speech.
There was a SAPI4, which was the predecessor to SAPI5, and shipped in Windows 2000 and as part of Microsoft Agent. It was similar to SAPI5. No-one uses it nowadays (this is almost certainly untrue, but as a general rule, it’s all SAPI5 now.)
Developers: The Desktop Windows API you use in .Net is System.Speech.Synthesis and the SpeechSynthesizer class, which is a wrapper round the SAPI5 COM object. You can use SAPI5 on Windows Server or desktop versions. SAPI5 is also available through a COM interface for C++ and other programming languages, Microsoft Speech Object Library.
Microsoft Speech API (SAPI) 5.3 on MSDN

Windows Mobile, Windows Runtime, or Bing Speech Services

Windows Mobile, which became Windows Phone, and is at the time of writing becoming Windows 10 Mobile, has a text-to-speech system. It’s “the text-to-speech you can use when you are writing Windows Phone Apps.” In some places on the Microsoft website it’s called the Bing Speech Service, but I don’t know how long that will last.

The interesting thing is that this system, and the voices that come with it, have landed on desktop Windows with the arrival of Windows 10. If you open up the Settings App in Windows 10, as opposed to the old Control Panel, you’ll find a Speech setting. This lists different voices from the SAPI5 list: on my Windows 10 UK English machine I have Microsoft Susan Mobile and Microsoft Heera Mobile. These are NOT SAPI5 voices. It’s ANOTHER speech API on your Windows 10 machine. It’s used by Windows Store Apps only (again, now called Universal Windows Platform apps).

Like SAPI5 voices, you get more by adding Languages to Windows. However, you can’t use the Languages and Language Packs in Control Panel: this gets you SAPI5 voices. You have to use the new Settings App and the Language settings. This gets you more Mobile voices.

These Mobile voices can now be used by normal Windows applications, but the developer will have to write support for them into their programs – they don’t show up in the SAPI5 system.
Conversely, SAPI5 voices cannot be used in Windows Store Apps, which can only use this Windows Mobile system.
You can find the installed voices in the registry, under HKEY_CURRENT_USER > Software > Microsoft > Speech Virtual and in HKEY_LOCAL_MACHINE > Software > Microsoft > Speech_OneCore (duplicated in HKEY_LOCAL_MACHINE > Software > WOW6432Node > Microsoft > Speech_OneCore for 32-bit machines.)
There’s ALSO an online Bing Speech service, also called Microsoft Translator, now called the Bing Speech API in Microsoft Cognitive Services. This is a web service you call with text to get audio files back with the speech you requested. So it’s not something you use on your Windows machine (though a website or App might).
Developers: the API you use in .Net is Windows.Media.SpeechSynthesis and the SpeechSynthesizer class. This works just like the .Net SAPI5 API, System.Speech.Synthesis, but it’s targeted at Windows Store Apps / Windows Runtime Apps / Universal Windows Platform Apps. So System.Speech.Synthesis is Desktop, and Windows.Media.SpeechSynthesis is Windows Runtime, and they are completely different systems. As of Windows 10 Anniversary Edition in 2016 you can access the Windows.Media.SpeechSynthesis API from desktop apps, so you should be able to access these voices – but you’ll have to write specific code for them, you can’t use the SAPI5 API.
You CAN actually get the Mobile voices to work with SAPI5, but you have to hack the registry, and since I might end trying to fix your computer when you’ve done this I’m not going to give you detailed instructions. Essentially, copy the Tokens registry key for the Mobile voice into the SAPI5 Voices key in HKLM > Software, do the same for 32-bit if necessary, and there you go!
Finally, also lurking in the same speech system are the Cortana voices, used by Microsoft’s Cortana digital personal assistant. They are also described in the same registry key but don’t show up in Windows.Media.SpeechSynthesis. Hacking the registry will probably enable them.

Microsoft Speech Platform

This is a set of voices and yet another speech system, but for use on Microsoft Windows Server. SAPI5 doesn’t work on Windows Server. They are intended to provide speech for servers, like automated voice menus when you call somewhere, that kind of thing. They aren’t intended for desktop programs.

This has Microsoft-only voices, and they won’t work with SAPI5 programs (or Windows Store Apps). A program could use them if the developer writes the code for it: the NVDA screenreader has done so, so you can install Microsoft Speech Platform and use the voices in NVDA.
You get the engine and the voices as installers from the Microsoft website, and install them on your server.
Developers: this uses the Microsoft.Speech API in .Net.
Microsoft Speech Platform on MSDN

Both BBC Radio programs now updated, including Windows XP support

Tonight I’ve updated BBC Live Radio to version 3.0, and I believe that it now works with the new BBC Radio live streams for Windows XP and later. Last week I updated BBC iPlayer Radio to version 9.1, and I believe that this now works for playing 7-day catch-up radio programmes using the new HLS format, again on Windows XP and later. Thanks for bearing with me, and let me know of any problems you find.

Here’s some notes about the new versions:

The BBC Live Radio programme is now using the new HLS streams for Windows 8 and later. Windows 7 and earlier won’t play these streams, so it falls back to the ShoutCast MP3 streams. These will be turned off by the BBC in 2016, on current plans, so Windows 7 and earlier will break again.
In BBC Live Radio at present I simply pick the first HLS stream available, which is probably not the best choice (non-optimum bitrate, or wrong international/not international version, or whatever).
The BBC iPlayer Radio programme simply converts the downloaded AAC file to MP3 to play it, using FFMPEG, which can take a little time, and there is no progress indicator yet. Users will probably think it has hung, but give it a little time.

BBC iPlayer Radio 9 released: still to fix AAC support

Both WebbIE BBC programs have now been updated to the new system, but they may not work yet if you have Windows 7 or earlier

Summary: both WebbIE BBC programs have now been updated to the new system, but they may not work yet if you have on Windows 7 or earlier.

Yesterday I released BBC iPlayer Radio 9. This new version works again with the BBC on-demand seven-day radio catch-up service, after the BBC shut off the old Windows Media service. It works the same except that instead of playing a programme immediately, it has to download it to your computer first. This can take a while, but downloaded programmes are saved for the next seven days so you only have to do it once.

However, there is another problem. The new AAC format used by the BBC works just great on my Windows 8 machine. But lots of users have contacted me to say that they get only a couple of seconds of audio on their machines before it stops, and they report it for both BBC iPlayer Radio and BBC Live Radio 2. It looks like on older versions of Windows, or Windows Media Player, or the K and KM versions of Windows, the BBC programmes won’t play.

Normally this is fixed by installing another piece of software called a codec or filter or plug-in and one user has reported exactly that. But until I get onto a Windows 7 machine next week in the test rig at work, I can’t confirm what versions of Windows are affected and I can’t identify the best plug-in to recommend.

Until then you have two options, both of which are quite technical, I’m afraid:

Install another media player that does support AAC files, like VLC. Use BBC iPlayer Radio 9, and wait for the programme to download, then stop playing. Then open the File menu (Alt and F) and select the new menu entry, “Open in external media player”. This will launch whatever program Windows thinks handles AAC files – hopefully VLC.
Try installing the codec that my user kindly suggested. I’m not naming him in case it goes wrong: remember, this is at your own risk! Orban AAC Plug-in – zip of Windows installer exe, unzip and run.

BBC Live Radio: local stations updated

I’ve now switched the all the BBC stations, national and local, to their new Audio Factory streams. Everything is working fine on my Windows 8.1 machine, but some points:

There are lots of different streams to choose from. UK or non-UK, and different bitrates (48,000 to 339,200), and different encodings (HE-AAC and AAC-LC).
I’ve chosen the non-UK streams because I have overseas users, although I understand that these streams will go quiet if the station is broadcasting something for which the BBC does not have international rights (e.g. sporting events). So I should probably do some detection, or provide an option, in the future.
This also means that I’m using the HE-AAC streams, for good or for ill, at a bitrate of 101,760. This might annoy some audiophiles, but I’m not one, so I’ll wait until someone complains.
Everything might be fine on my Windows 8.1 machine, but older Windows might not support the HE-AAC stream without installing a codec or other component. If you find nothing will play, do get in touch at support@webbie.org.uk.

Meanwhile, the BBC iPlayer Radio 8 is completely broken. Over the last week all the Windows Media streams that it used have been shut off. The good news is that lots of people complained to the BBC, so they’ve got in touch with me and promised to help. So far I’ve got the program to display the new catalogue, which means that you can still see programmes, but none of them play yet – and I don’t know why. I hope to get a fix sometime next week if they get back to me! (I can get the URL of the stream to play, but it doesn’t play in VLC or Windows Media Player).