hpr-knowledge-base/hpr_transcripts/hpr0959.txt

Episode: 959
Title: HPR0959: The Orca Screen Reader
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr0959/hpr0959.mp3
Transcribed: 2025-10-08 05:39:45

---

The program you're about to hear is a talk on Orca, given at the Northeast
canoe Linux Fest 2012. It normally would go into our Thursday queue as a
syndicated show and wait for its turn in there. But it's come to our attention
that Orca is very desperate for programmers right now and for help of
basically every kind. So we decided that that in itself was reason enough to
give it a time critical status and we bumped it ahead of the rest of our
Thursday shows. Please give this a listen and see if there's any way you can
help them out.
John Marie Diggs.
I'm not used to standing in front of a microphone. I'm really ADHD but I'm going to
try it. So the presentation title is the Orca screen reader, but that's a
little misleading because I'm here with an agenda.
Excellent. So we're going to talk very briefly about Orca, Orca 101. Then I want
to talk about how Orca does what it does because I want to turn everyone into
testers, hackers, assistive technology, developers, whatever. We have got the
worst bus factor problem in free desktop accessibility. It's pretty much if I
could hit by a bus and two other people could hit by a bus like all free desktop
accessibility is out the door. So I'm here to recruit. And that's why I'm going
to talk about getting involved. Okay, so Orca 101. Okay, so Orca is a screen
reader and just to make sure we're all starting in the same place. I mean, you
know, very briefly, Ruben said, you know, okay, here's Orca. We heard some stuff
talking. It's like, well, you know, does screen reader just mean thing that talks?
No, and that's like a big problem. So when we're talking about screen reader,
we're talking about a target population. These are zero blind or visually
impaired. Could I make Orca speak my incoming messages in pigeon? Yeah, that's
not what it's for. Can I make Orca work as a tool for people with learning
disabilities? Yeah, that's not what it's for. And the problem, in fact, in the past,
one of the biggest problems in Orca was we were because of this bus factor,
trying to be too many things for too many people. We used to drive, um,
GNOME mag. So people with magnification could track the focus in the car,
right? Which is just nonsense. And we're not doing that anymore.
Doesn't mean it doesn't need doing. It just means that's not what a screen reader
does. We need people to develop tools for users with learning disabilities.
They don't need the overhead of a screen reader. So we're focusing on users
who are blind or visually impaired. And, you know, you, you heard the, uh,
computerized voice talking and it's called a screen reader. We're not just
talking about synthesized speech. We're also talking about refreshable braille
and refreshable is literally what you think it means. If you've ever seen a braille
book, that's embossed braille. That's not refreshable braille.
Refreshable braille is hardware and pins go up and down. And so we can display
in real time, you know, what is on the screen? And some users are braille
lonely because they just like it that way. Some users are braille only because
they're deaf blind. Um, some users don't read braille and they're just using
the speech. Um, some users really want braille, but braille devices cost a huge
amount of money. And so they might not be able to afford it. So speech and or
refreshable braille. And the main thing we're doing, not the only thing we're
doing, is tracking the focus, tracking the carrot. You can, you know, press
alt F, get an application file menu, start arrowing down. And so a screen
reader, including Orca, needs to present where the focus is, where the
selection is. If you're typing or moving around in a document, we need to
present that new location, okay, in speech and or refreshable braille. So a lot
of what we're doing is interacting or excuse me, is presenting in speech and
or braille normal interaction with an application. Okay. And if the users not
in that particular widget or that particular document, it's probably not
relevant. You know, a lot of stuff is happening on your computer. Um, and I
should have done this, but we've got a full screen. You know, your clock, you can
have your, uh, operating system, not operating system. You just have
environment clock, just flight seconds. And by the way, we get an accessible
event every time you do that. Um, that is not of interest to the user. The user
normally cares about what he or she is in. But not always.
You know, if I, in fact, I do it all the time, I, I kick my
likes a bunch and get I'm very ADHD. And I'm always disconnecting my power
cord. I'm in a document, but that little thing that says,
hi, your battery's about to go dead. That's really relevant to me. It doesn't
have focus. And they get a get focus. But it's on the screen.
Um, incoming chat messages. If I'm having a conversation with you in
pigeon or whatever client I choose to use, I'm down in that little
editable text box below. But the stuff that really matters is not where I have
focus. So we make exceptions. There are things that go above and beyond
tabbing and arrowing and stuff like that. And we'll present that too.
Next slide. In addition, um, I mean, I don't know if you ever
thought about what it's what it's like to use a computer if you can't see what's
on the screen. But it's a lot slower. We take it so much
information visually. And a lot of it, we just filter right out.
It's not relevant. My scarves it. We can visually scan an entire screen really
quick. And you can see what those by bullets are. If you have to actually move
to the same, then it becomes just as much more inefficient.
Another problem that happens, getting back to say, arrowing through a menu.
If you know that you want the fifth thing down, you can arrow really fast.
But imagine if you had to hear, even though you're already on that fifth
thing, the first one and the second one and the third one. So we also have to
interrupt speech to make it more efficient. Because if the user's not on that
thing anymore, that's no longer relevant. Well, that has side effects.
Because if I am typing a message and you type a message and then you go,
oh, by the way, I have returned again. I mean, let's think of how we all use
instant messaging at IRC. No one types sentences anymore.
The type like three words he had entered and you type a few more words he had
enter. That incoming message can interrupt a previous message.
Well, crap, you can't see the previous message. Do I have to, you know, do
shift tab and move up there and go find it? Well, so screen readers will give you
various utilities for, again, people who are blind or
visually impaired to interact more efficiently with what's on the screen.
So Orca and other screen readers or Windows and I think voiceover doesn't, I'm
not sure, will let you review previous notifications. Because if, you know,
your instant message came in at the same time to, oh, crap, that your
battery's about to go dead. I need to go back to that previous
notification. There might be a key command for that.
Navigation by element type. Picture a Wikipedia page.
How do you read it? You look for the bold headings.
You scroll really fast. You move down three headings and then you start reading
irrelevant content. You can't do that just by
arrowing and tabbing and sighted users for the most part don't really need to
jump for heading, you know, heading by heading. You use your vision to do that.
So screen readers, you know, might give you a command in the screen reader where
you can hit H and move to the next heading and H and move to the next
heading and so on. Fun and formatting information.
How do you know that your text is, you know,
10 point monotype, of course, CIVA or whatever. I know it's bold and underlined.
Well, if it's in front of you, you'll look at it.
Okay, if you might go, well, it's bold and it's underlined.
Is it 10 point or is it 12 point? Well, that might be the toolbar.
It might be in a dialogue, but you can get that information really quickly.
Yeah, I could give focus to the toolbar tab around. Okay,
it's just going to take forever. So screen readers will typically give you a command
where you could press, you know, some magical hot can't say F and it will tell you
what point size it is if it's underlined and if it's bold and blah, blah, blah, blah, blah.
Okay, so these are the kind of commands I'm talking about.
Search the screen for an object. If you don't know, if you have a bunch of stuff,
you know, a bunch of widgets on the screen and someone, especially on the phone
or in an instant message is, well, you got to click on a thing called
through. And you know, how do I have to tell twice? Do I have to tell
where it's going? So screen readers can give you a way to search for
objects on the screen and jump to it right away.
Okay, so getting back to Orca. Orca is a screen reader. So again,
yeah, we kind of in the past got roped into the magnification business briefly.
Or actually for a long time, we are kind of got roped into the
learning disability business a little bit. We're working our way out of that.
We present information and speech and braille. I forgot the first point.
We're written in Python, which means again, really low barrier to joining the crew.
Python is so easy. By the way, as in the side, my background is lightness and visual impairment.
I worked at the Carroll Center for 15 years. If I can register screen reader,
y'all can write a screen reader. Okay, I thought it's easy.
Where? Oh, yeah. Okay. So just for the people who like really care about the technical stuff,
and we're going to actually get into that shortly. You know, the user's arrowing around in a menu.
Well, how do we know what the user's on? How do we know what text is selected? How do we know
what the font is? So it's greeners going to present all this nifty stuff. How?
Well, we talked to something called ATSPI, and we're now at the ATSPI too, which is basically the
D bus-based version instead of the core-based version. But it's the assistive technology surface
provider interface. That's what assistive technologies like Gorka and other assistive technologies,
if they were to exist, would interact with the applications and toolcats. The robotic voice you
heard during room of the presentation, that was speech dispatcher presumably with e-speech.
Speech dispatcher, orca talks to speech dispatcher, and just says, say this, and speech dispatcher has
all the knowledge of the different speech synthesizers and how to make them do whatever it is they do.
Unfortunately, and we'll come back to this, anyone who's like really into hardware and software
and speech and all that stuff, man, we got to do something about those robotic voices.
That's where the free software, one of the places free assistive technology really sucks
is the voice says, welcome to AARC, you know, that's gotta go. We don't have that problem when
one does. Braille translation, that's a whole other concept, if you're not familiar with Braille.
So, there are different languages have different combinations of words and letters and
syllables and all that stuff, and different words that appear like really, really often.
And the thing is Braille takes up a huge, huge amount of space. I remember going out to teaching
in various schools and sometimes you find the Braille dictionary and what, I mean, I have an OED,
all 20 volumes, but most people have like a single volume dictionary. And Braille takes up a whole
okay, Braille is huge. So, you want to be able to contract it to get, and remember,
refreshable Braille displays are really expensive. We got to get stuff in small, small spaces.
So Braille, not just, you know, computer Braille or anything like that, but just Braille has
contractions. So like in English, there's a single symbol, symbol for the IMG. There's a single
symbol for EED. And because these were all invented way back, I don't remember when, but basically
it was to transcribe the Bible. We have a K all by itself means knowledge. Okay, GD is good. So,
there's all these contractions, but obviously K for knowledge doesn't make sense in Spanish Braille.
And IMG is not a normal combination, although Spanish Braille don't think it's contracted at all,
but it's not a normal combination, that you see in other languages. Okay, so not only do we have
to contract stuff, we need to display an accented lighters. We need to display all this stuff
to the user in whatever language. We've got a really active Hungarian user. I know nothing
about Hungarian Braille, but think goodness. Live Lewis does. So, if you have Live Lewis, it takes
care of all the languagey stuff. It takes care of the contractions. The user just says, this is the
table I need. And again, Orchid doesn't have to worry about it. We just tell Live Lewis to good.
And then it has to actually talk to the hardware. That's something, a high level,
why I always say Braille API, but BRL API talks to what I call Braille TTY, which is BRL TTY.
And so all that stuff, Orchid doesn't have to worry about that. Orchid is able to deal with Live at a
very high level, which is awesome. Next slide. Okay, so I started to notice a theme here. People
keep going, blah, blah, blah, blah, blah, blah, blah, blah, blah, blah, blah, blah. And then we're
going to be yet another presentation like that. So very briefly, Orchid was started from scratch
in 2004 by Sun Microsystems. I joined the community, it's either 2005 and 2006, and actually
people this room would probably know better than I would. It's whenever the Commonwealth threatened
to switch the official document format to ODF, and then blind people complained. I mean, we didn't,
there was a lot of misunderstanding, but there was end result of that, as people started to show
it up at Perkins, Carroll Center for the Blind, places like that, to inform us about open document
format versus open, you know, what does all this stuff mean? And one of them was Sun Microsystems,
and they did this presentation on Orchid, I've seen your thing. Okay, I can make the screen reader.
You know, I'm teaching people, you know, every day, how do you use JAWS? I can't get my stupid bugs fixed.
It costs $1,200, and I'm like, wait a minute. I can like, you know, I take some computer science
after my first master's degree, you know, I kind of had a program, you know, I can make a screen
reader. Okay, I was totally sold, and I was in the community, whatever gear that was.
In 2007, we became the official screen reader of Gnome actually beating out the competition LSR
from IBM, and it was like, yay, you know, kind of friendly competition, and then IBM had a
big old riff, or no, excuse me, I think it was a strategy change. I forget the language they use.
It was really pretty language that means we're laying off our entire accessibility department,
except for some manager. So Gnome accessibility took a hit there.
That late 2007, that's what we started talking about Gnome 3, and I think that Rupin was the one
who mentioned, you know, when you change everything. Okay, that's a good way to, to, I mean,
I guarantee you, if you tell me you're going to change everything, you're going to break something
accessibility-wise. So it's like, oh gosh, Gnome 3. Oh, by the way, we're going to deprecate
Bonobo. Okay, so that means Gnome Mag, which Orca has to drive, is going away. We got to figure
something out about Magnification Gnome speech. Well, we're going to have to figure out how to
talk speech dispatcher. ATSPI is carpet base. That's going to go away. Oh, it is going to be really
non-performing over deep. Awesome. And right when all this is going on, some microsystems hadn't been
bought out yet, but they realized that financially they were in trouble, and there was riff after riff
after riff. And so the team just got lower and lower and lower and some microsystems, after I
be, you know, that was Gnome accessibility. So, you know, it was like, and then there were three,
and then there were two. And then 2009 was the official acquisition. It became legal. I think it was
late January 2010 and five days later, the rest of the accessibility paid people were gone,
and all of a sudden I became project lead. Keep in mind, I'm a teacher with a full-time job,
and then project lead. Gnome 3 almost went out the door and due to things, a few accessibility
things and a lot of other things didn't get out the door. And at least we didn't have to deal with
all those magnification speech problems, et cetera. 2011, it did come out. And then this is going
out live. So, between just us, it was an accessibility nightmare. Gnome 3, oh, was not accessible.
We tried. Some things just didn't get in on time, even though there were patches. This is a
problem with free software, and entire desktop environments made up of community numbers. A lot of
people with jobs paid to do it, but everyone's got their own, you know, it's just to scratch,
quote, unquote, their own company agendas, and things just didn't happen. Gnome 3 was an
accessibility nightmare. That's the bad news. 3, 2 came out. It was a lot better. She says kind of,
it was better. It really was, but it still wasn't good enough. And then in a funny twist of fate,
we had a good riff. At first, well, for about three hours, I was panicked, but my grant found
a position at the Carroll Center for the Blind ended. I was there for like 15 years. I went to my
normal telephone-based meeting, and it's like, are you sitting down? You never want to hear that
from your boss. But I got blessed Twitter. I tweet, and you know, I just got laid off. Oh my god.
And three hours that I had a job offer for. The gap, yeah, who was making a much bigger commitment
to accessibility. In fact, I would argue a bigger commitment than anyone else's, and I'm not saying
just as an employee who's drunk in the Kool-Aid, we really are investing quite a lot of
accessibility. And so now, my day job includes working on Orca. Next slide, please. And we have
another developer. In fact, Gnome, the new Gnome accessibility project late is Addigalia. He's
also the Gnome Shell accessibility developer. And so he and I just focused really, really hard to
pass a couple of buzz on Gnome Shell and Orca working together. And he did most of the work,
actually. I was just going, Addis isn't speaking right, and he'd come up with the fix. And
the fine folks at Red Hat, who are the ones really doing the Gnome Shell work, got these patches
reviewed and gave him feedback, and the patches are getting committed. And you know, Gnome 3.4,
you know, we can have the argument, oh, Gnome 3 is awesome. Oh, Gnome 3 is sex. I like Gnome 2.
Okay, forget all about. In terms of accessibility for users who are blind, Gnome 3.4 is going to be
awesome. Yay. So 2012 is looking up. We're going to start planning Gnome 3.6 soon, and whether or
not Trisco and other districts are going to go with that, I don't know. But it's good planning
time. She says looking at Ruben. So we've got to start talking. Other really neat things are happening.
Qt, it's one guy mind you again, bus factor of one. If this guy in Germany gets hit by a bus,
Qt accessibility is gone. But right now, he's been very, very careful with his travels. And Nokia is
funding a developer to make Qt accessible. Doesn't everyone have KDE accessibility like tomorrow?
In fact, Plasma is apparently, I don't want to say a nightmare. I like Plasma personally,
but accessibility-wise apparently is quite complex. But basic applications now are becoming
accessible with Orca. And I personally am not a Skype user because it's one of those closed
blobs. But I think users who are blind, if they want to, I think that's another freedom of choice.
And if you really want to use Skype, you should be allowed to do it. You should be able to do it.
And we're just now getting to the point where Skype plus Orca actually works.
So they shouldn't use Skype, but if they want to use Skype, it's now accessible more or less.
Thanks to the work done by Nokia. The XFCE developers, and they are, that's a whole desktop
environment with a small bus factor. There's like about five guys, and if they're all in the same
bus, we're in trouble. But they all by themselves. We don't have to go beat on their door and say,
you know, you need to make this accessible. They came to us and said, how do we do this? So they've
had some, some setbacks in terms of getting 4.10 out. But the next XFCE release is going to
hopefully be pretty darn accessible. They're working with us. So that's more or less three
environments. Orca users are going to be able to use, which is exciting. LXDE, don't know,
this GTK-based GTK, which we're going to get into momentarily, is accessible. So we just got to
figure out, you know, what kind of tweaks we need to do to get that working. And others, I don't know.
But I'm feeling really positive for 2012. Okay, so that is everything I want to say about
Orca. I mean, I want to talk about how Orca does what it does. Okay, and just very, very brief
history. And this is again, I was in the blindness biz for, you know, 15 years. And if you count what
I'm doing now more than 15 years. In the old days, and maybe he's still today, I don't know.
You'd launch the proprietary Windows screen reader and your cursor would start going really,
really fast. And it's like, why, you know, I never understood why that happened. I've someone
finally told me, that's how it's figuring out where the care it is. It's looking at the screen.
I remember you install, you know, one or two proprietary screen readers, you've installed
all of a sudden, you have no display access whatsoever. Because what was happening is they were
inserting some sort of hack to intercept whatever was being sent out to the video card. And that's
how screen reading used to work. Fortunately, we've never had to do that. And fortunately, even in
Windows, apparently, they're not doing that. So the bad news is that you could take out your system.
The good news is that a lot of times it worked. But from the point of view of screen reader,
which is again, why am you're recruiting? You didn't depend on anybody else. You did your
screen scraping, but it was everything was under your control. Okay, next slide.
We use, and even in Windows, but we at Oracle Land and in three desktop environments,
don't use scraping or tricks. We talked to the toolcats through the applications, through ATSP,
I'm going to give you a whole diagram in a minute. So none of that weird cursor flinky. Oh,
yeah, it's all orchid. You can't use your system anymore kind of stuff. That doesn't happen.
Okay. And what's better is we're not having to guess where the cursor is. We're asking where it is.
And we're being dulled. Okay. So that's like really, really awesome. It's really, really reliable.
We have, in fact, until just 3.4, the no-mixessability was just really sluggish. And it always was
sluggish. And part of that was just something happens. We're told about it. Okay, well now we've
got to find out what happened. Is that a focus menu or not a focus menu? Is that an incoming
chat mess? So we're having these things going back and forth over the wire. You know, we're getting
notified of stuff and we're asking questions. And it's just intensive work. And so sometimes
you get a performance out. Okay. But the good news slash bad news, double-edged shores here,
is we're depending on applications and toolkits doing the right thing. And if they don't expose
anything at all, we can't present anything at all. Now what's almost depending on your perspective
worse is if they expose focus information, I mean, we don't know. We just kind of pass it on to
the user. And so sometimes there are problems. Next slide. Okay. So here's just an overall diagram
of how all this stuff looks from my perspective. At the very, very top, I've got labor office,
fire box, financial epiphany, G-Edit, came out. So obviously applications beneath those,
what their toolkits are. And beneath those, that's where I'm really starting to care, is the
ATK implementation with the exception of QT, which I'll come back to. In the case of clutter and
GTK, they actually named it. So we had Gale and we have Callie. But it's still an ATK implementation.
ATK is the accessibility toolkit. Okay. And that's what toolkits do. To ultimately, it starts as
you can see, there's trickle and down to make their widgets and other, you know, font information.
All that neat stuff we want available and accessible to assistive technology. So we can
enter and pass it on to the user. I have been told by 20 different times why QT does not
implement ATK. It's talking directly to this thing called the bridge. And I keep forgetting,
in one of these days, I should just write it down, that I've got premature sonility or something
like that. Or maybe it's age-appropriate sonility. I can't remember why, but there's a really good
reason why the guide note, you didn't do it. He talks directly, he created a special bridge.
Everyone who implement ATK, it just lands in this thing called the ATK bridge. But ultimately,
the thing I care about is that ATSPI-2 rectangle. If it gets to ATSPI-2, I can get at it.
So I'm screen reader on the far left. And I've got a double-pointed arrow there because most of the
things when you care about when you're a screen reader is what's going on, where is the user?
But consider the case of when I said navigating by heading. In that case, I have to actually reposition
the carat to another location. So in that case, I am actually, it's not that often,
you know, Orca does actually manipulate the application. And we do it through ATSPI,
which works its way back up through the bridge to the toolkit, blah, blah, blah, blah.
Screen magnifier. We actually do have a good old Michelle magnifier that does not track focus
in the carat, but it's it's under to do list. But how would it find out where the carat is?
Well, let me back up. Why does it care where the carat is? So if you imagine magnifying your screen,
say just 2x magnification. Everything is twice as tall and twice as wide.
Okay? So area wise, you know, we need four times the space. And odds are you don't have four times
the monitor. So you're only seeing one fourth of the screen at a time. And what that means is if
you're typing really fast, you're pressing tab, you can actually move and frequently do move
right off the screen. So screen magnifier cares about where you are and finds it can find
how hopefully will eventually find out through ATSPI 2 where you are and could do some other
magic again through ATSPI 2 to say, is that on the screen or not? And if it's not on the screen,
move the magnifier so that you actually can see where you are. Okay? Speech for users with learning
disabilities for the most part is just going to be talking to them. There's probably not going to be
much manipulation. So that's an arrow just going down. Speech recognition, we don't have much
of a solution in GNOME. There's some really neat stuff called Simon that, right, it's very
QT-dependent. So GNOME is never officially going to adopt it, but GNOME users can install it.
Really cool stuff still still being developed. And in that case, assuming the user is just not able
to provide input to the computer, you can see what's on the screen. So you can actually like
click on widgets and things like that through ATSPI. Okay? So if I see an okay pattern, I could say,
okay. And then Simon can go off and do some magic and tell ATSPI to go find a click action.
To click on the thing. Okay? And testing tools. I don't know if you guys are familiar with LDTP.
I'm only aware of it and no see the developer in IRC. A lot of automated testing tools,
it's like nothing to do with accessibility per se does that automated testing and manipulation
of widgets and binding out where focus is through ATSPI. Another tool that we're going to talk
about very briefly, it is something I'm hoping I can convince everyone to start check it out through
their own applications and environments called Exercise or it is another testing tool. That's how I
use it. And it also finds out things through ATSPI too. And so Exercise or builds itself as the
interactive accessibility explorer. It's another Python app. Yay, people who are not, they want to be
hackers, but don't really have the background, you can hack on Exercise or okay? So very briefly,
because I'm going to actually do show and tell to make it all work out. You can view accessible
objects. You can examine what interfaces, so I guess I should define interfaces. Imagine a
progress bar. And you want to know that it's at 30%. So we have an accessible value interface.
If you want to know what the font information is, where the care it is, we have an accessible
text interface. We have different, you know, there's accessible table interface. We get
screen coordinates through the accessible component interface. So we can see for any given widget
and any given application, which of these interfaces are implemented at all. What information can
we find out about them? Okay, monitor accessibility events. So again, getting back to that menu case,
I down arrow. Well, sometimes it's got to tell me, you know, focus has changed. Selection has changed.
And for people who are normally programming, I mean, they're called signals. We call them events,
events or signals. And lastly, what's pretty cool is if you're kind of getting into, you know,
how does this accessibility kind of thing work? You're about to do a new feature, an org or whatever.
Sometimes it's nice to take Orca out of the picture and just kind of do those same sorts of
programming you do, but without Orca, like, being in the way. And you can do that to an
exercise. So I got pictures here. So on the, on the right, I got, no, on the left, I've got
G-Edit in the background. Hello world. And so I've got an exercise here, scrolled to G-Edit. So I see
the G-Edit accessible application. And these are all accessible objects, by the way. They correspond
to a widget. Okay, these aren't the widgets. These are accessible representation that we see of the
widget. And I can see the menu bar. I can see the file menu and edit menu and the view menu. And
I can see their roles. I can see a crazy lot of children, by the way. When I did these screenshots,
I do a bleeding edge dog fooding. This is for Dora 18. And I really don't think there's that
very accessible children. And you're going to see something even sillier momentarily. So
sometimes we see weird things like that. Normally in stable environments, we don't.
In fact, here's the work. Who could spot the weird thing? But what does this is me looking at,
so say the carrot moves. These are down arrow in line by line by line kind of thing. And I need to
present a new line to the user. Hello world. Well, I got to find out what the text is. Or if the
user wants to, you know, do the Orca modifier plus app. And I need to give them the fun information.
Well, where do I get that? And I get that through the accessible text interface. So to
exercise or without having to do anything other than just look, I can find out, you know, it is
accessible text interface implemented. Well, it seems to be. Do I have all these attributes
exposed? Yeah, do. The foreground color looks to be about the square of what I'd expect it to be.
But normally, colors are, you know, decimal, you know, like 255, 255, 255 RGB values. Why they're
in a 54 and 55,000? I'm not sure here that I'll file it by later. Next slide.
Events. So here is a case where the previous two G edit windows, I had hello. So what I had done
in between screenshots here is I highlighted or selected the LLO with it selected and typed a Y
to replace the LLO with Y to get a Hey world. And I got two events. I got an object text change
delete. The first detail too was the offset that started. The second one tells you how many
characters, which is in a way redundant because the third actually gives you the characters and
you could probably just count them. But anyway, that's that's the information I got. I know that
LLO is gone. And I immediately got a text change insert event that tells me that the character Y,
which is only one character, was inserted at offset two. Okay, these are these are these are the
kind of thing. This is how Orca does what it does. Okay, next one. And here's just an example of the,
you know, kind of thinking, well, let's say I wanted to present or move the character, do whatever,
you know, say I never implemented support for GTK. And I'm trying to figure out how this works.
This is me giving Python. So here's another alphabet soup. Pyat speed. So we got ATSPI,
slash ATSPI to the accessible technology service provider interface. Well, there's Python bindings,
which is, well, I guess technically they're bindings. They do it through introspection. So in a way,
it's not. But anyway, that is how Python assistive technologies and LDP and
Anxercizer interact with this information. Well, you get the icon console and Anxercizer,
you can start poking at the stuff yourself. Next slide. Okay, or you can roll your own listener.
And well, I've got the dead is simplest case. I mean, life is way more complex than this,
but again, I'm trying to recruit and convince people that you two can do this. And you can.
So all you have to do if you want to roll your own listener is you pick the event type. Okay,
let me example, we're going to use is again, pick picture, G edit, navigating through the menu.
And your screen reader, your job is to present those menu items. So there's a lot, a lot of
different types of events. They're listed in Anxercizer. They're listed in the accessibility
documentation. You pick the right ones. Okay, you figure out when you get that notification of
that event, what you're going to do with it. You tell the ATSPI registry that you care about it.
And here's, you know, here's the handler for it. And you start the registry up. So next slide.
So I'm hopefully able to do the lights and everything people can see this. So nine lines.
And that's counting the import and the pretty spaces I put to make it readable.
So I import pi ATSPI. I create a listener. It's jobs just going to be to run out the thing.
And I registered for a pretty generic event object state changed because it might be focused,
might be selected. I don't know. Started up the registry. Next slide. And I didn't put the output
here. And I knew this was just going to happen. I was always going to try to take a screenshot,
but to get it to fit on the slide, it would be too small to small. Everything starts spearing out
because if you're in an accessible environment, and you say, hey, I care about state changes,
you're getting anything that became focused, anything that became selected, anything that became
checked, anything that became visible. I mean, there's like a list of 20 things.
And you're getting it with the values true. And you get any values false. So I mean, just
what you know, you move from one mini-witem to another. That is a minimum of four events.
Okay. So you get like huge spewage and you go, oh, that was a mistake. I didn't want to do that.
So how do we narrow it down? Well, the good news is we can really listen to very specific,
or register for, very specific event types. The bad news is that's about all we can specify.
At least right now, I've got a RFE file, a system of technology really need to be able to
to fine tune what we ask for. We can't say, I only care about it if it's in G-Edit to the registry.
We can't say, you know, I only care about it if it's true, or I only care about it if it's a menu
item. I really wish we could. We can't. So what you have to do as an assistive technology is have
really efficient handlers for this. Okay. Find out right away. If it's not of interest, return.
It's right away. Because when you register for an event, you get it for every accessible object
in that environment, whether the application's focused or not. Next slide. Okay. So here's
take two of this, which is looks like 15 lines. Still was my nice, pretty blank lines.
So I did two things. I'm going to present them in reverse order. So at the bottom,
I mean, what could, out of all the different kinds of states that are changing?
Is your airline control menu? Well, keep our focus is moving. And, you know, I see the menu
becoming selected. So I guess state change focus, state change selected.
And my listener, I've found out what the application name is. And if it's not coming from G
Ed and it's not coming from menu, I don't care. I just want the rest. And the results are much,
much smaller. In fact, it fit on a slide. Okay. So that's what I got. So you could, I mean,
it takes a while if you've never looked at this format and output to take it in. When you look at it,
I mean, you can picture exactly what I did. Okay. After state change selected in the case of
state change, the first detail, ones and zeros here, tell you true or false. So the first line says,
something became selected. What was it? A menu item called new came from the application,
G out of the second event, something became unselected. The menu item. And I must have been
when I down arrowed because right after that, something became selected again, the open menu.
This is all I need, slight over simplification. This is all I need as an assistive technology
that wants to present menu changes to a user who's blind. You know, I still got to do some other
staff and I got to make sure it goes out in speech. I got to make sure it's relevant. It's that simple
folks. Okay. Next slide. Okay. And so here's just essentially what I said was pretty formatting to
to to gray out the stuff I don't care about. By the way, well, why don't I care about things
becoming unselected? These are not there anymore. These are the care about where he left.
You care about where he is now. Okay. So I just ignore the ones that, you know, we're unselected.
Present the ones that are what's up relevance to the user. We'll set the splite text and,
you know, they've, you didn't know what the splite text was. But again, when you look at the name
of those successful menu items, well, there it is. Okay. So my conclusions are as a screen reader,
I need to listen for object state change selected, where the first detail is one, meaning true.
Then I speak in brown name. And then my other thing is here I'd note to self because if I read
the handy, the handy, uh, ATK documentation, the definition of key for focus changing is that's,
you know, I forget the exact language, but basically keep when you down arrow in a menu,
that meets the official ATK definition of keyboard focus changing. It's like, why don't we get
in that event? What did GTK change? And I'm pretty sure I already found this in file a bug a
while ago, and I need to ping some developers about it. But this is, you know, a crash course
and accessibility 101. Okay. So we need you. So we got a bus factor of one and like so many
different groups. We have some ATs that don't exist. So people are like, oh, okay, you should also
be the, uh, assistive technology for people with learning disabilities, even though they can see
and don't need braille. Uh, excuse the snarkiness. Um, and so where there's, you know, this is a normal
join us kind of speech, all these different areas. You don't have to be able to program.
But let me give you a very specific case of why we need destined help. Um, and in many different
areas, but, um, Ubuntu, hopefully, um, whatever it is precise is going to come out accessible.
They broke like in the past month or so. So much stuff. So much stuff. Um, we have a lot of
Ubuntu users, um, and they're, they're angry too. The ARCA users are, they're, fortunately,
they're not complaining on your Oracle list anymore. They're complaining on your Ubuntu accessibility.
I mean, really nice people, the sweetest people, I mean, they're, they're dropping F-bombs.
Ubuntu kind of really blew it. And, you know, I've, I've, they're not sure. Everyone's asking the
question, well, how did you blow it this bad? You know, where's the regression testing?
Why aren't the Unity developers? Because Unity used to be accessible. It's like, apparently,
again, and it says it's being recorded. It's all going to be fine. Um, but, but, you know,
we're going to be pretty close to code for you to know why this stuff is broken. Um, we need more
people testing. And the problem here, the barrier to entry for a lot of users who are blind is,
if you only have one computer, and in order to actually test and interact with that computer,
you know, you need Unity working. You need Oracle working. You need ATSPI working. You need,
you need, you need, you need, you need, you need, you need. And if any one of those things breaks,
you can't use your system. It's not like you can turn on your vision and then diagnose it and
fix it, okay? That point you might have to reinstall your whole system. So users who are blind,
bless their hearts. They are testing throughout the process. But in terms of really, like,
grabbing Ubuntu and grabbing Fedora and grabbing, you know, insert this drum here, it's happening pretty
close to the release. We need people, you know, not necessarily, you can see, but people with, you know,
who understand how this stuff works, who maybe have a spare system or virtual machines,
who from all along the process, could test. And who know enough about, which is pretty much what I
showed you already, know enough about ATSPI, an advisor and writing, you know, listening for events,
that they can actually do some diagnosis and say, hey guys, this is why Oracle is not presenting
this field, not giving any events, or we don't think it's focused or whatever. We need a bunch
to help in this area. Anyone who is good at automated regression testing, we could use that too,
because if we had proper automated regression testing, both upstream and GNOME,
and for that matter, we had a recent regression in the Q accessibility and downstream and Ubuntu
and Fedora and blah, blah, blah, this stuff wouldn't happen. So it could be live functional testing.
It could be writing regression testing, but we definitely need help there.
We need bugs reported, you know, if we don't, you know, this field, if we don't know, it's broken,
you know, we can't fix it. Even at the toolkit level, there's like NX, you know,
it's slight exaggeration, but there was like one main guy doing DTK accessibility.
There is one main guy doing clutter accessibility. You know, we have got so many bus factors of one.
Okay. People fix and bugs, people implementing features. It would be amazing. Please join us.
By the way, if you write your own applications, please find out if they're accessible or not.
And if you're not, and you know, I'll give contact information briefly,
come find us. We'll help you make it accessible. We'll make, Trisco, like super, super accessible.
It'll be amazing. We want more people involved. Okay. But back to the free software community
experience, you know, typical spiel. Even if you can't code, we need documentation writers.
We need people translating and localizing community outreach. It sounds like, oh yeah,
we'll even you can join kind of thing. The biggest load off of two of our team members,
shoulders, one of them being mine is we got a marketing guy. He's actually got it for
the GNOME accessibility campaign running. He deals with getting quarterly reports and stuff
like that, you know, and release notes out. And all of a sudden, the other developer who does the
GNOME shell accessibility and me, we're actually able to focus on GNOME shell accessibility instead of
like marketing related, you know, bureaucratic things that really need doing. But, you know,
we just don't have time to do it all kind of thing. We need people in so many different areas.
Okay. And so why is speech synthesis my top area of choice? You heard me speak, right?
Enough said. Free speech synthesis. It's like it's awesome. It's there. It's in a lot of
different languages. And it sounds like the robot from Boston's base with less emotion. Okay.
The accessibility libraries, you know, again, these are things that are actively being developed
with like one person. Whatever your itch might be, you know, we need to come scratch it in accessibility.
The last one I want to point out, distros, I am on, here's my new kick. Everyone needs an
accessible installer. Ubuntu has an accessible installer. I don't think they broke that. They
they have, but like no one else might shouldn't say no one else. Do you have an accessible installer?
Trisco? Is there a? Okay, so you so you in here? Make sure it's not broken.
Yeah, but, you know, then try and it's pretty okay. It's not working right now because we have
some of our English talent. Okay. Okay. Okay. Yeah, a lot of them don't, you know, I mean,
I personally am a Fedora user. I like it. My other reason is it's so close to upstream that it makes
it really easy for me to do my GNOME-related work because, you know, I don't have to worry about
what's being passed. It's just all upstream goodness. And I would really love all these users
here dropping f-poms. It's like, you know, if you want to know for sure where ARCA works,
it works really great in Fedora. Where's the accessible installer? Kind of thing. So anyone
who wants who likes writing installers, we need all distros to have that. Next slide, please.
I don't expect everyone to like write these down, but there are a bunch of different active
accessibility communities with very few people in them, but whatever distro, whatever desktop
environment, whatever it's an environment you want to scratch an in, there's probably a group,
a very, very small group that would love to have you. And the last one, Google it. Okay. Or as I
forget who it was now, but earlier in the presentation, someone said, you know, you can start your
own community. That applies to accessibility too. Okay. If the thing you care about is I don't even know
you know, music editing software. Is it accessible? You know, if it's not, what is it going to take
to make it accessible? I think there's a lot of blind users who would love to use your music editing
software. But you know, we need to form a community around it, get people writing accessible software,
get people testing it, and so forth. So lastly, I've got the two long didn't read slide.
The neat thing, and it's not, you know, just makes it sound like good home accessibility is where it's
all at. I mean, historically, it was due to the whole sun, IBM, everyone invested in accessibility,
but it's just kind of like because of that history, I mean, the Mozilla guy, the Mozilla accessibility
developers, they're on pound a 11 Y. The cute developer, you know, KDE accessibility,
he's also in pound a 11 Y. The downs, the distros who really care about accessibility,
including Ubuntu. In fact, that's one of the, we have some active Ubuntu people who are there.
If you have any questions, if we don't know the answer, lots are that the person who doesn't know
the answer is actually hanging out in that room and to answer it for you. And if that's not the case,
we know who you need to talk to and are where the docs you need are. So it's, you know, it sounds
like I'm promoting an Elm here. It's just, this has happened to be where we've all converged.
So I talked really, really fast, but I knew I was going to run out of time. So other questions.
I've stunned you all into silence. I have that effect on people. I don't know why.
Okay, well, in that case, thanks.
You have been listening to Hacker Public Radio at Hacker Public Radio.
We are a community podcast network that releases shows every weekday Monday through Friday.
Today's show, like all our shows, was contributed by a HPR listener like yourself.
If you ever considered recording a podcast, then visit our website to find out how easy it really is.
Hacker Public Radio was founded by the digital dog pound and the
African-American computer cloud. HPR is funded by the binary revolution at binrev.com.
All binrev projects are proud to sponsor by lunar pages.
From shared hosting to custom private clouds, go to lunarpages.com for all your hosting needs.
Unless otherwise stated, today's show is released under a creative comments, attribution,
share a like, free.or license.
You're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to be able to do that, but you're not going to