Advertisement

AI can help a billion people, but Microsoft can't do it alone

Not even one of the largest and richest companies can tackle accessibility by itself.

"It cracks me up when I meet someone who says, 'Hey, I don't think I have people with disabilities in my company.' And that's when I know they've got people there that are not speaking up."

I'm sitting with Jenny Lay-Flurrie, Microsoft's chief accessibility officer. We've bogarted a few chairs in a hallway at the Washington State Convention Center in Seattle, where the company is holding its annual Build developer conference. Lay-Flurrie, who is deaf, speaks with a crisp English accent. Though she never signs when addressing me, she has brought a sign language interpreter to clarify the questions I'm asking, in case lipreading ever falls short. Together, we are among the few women I've seen at this tech convention.

None of this would normally be relevant -- not our genders nor Lay-Flurrie's deafness -- except in this case it is. For Microsoft, a giant in an industry not known for its inclusiveness, people with disabilities represent the next chapter in a quest to achieve equality.

Less than three hours earlier, Satya Nadella closed Build's day-one keynote not with coding demos but some remarks about humanity. The company plans to use artificial intelligence to help people with disabilities, he said, and will be investing $25 million in grants over five years. Throughout the keynote, closed captioning ran across the big screens in the auditorium.

This wasn't just a gesture of goodwill -- it's also good business. More than 1 billion people, or roughly 15 percent of the world's population, have a disability, according to the World Bank, encompassing what has to be one of the world's largest underserved markets. Disability is an umbrella term that includes lifelong impairments and temporary illnesses, including both physical ailments and mental health disorders. Some people are born with disabilities; many more acquire them later, often in old age.

"Disability is thus not just a health problem," the World Health Organization (WHO) concludes in its definition of the term. "It is a complex phenomenon, reflecting the interaction between features of a person's body and features of the society in which he or she lives."

The stakes are high indeed. According to a report from the US Department of Labor last summer, the unemployment rate for people with disabilities was 10.5 percent in 2016, compared with 4.6 percent for people without disabilities. Only one in 10 people with disabilities has access to assistive products, according to WHO.

Microsoft's Seeing AI app for iOS uses computer vision to do things like see currency, read handwriting and speak text.

If we accept WHO's definition of disability, then we need solutions that bridge the gap between humans and a world that wasn't necessarily designed for them. In that respect, tech companies have begun to step up. Over the past two years, Apple added a wheelchair mode to the Apple Watch, Google introduced an accessibility-checking app for developers (Microsoft already had one), Facebook made its site more accessible and added closed captioning for live videos, and the Consumer Technology Association added an accessibility-tech category to the Best of CES awards.

So far this year, Google Maps began suggesting wheelchair-accessible routes, Pinterest unveiled a higher-contrast mobile app, Airbnb made it easier to find accessible rentals and Apple proposed 13 emoji representing people with disabilities.

Microsoft, too, has been busy. Last summer the company introduced the Seeing AI app for iOS, which uses computer vision to read handwriting, identify currency, speak text and even suss out the emotion in people's facial cues. Soon the OS will get better-organized Ease of Access settings, with the ability to navigate menus using Narrator. Separately, Microsoft is testing eye control for users with limited mobility.

The benefits of AI go beyond accuracy and speed too: It's also cheap. Both Seeing AI and Microsoft's Translator app are free to download. And while it might be a stretch to envision a world where AI replaces Seeing Eye dogs, these tools might flip that 1-in-10 ratio of those who are without access to assistive tools.

Microsoft Accessibility team member Anne Taylor

All told, the company has several ways of sourcing feedback from the community it's trying to reach. There are Microsoft employees with disabilities, user forums, focus groups and the fantastically acronymed Disability Answer Desk (DAD!), which doubles as a dedicated technical-support line and a way for Microsoft to collect common complaints. And yet Lay-Flurrie insists that "no one would ever say we have enough feedback from people with disabilities." In part, that's because the ease with which someone can rate or review a product is itself a measure of how elegantly designed it is. "We desperately want more. We're trying to make it easy and streamlined for people to give us feedback."

But according to Lay-Flurrie, who has been with Microsoft in various roles since 2005, the company first had to rethink its own office culture. "Not just me, but a lot of people have helped to drive it over the time," she says. "We've been doing this stuff internally for quite some years. Getting our house in order and really working on that. And we've got a lot more to do."

If it seems that change has been coming faster lately, though, that's because it has. Lay-Flurrie says that when she entered the workforce some 20 years ago, closed captioning was in use but had yet to fulfill its promise. "I had somebody tell me that closed captioning was going to be a reality," she recalls. "Twenty years later, there's really not been much tangible progress to consumer-based products. AI has completely jumped that in the space of the last 18 months. Twenty-two years, I've been dogfooding this product to the point where it's accurate enough I can use it."

For Lay-Flurrie, that means few, if any, inaccurate captions. "For someone who relies on captions, someone who's deaf, that inaccuracy means it's not usable."

Even by the metric of accuracy, there's still work to do. "We are so not done," Lay-Flurrie adds. "Not every video has captions. And not every website is successful, not every product -- you can't walk into a supermarket right now and have a fully inclusive experience." Indeed, even lower-tech solutions can be hard to find in some parts of the world. Just last month, The New York Times reported that more than a billion people who need eyeglasses don't have them. Why should we expect AI or even closed captioning to become ubiquitous anytime soon?

Then there are other problems that remain entirely unsolved. For Lay-Flurrie, the holy grail is converting sign language to text. She also sees an opportunity for AI to assist people struggling with mental health problems like anxiety and post-traumatic stress disorder. Perhaps AI could track when you need drugs, she says, or recognize when you're heart rate is up or when you're flushed. "It knows the right things and the steps to take when for whatever reason you're not able to."

It's a compelling idea -- now someone needs to build it. "We haven't done a ton yet. I've got nothing to crow about," Lay-Flurrie says. "It's more a massive opportunity for AI, because there's so much knowledge." But it likely won't be Microsoft executing on the PTSD idea. "We know we're at the tip of the iceberg," she adds. "It really is democratizing, seeing where else we can drive impact more broadly. And the best and easiest way to do that is to empower others. Empower the developer crowd." Specifically, the AI for Accessibility program calls for a mix of grants, seed investments and expert advice, when it makes sense.

Microsoft has also been working with some of the same titans it normally calls its competitors, including Apple, Google and Engadget's parent company, Oath, among others. Accessibility advocates from these firms share best practices and frequently end up sitting next to one another on industry panels. "What grounds us is things like the unemployment rate," Lay-Flurrie says. "You can't have an unemployment rate that's double and then think this is a really big compete space. There's so much to do. Too much to do for one company alone."

Surely Microsoft will continue its efforts after this five-year program ends. Twenty-five million dollars isn't even a lot of money by the company's standards. Other than a commitment to accessibility, though, does Lay-Flurrie know what's next? "Oh no," she says. "The world was moving this way five years ago, but I don't think any of us anticipated quite the acceleration that we'd be in at this point."

Then Lay-Flurrie is reminded of something Nadella said in his keynote: The world is becoming a computer. Technology will just be that ubiquitous. "The one vision that keeps me grounded, that I think is going to be incredible, is if you imagine a table full of kids with multiple different disabilities," she says, "and instead of having to leave the classroom or use very big or bulky or specialized equipment, they're using the same equipment as anyone else, but in their own way. It's just mapping the technology to the individual as opposed to the other way around."

Click here to catch up on the latest news from Microsoft Build 2018!