Menu

Timesdelhi.com

May 26, 2019
Category archive

Blindness

ObjectiveEd is building a better digital curriculum for vision-impaired kids

in Apps/Blindness/Delhi/Education/Gadgets/Gaming/Hardware/India/objectiveed/Politics/TC/visual impairment/visually impaired by

Children with vision impairments struggle to get a solid K-12 education for a lot of reasons — so the more tools their teachers have to impart basic skills and concepts, the better. ObjectiveEd is a startup that aims to empower teachers and kids with a suite of learning games accessible to all vision levels, along with tools to track and promote progress.

Some of the reasons why vision-impaired kids don’t get the education they deserve are obvious, for example that reading and writing are slower and more difficult for them than for sighted kids. But other reasons are less obvious, for example that teachers have limited time and resources to dedicate to these special needs students when their overcrowded classrooms are already demanding more than they can provide.

Technology isn’t the solution, but it has to be part of the solution, because technology is so empowering and kids take to it naturally. There’s no reason a blind 8-year-old can’t also be a digital native like her peers, and that presents an opportunity for teachers and parents both.

This opportunity is being pursued by Marty Schultz, who has spent the last few years as head of a company that makes games targeted at the visually-impaired audience, and in the process saw the potential for adapting that work for more directly educational purposes.

“Children don’t like studying and don’t like doing their homework,” he told me. “They just want to play video games.”

It’s hard to argue with that. True of many adults too for that matter. But as Schultz points out, this is something educators have realized in recent years and turned to everyone’s benefit.

“Almost all regular education teachers use educational digital games in their classrooms and about 20 percent use it every day,” he explained. “Most teachers report an increase in student engagement when using educational video games. Gamification works because students own their learning. They have the freedom to fail, and try again, until they succeed. By doing this, students discover intrinsic motivation and learn without realizing it.”

Having learned to type, point and click, do geometry and identify countries via games, I’m a product of this same process and many of you likely are as well. It’s a great way for kids to teach themselves. But how many of those games would be playable by a kid with vision impairment or blindness? Practically none.

Held back

It turns out that these kids, like others with disabilities, are frequently left behind as the rising technology tide lifts everyone else’s boats. The fact is it’s difficult and time consuming to create accessible games that target things like Braille literacy and blind navigation of rooms and streets, so developers haven’t been able to do so profitably and teachers are left to themselves to figure out how to jury-rig existing resources or, more likely, fall back on tried and true methods like printed worksheets, in-person instruction, and spoken testing.

And since teacher time is limited and instructors trained in vision impaired learning are thin on the ground, these outdated methods are also difficult to cater to an individual student’s needs. For example a kid may be great at math but lack directionality skills. You need to draw up an “individual education plan” (IEP) explaining (among other things) this and what steps need to be taken to improve, then track those improvements. It’s time-consuming and hard! The idea behind ObjectiveEd is to create both games that teach these basic skills and a platform to track and document progress as well as adjust the lessons to the individual.

How this might work can be seen in a game like Barnyard, which like all of ObjectiveEd’s games has been designed to be playable by blind, low vision, or fully sighted kids. The game has the student finding an animal in a big pen, then dragging it in a specified direction. The easiest levels might be left and right, then move on to cardinal directions, then up to clock directions or even degrees.

“If the IEP objective is ‘Child will understand left versus right and succeed at performing this task 90 percent of the time,’ the teacher will first introduce these concepts and work with the child during their weekly session,” Schultz said. That’s the kind of hands-on instruction they already get. “The child plays Barnyard in school and at home, swiping left and right, winning points and getting encouragement, all week long. The dashboard shows how much time each child is playing, how often, and their level of success.”

That’s great for documentation for the mandated IEP paperwork, and difficulty can be changed on the fly as well:

“The teacher can set the game to get harder or faster automatically, or move onto the next level of complexity automatically (such as never repeating the prompt when the child hesitates). Or the teacher can maintain the child at the current level and advance the child when she thinks it’s appropriate.”

This isn’t meant to be a full-on K-12 education in a tablet app. But it helps close the gap between kids who can play Mavis Beacon or whatever on school computers and vision-impaired kids who can’t.

Practical measures

Importantly, the platform is not being developed without expert help — or, as is actually very important, without a business plan.

“We’ve developed relationships with several schools for the blind as well as leaders in the community to build educational games that tackle important skills,” Schultz said. “We work with both university researchers and experienced Teachers of Visually Impaired students, and Certified Orientation and Mobility specialists. We were surprised at how many different skills and curriculum subjects that teachers really need.”

Based on their suggestions, for instance, the company has built two games to teach iPhone gestures and the accessibility VoiceOver rotor. This may be a proprietary technology from Apple but it’s something these kids need to know how to use, just like they need to know how to run a Google search, use a mouse without being able to see the screen, and other common computing tasks. Why not learn it in a game like the other stuff?

Making technological advances is all well and good, but doing so while building a sustainable business is another thing many education startups have failed to address. Fortunately, public school systems actually have significant money set aside specifically for students with special needs, and products that improve education outcomes are actively sought and paid for. These state and federal funds can’t be siphoned off to use on the rest of the class so if there’s nothing to spend them on, they go unused.

ObjectiveEd has the benefit of being easily deployed without much specialty hardware or software. It runs on iPads, which are fairly common in schools and homes, and the dashboard is a simple web one. Although it may eventually interface with specialty hardware like Braille readers, it’s not necessary for many of the games and lessons, so that lowers the deployment bar as well.

The plan for now is to finalize and test the interface and build out the games library — ObjectiveEd isn’t quite ready to launch, but it’s important to build it with constant feedback from students, teachers, and experts. With luck in a year or two the visually-impaired youngsters at a school near you might have a fun new platform to learn and play with.

“ObjectiveEd exists to help teachers, parents and schools adapt to this new era of gamified learning for students with disabilities, starting with blind and visually impaired students,” Schultz said. “We firmly believe that well-designed software combined with ‘off-the-shelf’ technology makes all this possible. The low cost of technology has truly revolutionized the possibilities for improving education.”

An Xbox controller with a built-in braille display is Microsoft’s latest gaming accessibility play

in accessibility/Blindness/Braille/Delhi/Gadgets/Gaming/Hardware/India/Microsoft/Politics/vision impaired/xbox by

Microsoft has been leaning into accessibility in gaming lately, most visibly with its amazing Adaptive Controller, and a new patent suggests another way the company may be accommodating disabled gamers: an Xbox controller with a built-in braille display.

As you might expect, it’s already quite hard for a visually-impaired gamer to play some games, and although that difficulty can’t be entirely alleviated, there are definitely things worth doing. For instance: the text on screen that sighted people take for granted, documenting player status, items, onscreen dialogue or directions — how could these be read by a low-vision gamer who might be able to otherwise navigate the game world?

In many circumstances a screen reader is what a visually-impaired person would use to interact with this kind of data, but often that text is relayed to them in audio form, which is far less appealing an option when you’re in-game. Who wants to have a computer voice reading off your armor levels and inventory burden while you’re trying to take in the ambient environment?

There are already some braille display accessories for this kind of thing, but there’s nothing like having support direct from your console’s designer, and that’s what Microsoft has demonstrated with its patent for a braille-enabled controller.

The patent was filed last year and just recently became public, and was soon spotted by German tech site Let’s Go Digital; there have been no official announcements, though the timing is favorable for an E3 reveal. That said patents don’t necessarily represent real products in development, though in this case I think it’s worth highlighting regardless.

The braille Controller, as it’s referred to in the patent, is very much like an ordinary Xbox One gamepad, except on the back there appears to be a sort of robotic insect sticking out of it. This is the braille display, consisting of both a dot matrix that mechanically reproduces the bumps which players can run their fingers over, and a set of swappable paddles allowing for both input and output.

The six paddles correspond to the six dot positions on a braille-coded character, and a user may use them to chord or input text that way, or to receive text communications without moving their fingers off the paddles. Of course the mechanisms could also be used to send haptic feedback of other types, like directional indicators or environmental effects like screen shake. I wouldn’t mind having something like this on my controller, in fact.

Naturally this means games will need (and increasingly are including) a metadata layer for this kind of conversion of visual cue to auditory one, and vice versa, among many other considerations for gamers with disabilities. It’s on everyone’s minds but Microsoft and Xbox seem to be taking more concrete steps than the rest, so kudos to them for that. Hopefully their leadership in this space will help convince other developers and manufacturers to join up.

We’ll be sure to ask the Xbox team about their plans for this controller design and other accessibility improvements when we talk with them at E3 in June.

Blind users can now explore photos by touch with Microsoft’s Seeing AI

in accessibility/Apps/Artificial Intelligence/Augmented Reality/Blindness/Computer Vision/Delhi/Disabilities/India/machine learning/Microsoft/mobile/Politics by

Microsoft’s Seeing AI is an app that lets blind and limited-vision folks convert visual data into audio feedback, and it just got a useful new feature. Users can now use touch to explore the objects and people in photos.

It’s powered by machine learning, of course, specifically object and scene recognition. All you need to do is take a photo or open one up in the viewer and tap anywhere on it.

“This new feature enables users to tap their finger to an image on a touch-screen to hear a description of objects within an image and the spatial relationship between them,” wrote Seeing AI lead Saqib Shaikh in a blog post. “The app can even describe the physical appearance of people and predict their mood.”

Because there’s facial recognition built in as well, you could very well take a picture of your friends and hear who’s doing what and where, and whether there’s a dog in the picture (important) and so on. This was possible on an image-wide scale already, as you can see in this image:

But the app now lets users tap around to find where objects are — obviously important to understanding the picture or recognizing it from before. Other details that may not have made it into the overall description may also appear on closer inspection, such as flowers in the foreground or a movie poster in the background.

In addition to this, the app now natively supports the iPad, which is certainly going to be nice for the many people who use Apple’s tablets as their primary interface for media and interactions. Lastly, there are a few improvements to the interface so users can order things in the app to their preference.

Seeing AI is free — you can download it for iOS devices here.

Go to Top