A Mystery involving EVE Online, Dragon Naturally Speaking, and BLindness

August 20, 2020 3:29 PM

Hi, AskMe. This is a weird one. I'm trying to get into Eve Online, the infamous space MMO game, and running into blind alleys, no pun intended, at every turn. I'd appreciate any help doing so or figuring out what might be going on. Walls of text inside for sure.

So I'm totally blind, and use a screen reader to access all my game information. This has a fairly good OCR feature, but EVE has so much information that it doesn't do very well in this case. I heard rumors of a blind person who is able to play the game using Dragon naturally Speaking, a program which I don't really think of as a screen reader, in the sense I'm familiar with. I spoke to him at some length this morning and was left more mystified than before, to be perfectly honest.

HE claims that he is able to screen read the game information in real-time by moving his mouse over specific parts of the screen, which includes things like the overview panel, chat windows, and so forth. He also says that at least some of the info he gets is by scraping the relevant game log files, which are generated in real-time as he plays. That much I can understand.

The part that baffles me is the mouse reading of seemingly graphical elements. Dragon Naturally Speaking does not, as far as I know, have OCR support, and he insists that the program isn't scanning the game data but is somehow reading what's there in near-realtime. Complicating all of this is that he's not very good at explaining to me how his setup works in technical terms, it was done for him a long time ago by a relative who no longer plays the game. HE was talking about similar work done for Everquest back in the day, which gives you something of an idea of the vintage of the setup he uses.

I'm confused because as far as I know the Eve client just outputs via direct3D, with no kind of accessible data whatsoever. If there's any alternative, I've never found it.

I'm left confused because I don't doubt that this man is telling me the truth, and yet I feel like there's some fundamental piece of my understanding missing here. Needless to say when I use my mouse in the client the screen reader says nothing without OCR. HE claims not to use Dragon for its dictation abilities, which are about all I know it for.

It's always possible he's scamming me, because this IS eve, after all, but I can come up with no motive for doing so and am trying to figure out what he's going on about, if only because I want to see what I can do myself, but am unsure how to begin. I asked him about talking to the relative who did the initial setup, but he seemed dubious.

I've put in an email to the support people at the game company itself, but don't hold out much hope they'll be able to explain this any better than he did. IF anyone here knows anything about this game, or Dragon Naturally Speaking, or what might be going on here, I'd appreciate any pointers. :)

posted by Alensin to Computers & Internet (9 answers total)

caveat: I know very little about Eve or screenreaders, but I did a little bit of digging for you that I think might be helpful:

It looks like someone uses a screen reader called NVDA and an OCR plugin to play Eve: link, specific comment. Here's a link to the OCR plugin: link.

Perhaps they're using something called Dictation Bridge to connect Dragon and NVDA?
posted by suedehead at 4:05 PM on August 20, 2020

Response by poster: thank you for that, but I’m afraid that person is me. :-) I’m still trying to figure out what is going on with someone else‘s much more sophisticated setup.
posted by Alensin at 4:20 PM on August 20, 2020 [7 favorites]

This is just speculation (I'm not familiar with EVE Online), but I suspect that this person's software isn't really reading the logs, but rather the game's memory. What probably happens is, whenever the mouse hovers over an object, the game creates and handles a mouse-over event for that specific object. At that time, the "EVE screen reader" detects the event and reads some text related to the object. This could be anything, like the object's name, description, status, and so forth.

In the hope of tracking down clues, I tried to look into accessibility software for Everquest, but I wasn't able to find anything. If it's that old, I'm afraid this guy might be running one-of-a-kind custom software that his relative wrote in his spare time and didn't put up on Github or anywhere else. In that case, the most likely explanation for why it uses Dragon Naturally Speaking is because that just happened to be the TTS software this guy had at the time and was most comfortable using, so his relative wrote it specifically to hook into Dragon Naturally Speaking.
posted by J.K. Seazer at 4:27 PM on August 20, 2020 [1 favorite]

Response by poster: J.K. Seazer, thanks. I wondered about something like that, I know that EVE switched to a 64-bit client recently, which presumably would mess up any memory-reading implementation. I'll try to hit up the guy again sometime soon and see if he has any further insight, or could at least put me in touch with his relative.

Any other notions would be welcome. :)
posted by Alensin at 5:28 PM on August 20, 2020

I used to do screen reader testing, and remember how frustrating the technology can be, so I am reaching out along several different avenues to try and find you help. I will update the thread with what I hear.

One source says:
"It is possible the other guy is using JAWS + Dragon together maybe? I think the latest versions of JAWS support reading text images, not sure about life screen though."
posted by evilmonk at 5:30 PM on August 20, 2020 [2 favorites]

Hi Alensin, I'm a Jaws script writer with only a little knowledge of Dragon and no knowledge of Eve. I'm as puzzled as you why a blind user would use Dragon to access the screen. The best guess that I can make is that I think Dragon can sometimes read the tool tip text of objects so you can give a command like "click 'tool tip text'" and Dragon will click on the object with that text. I did a quick google search and that seems to work for web pages - maybe it works in Eve as well.

I'm guessing the relative might have done some Dragon macros to get that text and output it in a way the screen reader can announce it.

Jaws has an option to turn on tool tip announcements (the default is off). Open the settings manager, go to the Verbosity option and select the Configure Verbosity Levels for the verbosity level you use (Beginner is the default). In the item to be spoken list box go to the last item - Tool Tip - and check it.

Now you can move the mouse around the screen and hear tool tips when they appear. Note, this is different than mouse echo. Mouse echo just reads the screen text under the mouse. Because of the way tool tips work it is not possible to move the mouse pointer over the actual tool tip so Jaws won't read it any other way.

I'm not sure when Freedom Scientific added the tool tip verbosity but it wasn't there in very early versions of Jaws. It possible Jaws couldn't do the tool tips when they first set this up (or the user and relative didn't know about the option).

I've noticed your quest for access to video games before and follow your progress. I have one other suggestion for exploring the game environments. Look into BX by Doug Lee. He's another Jaws script writer and BX is a tool for exploring the screen. You can download it from the main page at dlee.org. It's fairly complex so check out the documentation before you put it on your system. It makes modifications to the Jaws default scripts but can be easily removed if it doesn't help you.

My email is in my profile if you want to follow up more outside this question.
posted by rsclark at 5:39 PM on August 20, 2020 [2 favorites]

Response by poster: Rsclark, Thanks for the BX mention. :) I'll grab that tonight and at least see what I can do with it. As far as I know Eve does tooltips but doesn't use the typical Windows mechanisms for displaying them, so Jaws/NVDA tooltip detection doesn't seem to work. It's why I suspect there's some other weirdness going on, though I'm still no closer to figuring out what that might be.

For anyone familiar with the game, this is the other player's killboard. I'm impressed considering all I can get out of it at the moment is random OCR nonsense with an occasional complete word.
posted by Alensin at 6:12 PM on August 20, 2020

Eve does have graphics, but it is rather infamously known as "flying spreadsheets". It's super complex and has a lot of depth. The graphics aren't super important to the core of the the game. Many things you fire at are so far away you really can't see, even if you are sighted. And yeah, space is really big and really empty. Most important things are listed in the overview panel. So while you may have a graphical representation, it will probably be as large as a single character or a cursor until you get close to it. So in that sense, it is a really good game to get into. So it it is on the screen, it's probably on the overview too.

I played for about 2 years. About 5 for World of Warcraft and about 5 for the MUDs that most graphical MMOs are based on.

Eve is pretty complex in terms of the things you can do. Not everyone flies spaceships in Eve, even though that it's major selling point. Some people stay in major cities, basically being a broker. Buying goods low and selling high. The low/null sec corporation is a pretty big part of Eve. This is where the big battle stories and scamming stories spawn from. Many of the super large battles are fought fleet on fleet. Meaning you'll be a part of a larger group and be told what to do and when to do it.

Have you checked out loglite?

I can't help you with the screen reader part though. Maybe try and get ahold of the guys relative?
posted by andryeevna at 11:16 PM on August 20, 2020 [1 favorite]

Best answer: So for anyone curious, I didn't figure out what the other fellow's solution to playing the game was, but I did re-examine my previous abortive attempt to get into the game. I contacted the developers behind one of the main bot engines, and they were able to create an "alternate UI," which runs in a web page and exposes, thus far, a readable overview window and the ability to theoretically click on anything. It uses memory reading but not injection so I Hope it won't run afoul of anyone.

It's really cool to finally be able to see the sheer scale of the star systems in the game, and begin to explore all that I can do.
posted by Alensin at 5:00 PM on August 22, 2020 [8 favorites]