Usage tutorials, examples, discussions of Apple's "Dwell Control"?
July 2, 2017 5:22 PM Subscribe
My father recently suffered a spinal stroke and is experiencing limited fine and gross motor skill mobility. He asked me to review the current state of affairs regarding accessibility and handsfree computing. I was pleased to note the introduction of an embedded head and gaze-tracking accessibility feature called "Dwell Control" in Sierra, which is the iteration of OS X he is currently using. However, Apple's help text for the feature is limited and somewhat cryptic. I have looked for demos or examples of people documenting their real world usage of the feature and struck out. Any leads?
There are a couple of independent head tracking programs available, iTracker and DwellClick, I found a single blog entry from 2014 describing a use-case and strategies for using them together.
I demoed iTracker and while it was far from perfect it was much more transparent and intuitive to me than I found Dwell Control, presumably as I am not a mobility-impaired person and am not familiar with extant and well-established strategies for handsfree computing using gaze and head tracking as primary mouse and keyboard-control input. I also found Dwell Control to be aggravatingly unpredictable in presenting me with feedback regarding my gaze location and if it was recognizing or tracking it, with the result that I continually attempted to exaggerate the intensity of my gaze directed at the machine.
This seems unlikely to be the intended usability strategy.
I look forward to learning more. Thank you for your links.
There are a couple of independent head tracking programs available, iTracker and DwellClick, I found a single blog entry from 2014 describing a use-case and strategies for using them together.
I demoed iTracker and while it was far from perfect it was much more transparent and intuitive to me than I found Dwell Control, presumably as I am not a mobility-impaired person and am not familiar with extant and well-established strategies for handsfree computing using gaze and head tracking as primary mouse and keyboard-control input. I also found Dwell Control to be aggravatingly unpredictable in presenting me with feedback regarding my gaze location and if it was recognizing or tracking it, with the result that I continually attempted to exaggerate the intensity of my gaze directed at the machine.
This seems unlikely to be the intended usability strategy.
I look forward to learning more. Thank you for your links.
You mention the help text not being super useful, but I'm not sure if you're referring to the built-in help within the OS, or if you've seen Apple's support article on the topic. It links out to a couple other pages as well, it seemed fairly well documented to me, but I didn't know this feature even existed until you mentioned it.
If you've already seen that, sorry! I hope you find a solution that works for you.
posted by daisy55 at 7:01 AM on July 3, 2017
If you've already seen that, sorry! I hope you find a solution that works for you.
posted by daisy55 at 7:01 AM on July 3, 2017
Response by poster: Yes, I have read that page muitiple times. In practice, it feels as if the machine with Dwell Contol active is randomly choosing when to notice and respond to gaze and head activity, or more accurately, to lack of head activity.
By comparison with iTracker, it feels as if I cannot predict where the head-tracking algorithm is placing the center of my theoretical gaze and therefore the location of the cursor.
With iTracker, it is immediately obvious when the algorithm has gotten lost and the mouse is decentered. With Dwell Control, it is less clear to me that that is an issue, presumably because from Apple's perspective, they have solved the gaze-centering problem.
My question is not so much how to activate the featureset or what the featureset is supposed to do. My question is "how does one use these features?" They feel randomly prone to inaccuracy and produce widely unexpected results. I cannot tell offhand if this is due to a complex featureset and unfamiliar user-interface or if it is because this is an underdone 1.0 feature.
The feature attracted a modcum of attention when initially announced for Sierra but does not appear to have generated a single instance of journalistic coverage, even in the disability press, since then, and Apple does not provide further illustrative or training materials that I have been able to discover.
posted by mwhybark at 8:08 AM on July 3, 2017
By comparison with iTracker, it feels as if I cannot predict where the head-tracking algorithm is placing the center of my theoretical gaze and therefore the location of the cursor.
With iTracker, it is immediately obvious when the algorithm has gotten lost and the mouse is decentered. With Dwell Control, it is less clear to me that that is an issue, presumably because from Apple's perspective, they have solved the gaze-centering problem.
My question is not so much how to activate the featureset or what the featureset is supposed to do. My question is "how does one use these features?" They feel randomly prone to inaccuracy and produce widely unexpected results. I cannot tell offhand if this is due to a complex featureset and unfamiliar user-interface or if it is because this is an underdone 1.0 feature.
The feature attracted a modcum of attention when initially announced for Sierra but does not appear to have generated a single instance of journalistic coverage, even in the disability press, since then, and Apple does not provide further illustrative or training materials that I have been able to discover.
posted by mwhybark at 8:08 AM on July 3, 2017
Response by poster: I'm keeping a giant Google doc open for notes and references as I work on this but will also stop by to leave some Dwell Control specific notes here.
First, looking again at that Apple support article, it begins with the words "With Dwell Control, you can use eye- or head-tracking technology to control the mouse to click, drag, and scroll items on the screen."
In the Preferences Pane itself, the description reads "Dwell Control allows the mouse to be controlled using head or eye tracking technology. Dwell, or hold the mouse still for specified amount of time, will trigger the Dwell action specified in Settings."
So, a few things. First, the language in the Pref pane is ungrammatical, which can be an indicator or a rushed release from Apple. Second, what do these two descriptions actually say?
They say that Dwell Control can be used WITH trackers. They do not say that Dwell Control is an embedded implementation of head tracking, which is what I was fundamentally misunderstanding.
When I enabled it on my father's laptop remotely, nether one of us could see the other's hands on our respective input devices, so we mistook each other's motion input for headtracking.
That said, my dad was very definitely able to move and position the mouse cursor via his trackpad. He does not (yet) have the motor control to click the trackpad or type. So understanding that Dwell Control is really a replacement for keypress and mouse click input, not a head tracking solution that incorporates clickless input, is a huge cognitive leap.
Today when I speak to him again I will demo Dwell Control with a better understanding of what it is intended to do. Hopefully, in conjunction with voice rec for text input, he'll be heading in the right direction.
posted by mwhybark at 2:02 PM on July 3, 2017
First, looking again at that Apple support article, it begins with the words "With Dwell Control, you can use eye- or head-tracking technology to control the mouse to click, drag, and scroll items on the screen."
In the Preferences Pane itself, the description reads "Dwell Control allows the mouse to be controlled using head or eye tracking technology. Dwell, or hold the mouse still for specified amount of time, will trigger the Dwell action specified in Settings."
So, a few things. First, the language in the Pref pane is ungrammatical, which can be an indicator or a rushed release from Apple. Second, what do these two descriptions actually say?
They say that Dwell Control can be used WITH trackers. They do not say that Dwell Control is an embedded implementation of head tracking, which is what I was fundamentally misunderstanding.
When I enabled it on my father's laptop remotely, nether one of us could see the other's hands on our respective input devices, so we mistook each other's motion input for headtracking.
That said, my dad was very definitely able to move and position the mouse cursor via his trackpad. He does not (yet) have the motor control to click the trackpad or type. So understanding that Dwell Control is really a replacement for keypress and mouse click input, not a head tracking solution that incorporates clickless input, is a huge cognitive leap.
Today when I speak to him again I will demo Dwell Control with a better understanding of what it is intended to do. Hopefully, in conjunction with voice rec for text input, he'll be heading in the right direction.
posted by mwhybark at 2:02 PM on July 3, 2017
There will be forums and discussion out there somewhere about this, but finding them can be hard.
I used to work with IT for blind people, and there was tons of discussions and support, but it didn't always turn up at the start of a Google search. Especially Apple stuff, because the really useful access tech news is drowned out by a ton of mainstream news sites posting a two paragraph story regurgitating the press release. And they'll never say "Yay it's great this feature has been included in a mainstream product. It does X ok, but is a bit meh on Y, and doesn't do Z at all. Which anyone who uses these features will tell you is a massive oversight" which is what you need. Yet those sites will happily do a 5000 word essay on why the new torch app is going to completely change the way we use our phones etc, etc...
I'm afraid it's been a while so I'm not so familiar with access tech resources any more, especially in the US. If you were in the UK I would be pointing you towards AbilityNet, they're nice people, so maybe drop them an email and see if they know of any resources. They do have some factsheets on their website, but none for gaze control that I could see.
posted by Helga-woo at 3:08 PM on July 3, 2017
I used to work with IT for blind people, and there was tons of discussions and support, but it didn't always turn up at the start of a Google search. Especially Apple stuff, because the really useful access tech news is drowned out by a ton of mainstream news sites posting a two paragraph story regurgitating the press release. And they'll never say "Yay it's great this feature has been included in a mainstream product. It does X ok, but is a bit meh on Y, and doesn't do Z at all. Which anyone who uses these features will tell you is a massive oversight" which is what you need. Yet those sites will happily do a 5000 word essay on why the new torch app is going to completely change the way we use our phones etc, etc...
I'm afraid it's been a while so I'm not so familiar with access tech resources any more, especially in the US. If you were in the UK I would be pointing you towards AbilityNet, they're nice people, so maybe drop them an email and see if they know of any resources. They do have some factsheets on their website, but none for gaze control that I could see.
posted by Helga-woo at 3:08 PM on July 3, 2017
Response by poster: Still haven't found the specific chatter I was looking for. I do have a bunch of interesting links to assistive technologies both for Mac and other platforms including a fully-featured head mouse system, SmartNav, from NaturalPoint, who also make and market the premier IR-based head tracking technology for PC gamers. The Mac version is marketed directly by their Mac dev partner, R. J. Cooper. The item detail and sales page loads with a default price of $0.00 which I misinterpreted to mean that the Mac version was discontinued but after exchanging emails with RJ, I realized that the dropdown selection will update with the price. $499 / 599 is steep but less than half the prices seen associated with other head tracking mouse analog products.
Finally this morning as I was gathering new links and mulling over my conversations with Dad about his hand strength and dexterity I realized that the specific action preventing him from interacting with his machine was just the click element of mousing (or track padding, but you know).
For a *really long time*, Mac OS has had a tap-to-click feature. I think they introduced it back when they first started adding elements of multitouch to the OS, so it's been in there for about a decade. It defaults to "off", and I haven't used it for years and had completely forgotten about it.
You locate it under System Preferences > Trackpad > Tap to click, and there are a range of other trackpad gestural and multitouch controls and features I mostly ignore, except for two-finger scroll.
So the upshot of this is that right now I don't think any special hardware is going to be needed at all, and we might not even need to enable Dwell Control to get around his limited finger and hand strength. I'm a little embarrassed that it took me two weeks to put all the pieces together in my head.
I'm still going to *demo* iTracker to him, largely because even though it's an imperfect piece of technology, putting on my machine here at home to see what it did and how well it worked provoked one of those increasing few-and-far-between moments of amazed joy when using a computer, the kind where you jump up and go get someone to show the cool thing to - in this case, my wife, who also yelped in surprise.
posted by mwhybark at 11:58 AM on July 4, 2017 [1 favorite]
Finally this morning as I was gathering new links and mulling over my conversations with Dad about his hand strength and dexterity I realized that the specific action preventing him from interacting with his machine was just the click element of mousing (or track padding, but you know).
For a *really long time*, Mac OS has had a tap-to-click feature. I think they introduced it back when they first started adding elements of multitouch to the OS, so it's been in there for about a decade. It defaults to "off", and I haven't used it for years and had completely forgotten about it.
You locate it under System Preferences > Trackpad > Tap to click, and there are a range of other trackpad gestural and multitouch controls and features I mostly ignore, except for two-finger scroll.
So the upshot of this is that right now I don't think any special hardware is going to be needed at all, and we might not even need to enable Dwell Control to get around his limited finger and hand strength. I'm a little embarrassed that it took me two weeks to put all the pieces together in my head.
I'm still going to *demo* iTracker to him, largely because even though it's an imperfect piece of technology, putting on my machine here at home to see what it did and how well it worked provoked one of those increasing few-and-far-between moments of amazed joy when using a computer, the kind where you jump up and go get someone to show the cool thing to - in this case, my wife, who also yelped in surprise.
posted by mwhybark at 11:58 AM on July 4, 2017 [1 favorite]
This thread is closed to new comments.
posted by sevensnowflakes at 12:09 AM on July 3, 2017