Requesting support for inverted use

[mrow1109]

Hi Im guessing this is probably very hard since it is not featured. But i among others would greatly appreciate support from the Leap team for using the controller in an inverted manner. We are using the controller to help people who have suffered stroke and other problems. The software we have works very well. But we want to add some new things that could be very helpful however in order to do this we need to the leap to be able to recognize a hand consistently while the leap is inverted currently sometimes the leap flips it back and forth.

[katem]

Hi @mrow1109 --

Rehabilitation/physical therapy is definitely a really intriguing use case for our technology. We've seen some interesting gamification methods, notably Visual Touch Therapy, one of our Leap AXLR8R participants.

We're always happy to consider new features for the SDK -- can you provide a bit of clarification on A) What specifically you're looking to achieve with an inverted mode, and B) what language you're looking to build this experience in.

[mrow1109]

Currently were doing a very similar project for our senior design. The project has been ongoing for the last two years. We have a suite of games created in java script. We use the leap in most cases in an inverted fashion.We are working with the occupational therapy department on our school. We provide them with the system and they test it on patients. The problem is the majority of these patients who have suffered from stroke, or cerebral palsy, or other conditions that affect their upper limbs, is that they are unable to keep their arm elevated for a long amount of time. So what we have done is we set up a play area where they are able to keep their arm in a rested position while they play the games. The therapists wanted this because it can help with the slow start that is needed in the rehab process. In the inverted mode we are able to capture motion fine and surprisingly we can capture grab strength fairly well. However for gestures and getting the leap to recognize other objects is not possible. The other big problem is that when it is inverted although it captures the motion well, it doesnt translate it that well. It causes finicky movements and it doesn't appear that smooth on screen (this specifically happens for games that have 2 axis of motion).

[mrow1109]

support for the controller being propped on its side could also be very interesting.

[Joe_Ward]

Hi @mrow1109,

Can you elaborate a bit on what you mean when you say that the Leap Motion device "captures the motion well, but doesn't translate it that well." This seems to be a bit self-contradictory, so a little more explanation could help us understand your issues better. Are these observations using the V2 tracking system or the older V1?

Also, have you tried covering the background of the play area with a material that would absorb rather than reflect IR light? That could potentially help quite a bit to increase the contrast between the tracked hands and the background.

Thanks,
Joe

[mrow1109]

That is quite contradictory sorry about that. Before I begin id like to point out we are using a mat thigh absorbs the infrared light and we are using v2. The problem is that when the leap is inverted it see the top of the hand as the bottom of the hand. So if you look in the visualizer doing movement the hand flips back in forth to upside down and then normal. This i believe causes rapid movement that doesnt translate well in the game. For our signle axis games this isnt a big issue however in games with two axis this causes a lot of movement issues, and since it happens often buffering the movement unfortunately exposes the problem even more. Being upside down the leap is able to see were going left right up down forward backward well it just at the same time gives us back extra information about the movement when the hand is captured erraticly that causes finicky movement in our games.it would be very helpful to us if when the leap is above the hand it could see the top of the hand rather than trying to force the hand it sees to be the bottom of the hand.

[Joe_Ward]

The Optimize for HMD API should help with this. When you activate this mode, the Leap Motion software expects to see the back rather than the front of the hand and should initialize the hand models more appropriately.

[mrow1109]

We have tried this, and it doesnt help as much as we would like and or we would think. We still get the same issues.

[Joe_Ward]

I discussed your scenario with one of our engineers and discovered that the current heuristics used when the Optimize for HMD mode is active make some assumptions that are incompatible with a device mounted on a rigid stand. Theoretically, mounting the hardware on the user's head should help, as funny as that sounds. Around the office, people use devices mounted on eyeglass frames or headbands to test the HMD mode (since the supply of proper HMD displays is limited). I don't know whether this is workable in the therapeutic environment or not, but it may at least be worth exploring.

[mrow1109]

We will explore this however its not necessarily ideal. How challenging do you guys think it would be to create a prototype switch that enables the leap to be above or below the hand?

[Joe_Ward]

It is technically feasible, but wouldn't be a small engineering effort. So not something I would hope for in the short term, certainly.

[1337]

Why letting Leap assume its position and not setting it to a fixed one (via Settings... unchecking Auto-orient Tracking)?

[mrow1109]

Joe I hope to see it soon, but wont cross my fingers.

[mrow1109]

Thanks for all the feedback, I appreciate the quick and helpful response that leap always provides!

[sbucca]

It is technically feasible, but wouldn't be a small engineering effort. So not something I would hope for in the short term, certainly.

Hi there, I am also wanting to use the Leap Motion be facing the palms of the user, rather than the top of the hands. Is there any support for this application? Simply transforming the camera position/scale provides unreliable resutls