Gesture recognizer

[meischder]

Hi there,

are there any good gesture recognizers that work with leap motion in an unity project?
The standard gestures are not sufficient for my project, so I actually need a system which is able to learn new gestures and to recognize them in use. Or alternatively, is there any answer to this thread?

Thanks

[Joe_Ward]

It looks like this library has all the tools you need: https://github.com/accord-net/framework/wiki/Classification, although it doesn't put those tools together specifically into a gesture recognition system -- that part is up to you.

The Gesture Recognition Toolkit library is in C++, but has what looks like a reasonable overview that could help you figure out which tools you might use from the above C# library and how to put them together. You could also potentially wrap the GRT toolkit so that it can be used in Unity via C#, if that suits your needs better.

Also, for less rigorous approaches, you could look at this Objective-C Gesture Recognizer and the JavaScript Leap Trainer. While not C#, they could be pragmatic examples of ways to approach the problem.

[Kulwinderjeet]

Hi There,

I know it is pretty old conversation, but I was curious to know if JavaScript Leap Trainer worked out good enough for multiple gesture training? I have tried JavaScript Leap Trainer to use more than 2 gestures in Web-GLenvironment, but those does not work really well. One of the gesture is often wrongly predicted as a another gesture. It would be really helpful if I can get any feedback on this. I need to use gestures in Web-GL environment.

Thanks

[meischder]

As far as I remember, the recognition of the gestures I created worked well enough in my bachelor thesis. However I'm not sure how many gestures had to be recognized by the JS-Trainer.

How many gestures are you using in your environment? How many learning-cycles are you using when you train the gestures?

In general it seems that the two gestures that are mixed up are not different enough. So changing a gesture may be sufficient (but this also depends on the number of gestures).

Greetings

[Kulwinderjeet]

@meischder Thank you for the feedback. I am not training too many gestures. I have started with 2-3 gestures but those mixed up a lot many time. In each gesture training I have used 5-20 training samples. How many training sample would you recommend?

Also, I believe you are not referring the learning-cycles as retrain a gesture. I could be wrong but when you retain a gesture, it does not utilize the previously trained data. It retrain the gesture from scratch.

Thanks

[meischder]

With learning-cycles I mean the number of repetitions you made when training a certain gesture. Afaik the trained samples will be overwritten when retraining the gesture but if you want to you should be able to concatenate samples from multiple training-sets by using the import/export-functionality

For 2-3 gestures the recognizer should be sufficient so I guess that your gestures are to similar-looking. This could also be due to the $P-logic (it actually just sums up the minimal distances of every sample-point to the closest point in the reference-point-cloud). E.g. if you have to circles, one starting at the top going to the left and another one starting at the bottom going the the left, they could be hard to differentiate.

It could also be helpful to play around with some options (i.e. the threshold).

[Kulwinderjeet]

@meischder, Thank you for the response.