Using data from websocket to build a Hand Object in C#

[519396593]

I try to implement a remote application. My platform is Unity.
This needs to connect the leap motion controller with a virtual machine. After I send the leap motion data from virtual machine to the host machine. I don't know how to create a Hand Object from these data. Is there a easy way to do this?

[Joe_Ward]

The Hand object has a constuctor function that you can call.

[519396593]

I think you mean this constuctor,

but the data I receive is a string format with JSON type.
Does that mean I should parse this JSON data and then use this constuctor?

[Joe_Ward]

Yes, you have to convert the JSON into numbers and objects.

[519396593]

I really try hard to do this JSON parsing and build a new hand Object.
However, these parameters which this constuctor needs contains not only float and vector but also Arm and Finger which still need to use a constuctor. And there are even some parameters can't get directly. It is really complex! Is there an easy way to build a hand object from JSON?

[519396593]

or do you have some existing code or Link method?
Thank to Your reply very much!

[519396593]

And there exists one more problem. when i add "/v7.json", want to get a higher json data. It just receives null....

I have tried the v1.json ,v2.json and v3.json can work but the higher version can't be.

[Joe_Ward]

There is no v7 protocol. v6 is the highest.

Ultimately, to construct a Hand object, you have to call the Hand constructor, so there is no simpler way. I don't know of any existing code that converts the frame or hand JSON to C# objects.

[519396593]

All right, Thank you very much! That's fine.

[519396593]

Sorry for that I get a new trouble. These Units of JSON from Both Remote PC and Local PS are millimeters ( It seems millimeters. It's about 10~100). However, The units of the data I get from Leap Motion Controller Unity directly seems is meters. I have no idea what makes the change of the units.
By the way, the asset I used is "Hand Module"

[Joe_Ward]

The LeapServiceProvider applies a transform to the frame of data which transforms the coordinates from the standard Leap Motion frame of reference to the Unity scene world coordinates -- including converting from millimeters to meters.

If you are just trying to feed the Websocket JSON data into our Unity assets, you might want to start with writing your own Provider implementation that uses the WebSocket data. This still involves constructing hands and fingers, etc, but you would then put them in a frame object that is transformed by your provider and made available to the rest of the assets.

[519396593]

Thank you very much. I have got all the communication work down. But there exists a problem while I make a Quaternion from the Arm.Basis.
This LeapQuaternion is used to build an Arm Object. Basis comes from the JSON Data directly.
The result seems always wrong.
I check the documetation. It seems that this basis of right hands is right-hand coordinate, left hand is left-hand coordinate.
Is that correct?
I can't a function from the Leap API Can do this transform, so I use the function of Unity.
Is there exist a problem?

(2) when I do use this Quaternion calculated by this function to build a Arm Object. The Basis Var of the Arm Object is different from the Basis I used to calculate the Quaternion.

[Joe_Ward]

They changed the convention of having different basis types for each hand in the Orion C# library. I'm not actually sure whether this was changed for the JSON data coming from the web socket or not. If you download the Leap Motion SDK (not the Unity assets), you can look at the C# source code. If you look in the ServiceFrameFactory class, you can see how all the hands are built -- from structs marshaled from C in this case. This won't have a basis conversion for the right/left hand issue, but that is just a matter of changing a sign of one of the basis vectors (or two), if I recall correctly.

Note that I'm no longer with Leap Motion, so I won't be able to provide any further assistance. Good luck!

[519396593]

Thank you! I have known a lot from you.
Thank you again.

[fishsticks]

@519396593 Were you able to get this working? Like you said, there's no straight-forward way to do this. The entire JSON is complex to reproduce in Unity.

Thanks.

[fatislord]

Hi, I also has the same problem. I want to sample the hand data on a host PC and rebuild the hand on a client PC by transferring the sampled data using network. I have finished the data transmission, but I don't know where is the best place to insert my data. I mean to replace the original data of client PC with the data transferred from host PC. I have tried this in HandModelManager and LeapServiceProvider scripts, but the problem is that the original data wasn't replaced if my hand appeared above the client Leapmotion, and hand model would be disabled if I disconnect or move client Leapmotion far away. So do you have ideas to solve these problems?

My platform is Unity too.

Many Thanks!

[andrewchilicki]

Hello,
I am currently trying to do the same as well but am stuck at transferring the incoming data to the LeapMotion Unity scripts. Please let me know if anyone has successfully got it running! Thank you!

[Hyro]

Hey there! If you're interested, I've done that recently after a discussion with andrewchilicki, and it's on Github : https://github.com/HyroVitalyProtago/LeapMotionRemoteUnity
Not cleaned yet because I was prototyping, but it's working
I'll probably submit a pull request after cleaning it to the Github Leap Motion Unity Modules