Replay hand movement from stored tracking data unity

[kartheekgvms]

Hey all,

I want to replay the hand movement in unity from stored motion data. I think this is very similar to record and playback plugin in 'leapjs'. but I am not sure if we can do this unity. I tried serialization and deserialization of frame data in unity. but, it only gives me byte data. Is there a way that i can get the motion data instead of the byte data and replay my hand movements from it in unity??

Any help at all would be much appreciated.

Thanks
Kartheek.

[weeba2933]

Hi Kartheek,

The byte data includes the hand positions, and all the API data of that frame. The ability to record and playback is already in the Hand Controller! Examples can be found at:

Basic Demo Record Scene:
LeapMotionCoreAssets/Assets/LeapMotion/Scenes/RecordAndPlayback.unity

Basic Demo Script to Record/Save:
LeapMotionCoreAssets/Assets/LeapMotion/LeapMotion/DemoResources/Scripts/RecordingControls.cs

After you've recorded, you can enable the Record/Playback as shown above, and copy the recorded file to the Resources folder and select it in the Recording File field.

To play it manually, you just need to call handController.PlayRecording() and it'll start playing the recorded file based on the Graphics and Physics model in the Hand Controller.

Hope this helps! If you're stuck anywhere, let me know.

Cheers,
Wilbur

[kartheekgvms]

Hi Wilbur,

Thanks for the reply. I just have two more questions regarding this:

1.Can we store the API data of a frame in any human readable format such as txt file,Json etc. instead of bytes into a file in unity?
2. If we can store the API data in human readable format, can we change the storage structure ( i.e can we only store tracking data like Handlist,Gesturelist which are mainly required for hand movement ) and replay the hand movment from the data file in unity??

Really appreciate your help.

Thanks,
Kartheek.

[zars15]

You could always manually store positions and other info you find nescessary. I've done something similar, in order to get Leap Motion hand and finger data in software, that doesn't support leap.

[user-295940283]

Basic Demo Record Scene:
LeapMotionCoreAssets/Assets/LeapMotion/Scenes/RecordAndPlayback.unity

Basic Demo Script to Record/Save:
LeapMotionCoreAssets/Assets/LeapMotion/LeapMotion/DemoResources/Scripts/RecordingControls.cs

Are these available for (or will work with) Orion?

[Joe_Ward]

No, we haven't implemented a record and playback feature in Orion.

[eskeeper]

Any news for supporting a record/playback feature in Orion? I tried implementing it by myself, but "Serialize" property in the "Frame" class doesn't work like that in V2. It seems the Frame is not set as Serializable in Orion. I could still export the Frame object into a xml format to save it into a file, but the Frame object being loaded from a xml file doesn't work.

[fahn]

can anyone provide me with a version, in which this still works? the asset is not included anymore.
Thanks in advance!

[antoson]

The assets are still available on Leap Motion github repo, in folder Experimental/Playback, which you can download and import into your Unity project.