[Homogeneous transformation matrix] Dual-Leap measurement with two computers

[gondsieg]

Hello, I am trying to make a Multi-Leap system to obtain the vibrations of a hand (Parkinson's diagnosis). I am having problems with the implementation of the homogeneous matrix transformation for the "calibration" from one hand to the position of the other hand. This is done in a Sciencedirect article:

https://www.sciencedirect.com/science/article/pii/S2468232216000111#sec3

Please help, thanks!

[HOTTHANDS]

Sounds like a wonderful endeavor.
Not sure how this forum can help because it sounds like you are having a maths problem, not a device problem. Suggestions:
1. In terms of the LEAP devices, make sure you have properly oriented them using the green light on the side as your guide. LEAP uses a RIGHT-HANDED 3D Cartesian Coordinate system so orientation is important. You will need to match the distances and angles exactly as described. Also assure there are no lights directly overhead and you are indoors as the sensors are infrared.
2. If you find the diagram and the algorithm are not working for you, the best course of action is to contact the Principle Investigator at the University... This was supposed to be a peer reviewed paper. They need to know if it doesn't work.
3. There is also a robust Computer Vision community out there that can help you with your algorithms, transforms and possible machine learning work. https://www.learnopencv.com/about/ This is the most friendly computer vision site I have found.
4. When LEAP doesn't want to work, you can reset the service or reinstall the service software and it recovers. Remember that this is still a Beta ...

[gondsieg]

Thank you for responding, I've been working on this for at least 11 months and this is the most complex, since in the research it doesn't clarify the point about the application of the matrix (it doesn't give information about its mathematical/programmable procedure). I'll keep trying to contact your principle investigator. Thanks again!

[HOTTHANDS]

It saddens me that you have been working on this so long with such frustration.

I was honestly not very impressed with the paper and would encourage you to reach out to the computer vision community and perhaps learn a bit from the other link I shared.

Regarding the use of LEAP in 2018, the math that translates the image to the rendered hand object is already done for you.

If your entire purpose is to identify Parkinsonian hand movements using hand objects, you can start by using a single LEAP camera from the desk or bottom point of view to minimize occlusion for outstretched hands.

By tracking the displacement and velocities of specific points in the hand objects over time, you should be able to scrub outliers from glitchy tracking (curve smooth) and gather replayable demonstrations of an individual's Parkinsonian movement profile.

LEAP can track finger/thumb tips, palms position and velocity to render as a Unity3D hand object. No visor or math needed.

LEAP Orion 4 Unity3d kit can record the hand movement profiles AND apply the Unity3D machine learning/AI capabilities to help you find patterns.

Then, adding hand profiles from cameras placed at various angles becomes a hand object problem, not a computer vision problem.

I think leveraging existing technology it will save you from the math heavy lifting entirely and move you toward your problem of interest.