I just uploaded a video of the subject app.
3D models are typically derived from CAD, but the conversion (including tessellation) to a suitable polygon format has to occur before importing them in the manipulation app and is user's responsibility.
Details, some of which have also been given in the youtube video page, are reported below:
- Leap Motion Controller bone orientations are the only kind of data used to map to virtual hand articulated movements due to generally different sizes between the two (see also the stimulating discussion 2);
- Tracking data are not applied directly to the virtual hand, yet by a spring-damper approach by Borst-Indugula;
- Palm and forearm bodies are "attracted" by 6DoF LeapMotion data (position and quaternion orientation), whilst phalanxes are only attracted by the corresponding LeapMotion 3DoF position data. No torque spring-damping on phalanxes is applied in order to keep the dynamics world more stable since of the high degree of articulation of the hand-arm system, whose kinematic chain gets even longer when a hand touches world bodies as objects to manipulate which, in turn, can be possibly joined to other bodies by some ODE joint.
- Accordingly with using a physics (rigid-body dynamics) engine, manipulation is only based on physics world concepts and mechanisms: 3D trimesh geom shapes of bodies (including the #17 bodies of each hand-arm systems, left and right), force transfer through collision response, friction, joints between pairs of bodies, influence of field forces (e.g. gravity) - no pose- or gesture-based snap or special cases.
A simple use of the app is putting an object on a table and start interacting with it (object mass properties are calculated internally). More interesting manipulations can be set up by defining more details of the dynamics world in a ascii file (static joints, and dynamic "snapping" joints useful for assembly-disassembly, e.g. to plug/unplug a connector to/from an electrical appliance). For the first use case - casual user - I'm devising a sort of wizard for guiding the user in those basic 3-4 steps needed to import and place object to start manipulation.
Presently I'm considering submitting the app to LM App Store, yet the effort to "finish" the app for the official public distribution risks to confirm the 80/20 rule (80% of the time is spent on the 20% of the activities). For now who should be interested to give it a try can request it to me.
To LM, thanks for continuing improving the LeapMotion and stimulating user, and for developer experience exchange.
Paolo
Physics engine library: ODE (old repository http://sourceforge.net/projects/opende, new one https://bitbucket.org/odedevs/ode).
3D asset import library: Assimp.
Scene graph library: Plib (w/ custom VBO exts and shader).
Related community topics:
Is there an app for showing cad models
V2 rigged hand model - about the hand size