Leap motion Unity Add Hand delay

[ayanez]

Is there a way to add a delay to the hands. I plan on doing psychological experiment and would like to add some sort of delay to the hands to see how people react.

Thanks!

[Joe_Ward]

If it is a very short delay less than .5 seconds, then the easiest way is probably to alter the HandController script to get past frames rather than the current frame. The Controller object maintains a frame history buffer of 60 frames. Controller.Frame() gets the most recent frame; Controller.Frame(10) gets the tenth newest frame, and so on. At 120fps, this would give you a delay of up to .5 seconds. Obviously it would be frame rate dependent, but you could adjust the timing by looking at the Frame timestamps, if desired. For a longer delay, save frames in your own buffer and feed those into the HandController instead.

To do this, look at the HandController script for the GetFrame() function:

public Frame GetFrame() {
    if (enableRecordPlayback && recorder_.state == RecorderState.Playing)
      return recorder_.GetCurrentFrame();

    return leap_controller_.Frame();
}

To use the Leap's built-in history buffer, just add a delay variable to the last line:

 return leap_controller_.Frame(delay);

where delay is the number of frames back (up to 59).

For the longer delay, you would have to add additional code store the frames from the Leap service into a list and then get the desired frame from your list. Something like:

List<Frame> buffer = new List<Frame>();
int maxLength = 240;
public Frame GetFrame() {
    if (enableRecordPlayback && recorder_.state == RecorderState.Playing)
      return recorder_.GetCurrentFrame();

    buffer.Insert(0, leap_controller_.Frame());
    if(buffer.Length >= maxLength) buffer.RemoveAt(buffer.Length - 1);

    return buffer[delay];
}

Note that I haven't compiled that code, so there could be errors, but it should give you the general idea. Notice that the Record/Playback feature is doing essentially the same thing, but with serialized frames rather than frames stored in a list.

[ayanez]

This works great!

thanks!

( For anyone who wants to use the code above change .Length to .Count)

[Froehner]

Hello, this works great also for me. But this solution only delays the virtual hand. Do you know how to delay also the objects, which are grasped by the virtual hand?

[AndrewHodrien]

Does anyone know how to introduce a random delay between a range of milliseconds (e.g., from 200-500ms, or 0.2-0.5seconds) each time the hand is moved?

For example, the hand is closed and automatically has a delay of a value within that range, then when opened it has a different value within that range.

I am using Leap in Unity. This is urgent for a PhD study so any advice I would appreciate it, thanks.

[Zacryon]

Thanks for your answer. Since this is now a couple of years ago and a lot has changed. I wanted to achieve such a delay functionality with the current version of the Leap Motion SDK (4.4.0 Core Assets). My idea was similar to yours: Just use a buffer but in conjunction with timers. First I tried to modify HandModelManager for that purpose, and then some other scripts, without success. Finally I found a solution by modifying the LeapServiceProvider and Controller classes.

I've uploaded these scripts here Controller.cs and here LeapServiceProvider.cs so you can use them as you wish. At this moment I deactivated the interpolation mode, since I don't need to use it and I didn't want to spend time on figuring out how to adapt the interpolation mode for a delay functionality anyway. The most important points I'll explain in the following:

The trick is to use two buffers. One for frames and one for delay timers. Those must be used in one of the Frame(...) functions of the Controller class. I decided to modify Frame(Frame toFill, int history = 0) since the other one does call this function anyway.

This is the code in Controller.cs:

public void Frame(Frame toFill, int history = 0)
{
    if (!delayActive)
    {
        LEAP_TRACKING_EVENT trackingEvent;
        _connection.Frames.Get(out trackingEvent, history);
        toFill.CopyFrom(ref trackingEvent);
    }
    else
    {
        LEAP_TRACKING_EVENT trackingEvent;
        _connection.Frames.Get(out trackingEvent, history);
    
        if (delayedFrame != null)
            toFill.CopyFrom(delayedFrame);
        else
            toFill.CopyFrom(ref trackingEvent);

        frameBuffer.Enqueue(new Frame().CopyFrom(ref trackingEvent));
        Timer delayTimerFrame = new Timer(delay);
        delayTimerFrame.Elapsed += FrameTimerElapsedHandler;
        delayTimerFrame.AutoReset = false;
        delayTimerFrame.Start();
        timerBuffer.Enqueue(delayTimerFrame);
    }
}

The delayActive value is set either via a function I added or via a modified constructor of the Controller class. If it is set to false it will just do what the function was originally doing. Otherwise it will be checked if a delayedFrame is already available. If not then the hands keep getting updated in Unity as if no delay was there. This is until the first delayedFrame is filled after a given delay with a corresponding buffered frame.

The frames are buffered in the frameBuffer, which is a Concurrent Queue. Then a new Timer is instantiated with the delay time given in milliseconds. For this particular new timer object an event handler is assigned, which I will detail later. This handler is called as soon as the timer elapses. Then just prevent that the timer automatically restarts and start the timer. The object is then buffered in a timerBuffer, also a Concurrent Queue. I used concurrent queues due to their increased thread safety.

Then I have added this piece of code also in the Controller class:

protected void FrameTimerElapsedHandler(System.Object sender, EventArgs evtArgs)
{
    if (delayedFrame == null)
        delayedFrame = new Frame();

    Frame timerElapsedTmpFrame;
    if (frameBuffer.TryDequeue(out timerElapsedTmpFrame))
            delayedFrame.CopyFrom(timerElapsedTmpFrame);

    Timer tmpTimer;
    if (timerBuffer.TryDequeue(out tmpTimer))
    {
        tmpTimer.Elapsed -= FrameTimerElapsedHandler;
        tmpTimer.Dispose();
    }
}

This is the handler for timer elapsed events. This is where the delayed frame gets instantiated for the first time (it is set to null if the Controller object is constructed or if the delay mode is deactivated). Then We just get the first element of the frameBuffer queue and assign it to the delayedFrame. Afterwards we dispose the first now no longer used element in the timerBuffer queue.

In the LeapServiceProvider class also some lines had to be added resp. changed:

public void EnableDelayHandler(System.Object sender, EventArgs evtArgs)
{
    if (_leapController == null)
        createController();

    enableDelay = true;
    _leapController.SetDelayActive(enableDelay);
}

public void DisableDelayHandler(System.Object sender, EventArgs evtArgs)
{
    enableDelay = false;
    if (_leapController != null)
        _leapController.SetDelayActive(enableDelay);
}

These are event handlers which can be called by resp. triggered by another class. I use them to set/unset the delay mode when a GUI button is pressed. As you can see a new controller is created if there is none. This actually utilizes a function and object name ( _leapController ) already implemented. The internal enableDelay variable, which is public by the way, is adjustet according to the commanded mode. And then the mode is effectively set by calling the SetDelayActive(bool) function which I've added to the Controller class. The neat thing about the public delay bool is that you can set/unset the delay mode (as well as the delay duration) via the Unity inspector.

I changed the access modifier for _useInterpolation to public and set it to false by default, due to the reasons mentioned above. It is possible to activate the interpolation mode via the Unity inspector though, but then the delay mode won't operate and all settings regarding the delay mode are ignored. Currently the default value for the delay is set to 350 milliseconds. But you can change that as you like either via the source files or via the Unity inspector to any value within the (positive) range of the double type. There is no assertion for a positive value yet, so be aware! The rest of my changes in these two scripts are simple and self explanatory.
Also keep in mind that the longer the delay duration is the more resource demanding this will be due to the buffering of frames and timers.

Have fun with it!