Thank you for raising a separate support ticket as well. I am posting this here to help anyone else in the community that encounters the same issue.
I recommend watching this video made by the VTuber community that will help to guide you through the setup and configuration process for VSeeFace to improve how the hand tracking is applied to your avatar.
I also recommend using V4.1 as it is our current full release and V5 Gemini is currently an early access developer preview.