Skip to content

Hand Motion Capture from a 3D Leap Motion Controller for a Musculoskeletal Dynamic Simulation implemented in Python

License

Notifications You must be signed in to change notification settings

seanschneeweiss/RoSeMotion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

No Maintenance Intended

ROSE Motion

Zenodo Code Archive: DOI MDPI Sensors Article: DOI

LeapMotion recording, export to BVH and Anybody ROSE Motion

Structure / Actions

  • Record Plug in the Leap Motion Controller and make a recording of the right hand
    • Settings
      • Setting "Frames per second" defines the minimum time delta between to recorded frames
      • Checking "Animate" will open the bvh animation after recording, a slider can be used to iterate through the frames
      • Setting the basis
    • BVH Export
      • Setting "Write BVH-File" will export the recorded motion to a BVH file defined in the next setting
      • Choose the filepath and name in "BVH File"
      • Setting "BVH Channels" will export either the channels XRotation, YRotation, ZRotation or also XPosition, YPosition, ZPosition
    • Interpolation Vector
      • Setting "Write interpolation files for AnyBody" will export the files
        • Elbow.any (pronation angle)
        • Finger[1-5].any (angles for all finger joints)
        • FingerLength.any (scaling of the finger lengths based on Leap Motion recording)
        • TimeSeries.any (equally spaced time points between 0 and 1)
        • Wrist.any (abduction and flexion angles for wrist)
      • Choose the template directory, the AnyBody interpolation files are written based on those files
      • Choose the output directory, here the above mentioned files will be saved to
  • AnyBody (repository)
    • Source files
      • Choose "exisiting vector files" if interpolation files are already in the project folder /Model/InterpolVec
      • Choose ".bvh file" to convert it to the interpolation files based on the templates in config/anybody_templates/ and copy into the AnyBody project folder /Model/InterpolVec
      • Choose "Source (.any)" to copy all .any files into the AnyBody project folder /Model/InterpolVec
      • Setting "HAND.Main.any" defines the main model file of the AnyBody project, which should be loaded for the analysis
      • Setting "Start Frame" will define the first frame to start with (cut off the frames before that). Leaving this option empty will set the first frame to 1
      • Setting "End Frame" will define the last frame to end with (cut off the frames after that). Leaving this option empty will set the last frame to end
    • Operations
      • Select the operations which should be executed in AnyBody
      • Setting "Time steps" will rewrite alls lines which match nStep = xx; in the main model file. (e.g. nStep = 50;)
    • Results
      • Selecting "plot after the analysis" will open an interactive plot for the results from the AnyBody analysis (joint angles)
      • Setting ".anydata.h5 file" will save the results from the AnyBody anaylsis to the specified file
      • Selecting "Open AnyBody" will open the AnyBody GUI after the analysis and will load the .anydata.h5 to make a replay available
  • Converter
    • Convert a given bvh file to the interpolation files used for AnyBody based on the templates in config/anybody_templates
  • Animation
    • Open a bvh file to animate it, a slider can be used to iterate through the frames

Basis setting

  • AnyBody initial basis -> select for correct movement within AnyBody
  • Leap Motion first frame basis -> select for exporting to BVH and use in other applications

Setup

Windows

pip install -r requirements.txt

You might have to use pip3 instead, depending on your setup.

  • Start RoSeMotion
python app/main.py

Following python packages and versions were tested:

  • Gooey (1.0.8.1)
  • AnyPyTools (1.7.8)
  • Matplotlib (3.3.3) (only for AnyBody results)
  • Numpy (1.19.5)
  • Pandas (1.1.5)
  • pywin32 (301) (only for window size automation)
  • pywinauto (0.6.8) (only for window size automation)
  • Scikit-learn (0.23.2) (only for BVH animation)
  • wxPython (4.1.2a1.dev5330+3e6be81d)

Credits