uxvirtual
All work

StretchSense Hand Engine

Leading motion capture for hands

Role

Product Lead, React Developer

Services

Product Management, Mentorship, Team Leadership, Integrations Architecture, SDK Development, IoT Development

Project Description

Hand Engine 3.0 is the premiere hand motion capture software for StretchSense's Studio and Pro Fidelity gloves.

Features

  • High data throughput to up to 12 devices per PC
  • RxJS state management
  • Export data via gRPC to Unity / Unreal
  • Export to mocap software
  • Export data to VTubing software via VMC
  • Retargeted OSC animation output to any app
  • Express calibration
  • Advanced pose-based calibration

A core feature launched in Hand Engine 3.0 was the introduction of an express calibration process. This reduced setup time for mocap actors down to 30 seconds for basic usage, using a reduced set of range of motion poses.

Leading the NZ dev team

My primary role on this project was spending two years leading and mentoring our New Zealand based development team, while reporting to our Head of Engineering and interfacing with our product leadership and sales teams. I led the NZ software team's development through the release of Hand Engine 3.0 and post launch support through various feature, plugin and SDK releases. I specifically mentored our intern and junior team members in agile dev practices and was the hiring manager for two additional NZ based staff members. Our junior team members were empowered to be able to contribute additional features to the app, such as improvements to the pose editor and state management. I also contributed the hiring process for several other dev roles as we launched our UK based development team in Edinburgh - these particular roles were critical to our success as the company pivoted into making new products for indie mocap, VTubing and the XR industries.

Post-Launch Outcomes

The launch of Hand Engine 3.0 fulfilled business requirements of reducing technical debt that had accrued in Hand Engine 1.0 and 2.0 and also improved the customer onboarding experience. This new version implemented RxJS stores and a publisher subscriber pattern for state management to make maintenance and development of newer features easier.

Post-launch the development team continued to reduce technical debt and improve GPU and CPU usage in the app, while I lead various integration features into third party motion capture software such as NaturalPoint's OptiTrack.

I lead the following initiatives in our software development team:

  • Created a sensor data visualization tool which was used for internal hardware QA and our support team to decrease customer troubleshooting time.

  • Implemented a gRPC based API which was used for communication between StretchSense's first party plugins and hardware devices.

  • Created documented APIs and release processes for our dev team and QA staff.

  • Published public-facing guides and API specs for a technical audience.

  • Lead initiatives in our QA and development teams to setup CI/CD pipelines, improving our release cycles.

  • Hired and onboarded additional dev team members with adaptive technical skillsets to meet the changing requirements in a fast-paced startup environment, as we moved towards supporting Unity based VTubing, social VR platforms and XR products.

  • Contributed product direction for a user-friendly firmware updater feature built into the app, reducing the need for a cumbersome legacy firmware update process.

Foundation for the future

Eventually Hand Engine was deprecated in StretchSense's newer Reality XR Glove products, as advancements made by our UK based dev team allowed newer products to work without the need for the Hand Engine software, greatly simplifying the setup and software stack for a mass market. Many of the core interoperability features and express calibration UX were ported over to StretchSense's Unity based Reality XR Train and XR Game apps.