African-Robotics-Unit/AcinoSet

Camera calibration & 3D pose estimation tools for AcinoSet # AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild Daniel Joska, Liam Clark, Naoya Muramatsu, Ricardo Jericevich, Fred Nicolls, Alexander Mathis, Mackenzie W. Mathis, Amir Patel AcinoSet is a dataset of free-running cheetahs in the wild that contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. We utilize markerless animal pose estimation with DeepLabCut to provide 2D keypoints (in the 119K frames). Then, we use three methods that serve as strong baselines for 3D pose estimation tool development: traditional sparse bundle adjustment, an Extended Kalman Filter, and a trajectory optimization-based method we call Full Trajectory Estimation. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided. We believe this dataset will be useful for a diverse range of fields such as ecology, robotics, biomechanics, as well as computer vision. ### AcinoSet code by: - Liam Clark | Ricardo Jericevich | Daniel Joska | Naoya Muramatsu ## Prerequisites - Anaconda - The dependecies defined in conda_envs/*.yml ## What we provide: - 7,588 ground truth 2D frames - 119,490 processed frames with 2D keypoint estimation outputs (H5 files as in the DLC format, and raw video) - this is currently organized by date > animal ID > "run/attempt" - 3D files that are processed using our FTE baseline model. These can be used for 3D GT. - these files are called `fte.pickle`, have a related `(n)_cam_scene_sba.json` file, and can be loaded in the GUI. - A GUI to inspect the 3D dataset, which can be found here The following sections document how this was created by the code within this repo: ### Pre-trained DeepLabCut Model: - You can use the `full_cheetah` model provided in the DLC Model Zoo to re-create the existing H5 files (or on new videos). - Here, we also already provide the videos and H5 outputs of all fram …