Mouse Grooming Behavior Data

Neural Network Annotated Training Data

All training data is contained within a single HDF5 file. This format is similar to having individual files (called datasets) inside folders (called groups).
Interaction with this data is easiest using either HDFView or programmatically with H5PY.
The structure of the data is as follows:


Example image of Grooming HDF5 Dataset

  • First level grouping is Train/Validation split
  • Second level grouping is by Video Clip
  • Each video contains 5 datasets
    • nframe
      • Number of frames in this video
      • Shape: 1
    • ┬ávideo
      • Raw Video
      • Shape: nframes x 112 x 112
    • label
      • Labels for each frame
        • 0 = not grooming, 1 = grooming
      • Shape: nframes
    • mask
      • Information for whether or not annotators agreed
        • 0 = disagree, 1 = agree
        • When annotators disagree, label contains the values from the first person to annotate the frame
      • Shape: nframes
    • nlabelers
      • Number of annotators that have labeled the video clip
      • Shape: 1

Link to the annotated dataset:

Neural Network Training Code

We distribute the code we used for creating a dataset as well as training individual models.
Please read the documentation over on github for further information.

Neural Network Inference Code

Since we combine multiple networks to form a consensus, we share the code to take that combined network and apply it to new videos. Additionally, since the training dataset contains cropped 112×112 frames of the mouse, we also attach our tracking network to the front such that this inference code operates on 480×480 videos.

The code we use to run inferences is on our github:
We also release the singularity container containing the our tracking and grooming model along with the inference code in a simple-to-use format:
Example usage: singularity run –nv GroomingInferRelease.sif Input_Movie.avi
The input movie must be a 480×480 video and appear visually similar to our training dataset. Since we do not employ distortion augmentation in this model, even slight differences in lighting can cause issues with performance.