Biopsychology - Advanced Tools and Emerging Techniques

Neural Activity and Behavioral Tracking Introduction: Why Track Behavior and Neural Activity Together? Understanding how animals behave requires us to measure what they're actually doing. For decades, researchers relied on subjective descriptions or manually coded behaviors—a tedious and error-prone process. Modern neuroscience has shifted toward objective, quantitative measurement of animal movements combined with simultaneous neural recordings. This creates a powerful framework: we can directly link specific neural activity to specific behaviors, uncovering the neural circuits that drive action. The tools and methods described here represent a revolution in how neuroscience approaches this problem. They use computer vision and machine learning to automatically track animal body movements with precision and speed. This section introduces the key technologies that make this possible. Behavioral Tracking: From Simple Positions to Complex Dynamics The Challenge: From Manual Coding to Automated Tracking Traditionally, neuroscientists watched video recordings and manually labeled what an animal was doing—"this was grooming, that was running." This process was: Time-consuming: A 10-minute video might take hours to code Subjective: Different researchers might label the same behavior differently Limited: Researchers could only track gross behaviors, missing subtle movements Modern behavioral tracking overcomes these limitations by using cameras and computer vision to automatically measure an animal's body position in three-dimensional space. Whole-Body 3D Kinematic Recording The most comprehensive approach to behavioral measurement captures the complete three-dimensional position of the entire body. Marshall and colleagues (2021) developed a system that continuously records a freely moving rodent's full behavioral repertoire in 3D space. What makes this powerful: Complete coverage: Every major body part is tracked simultaneously Natural behavior: Animals move freely without constraints High temporal resolution: Captures rapid movements and precise timing 3D information: Provides depth information, not just 2D position on a camera view This whole-body approach provides the richest possible behavioral data, though it also requires more computational power to analyze all that information. Markerless Pose Estimation: DeepLabCut Traditionally, tracking an animal required placing physical markers (like reflective dots) on the body—a process called "marker-based tracking." DeepLabCut, developed by Mathis and colleagues (2018), changed this entirely with markerless pose estimation. How DeepLabCut works: DeepLabCut uses deep learning (specifically, a type of neural network trained on labeled video frames) to automatically identify user-defined body parts in video footage—things like the head, paws, tail tip, or whiskers. The researcher simply: Records videos of the animal Manually labels a few hundred frames (marking where specific body parts are) Trains the neural network on these labeled examples Applies the trained network to all remaining video frames automatically Why this matters: No markers needed: Eliminates the need to attach anything to the animal User-defined: You choose which body parts to track based on your research question Generalizable: A network trained on one animal often works for others Accessible: Has become a standard tool across behavioral neuroscience labs The key innovation is that a trained neural network can "learn" what each body part looks like and find it automatically in new videos, even under challenging lighting or when the animal is partially obscured. Multi-Camera 3D Pose Estimation: Anipose While DeepLabCut operates on single camera views, Anipose (Karashchuk et al., 2021) extends this to multiple synchronized camera angles simultaneously. The advantage of multiple cameras: When you film from just one angle, you lose depth information. A paw might move forward or sideways—the single camera can't tell which. With multiple cameras filming the same behavior from different angles, you can triangulate the true 3D position of each body part, much like how your two eyes give you depth perception. Anipose provides: Robust 3D reconstruction: Combines information from multiple camera views to determine exact spatial positions Handling occlusions: If one camera's view of a body part is blocked, another camera's view can compensate Markerless operation: Uses the same computer vision principles as DeepLabCut but extends them to 3D space Specialized Tracking: Orofacial Movements and Facemap Not all behaviors are equally important for all research questions. Some neuroscientists care deeply about facial and mouth movements—whisker position, eye movements, tongue protrusion, and other orofacial behaviors. Syeda and colleagues (2024) developed Facemap, a specialized framework for high-resolution tracking of facial movements. What makes Facemap unique is that it doesn't just track position—it models how this movement data relates to neural activity patterns. In other words, Facemap is designed to answer questions like: "Which neurons fire when the whiskers move in a particular direction?" Why facial tracking matters: The face is densely innervated with sensory receptors and motor control Facial expressions convey social and emotional information Whisker movements are crucial for rodent tactile exploration From Tracking to Understanding: Machine Learning and Behavioral Classification The Classification Problem Once you've tracked an animal's body position across thousands of frames, you face a new challenge: what does all this mean? A sequence of paw positions, body angles, and head orientations might represent "grooming," or it might represent something else entirely. This is where machine learning classification comes in. Simple Behavioral Analysis: SimBA SimBA (Simple Behavioral Analysis, Goodwin et al., 2024) tackles the challenge of automatically identifying and classifying complex animal behaviors using machine learning. The workflow: Obtain pose data: Use a tool like DeepLabCut to get body positions Label examples: Manually identify a subset of video frames as containing the behavior of interest (e.g., "this is grooming," "this is not grooming") Train a classifier: Use machine learning to learn the pattern that distinguishes that behavior from others Apply to all data: Classify all unlabeled frames automatically Interpret results: SimBA provides explainability—you can understand why the algorithm made each classification Critical advantage—explainability: Many machine learning systems function as "black boxes," where even the developer doesn't know exactly what features the algorithm is using. SimBA prioritizes interpretability, so neuroscientists can understand which aspects of the pose data (which body part positions or movements) actually drive the classification. This is essential in neuroscience, where you want not just to identify behaviors but to understand them mechanistically. Sequential Dynamics: Keypoint-MoSeq There's a subtle but important distinction between static pose and dynamic movement. A still frame showing an animal in a specific posture tells you something, but the way that posture changes over time tells you something else. Keypoint-MoSeq (Weinreb et al., 2023) goes beyond classification to model pose dynamics—the patterns of how body positions change over time. Key concept—sequential modeling: Rather than looking at individual frames independently, Keypoint-MoSeq uses statistical models that understand sequences of poses. It identifies recurring patterns of movement—the characteristic way a body transitions through different positions. These movement patterns, called "motifs," can be thought of as the building blocks of behavior. Why this matters for neuroscience: Neural circuits don't just respond to static positions—they generate and control movement sequences. By understanding the underlying pose dynamics, you're getting closer to understanding the neural code that actually produces behavior. Keypoint-MoSeq bridges the gap between "here's where the paw is" and "here's how the paw is moving through space." Computational Modeling in Neuroscience <extrainfo> The outline also references broader computational neuroscience resources. While these aren't specific tools for behavioral tracking, they provide essential context for understanding why we develop and use these technologies. What is Computational Neuroscience? Computational neuroscience uses mathematical and computer models to understand how the nervous system works. Rather than just measuring neural activity or behavior, it creates formal models that simulate how neural circuits might produce observed behaviors. Key principle—closing the loop: Computational models work best when they engage in an iterative cycle: Make observations (neural activity and behavior) Build a model that explains those observations Use the model to make predictions Test predictions experimentally Refine the model based on new results Behavioral tracking tools like DeepLabCut and Anipose are crucial for step 1 (making precise observations) and step 4 (testing predictions about how specific movements should relate to specific neural activity patterns). Theoretical Integration The textbook The Computational Brain (Churchland & Sejnowski, 2016) exemplifies the modern approach: integrating theoretical models with experimental data. It emphasizes that understanding neural computation requires both: Empirical data: Measurements of what the brain actually does Theoretical frameworks: Mathematical models that explain how measured neural activity produces behavior Behavioral tracking provides the detailed empirical data on behavior that these computational models need. </extrainfo> Connecting the Tools: A Complete Pipeline To fully understand how these methods work together, imagine a concrete neuroscience experiment: Scenario: A researcher studies decision-making in mice, recording neural activity while animals choose between two options. Capture behavior: Use Anipose with multiple cameras to get precise 3D tracking of the animal's approach movements, head turns, and sniffing patterns Extract meaningful behavior: Use SimBA to automatically classify different types of decision behaviors (hesitation, commitment to a choice, etc.) Analyze dynamics: Apply Keypoint-MoSeq to understand the characteristic movement sequences associated with different decision types Link to neural activity: Compare neural firing patterns (measured simultaneously with the behavioral tracking) to specific movement patterns Build a model: Use computational methods to create a model predicting neural activity from tracked behavior and vice versa Each tool in this pipeline serves a specific purpose, but their power comes from how they integrate with each other. Raw video is transformed into interpretable behavior, which is then linked to neural mechanisms. Summary: Why These Methods Matter The tools and approaches discussed above represent a fundamental shift in behavioral neuroscience: Objectivity: Behavior is measured quantitatively rather than subjectively Completeness: Multiple body parts or entire bodies tracked simultaneously Speed: Automated analysis processes hours of video in seconds Integration: Behavioral data can be directly paired with neural recordings Interpretability: Machine learning methods provide explanations, not just predictions Together, these technologies enable neuroscientists to ask and answer questions that were impossible a decade ago: exactly how do neural circuits control behavior at fine temporal and spatial resolution? The answer requires precisely quantifying what animals are doing—and that's what these tools make possible.