Fruit Fly Behavioral Tracking System

This was for a capstone project where my team built a fruit fly tracking tool for a researcher who needed "assay" set up. More specifically, a way to record flies moving through tubes to get real distance and velocity rather than manual methods they used before.

We used OpenCV for the actual detection, which was new to most of us going in, so a good chunk of the early time was learning how contour detection and adaptive thresholding actually work relative to our needs. The bigger surprise was how much the project ended up depending on things outside the code. Our sponsor's videos were shot in a way that made motion tracking a lot harder than we expected, so we ended up designing a physical prop for the recording setup just to make the footage consistent enough to track reliably. That, plus not having a fully locked down video recording workflow, meant a fair amount of development happened without knowing exactly what we needed to build for. We also built a full GUI on top of the tracking logic so that researchers with zero technical background could actually run it themselves, not just us. By the end, it cut the time to process a video down by roughly 65% compared to the manual method our sponsor was using before.

The biggest thing I'd want to revisit is the tracking approach itself, and I think it's actually one of the more interesting parts of the project. Contour detection with adaptive thresholding got us far, but it still had clear flaws. Primarily, it needs presets and manual calibration to compensate for lighting and setup differences between sessions. Getting it tuned well enough to be reliable across real research footage was a large chunk of the work as well. Where I'd take it next is toward something closer to "plug and play," where the tracker adapts to lighting and setup changes on its own instead of needing a human to recalibrate and click ROIs every time. That's a harder problem, and figuring out the right approach, whether that's a different CV technique or something learned, is the kind of thing I'd want to dig into given more time.

The tool is done and usable now, and our sponsor is planning to hand it off to future capstone groups to keep building on. Path visualization, drawing the fly's actual movement trail over the video, was one of the more obvious next features, but it was explicitly out of scope for what our sponsor needed. I'm still working with our sponsor post-grad, since he wants to include a lot of these results in a future research paper. Overall, it's in a solid state for something that's meant to keep evolving past us.