A new AI system called BioCoach can watch you exercise and provide real-time, personalized coaching on your form, addressing a gap left by fitness apps that offer only generic feedback. Developed by researchers at Drexel University and Michigan State University, the prototype combines computer vision, biomechanical modeling, and language AI to deliver specific, anatomy-based corrections during workouts.

The motivation is practical. During the COVID-19 pandemic, when many people shifted to home workouts, the U.S. Consumer Product Safety Commission reported a 48% rise in injuries related to at-home exercise. Without access to knowledgeable coaches or trainers, maintaining proper form becomes difficult, and small mistakes compound into injury risk.

What Makes BioCoach Different From Other Fitness Apps?

Most fitness coaching apps provide generic encouragement or vague feedback like "keep going." BioCoach takes a fundamentally different approach by grounding its guidance in actual body mechanics. The system analyzes exercise videos through two complementary information streams: one captures visual appearance and motion patterns, while the other estimates 3D skeletal movements and body shape, giving the program access to joint angles, ranges of motion, and exercise phases.

Before offering feedback, BioCoach identifies which joints matter most for each specific exercise. For squats, it focuses on hips, knees, and ankles; for push-ups, it prioritizes shoulders, elbows, and wrists. This targeted approach allows the system to provide detailed, anatomy-specific corrections rather than one-size-fits-all advice.

"Many people who exercise at home with videos and apps don't get high-quality assessment of their movements. Feedback is often too generic or simply encouragement but no actual form coaching. Our goal with BioCoach is to provide timely, specific cues grounded in body motion, closer to the kind of guidance a knowledgeable coach would give," said Feng Liu, an assistant professor in Drexel's College of Engineering and Computing who led the research.

Feng Liu, Assistant Professor, Drexel University

How Did Researchers Train and Test the System?

The team started with the Qualcomm Exercise Video Dataset (QEVD), a publicly available collection of hundreds of hours of exercise footage with time-stamped coaching feedback. However, the original annotations were too brief, offering only short comments like "lower your body more." The researchers re-annotated over 200 videos with more than 2,400 detailed notes, specifying biomechanical targets such as "increase elbow flexion to 90 degrees at the bottom" and explaining the reasoning, such as "increase hip/knee flexion to distribute load".

This enhanced dataset became the training ground for BioCoach's language model. Because the time stamps were preserved, the researchers could evaluate not only whether the feedback was accurate but also whether the system responded at the right moment during the exercise.

To validate BioCoach, the team tested it against video-language AI programs from leading research and industry organizations, including NVIDIA, ByteDance, Alibaba, Salesforce, OpenAI, MIT, Shanghai Jiao Tong University, Chinese University of Hong Kong, Peking University, and Peng Cheng Laboratory in China.

How Did BioCoach Perform Against Top Competitors?

When tested on the original QEVD dataset, BioCoach outperformed its nearest competitor, Stream-VLM (a program created by MIT and NVIDIA researchers), in text quality and judged correctness, though its timing score was slightly lower. However, when feedback was graded against the more detailed annotations the team created, BioCoach surpassed Stream-VLM across all metrics, showing particularly strong improvements in biomechanical correctness and detailed, anatomy-specific feedback.

The results suggest that adding explicit 3D kinematics and biomechanical context improves the quality and interpretability of real-time exercise feedback without substantially reducing responsiveness. This is a significant finding because it demonstrates that specialized domain knowledge can enhance AI systems in ways that general-purpose models cannot easily replicate.

"It was encouraging to see that BioCoach was able to perform so well against programs made by some of the top researchers and companies in the AI field. This is still a prototype, but it shows how combining computer vision with structured biomechanical reasoning can make AI coaching systems more useful and easier to inspect," Liu stated.

Feng Liu, Assistant Professor, Drexel University

Steps to Implement AI-Powered Exercise Coaching

• Integrate Biomechanical Modeling: Rather than relying solely on visual pattern recognition, systems must incorporate 3D skeletal tracking and joint angle estimation to provide anatomically grounded feedback.
• Create Exercise-Specific Datasets: Training data must include detailed annotations of proper form, biomechanical targets, and explanations of why corrections matter, not just generic encouragement.
• Combine Multiple AI Approaches: Pairing computer vision with language models allows systems to translate movement analysis into clear, personalized coaching cues that users can understand and act on.
• Preserve Temporal Information: Time-stamped annotations enable systems to deliver feedback at the precise moment when a form error occurs, improving responsiveness and relevance.

What's Next for BioCoach and AI Coaching?

The team plans to enhance BioCoach further by enabling it to estimate joint reaction forces and muscle activation patterns from video. This capability would allow the system to detect subtle compensatory movements that could lead to injury, even when the primary movement appears correct. Such advances could transform how people receive physical therapy and exercise guidance outside of in-person sessions.

"We believe this work could ultimately support exercise and physical-therapy apps that extend the expertise of human coaches and trainers between in-person sessions. A future system could help users receive more specific, timely feedback when they practice on their own, while still keeping human experts in the loop," Liu explained.

Feng Liu, Assistant Professor, Drexel University

The research was supported by the National Science Foundation and involved collaborators from both Drexel and Michigan State University. The team presented BioCoach at the Conference on Computer Vision and Pattern Recognition, hosted by the Institute of Electrical and Electronics Engineers and the Computer Vision Foundation in June. The full paper is available on arXiv, providing detailed technical specifications for researchers interested in replicating or extending the work.

This development highlights a broader trend in AI research: specialized systems that combine domain expertise with machine learning often outperform general-purpose models on specific tasks. As AI continues to advance, the most practical applications may come not from scaling up large language models, but from thoughtfully integrating them with structured knowledge about the problem at hand.