Before we close out this series on AI video coaching in real life, I should say we’ve built a lot more at TNNS HEAD. We’ve built tools for finding events, put together a library of more than eight hours of coaching videos, and built a decent college pathway feature with more than a thousand programs organized by level. We’ve also recently started working with a university tennis team on ways to use AI in its program. There’s no shortage of great applications of AI to tennis. This particular problem doesn’t want to be solved. For now.
Here’s why.
There are plenty of great tennis apps and services. We were trying to do something very specific: produce useful coaching from the handheld recordings people can actually take in tough conditions, with a fence in the way, a player half out of frame, or an opponent who’s barely visible on the far side.
Mounted cameras? No problem.
Serve and groundstroke mechanics from a single angle? Sure.
For us, the economics of solving the handheld problem didn’t make sense yet. By the end, we were asking for one useful coaching insight in more than 90 percent of reports. We still couldn’t get there reliably enough to keep pushing.
I expect Elorian Glimpse will get better. There will be competitors, and frontier models will get better at the sort of work we’ve been doing with computer vision. I expect we’ll be able to dust off this work and take another run at it. Maybe six months, maybe 18. If the models keep improving at this pace, that difficulty line should come down. We’ll be ready.
What we learned
Vision and reasoning have advanced at an insane pace.
We spent a lot of time labeling data in the first half of 2026 and very little time doing it by hand later in the year. Astra did most of the first pass for us. We still reviewed the labels, but we spent a lot less time clicking on court-line intersections and trying to mark the center of a blurry tennis ball. Game changer. What Elorian is doing is really cool, too. It’s one reason I expect that curve to look different before long.
OG CV still has a role.
Segmentation, tracking, and pose models may be less necessary someday. For now, they’re too good and too efficient in many cases to ignore. In our Glimpse tests, giving the model player positions helped it put the opponent at the far baseline. It still thought the players were picking up balls while they were playing the point. Giving it the sequence of events helped it make better sense of the exchange. We still had to check whether the advice it gave followed from what actually happened.
What we can do on the phone is advancing fast.
We capped videos at five minutes, and even a few minutes at 60 frames per second makes for a big upload. Tennis courts often have lousy connections. And at least here in Southern California, they get hot enough to put a phone in thermal trouble in no time. Some computer vision tasks, like estimating pose, can already run on the phone. In our setup, the main reasoning and heavier work, including court geometry, still ran on the server. I suspect newer phones and more efficient models will eventually handle much more of it. That would cut out the upload and the wait for results to come back from the server. In our tests, the round trip often took more than 15 minutes.
Check the footage while the person filming can still fix it.
Moving capture onto the iPhone let us check framing, player size, and camera readiness before the point. If the far court is out of frame, there’s very little the server can do about it. Moving to mobile was the right call for us. If I were starting over, I’d get a phone onto a court much sooner.
Recording at 60 fps and showing the model 60 fps are separate decisions.
The camera, live tracker, and coaching model each worked at different rates. We kept the detail in the original recording, then gave the model denser frames around likely action, with enough before and after to understand it. Audio helped us decide where to look, even when we couldn’t tell which court a sound came from. What happens if you downshift to 30 fps? You get half as many chances to see the ball. Give the camera a longer exposure, too, and the ball can turn into a blurry comet on the near court and dissolve into nothing on the other side.
Put everything on the same clock.
Audio, player tracks, motion data, and coaching playback all had to line up with the saved video. Actual frame timestamps mattered. Even a rule as simple as “keep looking for the player for ten frames” changed meaning when we moved from 30 to 60 fps. This one bit us for a while. It took us too long to realize how many problems it was causing.
Finding a person is easier than following the right person.
The player you tapped could leave the frame and come back while someone else was standing there. We even had a case where the server assigned the same detection to both sides of the net. Zooming in helped the near-player tracker and could cost us the opponent we needed to understand the point. We tried several models. Detecting a person was the easy part. Keeping track of which person they were sent us down a much deeper rabbit hole than we expected.
A court overlay can look right while the measurements are way off.
Using evidence along court lines helped us fit the court when few intersections were visible. But in one clip, about two pixels of jitter translated into nearly a meter of apparent movement at the far baseline. That’s a big error if you’re trying to calculate someone’s speed. We saw the same kind of thing with pose and ball tracking. A few pixels can produce outsized whoopsies.
Finding more balls in individual frames can still leave you with no usable track.
Fine-tuning WASB raised candidate recall from 58 to 84 percent on four held-out clips. The same experimental tracker produced zero confirmed tracks with either version. It still had to tell which speck was the ball across frames. Sometimes it drew a perfectly smooth line between two unrelated detections.
The IMU measures the phone. Stabilization changes the picture.
Electronic stabilization could shift and crop the video independently of the motion the phone recorded. Turning it off made the relationship more predictable. We still needed the image and the motion data together to tell when it was safe to smooth the court fit.
The human brain is amazing.
Trying to get a model to see what a person sees, then figure out what to say about it, gave me a new appreciation for what our brains do without us even noticing. We can lose sight of a player or a court line and still follow the point. A good coach can see half a dozen things wrong with a swing and know which one is worth bringing up with that player, right then. It took trying to recreate that to realize just how blessed we are.