Capturing outside object

Question

Created Jun ’20

Replies 4

Boosts 0

Participants 4

Loved the example of Body Pose Model using Create ML, however I would also like to track an external object such as a ball that the person is interacting with. How can I detect and track the ball along with the body pose using Create ML?

Ref: https://developer.apple.com/videos/play/wwdc2020/10043/

Answered by ForumsContributor in

Boost

Answer 1

atineoSE OP

Jun ’20

In the video at 1:57 it says that this Create ML template only works with human body, not with objects. So it looks like it doesn't cover your usecase.

0

Answer 2

Engineer OP

Apple

Jun ’20

Accepted Answer

You can combine Action Classification with other Vision APIs to reason about a person's actions and interactions with objects.

Please check out the "Explore the Action & Vision app" session on June 25th, where you can

Learn how we designed the Action & Vision app using Object Detection and Action Classification in Create ML along with the new Body Pose Estimation, Trajectory Detection, and Contour Detection features in the Vision framework.

See: https://developer.apple.com/videos/play/wwdc2020/10099/

You can checkout the Vision Trajectory and Contour Detection APIs ahead of time here:
https://developer.apple.com/documentation/vision/identifying_trajectories_in_video

1

Answer 3

Frameworks Engineer OP

Apple

Jun ’20

Thank you!. Yeah, I think it is doable. You need to combine with other Vision APIs to handle the ball, such as the tracker, or object detector (to detect the ball only).

1

Answer 4

Frameworks Engineer OP

Apple

Jun ’20

Your use case sounds similar to this coming session that uses such action classifier and interacts with other objects.

Ref:

Vision & Action: https://developer.apple.com/videos/play/wwdc2020/10099

We also have an Article & sample code for this: https://developer.apple.com/documentation/vision/building_a_feature-rich_app_for_sports_analysis)

1