Create ML with SwiftUI

Hi there; great question!

Create ML & Core ML, together with the Vision API provide the right technologies to do real-time image recognition.

Because CoreML models are accelerated using the neural engine or GPU, they can provide multiple predictions per second (I recommend 2-5 fps) - which will create a smooth user experience.

I have personally built apps using SwiftUI and CoreML successfully, but for the moment I'm going to point you at some official Apple Resources such as the great demo apps and WWDC videos that talk about how to do this with with UIKit.

The other key technologies you will need to build your app are AVFoundation - which gets video frames from the camera and can display it onscreen, and the Vision framework

Demo App

Here's a demo app which uses AVFoundation and Vision to do machine learning on the frames of live video. It uses UIKit, not swiftUI.

https://developer.apple.com/documentation/createml/detecting_human_actions_in_a_live_video_feed

Videos

For context, I recommend these WWDC videos (I'm in one of them):

• https://developer.apple.com/wwdc20/10043 Build and Action Classifier with CreateML
• https://developer.apple.com/wwdc20/10673 Explore Computer Vision APIs
• https://developer.apple.com/wwdc21/10039 Classify Hand Poses and Actions with Create ML

Technologies

This documentation should help you get familiar with some of the technologies involved:

https://developer.apple.com/documentation/vision/classifying_images_with_vision_and_core_ml

https://developer.apple.com/documentation/vision/tracking_multiple_objects_or_rectangles_in_video

Using SwiftUI

I don't have any links to official demos for SwiftUI and AVFoundation, but please try searching the web for examples.

https://developer.apple.com/av-foundation/