Video Background Removal

Question

Created Sep ’24

Replies 5

Boosts 0

Participants 3

I am searching for a method to remove background from a video. it can be from camera Session fileOutput url or from photo library.

I was able to accomplish live preview of removed background with the depth data and some metal framework code from the example Enhancing Live Video by Leveraging TrueDepth Camera Data. However I count figure out a way to save this as a video so that I can upload it.

Also this method is using over 150% of cpu ( Xcode cpu usage ), which seems to be quite a lot and the device is getting heated up so fast and drops the frames when It hot.

I also found something similar from GitHub using CoreML example by Dmitry Voitekh which only uses less than 40% cpu.

Any information regarding this will be helpful.

Objective : Remove Background from video and save it

Answered by DTS Engineer in 802753022

Hello @alansmathew,

You should consider using the Vision framework for this task, you might want to use GenerateForegroundInstanceMaskRequest to remove the background from each frame as part of your video editing pipeline.

Best regards,

Greg

Boost

Answer 1

DTS Engineer OP

Apple

Sep ’24

Recommended

Hello @alansmathew,

You should consider using the Vision framework for this task, you might want to use GenerateForegroundInstanceMaskRequest to remove the background from each frame as part of your video editing pipeline.

Best regards,

Greg

1

Answer 2

alansmathew OP

Sep ’24

Thank you so much for prompt feedback. I accidentally came across this and I saw this is still in beta, also only have supports from iOS 18.

if possible I would like to know if there is any other alternatives for this problem.

0

Answer 3

DTS Engineer OP

Apple

Sep ’24

Accepted Answer

Hey @alansmathew,

Vision framework introduced a redesigned Swift API in iOS 18, which includes GenerateForegroundInstanceMaskRequest.

However, the same functionality is available via VNGenerateForegroundInstanceMaskRequest in iOS 17.

There is no Vision API specifically for background removal prior to iOS 17.

Best regards,

Greg

1

Answer 4

Houchen OP

Sep ’24

詐騙集團

0

Answer 5

Houchen OP

Sep ’24

Apple協助詐騙

0