Overlaying 2 videos on top a background image

Greetings,


Happy New Year 2019!


I am trying to overlay 2 or more vidoes like Picture In Picture functionality or like group Facetime call. My end goal is store this composed video into a video library.


I developed an iPad App to compose this visually. I can place mutliple frames on screen, zoom, move, rotate and select video to play in the frame. Then request to convert this into a video. So I know what I want from the final video and I check in camera roll to what is the final output of AVVideoExportSession. At present there do not match.


I able to place more than one videos at different sizes in a single video using AVMutalebleComposition and AVVideoComposition classes.


However I am facing challenge with following things what are yet to be done:


  1. Add a background image instead of default black color. I set the background color on AVMutableVideoCompositionInstruction to clear color but there was no impact. I didn't find any such property on AVVideoComposition.
  2. I want the multiple vidoes to be at different location on screen with different size and orientation. I am trying using CGAffineTransform on AVMutableVideoCompositionLayerInstruction to move, rotate and scale. There is some effect but not the one that I desire. I am tring the right order of transform. ☹
  3. I also want to add a mask layer on top of videos so that the videos will be played in different shapes like heart, diamond, or any picture frame shapes.


For the seond and third points I am trying with AVVideoCompositionCoreAnimationTool. I get the bacground Image but videos has black color and that is covering the backgound image. I discovered this by reducing the render size of video composition.


Hopefully I explained my question and look forward to your answer.


Thanks and regards,

Pankaj