SKVideoNode not working for FairPlay protected HLS

Is SKVideoNode working with FairPlay protected HLS?


I am trying to use a SKVideoNode as diffuse content of a SCNPlane, but am receiving the following error:

{Error Domain=NSOSStatusErrorDomain Code=-12852 "(null)"}, NSLocalizedFailureReason=An unknown error occurred (-12852), NSLocalizedDescription=The operation could not be completed}

the underlying error only shows some stats, e.g.:

#Version: 1.0
#Software: AppleCoreMedia/1.0.0.13G36 (iPhone; U; CPU OS 9_3_5 like Mac OS X; en_us)
#Date: 2017/03/24 14:37:50.050


Here the code I'm using:

- (void)viewDidLoad
{
  [super viewDidLoad];

  SCNScene *scene = [[SCNScene alloc] init];

  SCNNode *cameraNode = [SCNNode node];
  cameraNode.camera = [SCNCamera camera];
  [scene.rootNode addChildNode:cameraNode];

  cameraNode.position = SCNVector3Make(0, 0, 0);

  SCNNode *lightNode = [SCNNode node];
  lightNode.light = [SCNLight light];
  lightNode.light.type = SCNLightTypeOmni;
  lightNode.position = SCNVector3Make(0, 10, 10);
  [scene.rootNode addChildNode:lightNode];

  SCNNode *ambientLightNode = [SCNNode node];
  ambientLightNode.light = [SCNLight light];
  ambientLightNode.light.type = SCNLightTypeAmbient;
  ambientLightNode.light.color = [UIColor darkGrayColor];
  [scene.rootNode addChildNode:ambientLightNode];

  SCNView *scnView = (SCNView *)self.view;

  scnView.scene = scene;
  self.scene = scene;

  scnView.allowsCameraControl = YES;
  scnView.showsStatistics = YES;
  scnView.backgroundColor = [UIColor greenColor];

  // (standard AVPlayer setup and FairPlay stuff)
  // - set up AVPlayerItem using an AVURLAsset that has its resourceLoader set
  // - the resource loader handles the FairPlay chain correctly and the resource loading request
  // finishes loading with no error
}

- (void)newPlayerItemReceived:(nonnull MYPlayerItem *)playerItem
{
  playerItem.delegate = self; // MyPlayerItem simply inherits from AVPlayerItem and deals with observing its status.

  if (self.player != nil)
  {
  [self.player replaceCurrentItemWithPlayerItem:playerItem];
  }
  else
  {
  self.player = [AVPlayer playerWithPlayerItem:playerItem];
  }

// I would normally set the size using the CVPixelBuffer retrieved from a videoOutput attached to the playerItem
// but it appears that FairPlay protected HLS causes copyPixelBufferForItemTime to always return NULL.
// Again, I tested my videoOutput setup with unprotected HLS and it works as expected.

  CGSize videoSize = CGSizeMake(480, 320);
  SKVideoNode *videoNode = [SKVideoNode videoNodeWithAVPlayer:self.player];

  SKScene *spriteScene = [SKScene sceneWithSize:CGSizeMake(480, 320)];
  videoNode.position = CGPointMake(spriteScene.size.width*0.5f, spriteScene.size.height*0.5);
  videoNode.size = spriteScene.size;
  videoNode.yScale = -1.f;
  [spriteScene addChild:videoNode];

  SCNPlane *plane = [SCNPlane planeWithWidth:4.8 height:3.2];
  SCNNode *planeNode = [SCNNode nodeWithGeometry:plane];
  [self.scene.rootNode addChildNode:planeNode];
  planeNode.position = SCNVector3Make(0, 0.35, -5);

  plane.firstMaterial.diffuse.contents = spriteScene;

  SCNView *scnView = (SCNView *)self.view;
  scnView.playing = YES;

// Using an AVPlayerLayer as part of an UIView here instead of SKVideoNode works:
// it displays the FairPlay protected video correctly

// Otherwise using an unprotected stream works as well and shows as excepted on the plane.
}

- (void)playerItemReadyToPlay:(nonnull MYPlayerItem *)playerItem
{
  [self.player play];
}



All this makes me think that SKVideoNode doesn't yet support FairPlay protected HLS and that protected streams do not expose their CVPixelBufferRef. However, since I cannot find any documentation about this there is no way I can be 100% sure.


So far all my tests have indicated that for FairPlay protected HLS:
- using AVPlayerLayer as part of an UIView works

- using SKVideoNode doesn't work

- retrieving the CVPixelBufferRef doesn't work

- drawing the AVPlayerLayer in another CGContext doesn't work


It can't be the HDCP-LEVEL in the manifest, right?

https://tools.ietf.org/html/draft-pantos-http-live-streaming-20

The value is an enumerated-string; valid strings are TYPE-0 and NONE. This attribute is advisory; a value of TYPE-0 indicates that the Variant Stream could fail to play unless the output is protected by HDCP Type 0 [


Exposing the pixel buffer via the API cannot be considered 'output' or does it?


(I'm sorry about this useless edit -- didn't know it would have to be approved again by a Moderator)

Accepted Reply

Hey,


HDCP is enforced by default when using FairPlayStreaming. However, I don't believe that this is the cause of the erorrs you see.


It seems like if what you propose actually worked, then the app could effectivly make a copy of the streaming content? (albeit a low grade one).

This is at odds with the reason for applying DRM in the first place, i.e. the content owner or licensee wants to prevent unauthorised copying. One of the implicit requirements around a client-side DRM implementation, is that it should attempt to secure the video decode/rendering pipeline, so that decrypted video buffers cannot be 'scraped' for instance. Thus I supect this is related to the lower level security restrictions implemented by Apple.

Replies

Hey,


HDCP is enforced by default when using FairPlayStreaming. However, I don't believe that this is the cause of the erorrs you see.


It seems like if what you propose actually worked, then the app could effectivly make a copy of the streaming content? (albeit a low grade one).

This is at odds with the reason for applying DRM in the first place, i.e. the content owner or licensee wants to prevent unauthorised copying. One of the implicit requirements around a client-side DRM implementation, is that it should attempt to secure the video decode/rendering pipeline, so that decrypted video buffers cannot be 'scraped' for instance. Thus I supect this is related to the lower level security restrictions implemented by Apple.

Hi there,
thank you for your answer.

"It seems like if what you propose actually worked, then the app could effectivly make a copy of the streaming content?"

It could if the pixel buffer ref was exposed, yes.

And it might be possible if Apple's own SKVideoNode is not secured against copying (haven't researched into it).


I suspect that maybe Apple's Scene Kit/Sprite Kit might not be considered sufficiently secure by Apple at this stage and so the only way that Apple's API allows to display FairPlay protected HLS visual media at the moment is via AVPlayerLayer. However, since AVPlayerLayer only works in conjunction with 2D display methods this would effectively mean that FairPlay protected HLS is not yet supported in 3D on Apple's platforms -- at least officially, there might be a way using IOSurface but using IOSurface will have your app rejected.


It would be awesome if somebody could prove me wrong :/

did you ever get an answer to this?