I developed a small drawing application for the iPad in Swift using Metal. It supports drawing Lines with the Apple Pencil, erase them, select them and so on. Everything is drawn onto an offscreen texture so I don't need to redraw all of them. The App also supports zooming and panning. If you zoom you have to redraw everything. And there begins the problem. At the moment if I pan my canvas I simply draw the offscreen texture at zoom level 1.0 and pan it around. So it looks very pixelated because I zoom into the offscreen texture without redrawing the offscreen texture itself. The problem is I can't redraw the whole offscreen texture every frame while panning because for performance reasons. My idea was to implement a tile based progressive rendering technique. I thought I could split up the screen into tiles with size of for example 256x256 pixels and if I pan I can move those tiles and only the new visible ones I need to render. But I don't really know where to start. I do not know how to render the lines onto those tiles. every line has a single VertexBuffer at the moment storing the triangles. I thought maybe you can use multiple color Attachments for The Render Encoder to draw the lines. So every colorAttachements[n].texture would be a tile. And maybe through the help of Viewport Culling? I don't have much experience in this area so I have no idea. I also found the Sparse Texture feature, which looked like it would solve my problem. But I need to support iPads that do not support this feature. Has someone else made something similar ? Or has any example code for this? Or has a complete different Idea that could help? Thank you very much for your help!

I am developing a drawing app that supports multiple canvases for drawing. I manage these canvases using a collection view with paging enabled. Each collectionViewCell is a canvas and has the size of the collectionView. The setup is similar to a PDF viewer where you can annotate pages with my custom drawing engine. I currently have a custom panning gesture for the CanvasView as well as the panning gesture of the collectionView. I use the UIGestureRecognizerDelegate methods to allow the transition from the canvas panning gesture to the collectionView panning gesture when reaching the end of the canvas. I achieve this by setting the canvas gesture state to .cancelled: func gestureRecognizer(_ gestureRecognizer: UIGestureRecognizer, shouldRequireFailureOf otherGestureRecognizer: UIGestureRecognizer) -> Bool { if otherGestureRecognizer == panGestureCellRecognizer { return true } return false } @objc private func didPan(_ gesture: UIPanGestureRecognizer) { let canvas = canvasViews[collectionView.currentCanvas] switch gesture.state { case .began: canvas.touchBegan(gesture); canvas.redraw() case .changed: let canPan = canvas.touchMoved(gesture) if !canPan { gesture.state = .cancelled } canvas.redraw() default: return } } My issue arises when the collectionView panning gesture is active, and I pan back to the same page; I am unable to activate the panning gesture of the CanvasView. In other words, I can only transition once from the canvas panning gesture to the collectionView panning gesture. Ideally, I want to be able to go back if, on the next canvas, I can pan again on the CanvasView itself. Note that my CanvasView is a custom Metal View, not a UIScrollView. Does anyone have suggestions for correctly handling multiple panning gesture recognizers in this scenario? Thank you in advance!

Hello everyone, I am currently in the process of developing an app that integrates both UIKit and Metal. Within my app, I have a UIKit View and overlaying it, a transparent Metal View. My primary goal is to sync UIKit and Metal rendering. I've adopted the following configuration for my MetalLayer: func setupMetalLayer() { metalLayer = CAMetalLayer() metalLayer.device = MetalEngine.device metalLayer.framebufferOnly = true metalLayer.drawableSize = CGSize(width: frame.width * scale, height: frame.height * scale) metalLayer.frame = self.frame metalLayer.delegate = metalLayerDelegate metalLayer.needsDisplayOnBoundsChange = true metalLayer.isOpaque = false metalLayer.presentsWithTransaction = true metalLayer.maximumDrawableCount = 3 } In particular, setting metalLayer.presentsWithTransaction = true appears to be critical. Here is a simplified version of my rendering function: func render() { self.frameBoundarySemaphore.wait() self.currentFrameIndex = (self.currentFrameIndex + 1) % Renderer.kMaxInflightBuffers let commandBuffer = MetalEngine.commandQueue.makeCommandBuffer() guard let drawable = metalView?.metalLayer.nextDrawable(), let commandBuffer = commandBuffer, let metalView = metalView else { fatalError("Rendering failed") } let rpd = makeRenderPassDescriptor(drawableTexture: drawable.texture) let renderCommandEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: rpd)! renderCommandEncoder.setViewport(MTLViewport(originX: 0.0, originY: 0.0, width: Double(metalView.viewportSize.x) * metalView.scale, height: Double(metalView.viewportSize.y) * metalView.scale, znear: 0.0, zfar: 1.0)) commandBuffer.addCompletedHandler { _ in self.frameBoundarySemaphore.signal() } for objects in renderObjects { objects.render(renderEncoder: renderCommandEncoder) } renderCommandEncoder.endEncoding() commandBuffer.commit() commandBuffer.waitUntilScheduled() drawable.present() } This configuration performs smoothly when I update my UIKit View and call the render method simultaneously. However, I've run into an issue when I use (in my case) the Apple Pencil to only update the Metal content. If I call my render function, the content isn't always rendered. It does update if I perform an action that triggers UIKit updates (note that setNeedsDisplay() is insufficient). However, if I call render() twice in a row, the update does occur. An alternate solution I've found is to temporarily set metalLayer.presentsWithTransaction to false. This workaround seems viable, but I'm uncertain if I might end up rendering an outdated frame when I call render(). I am seeking insights on how presentsWithTransaction operates and how to trigger a content update effectively. The existing documentation hasn't been able to address my query satisfactorily. I would appreciate any insights or solutions from anyone who has experienced a similar issue or understands more about this topic. Thanks in advance