SceneKit "wait drawable" on iPhone only

I have an AR app built with ARSCNView. The scene's pretty complex and it works very well until I notice that dragging some contents in the AR scene is not smooth. So I start profiling the app on different devices. The result is very confusing. First, on my iPad Pro (A10X), the app works really great and it takes less than 2ms for each frame. But on my iPhone XS Max (A12), almost all frames will have a long "waiting drawable" time, and each frame will take up the full 16ms. The NSRunLoop thread will be blocked until the drawable is available. As they're running the same code, so I guess it's not a problem in my code. Does anyone have any idea on why it's different for different devices or better, a way to work around it? Thanks!