SCNKit didBegin contact sometimes comes after didEnd contact

In an ARKit game using SceneKit, I’m detecting collisions between the real user and walls that I build out of SCNBoxes. I have a cylinder that follows the device’s pointOfView to accomplish that.

The node of boxes is set as dynamic and the cylinder is set as kinematic, that way when the user moves and collides with the boxes, the boxes will move along. All of the above works exactly right.

I use both physicsWorld didBegin and didEnd contact delegate methods because I need to know both when collisions start and when they end to disable and re-enable a certain button. So I have standard code like the following.

In didBegin contact I hide the button as such:

func physicsWorld(_ world: SCNPhysicsWorld, didBegin contact: SCNPhysicsContact) {
    
    if contact.nodeA.physicsBody?.categoryBitMask == BodyType.wallsCategory.rawValue || contact.nodeB.physicsBody?.categoryBitMask == BodyType.wallsCategory.rawValue {
        
        freezeButton.isHidden = true
    }
}

Then in didEnd contact I re-enable the button like that:

func physicsWorld(_ world: SCNPhysicsWorld, didEnd contact: SCNPhysicsContact) {
    
    freezeButton.isHidden = false
}

Most of the time this works fine, however, sometimes the didBegin method runs last, even though I clearly moved away and there are no more collisions so my app still thinks that we’re in a collision state and my button isn't restored. I have SCNDebugOptions.showPhysicsShapes as debugOptions so I can clearly see that the cylinder is not touching the walls at all. Yet, through debugging the code I can see that sometimes didBegin was run last.

I then need to cause a new collision then walk back to have the didEnd function be called and get my button back. But I cannot expect the user to perform such a workaround because my game is buggy.

I'm not using the didUpdate contact call as I don't need it.

How come didBegin is called last when clearly the collision has ended? What could cause this to happen like that? And what could I do to fix this?

I am thinking of a workaround which would check if some time has passed since the last didBegin call that would indicate this is an old collision.

Or is there something else I could check if a collision is currently still in progress? AFAIK, SceneKit doesn't have a function similar to allContactedBodies as in SpriteKit

Is there something else I could use to check that 2 bodies are still in contact or not?

Thanks!