[VisionOS] Entity spawning during the runtime won't respond to my gesture

I'm creating an immersive experience with RealityView (just consider it Fruit Ninja like experience). Saying I have some random generated fruits that were generated by certain criteria in System.update function. And I want to interact these generated fruits with whatever hand gesture.

Well it simply doesn't work, the gesture.onChange function isn't fire as I expected. I put both InputTargetComponent and CollisionComponent to make it detectable in an immersive view. It works fine if I already set up these fruits in the scene with Reality Composer Pro before the app running.

Here is what I did

Firstly I load the fruitTemplate by:

let tempScene = try await Entity(named: fruitPrefab.usda, in: realityKitContentBundle)
fruitTemplate = tempScene.findEntity(named: "fruitPrefab")

Then I clone it during the System.update(context) function. parent is an invisible object being placed in .zero in my loaded immersive scene

let fruitClone = fruitTemplate!.clone(recursive: true)
fruitClone.position = pos
fruitClone.scale = scale
parent.addChild(fruitClone)

I attached my gesture to RealityView by

.gesture(DragGesture(minimumDistance: 0.0)
            .targetedToAnyEntity()
            .onChanged { value in
                print("dragging")
            }
            .onEnded { tapEnd in
                print("dragging ends")
            }
         )

I was considering if the runtime-generated entity is not tracked by RealityView, but since I have added it as a child to a placeholder entity in the scene, it should be fine...right?

Or I just needs to put a new AnchorEntity there?

Thanks for any advice in advance. I've been tried it out for the whole day.