NSCollectionView deselect cell on click

In my project, I've used NSCollectionView with a custom cell item. I want only a single selection on collectionView.

In the collectionViewItem I've many textFields & buttons. I'm able to select a specific cell on click. But I'm not able to deselect again clicking on same cell.

Though I have used following code to deselect cell:

var clickGesture: NSClickGestureRecognizer!
    
    override var isSelected: Bool {
        didSet {
            if isSelected {
                mainBox.borderType = .lineBorder
                //shortContentBox.fillColor = NSColor(red: 72/255, green: 112/255, blue: 255/255, alpha: 0.05)
                mainBox.addSelectionShadow()
                addGestureRecognizer()
            } else {
                mainBox.borderType = .noBorder
                //shortContentBox.fillColor = .controlColor
                mainBox.removeShadow()
                removeGestureRecognizer()
            }
        }
    }
    
    func addGestureRecognizer() {
        clickGesture = NSClickGestureRecognizer(target: self, action: #selector(onItemClick(_:)))
        clickGesture.buttonMask = 0x1
        clickGesture.numberOfClicksRequired = 2
        view.addGestureRecognizer(clickGesture)
    }

    func removeGestureRecognizer() {
        view.removeGestureRecognizer(clickGesture)
        clickGesture = nil
    }

    @objc func onItemClick(_ sender: NSGestureRecognizer?) {
        collectionView?.deselectAll(nil)
    }

My problem is, now I'm not able to click on textField or button, after selecting the cell. While I'm able to click on textField when the cell is not selected.

Any suggestions? How do I achieve selection & deselection on click?