emulate mouse click

I need to emulate a left mouse click on my Mac. Before I upgraded to Big Sur 11.5, the following code worked just fine - but now it no longer does. What happened, and what's the remedy? In a related question: how does one emulate a double-click?

        let position = NSPoint(100,75)         CGDisplayMoveCursorToPoint(0, position)         let source = CGEventSource.init(stateID: .hidSystemState)         let eventDown = CGEvent(mouseEventSource: source, mouseType: .leftMouseDown, mouseCursorPosition: position , mouseButton: .left)         let eventUp = CGEvent(mouseEventSource: source, mouseType: .leftMouseUp, mouseCursorPosition: position , mouseButton: .left)         eventDown?.post(tap: .cghidEventTap)         usleep(50_000)         eventUp?.post(tap: .cghidEventTap)

Answered by Claude31 in 682892022

Did you try to send mouseDown event twice, with no delay ?

This thread starts to be a bit too long. You should close it (mark one of my answers that suits best) and open a new one for double click if this doesn't work.

  • Indeed, the following works perfectly on MacOS 10.15.7.
    func switchToAppAndClick(withWindow windowNumber: Int32) {
        let options = CGWindowListOption(arrayLiteral: CGWindowListOption.excludeDesktopElements, CGWindowListOption.optionOnScreenOnly)
     let windowListInfo = CGWindowListCopyWindowInfo(options, CGWindowID(0))
        guard let infoList = windowListInfo as NSArray? as? [[String: AnyObject]] else { return }
        if let window = infoList.first(where: { ($0["kCGWindowNumber"] as? Int32) == windowNumber}), let pid = window["kCGWindowOwnerPID"] as? Int32 {
            let app = NSRunningApplication(processIdentifier: pid)
            app?.activate(options: .activateIgnoringOtherApps)
            let inWindowClickpoint = CGPoint(x: 100  , y:200) // a clickable point in the targeted window coords
            print("creating eventMouseDown and Up events")
            let source = CGEventSource.init(stateID: .hidSystemState)
            let position = CGPoint(x: 75, y: 100)
            let eventDown = CGEvent(mouseEventSource: source, mouseType: .leftMouseDown, mouseCursorPosition: position , mouseButton: .left)
            let eventUp = CGEvent(mouseEventSource: source, mouseType: .leftMouseUp, mouseCursorPosition: position , mouseButton: .left)
            print("posting \(eventUp)")
            eventDown?.post(tap: .cghidEventTap)
            Thread.sleep(forTimeInterval: 0.1)
            print("posting \(eventDown)")
            eventUp?.post(tap: .cghidEventTap)
        }
    }

    @IBAction func tryToActivateWindow(_ sender: NSButton) {
        print("begin -----------")
        print(self.view.window?.windowNumber)

        let mousePosition = NSPoint(x: 205,y: 200) // This corresponds to a visible window that's not part of this App
        print("Mouse coordinates (\(mousePosition.x),\(mousePosition.y))")
        let screenHeight = NSScreen.main!.frame.height
        print("Screenheight: \(screenHeight)")
        var screenPosition: NSPoint = mousePosition
        screenPosition.y = screenHeight - mousePosition.y
        print("Screen coordinates (\(screenPosition.x),\(screenPosition.y))")
        let numberOfHitWindow = NSWindow.windowNumber(at: screenPosition, belowWindowWithWindowNumber: 0)
        print("Window hit: \(numberOfHitWindow)")

        CGDisplayMoveCursorToPoint(0, mousePosition) // Just for visual reference

        switchToAppAndClick(withWindow: Int32(numberOfHitWindow))
    }
  • So, I tried on BigSur 11.5.

I was asked to authorise accessibility control. What I did. But then it does not work.

  • Hopefully, I found this:

https://stackoverflow.com/questions/65073294/is-it-possible-to-simulate-click-using-cgevent-for-ios-app-on-mac-with-m1

Which explains:

After the last Big Sur update this code works, but you need to focus window first time.

if(firstTime) testClick()
testClick()
  • So I adapted the code. No need to test for firstTime. And now, IT WORKS ! Strange, but it works:
    func testClick() {
        
        let source = CGEventSource.init(stateID: .hidSystemState)
        let position = CGPoint(x: 75, y: 100)
        let eventDown = CGEvent(mouseEventSource: source, mouseType: .leftMouseDown, mouseCursorPosition: position , mouseButton: .left)
        let eventUp = CGEvent(mouseEventSource: source, mouseType: .leftMouseUp, mouseCursorPosition: position , mouseButton: .left)
        print("posting \(eventUp)")
        eventDown?.post(tap: .cghidEventTap)
        Thread.sleep(forTimeInterval: 0.1)
        print("posting \(eventDown)")
        eventUp?.post(tap: .cghidEventTap)
    }
    
    func switchToAppAndClick(withWindow windowNumber: Int32) {
        let options = CGWindowListOption(arrayLiteral: CGWindowListOption.excludeDesktopElements, CGWindowListOption.optionOnScreenOnly)
     let windowListInfo = CGWindowListCopyWindowInfo(options, CGWindowID(0))
        guard let infoList = windowListInfo as NSArray? as? [[String: AnyObject]] else { return }
        if let window = infoList.first(where: { ($0["kCGWindowNumber"] as? Int32) == windowNumber}), let pid = window["kCGWindowOwnerPID"] as? Int32 {
            let app = NSRunningApplication(processIdentifier: pid)
            app?.activate(options: .activateIgnoringOtherApps)
            let inWindowClickpoint = CGPoint(x: 100  , y:200) // a clickable point in the targeted window coords
            print("creating eventMouseDown and Up events")
            testClick()
            testClick()     // <----- CALL IT TWICE !!!!
        }
    }

    @IBAction func tryToActivateWindow(_ sender: NSButton) {
        print("begin -----------")
        print(self.view.window?.windowNumber)

        let mousePosition = NSPoint(x: 205,y: 200) // This corresponds to a visible window that's not part of this App
        print("Mouse coordinates (\(mousePosition.x),\(mousePosition.y))")
        let screenHeight = NSScreen.main!.frame.height
        print("Screenheight: \(screenHeight)")
        var screenPosition: NSPoint = mousePosition
        screenPosition.y = screenHeight - mousePosition.y
        print("Screen coordinates (\(screenPosition.x),\(screenPosition.y))")
        let numberOfHitWindow = NSWindow.windowNumber(at: screenPosition, belowWindowWithWindowNumber: 0)
        print("Window hit: \(numberOfHitWindow)")

        CGDisplayMoveCursorToPoint(0, mousePosition) // Just for visual reference

        switchToAppAndClick(withWindow: Int32(numberOfHitWindow))
    }

That works!

That said, I experimented a bit more, and it turns out the problem may be elsewhere than where we were looking:

Every time I make changes to the code and recompile, I have to go to Preferences-Security and Privacy-Accessibility and de-check and re-check my App's permissions there. If I don't, the code only activates the window but doesn't activate any controls on there (note that I used a DispatchQueue call for safety, as these functions may be called by other threads than the main UI)

With that, I tried my original code for left- and right click, and it works fine once I de-check and re-check the App's permissions in Preferences after each compile:

func leftClick() {
    DispatchQueue.main.async {
        let source = CGEventSource.init(stateID: .hidSystemState)
        let position = self.correctedCursorPosition // use your own CGPoint location here
        let eventDown = CGEvent(mouseEventSource: source, mouseType: .leftMouseDown, mouseCursorPosition: position , mouseButton: .left)
        let eventUp = CGEvent(mouseEventSource: source, mouseType: .leftMouseUp, mouseCursorPosition: position , mouseButton: .left)
        eventDown?.post(tap: .cghidEventTap)
        //            usleep(50_000) // there's no need to sleep, it turns out
        eventUp?.post(tap: .cghidEventTap)
    }
}

func rightClick() {
    DispatchQueue.main.async {
        let source = CGEventSource.init(stateID: .hidSystemState)
        let position = self.correctedCursorPosition // use your own CGPoint location here
        let eventDown = CGEvent(mouseEventSource: source, mouseType: .rightMouseDown, mouseCursorPosition: position , mouseButton: .right)
        let eventUp = CGEvent(mouseEventSource: source, mouseType: .rightMouseUp, mouseCursorPosition: position , mouseButton: .right)
        eventDown?.post(tap: .cghidEventTap)
        //            usleep(50_000)   // there's no need to sleep, it turns out
        eventUp?.post(tap: .cghidEventTap)
    }
}

By the way, using your code, one call to testClick works fine, too. In fact, two calls works too for activating controls on the targeted window (I guess one click or two doesn't matter for most controls), but using two testClick calls on e.g. the Apple icon in TopLeft produces two clicks, so that you're activating and immediately de-activating the dropdown. Fortunately, one call to testClick works fine (at least for me) both for controls in other windows as for the Apple dropdown etc.

That leaves me with the double-click as still being an issue, unfortunately. I tried using your original code, where I can change the clickCount (tried both 1 and 2), instead of let eventDown = CGEvent(mouseEventSource: source, mouseType: .leftMouseDown, mouseCursorPosition: position , mouseButton: .left) , etc. No luck, though...

let newMouseDownEvent = NSEvent.mouseEvent(
    with: .leftMouseDown,
    location: position, //NSPoint(x: 205,y: 200), // this should be in the hitWindow
    modifierFlags: NSEvent.ModifierFlags.control,
    timestamp: ProcessInfo.processInfo.systemUptime,
    windowNumber: numberOfHitWindow,
    context: nil,
    eventNumber: 0, //don't know what else to use
    clickCount: 1,
    pressure: 1)
self.perform(#selector(NSResponder.self.mouseDown(with:)), with: newMouseDownEvent)

Thread.sleep(forTimeInterval: 0.1)

let newMouseUpEvent = NSEvent.mouseEvent(
    with: .leftMouseDown,
    location: position, //NSPoint(x: 205,y: 200), // this should be in the hitWindow
    modifierFlags: NSEvent.ModifierFlags.control,
    timestamp: ProcessInfo.processInfo.systemUptime,
    windowNumber: numberOfHitWindow,
    context: nil,
    eventNumber: 1, //don't know what else to use
    clickCount: 1,
    pressure: 1)
self.perform(#selector(NSResponder.self.mouseUp(with:)), with: newMouseUpEvent)
Accepted Answer

Did you try to send mouseDown event twice, with no delay ?

This thread starts to be a bit too long. You should close it (mark one of my answers that suits best) and open a new one for double click if this doesn't work.

Great work! Highly appreciated, thanks again! Learned a lot - and to think that one little checkbox in Preferences was the culprit!

emulate mouse click
 
 
Q