Is PencilKit Copy to Text available to developers?

I have been playing with samples, but I found the text recognition only works with Apple's note app. Using PencilKit sample will not trigger the recognition.

Is it a private API?

If it's not available to developers, is there an approach that could achieve similar hand writing recognition for developers?



Up vote post of SeaOverload Down vote post of SeaOverload
1.3k views
Posted by
SeaOverload
Reply
Add a Comment

Replies

I tested a bit more. I realized the handwriting seems like relying on a on-device ML model. The offline recognition result is great but not perfect yet. If that's the case, are we encouraged to use a third part online recognition engine by providing them with the PKStrokes?

I thought about this approach last year, but personally still prefer to use a offline model if possible to keep user's note on device. I tested the Vision's text recognition model, but that one does not work well with handwriting text. I'm wondering if Apple's offline text recognition model will be built into the framework in the future?
Posted by
SeaOverload
Up vote reply of SeaOverload Down vote reply of SeaOverload
Add a Comment
Copy as Text is only available in Notes in iOS 14.0. You could use the path from PKStrokes and train your own CoreML model. You can also check out: https://developer.apple.com/videos/play/wwdc2019/420/

Posted by
Frameworks Engineer
Up vote reply of Frameworks Engineer Down vote reply of Frameworks Engineer
Add a Comment