Is PencilKit Copy to Text available to developers?

Question

Created Jun ’20

Replies 2

Boosts 0

Views 1.5k

Participants 2

I have been playing with samples, but I found the text recognition only works with Apple's note app. Using PencilKit sample will not trigger the recognition.

Is it a private API?

If it's not available to developers, is there an approach that could achieve similar hand writing recognition for developers?

Boost

Answer 1

SeaOverload OP

Jun ’20

I tested a bit more. I realized the handwriting seems like relying on a on-device ML model. The offline recognition result is great but not perfect yet. If that's the case, are we encouraged to use a third part online recognition engine by providing them with the PKStrokes?

I thought about this approach last year, but personally still prefer to use a offline model if possible to keep user's note on device. I tested the Vision's text recognition model, but that one does not work well with handwriting text. I'm wondering if Apple's offline text recognition model will be built into the framework in the future?

0

Answer 2

Frameworks Engineer OP

Apple

Jun ’20

Copy as Text is only available in Notes in iOS 14.0. You could use the path from PKStrokes and train your own CoreML model. You can also check out: https://developer.apple.com/videos/play/wwdc2019/420/

0