I'm using Numbers to build a spreadsheet that I'm exporting as a CSV. I then import this file into Create ML to train a word tagger model. Everything has been working fine for all the models I've trained so far, but now I'm coming across a use case that has been breaking the import process: commas within the training data. This is a case that none of Apple's examples show.
My project takes Navajo text that has been tokenized by syllables and labels the parts-of-speech.
Case that works...
Raw text:
Naaltsoos yídéeshtah.
Tokens column:
Naal,tsoos, ,yí,déesh,tah,.
Labels column:
NObj,NObj,Space,Verb,Verb,VStem,Punct
Case that breaks...
Raw text:
óola, béésh łigaii, tłʼoh naadą́ą́ʼ, wáin, akʼah, dóó á,shįįh
Tokens column with tokenized text (commas quoted):
óo,la,",", ,béésh, ,łi,gaii,",", ,tłʼoh, ,naa,dą́ą́ʼ,",", ,wáin,",", ,a,kʼah,",", ,dóó, ,á,shįįh
(Create ML reports mismatched columns)
Tokens column with tokenized text (commas escaped):
óo,la,\,, ,béésh, ,łi,gaii,\,, ,tłʼoh, ,naa,dą́ą́ʼ,\,, ,wáin,\,, ,a,kʼah,\,, ,dóó, ,á,shįįh
(Create ML reports mismatched columns)
Tokens column with tokenized text (commas escape-quoted):
óo,la,\",\", ,béésh, ,łi,gaii,\",\", ,tłʼoh, ,naa,dą́ą́ʼ,\",\", ,wáin,\",\", ,a,kʼah,\",\", ,dóó, ,á,shįįh
(record not detected by Create ML)
Tokens column with tokenized text (commas escape-quoted):
óo,la,"","", ,béésh, ,łi,gaii,"","", ,tłʼoh, ,naa,dą́ą́ʼ,"","", ,wáin,"","", ,a,kʼah,"","", ,dóó, ,á,shįįh
(Create ML reports mismatched columns)
Labels column:
NSub,NSub,Punct,Space,NSub,Space,NSub,NSub,Punct,Space,NSub,Space,NSub,NSub,Punct,Space,NSub,Punct,Space,NSub,NSub,Punct,Space,Conj,Space,NSub,NSub
Sample From Spreadsheet
Solution Needed
It's simple enough to escape commas within CSV files, but the format needed by Create ML essentially combines entire CSV records into single columns, so I'm ending up needing a CSV record that contains a mixture of commas to use for parsing and ones to use as character literals. That's where this gets complicated.
For this particular use case (which seems like it would frequently arise when training a word tagger model), how should I properly escape a comma literal?
Natural Language
RSS for tagAnalyze natural language text and deduce its language-specific metadata using Natural Language.
Posts under Natural Language tag
11 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
addition of Central Kurdish language support for Text-to-Speech (TTS) and VoiceOver functionality on Apple products. Our TTS model boasts an impressive 99.9% accuracy, making it a highly reliable tool for this purpose.
This initiative would bring meaningful benefits to over 10,000 visually impaired and more than 40,000 illiterate individuals in the Kurdistan Region of Iraq, empowering them to access digital information, navigate devices, and perform tasks more independently.
The integration of Central Kurdish VoiceOver support would make a significant difference in improving accessibility and quality of life for these individuals, promoting inclusivity and digital literacy in the region.
what am I not understanding here.
in short the view loads text from the jsons descriptions and then should filter out the words. and return and display a list of most used words, debugging shows words being identified by the code but does not filter them out
private func loadWordCounts() {
DispatchQueue.global(qos: .background).async {
let fileManager = FileManager.default
guard let documentsDirectory = try? fileManager.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: false) else { return }
let descriptions = loadDescriptions(fileManager: fileManager, documentsDirectory: documentsDirectory)
var counts = countWords(in: descriptions)
let tagsToRemove: Set<NLTag> = [
.verb,
.pronoun,
.determiner,
.particle,
.preposition,
.conjunction,
.interjection,
.classifier
]
for (word, _) in counts {
let tagger = NLTagger(tagSchemes: [.lexicalClass])
tagger.string = word
let (tag, _) = tagger.tag(at: word.startIndex, unit: .word, scheme: .lexicalClass)
if let unwrappedTag = tag, tagsToRemove.contains(unwrappedTag) {
counts[word] = 0
}
}
DispatchQueue.main.async {
self.wordCounts = counts
}
}
}
iOS17.4.1及以上,系统语言是中文,关闭系统NFC,app内调用CoreNFC弹窗显示英文(正常应该显示中文,在iOS17.4.1是正常显示中文的)
I'm trying to cast the error thrown by TranslationSession.translations(from:) as Translation.TranslationError. However, the app crashes at runtime whenever Translation.TranslationError is used in the project.
Environment:
iOS Version: 18.1 beta
Xcode Version: 16 beta
yld[14615]: Symbol not found: _$s11Translation0A5ErrorVMa
Referenced from: <3426152D-A738-30C1-8F06-47D2C6A1B75B> /private/var/containers/Bundle/Application/043A25BC-E53E-4B28-B71A-C21F77C0D76D/TranslationAPI.app/TranslationAPI.debug.dylib
Expected in: /System/Library/Frameworks/Translation.framework/Translation
I have a macOS application with a minimum version of macOS 12.0. I need to be able to get the current keyboard region designator.
Example: The user selects a input source of English Canadian. What I want as a result of this fact is en-CA locale identifier.
I get the current keyboard language with the following code
func keyboardLanguage() -> String?{
let keyboard = TISCopyCurrentKeyboardInputSource().takeRetainedValue()
let languagesPtr = TISGetInputSourceProperty(keyboard, kTISPropertyInputSourceLanguages)!
let languages = Unmanaged<AnyObject>.fromOpaque(languagesPtr).takeUnretainedValue() as? [String]
return languages?.first
}
This returns the language as en, but I don't see how I can get the region from Text Input Sources. I can get the input source id
let keyboard = TISCopyCurrentKeyboardInputSource().takeRetainedValue()
let idPtr = TISGetInputSourceProperty(keyboard, kTISPropertyInputSourceID)!
let id = Unmanaged<AnyObject>.fromOpaque(idPtr).takeUnretainedValue() as? String
print(String(describing: id))
This prints com.apple.keylayout.Canadian which points to the Canadian region but is not a region designator. I can possible parse this id and map it to a region designator but first I'm not sure if I will capture all of the regions and secondly what happens if the format of the id changes? If someone can point to the correct API to use it will be much appreciated.
I was wondering if there is a quick way to convert a model trained with the open source CRFSuite for use with NLTagger?
It seems like retraining should be possible but was wondering if automatic conversion was supported?
i'm trying to create an NLModel within a MessageFilterExtension handler.
The code works fine in the main app, but when I try to use it in the extension it fails to initialize. Just this doesn't even work and gets the error below.
Single line that fails.
SMS_Classifier is the class xcode generated for my model. This line works fine in the main app.
let mlModel = try SMS_Classifier(configuration: MLModelConfiguration()).model
Error
Unable to locate Asset for contextual word embedding model for local en.
MLModelAsset: load failed with error Error Domain=com.apple.CoreML Code=0 "initialization of text classifier model with model data failed" UserInfo={NSLocalizedDescription=initialization of text classifier model with model data failed}
Any ideas?
Is there a way to extract the list of words recognized by the Speech framework?
I'm trying to filter out words that won't appear in the transcription output, but to do that I'll need a list of words that can appear. SFSpeechLanguageModel.Configuration can be initialized with a vocabulary, but there doesn't seem to be a way to read it, and while there are ways to create custom vocabularies, I have yet to find a way to retrieve it.
I added the Natural Language tag in case the framework might contribute to a solution
I am working on an app that pulls data from weatherKit, including the conditionCode property, the content of which is displayed to the user. I wish to localize the data pulled from weatherKit but when pulling data from:
weatherkit.apple.com/api/v1/weather/de/{latitude}/{longitude}
The conditionCode and other strings is in english. Same is true if the language parameter is set to es, ja or something else.
Am I doing something wrong or is localization yet to be supported in weatherKit? I can't find any documentation on this.
I am using NLTagger to tag lexical classes of words, but it suddenly just stopped working. I boiled my code down to the most basic version, but it's never executing the closure of the enumerateTags() function. What do I have to change or what should I try?
for e in sentenceArray {
let cupcake = "I like you, have a cupcake"
tagger.string = cupcake
tagger.enumerateTags(in: cupcake.startIndex..<cupcake.endIndex, unit: .word, scheme: .nameTypeOrLexicalClass) { tag, range in
print("TAG")
return true
}