I've just started working with the Natural Language framework, and with NLTagger. I believe that NLTagger will only give reliable results when it has the context of an entire sentence as input; and perhaps it requires even more context than that. Hopefully some experts will be able to tell us about how the model works.
I'm having my own troubles with NLTagger, which I'm discussing in a separate topic.
A question, to quench my curiosity.
What do you get with
in the weeds
we are in the weeds.
Aha. Both "in the weeds" and "we are in the weeds" get lemmatized to "weed". Much better, thanks.
Worse, my code was looking at the wrong index, so "IN THE WEEDS" was lemmatizing “IN”. not “WEEDS”. And “Indiana” is not the correct lemmatization here, but it's not insane.