2 ms·
This looks like a fun visualization of various NLP techniques to parse sentences, but as far as I understand, only the tokenization is relevant to LLMs. Perhaps
by pamelafox 1y ago
This looks like a fun visualization of various NLP techniques to parse sentences, but as far as I understand, only the tokenization is relevant to LLMs. Perhaps it's just mis-titled?
I actually worked on a similar tree viewer as part of an NLP project back in 2005, in college, but that was for rule-based machine translation systems.
Chapter 4 in the final report:
https://www.researchgate.net/profile/Declan-Groves/publication/237232513_Final_Report_of_the_2005_Language_Engineering_Workshop_on_Statistical_Machine_Translation_by_Parsing/links/0c960528381020875f000000/Final-Report-of-the-2005-Language-Engineering-Workshop-on-Statistical-Machine-Translation-by-Parsing.pdf https://www.researchgate.net/profile/Declan-Groves/publicati...