Diving into Natural Language Processing

The ultimate goal for me is to combine my cognitive linguistic background with the new knowledge in software development and dive into the world of NLP. With hundreds of NLP APIs and SoS readily available to be integrated into your apps reaching this goal becomes plausible. Here I will give a quick and rough introduction to the world of NLP.

Back in college when I was getting my minor in psychology, NLP was all a rage but it was referring to a completely different thing — Neuro-linguistic Programming. Now the NLP acronym is back in the spotlight but with a different meaning attached to it — Natural Language Processing.

In a nutshell, NLP helps computers understand human language. And human language is a weird concept. Not only you have syntax, grammar, phonetics which are somewhat straightforward with the digestible set of rules, you also have semantics… and this is a completely different animal. A neural network has to be really smart to understand all stylistic devices like metaphor, hyperbole, metonymy, anaphora with epiphora, synecdoche, etc.

NLP is a key component of artificial intelligence (AI) and relies on machine learning, a specific type of AI that analyzes and makes use of patterns in data to improve a program’s understanding of speech. We can take Scott Simms’ categorization of common uses of NLP as a base and build on it:

Common uses of NLP

1. Sentence segmentation (tokenization), stemming (create a word stem for each token), stop words removal, part-of-speech tagging. Hence, NLP is used to analyze parts of a sentence to better understand the grammatical construction of the sentence. Libraries: NLP Toolkit, NodeNatural.
2. Deep analytics. Involves the application of advanced data processing techniques in order to extract specific information from large or multi-source data sets. Deep analytics is particularly useful when dealing with precisely targeted or highly complex queries with unstructured and semi-structured data. Libraries: Watson Discovery
3. Machine translation. One human language is automatically translated into another human language. Tools: Google Translate, Microsoft Translator.
4. Text-to-speech and speech-to-text. Convert audio and voice into written text and text into natural-sounding audio. APIs: Watson Text-to-Speech, Watson Speech-to-Text.
5. Natural conversations. Build conversational interfaces into your applications (chatbots). APIs: Amazon Lex, Watson Conversation.
6. Automatic text summarization. Produce a readable summary from a large chunk of text. Document categorization.
7. Content classification. Understand the overall sentiment expressed in a block of text. APIs: Google Cloud Natural Language
8. Sentiment analysis. Understand the overall sentiment expressed in a block of text. APIs: Google Cloud Natural Language, Watson Personality Insights, Watson Tone Analyzer.
9. Semantic analysis. Contextual intelligence, audience profiling, and segmentation. APIs: Leiki.

In a word, NLP helps computers to help us to make sense of what we say. This is just a gist of all the fun things that NLP provides. Next step is building apps using the above-mentioned libraries, APIs, and services.

Let's build NLP things

My first take at combining NLP APIs in one App is a tool that I built as my graduation project at Flatiron School: Linguistic Analysis (linguistic-analysis.herokuapp.com).

It is a lexical, semantic, and text analytics tool for linguists, copywriters, and creative writers, utilizing NLP APIs. Built on React/Redux it utilizes 9 language processing APIs:

Full code is available on my GitHub, and you can see a video demo on YouTube.

// Learn. Love. Code.