Natural Language Processing (NLP)

Natural Language Processing (NLP) refers to AI method of communicating with an intelligent system using a natural language such as English. Intelligent system could be a robot or backend software.

Processing of Natural Language is required when an intelligent system needs to perform as per the instructions provided or when the decision is needed from the interlligent system like clinical expert system.

The field of NLP involves making computers to perform useful tasks with the natural languages humans use.

Ex: Chat bot or automated chat engine which could be a replacement for customer care.

The input and output of an NLP system can be:

• Speech
• Written Text

Components of NLP

There are two components in NLP -

1. NLU
2. NLG

NLU (Natural Language Understanding)

NLU is the process of

• Mapping the given input in natural language into useful representations
• Analyzing different aspects of the language

NLG ( Natural Language Generation)

NLG is the process of producing meaningful phrases and sentences in the form of natural language using internal representation. The process involves

• Text Planning - It includes retrieving the relevant content from knowledge base.
• Sentence Planning - It includes choosing required words, forming meaningful phrases, setting tone of the sentence.
• Text Realization - It is mapping sentence plan into sentence structure.

Though, both NLU and NLG are equally important for NLP, NLU is considered to be more complex to implement.

Difficulties in NLU

NL has an extremely rich form and structure. It is very ambiguous. There are different levels of ambiguity - 

• Lexical ambiguity − It is at very primitive level such as word-level.
• Syntax Level ambiguity − A sentence can be parsed in different ways.
• Referential ambiguity − Referring to something using pronouns.
• One input can mean different meanings.
• Many inputs can mean the same thing.

Let's discuss about the steps involved in NLP.

I wanted to stop this article here as the next topic would be about the parsing algorithms as part of the Syntactic Analysis. Following are the list of methods for the analysis.

• Context-Free Grammar
• Top-Down Parser

I can detail about these methods in the next article that I publish.