AI-Natural Language Processing - Revision history

Paulreed: /* Unit 8: Natural Language Processing */

2025-01-04T18:07:48Z

Unit 8: Natural Language Processing

← Older revision		Revision as of 13:07, 4 January 2025
Line 1:		Line 1:
	= ~~Unit 8:~~ Natural Language Processing =		= Natural Language Processing =

	== Contents ==		== Contents ==

Paulreed at 18:07, 4 January 2025

2025-01-04T18:07:09Z

← Older revision		Revision as of 13:07, 4 January 2025
Line 20:		Line 20:
	* "Alice is a student of physics" → [("Alice", NNP), ("is", VBZ), ("a", DT), ("student", NN), ("of", IN), ("physics", NNS)].		* "Alice is a student of physics" → [("Alice", NNP), ("is", VBZ), ("a", DT), ("student", NN), ("of", IN), ("physics", NNS)].
	* Parsing: Determining all the syntactical relations of words in a sentence. Parse trees help represent these relations (refer to Figure 1).		* Parsing: Determining all the syntactical relations of words in a sentence. Parse trees help represent these relations (refer to Figure 1).
			[[file:natural_langauage_processing1.png]]
	==== Semantics ====		==== Semantics ====
	Semantics deals with the meaning of words, sentences, and texts. Common tasks include:		Semantics deals with the meaning of words, sentences, and texts. Common tasks include:
Line 28:		Line 28:
	* Machine Translation: Automatically translating text between languages.		* Machine Translation: Automatically translating text between languages.
	* Topic Classification: Detecting topics or subjects within texts.		* Topic Classification: Detecting topics or subjects within texts.
			[[file:natural_langauage_processing2.png]]
	==== Speech ====		==== Speech ====
	NLP tasks related to voice include:		NLP tasks related to voice include:

Paulreed at 04:53, 4 January 2025

2025-01-04T04:53:14Z

← Older revision		Revision as of 23:53, 3 January 2025
Line 1:		Line 1:


	~~==Example: A Dialogue System==~~
	A dialogue system (DS) is an NLP-based application capable of holding conversations with humans. An example is ChatGPT, which allows human-like text-based conversations. Components of a modular DS architecture include:
	* Acoustic Turn-Taking (ATT): Detecting when a user finishes speaking.
	* Automatic Speech Recognition (ASR): Converting speech to text.
	* Sentiment Analyzer (SA): Determining emotional tone.
	* Dialogue Manager (DM): Generating appropriate system responses.
	* Text-to-Speech Synthesizer (TTS): Rendering speech output.

	~~Two operating modes can function simultaneously:~~
	~~1. Long Loop: User → ATT → ASR → Sentiment Analysis → DM → TTS.~~
	~~2. Short Loop: User → ATT → DM → TTS.~~


	= Unit 8: Natural Language Processing =		= Unit 8: Natural Language Processing =

Paulreed at 04:52, 4 January 2025

2025-01-04T04:52:45Z

Show changes

Paulreed at 04:50, 4 January 2025

2025-01-04T04:50:33Z

@@ Line 104: / Line 104: @@
 ==Conclusion==
 Natural Language Processing is a transformative technology with applications spanning translation, sentiment analysis, and dialogue systems. By leveraging advances in machine learning and computational power, NLP continues to expand its real-world impact.
 [[category:Artificial Intelligence]]

Paulreed at 04:42, 4 January 2025

2025-01-04T04:42:58Z

← Older revision		Revision as of 23:42, 3 January 2025
Line 104:		Line 104:
	==Conclusion==		==Conclusion==
	Natural Language Processing is a transformative technology with applications spanning translation, sentiment analysis, and dialogue systems. By leveraging advances in machine learning and computational power, NLP continues to expand its real-world impact.		Natural Language Processing is a transformative technology with applications spanning translation, sentiment analysis, and dialogue systems. By leveraging advances in machine learning and computational power, NLP continues to expand its real-world impact.

			[[category:Artificial Intelligence]]

Paulreed at 04:42, 4 January 2025

2025-01-04T04:42:18Z

Paulreed at 04:41, 4 January 2025

2025-01-04T04:41:40Z

← Older revision		Revision as of 23:41, 3 January 2025
Line 75:		Line 75:
	==Glossary==		==Glossary==
	1. Natural Language Processing (NLP): The field of AI focused on enabling computers to understand and process human language, both spoken and written.		1. Natural Language Processing (NLP): The field of AI focused on enabling computers to understand and process human language, both spoken and written.

	2. Tokenization: Breaking text into smaller components, such as words or phrases.		2. Tokenization: Breaking text into smaller components, such as words or phrases.
	3. Stemming: Reducing words to their root form by removing prefixes and suffixes.		3. Stemming: Reducing words to their root form by removing prefixes and suffixes.

Paulreed at 04:40, 4 January 2025

2025-01-04T04:40:57Z

@@ Line 1: / Line 1: @@
 ==What is Natural Language Processing?==
 Despite its complexity, there exist common patterns that can be exploited by computers to automatically perform human-like activities related to verbal communication. This is the goal of natural language processing (NLP), a discipline that combines linguistics and computing science to emulate our capacity to manage language.
@@ Line 54: / Line 53: @@
 . **Short Loop**: User → ATT → DM → TTS.
-==Introduction to Machine Translation==
+==ChatGPT Interaction Example==
-Machine translation (MT) uses statistical methods trained on aligned parallel corpora to translate text. BLEU (Bilingual Evaluation Understudy) is a common metric to evaluate translation quality, with scores ranging from 0 (poor) to 1 (perfect match).
+### **1. Key Language Signals for Frustration**
-===Challenges===
+==Glossary==
-Multiple valid translations can exist for a sentence, e.g., "Alice is a physics student" and "Alice is a student of physics." BLEU scores reflect similarity to reference translations rather than correctness.
+. **Natural Language Processing (NLP):** The field of AI focused on enabling computers to understand and process human language, both spoken and written.
-==Summary==
+==Conclusion==
-Natural Language Processing (NLP) is a key AI field enabling computers to understand and process human language. Applications include language translation, sentiment analysis, chatbots, and more. Advances in computing, algorithms, and linguistic datasets continue to expand its capabilities and real-world relevance.
+Natural Language Processing is a transformative technology with applications spanning translation, sentiment analysis, and dialogue systems. By leveraging advances in machine learning and computational power, NLP continues to expand its real-world impact.

Paulreed at 04:34, 4 January 2025

2025-01-04T04:34:10Z

Show changes

← Older revision		Revision as of 23:42, 3 January 2025
Line 77:		Line 77:

	2. Tokenization: Breaking text into smaller components, such as words or phrases.		2. Tokenization: Breaking text into smaller components, such as words or phrases.

	3. Stemming: Reducing words to their root form by removing prefixes and suffixes.		3. Stemming: Reducing words to their root form by removing prefixes and suffixes.

	4. Lemmatization: Reducing words to their base form using vocabulary mapping.		4. Lemmatization: Reducing words to their base form using vocabulary mapping.

	5. Part of Speech (POS) Tagging: Identifying the grammatical role of words in a sentence.		5. Part of Speech (POS) Tagging: Identifying the grammatical role of words in a sentence.

	6. Named Entity Recognition (NER): Identifying and classifying entities in text, such as names, dates, and places.		6. Named Entity Recognition (NER): Identifying and classifying entities in text, such as names, dates, and places.

	7. Sentiment Analysis: Determining the sentiment or emotion behind a text.		7. Sentiment Analysis: Determining the sentiment or emotion behind a text.

	8. Machine Translation (MT): Translating text from one language to another.		8. Machine Translation (MT): Translating text from one language to another.

	9. Word Cloud: A visual representation of word frequency in a dataset.		9. Word Cloud: A visual representation of word frequency in a dataset.

	10. BLEU (Bilingual Evaluation Understudy): A metric used to evaluate the quality of machine translation.		10. BLEU (Bilingual Evaluation Understudy): A metric used to evaluate the quality of machine translation.

	11. Corpus/Corpora: A collection of linguistic data for training machine learning models.		11. Corpus/Corpora: A collection of linguistic data for training machine learning models.

	12. Deep Learning: A subset of machine learning using neural networks.		12. Deep Learning: A subset of machine learning using neural networks.

	13. Dialogue System (DS): NLP applications enabling human-computer conversations.		13. Dialogue System (DS): NLP applications enabling human-computer conversations.

	14. Exploratory Data Analysis (EDA): Analyzing and summarizing data visually or statistically.		14. Exploratory Data Analysis (EDA): Analyzing and summarizing data visually or statistically.

	==Conclusion==		==Conclusion==
	Natural Language Processing is a transformative technology with applications spanning translation, sentiment analysis, and dialogue systems. By leveraging advances in machine learning and computational power, NLP continues to expand its real-world impact.		Natural Language Processing is a transformative technology with applications spanning translation, sentiment analysis, and dialogue systems. By leveraging advances in machine learning and computational power, NLP continues to expand its real-world impact.