13.1 Fundamentals of computational linguistics

Core Concepts and Applications of Computational Linguistics

Computational linguistics brings together linguistics and computer science to build models that can process and understand human language. It provides the foundation for technologies like search engines, virtual assistants, and translation tools. Understanding these fundamentals also sheds light on how language itself works, since building a system that "understands" language forces you to be precise about things humans do intuitively.

Core Concepts of Computational Linguistics

At its heart, computational linguistics develops formal models of how language works and then implements those models as algorithms. A few core tasks define the field:

• Language modeling predicts the probability of word sequences. For example, given "The cat sat on the ___," a language model assigns higher probability to "mat" than to "algorithm." This underpins almost every other NLP task.
• Parsing analyzes sentence structure to determine grammatical relationships between words. It's how a system figures out that in "The dog chased the cat," dog is the subject and cat is the object.
• Machine translation automatically converts text from one language to another (e.g., Google Translate).
• Speech recognition converts spoken language into text, while text-to-speech synthesis does the reverse.

These core capabilities feed into practical applications you've probably used:

• Information retrieval locates relevant documents in large datasets. Every time you use a search engine, you're relying on this.
• Sentiment analysis determines the emotional tone of text, such as classifying product reviews as positive or negative.
• Question answering systems provide specific answers to queries rather than just returning a list of links.
• Chatbots and virtual assistants like Siri and Alexa simulate conversational interaction by combining several of these tasks at once.

Computational Linguistics vs. Natural Language Processing

These two terms get used interchangeably, but they have different emphases.

In practice, the boundary is blurry. Computational linguistics provides the theories and models; NLP implements them in systems people actually use. Both fields draw on the same toolkit:

• Machine learning algorithms that detect patterns in language data
• Statistical analysis of large text collections (corpora) to build probabilistic models
• Rule-based systems that encode linguistic knowledge as formal grammars or logical rules

Think of computational linguistics as the science and NLP as the engineering. They constantly feed into each other.

Challenges in Language Modeling

Human language is messy, and that messiness creates real problems for computational systems. The biggest challenges fall into a few categories.

Ambiguity is the single most persistent problem:

• Lexical ambiguity occurs when a word has multiple meanings. "Bank" could mean a financial institution or the edge of a river. A system needs context to pick the right one.
• Syntactic ambiguity arises when a sentence can be parsed in more than one way. "I saw the man with the telescope" could mean you used the telescope to see him, or he was the one holding it.
• Semantic ambiguity produces multiple interpretations of meaning. "The chicken is ready to eat" could mean the chicken is cooked and ready to be eaten, or the chicken (the animal) is hungry.

Context-dependence makes things harder still. Understanding many sentences requires world knowledge and common sense reasoning that's difficult to encode in a system. If someone says "It's cold in here," they might be requesting that you close a window, not just reporting the temperature.

Language variation and change also pose challenges:

• Dialects and regional differences complicate processing (e.g., "soda" vs. "pop" vs. "coke" for the same drink)
• Slang and informal language evolve quickly, making static models outdated
• New words (neologisms) appear constantly and require model updates

Multimodal communication adds another layer. Humans convey meaning through gestures, facial expressions, and tone of voice. Text-based systems lose all of that information, and even speech-based systems struggle to capture the full picture.

Finally, scalability is a practical concern. Processing large volumes of text or speech demands efficient algorithms, and real-time applications like live translation require careful optimization of computational resources.

Syntax, Semantics, and Pragmatics in Computational Linguistics

These three levels of linguistic analysis map directly onto how computational systems try to understand language. Each level tackles a different aspect of meaning.

Syntax deals with sentence structure. Computational systems use parsing techniques to figure out how words relate to each other grammatically.

• Constituency parsing breaks a sentence into nested phrases (noun phrases, verb phrases, etc.). For "The big dog chased the cat," it identifies "The big dog" as a noun phrase and "chased the cat" as a verb phrase.
• Dependency parsing maps direct relationships between individual words, such as identifying that "dog" is the subject of "chased."
• Grammar formalisms like context-free grammars provide the formal rules these parsers use.

Semantics deals with meaning at both the word and sentence level.

• Lexical semantics studies relationships between word meanings: synonyms, antonyms, hyponyms (e.g., "dog" is a hyponym of "animal").
• Compositional semantics analyzes how word meanings combine to produce sentence-level meaning.
• Semantic role labeling identifies who did what to whom in a sentence. In "The cat chased the mouse," cat fills the agent role and mouse fills the patient role.

Pragmatics goes beyond literal meaning to examine how context shapes interpretation.

• Discourse analysis looks at how sentences connect across a longer text to form coherent meaning.
• Inference and implicature capture the additional meaning listeners derive from context. If someone asks "Can you pass the salt?" the literal meaning is a question about ability, but the pragmatic meaning is a request.
• Speech acts (also called illocutionary acts) analyze the goal behind an utterance: is the speaker making a promise, issuing a command, or asking a question?

These three levels don't operate in isolation. The syntax-semantics interface explores how structural choices affect meaning (e.g., passive vs. active voice changes emphasis). Semantic parsing maps natural language onto formal meaning representations that a computer can reason about. And pragmatic interpretation layers context on top of both syntactic and semantic analysis to arrive at what a speaker actually means.