Probabilistic models have thoroughly reshaped computational linguistics and continues to profoundly change other areas in the scientific study of language, ranging from psycholinguistics to syntax and phonology and even pragmatics and sociolinguistics. This change has included (a) qualitative improvements in our ability to analyze complex linguistic datasets and (b) new conceptualizations of language knowledge, acquisition, and use. For the most part, these changes have occurred in parallel, but the same theoretical toolkit underlies both advances. In this lecture I give a concise introduction to this toolkit, covering the fundamentals of contemporary probabilistic models in the study of language, with examples including phoneme identification, perceptual magnet effects, and simple hierarchical models. This lecture includes content of theoretical interest in its own right, as well as tools and concepts that are fundamental to the other three lectures of the series.
Human language comprehension poses some of the deepest scientific challenges in accounting for the capabilities of the human mind. In this lecture I describe several major advances we have recently made in this domain that have led to a state-of-the-art theory of language comprehension. First, I describe a detailed expectation-based theory of real-time language understanding, surpri
In constructing theories of linguistic meaning in context it has been productive to distinguish between strictly semantic content, or the “literal” meanings of atomic expressions (e.g., words) and the rules of meaning composition, and pragmatic enrichment, by which speakers and listeners can rely on general principles of cooperative communication to take understood communicative intent far beyond literal content. Major open questions remain, however, of how to formalize pragmatic inference and characterize its relationship with semantic composition. In this lecture I describe recent work within a Bayesian framework of interleaved semantic composition and pragmatic inference. First I show how two major principles of Levinson’s typology of conversational implicature fall out of our models: Q(uantity) implicature, in which utterance meaning is refined through exclusion of the meanings of alternative utterances; and I(nformativeness) implicature, in which utterance meaning is refined by strengthening to the prototypical case. Q and I are often in tension; I show that the Bayesian approach derives quantitative predictions regarding their relative strength in interpretation of a given utterance, and present evidence supporting these predictions from a large-scale experiment. I then describe more complex applications of the theory to key cases of compositionality, focusing on two of the most fundamental building blocks of semantic composition, the words “and” and “or”.