Context-free grammars are formal systems used to define the syntax of programming languages and natural languages, consisting of a set of production rules that describe how symbols can be combined to generate strings. They are essential in computational linguistics as they provide a structured way to analyze and parse language, allowing for the creation of algorithms that can process and understand linguistic structures.