Terminal nodes are the endpoints of a tree diagram used in syntactic structure representation, where they denote the actual words or morphemes of a language. These nodes are crucial in understanding how larger linguistic units, such as phrases and sentences, are constructed, providing insight into the relationships between different parts of speech and their grammatical roles.