Like this:
* Two related varieties are normally considered varieties of the same language if speakers of each variety have inherent understanding of the other variety at a functional level (that is, can understand based on knowledge of their own variety without needing to learn the other variety).
* Where spoken intelligibility between varieties is marginal, the existence of a common literature or of a common ethnolinguistic identity with a central variety that both understand can be a strong indicator that they should nevertheless be considered varieties of the same language.
* Where there is enough intelligibility between varieties to enable communication, the existence of well-established distinct ethnolinguistic identities can be a strong indicator that they should nevertheless be considered to be different languages.
It’s all pretty confusing, but it gives you something to mull over in your spare time if you choose. I’ve been mulling over this stuff for months now.
The last refers to cases like the Croat, Serbian and Bosnian languages.
Marginal intelligibility in the second case refers to intelligibility of about 90%. For instance, Galician and Portuguese have about 85% intelligibility, but Galician has its own literature, so that’s a point in its favor.
The first is crucial. Inherent intelligibility means virgin ears. Everything else is called “bilingual learning” and doesn’t count. If you understand the other lect because you’ve been hearing it forever, that means you have learned a new lect via language learning. Inherent intelligibility means exposing people who have never heard the lect before to the new lect and then seeing how much they can get.
As you can see, it’s not exactly rocket science and there’s fudging everywhere. That’s what physics envy is all about. The social sciences are fuzzy by nature. Why? We deal with human nature, and that’s not so quantifiable right there, mysterious critters that we are and all. If that bothers you, put on a smock and head for the lab instead. Nobody’s forcing you to get a Sociology degree.
Stuff like how our bodies work (medicine) is a lot more hard science-ish, but that’s getting out of the human nature thing.