I have suggested previously that intelligibility testing is the best, and really the only, way to scientifically attempt to divide languages from dialects.
Dividing based on intelligibility is a more scientific concept to the dialect/language concept than the sociolinguistic or political concepts currently used that have resulted in chaos surrounding the language/dialect question.
The resulting chaos has caused linguists to throw up their hands. Many now take the weird and soft science position that there is no way to tell a language from a dialect. This means that English and Mandarin may well be dialects of one language or California English and Massachusetts English are possibly separate languages. Make sense?
Focusing merely on intelligibility and nothing else turns the language/dialect question from its current senselessness towards a more solidly scientific basis.
There are other ways to distinguish language from dialect, but these just make things messier.
One way is structural divergence. Structural divergence is fairly well correlated with intelligibility, but not completely. Some divergent lects are quite intelligible, for instance, Turkish and Gaguaz. If you split structurally divergent yet intelligible languages, you run into the strange situation where speakers of two completely different languages can understand each other perfectly.
Dividing languages on sociological or, worse, political grounds is almost thoroughly anti-scientific. You run into odd cases, such as with Galician and Mirandese, where people of two intelligible lects wish to insist that they speak different languages in order to cynically acquire state funds and recognition, or because they dislike the other group, or maybe live in a different country than the other group.
You also run into odd cases like various unintelligible Mayan languages of Guatemala who insist that they all speak dialects of one language because this is the way they have always referred to these lects, or in order to preserve the unity of the language. In this case, you run into the bizarre case of speakers of two different dialects of a single language who can’t understand each other at all. If they can’t understand each other, who says they are speaking the same language?
Political reasons are much worse. States typically deny that minority languages spoken within their borders are languages in order to preserve the unity of the state and ward off fears of separatism and ethnic conflict. That linguists acquiesce to such blatantly fascist demands with a shrug of the shoulders is disturbing. States such as Sweden have recently engaged in gross manipulation of the ISO code process in order to deny language rights to minority tongues. It is disturbing that SIL caved in to the demands of the Swedish state so easily.
All in all, intelligibility is really the only way to go, and avoids all of these other anti-scientific minefields. A So if Ethnologue are mad splitters, so are the specialist authors themselves. The attacks on SIL and Ethnologue are poorly informed and typical of excessive emotionalism and fanaticism that has overtaken Linguistics recently and threatens to make it into yet another joke soft science social science. It is interesting that the same wild-eyed screamers who oppose lumping in genetic classification (opposition to say, Penutian and Altaic) are the same snarks who sneer that Ethnologue excessively splits.
The one thing that they have in common is the typical soft science dodge that we can’t prove much of anything about anything. We can’t prove what’s a language and what’s a dialect, so leave it alone. We can’t prove any more language families due to time depth or the weather or whatever, so let’s stop making any more families until we sit down, relax for 20-30 years over cups of coffee and get this stuff all sorted out.
The other main attack on Ethnologue is bizarre. It’s based on the fact that Ethnologue is run by Christian missionaries who translate the Bible into many languages. There are many atheists, usually very leftwing atheists, in the academic field of Linguistics, and it’s clear that their sneering contempt for SIL is based on the fact that they are unapologetically religious.
The bizarre insinuation is also made that since they Bible-translating missionaires, they could not possibly be competent linguists. How strange. Why can’t one be a Bible-translating missionary and a linguist at the same time? And other than that this is an obvious ad hominem attack, what’s so bad about being religious anyway? This sort of condemnation reminds one of the former USSR. Surely being a religious believer should not disqualify one from being a competent linguist!
Another strange and ultra-leftwing attack on SIL is that they somehow are CIA spies of some sort. I’m not competent to respond to that attack.
However, some Indian organizations in Latin America have protested the organization and tried, sometimes successfully, to get them banned from their nations, usually on grounds of trying to convert Indians to Christianity. Somehow the fact that SIL was banned from various backwards, dysfunctional banana republics for “trying to convert the Indians” is evidence that they can’t possibly do competent Linguistics.
How bizarre. Competent scholars are regularly banned from silly nations for sorts of strange political reasons that have nothing to do with scholarship. Once again, linguists appear to be siding with fascist-like states and opposing scholars.
Harold Hammarström, a man working in computational linguistics, which I would hope is about as scientific as our field gets, offers some hope to steer our field back to a more scientific path and reclaim some territory from the soft science mush-heads.