How much can you believe scientific journals that discuss dizziness or hearing issues ?

Timothy C. Hain, M.D. • Last edited: July 26, 2020

This site contains references to a great many journals. It is important to realize that not all of these sources are as believable as others. In other words, the proportion of "fake news", is higher in certain types of publications than others.

Some of these journals are called "predatory journals". The idea here is that the goal of the journal is a commercial one rather than an attempt to improve the world's fund of knowledge. The individuals paying to have their writing published are being "preyed upon" by the organizers of the journal. According to the New Yorker, in an article written by "Alan Burdick" -- (a writer), " (https://www.newyorker.com/tech/elements/paging-dr-fraud-the-fake-publishers-that-are-ruining-science) A Finnish study found that, between 2010 and 2014, the number of articles published by predatory journals grew from fifty-three thousand to almost half a million." Mr. Burdick does not provide a reference to the "Finnish study", but anyway, one gets the general idea here.

The validity of a source of information depends on many factors. Lets think about a few of these.

1. What type of writing is this anyway ?

• Webpages, social media posts, community newsletters are "all over the place". Most are low credibility. Newspapers or similar entities that do not name a source for their writing, are not credible. Web pages written by professional writers who have no content expertise are particularly likely to be "fake news". For example, we have seen articles claiming to have a "cure for Meniere's disease", which sadly enough, simply does not exist. Articles that claim to help tinnitus using ear drops, magnets, or electrical devices that they sell, are another general category.
• Abstracts -- information "published" in abstracts is generally not reviewed by other experts in the field, and for this reason are usually not trustworthy. There may be something of great value in an abstract, but it is hard to be sure. These are not evidence, they are just ideas.
• Entities that call themselves "journals" are more credible. See commentary below about journal articles starting with item 3.
• Books vary
  • Popular books, often depend on wild assertions to sell more copies. Most popular books are very unreliable.
  • Thinly distributed scientific books often are the result of collecting together of writings from individuals who could not get their material published in "real journals".
  • Textbooks about "fuzzy" subjects such as social policy and intrinisically containing content that cannot be verified are generally just collections of opinions. An example of this are papers published in the psychology field, which according to studies, are generally unverifiable.
  • Textbooks, with a large audience, and containing contant that can be verified (such as mathematics, or science) are usually reliable.
• Is it "open access" ? Open access journals have mushroomed in the last decade. Open access journals are frequently more "sketchy" than conventional journals. They may have little or no credible review process, and they make money from fees paid by the foolish authors. The general term for these types of journals are "Predatory". Beall's list, at https://beallslist.weebly.com/, is a collection of "predatory journals and publishers". Somewhat more reputable open access journals include Hindawi and PLOS (see below).
• Does the title say that it is speculation ? For example, if an article says it is a "preliminary study", or a "hypothesis", they are telling you that the material should be considered with some caution.
• Is this a case report ? Case reports are often just noise.
• Is this paper written in another language. It is difficult to be sure of a reference quality if you can't read a paper because it is written, lets say, in cyrillic. Usually this comes up for papers written in either German or Japanese.

2. Expertise of writer --

• Do we know who the writer is ? Articles having no given author (such as in some news magazines like the "Economist" or newspapers where there is no author), should not be trusted without verification.
• Is the writer a scholar, or at least do they provide references to articles by scholars ? With no source, an article is opinion and should not be trusted.
• Does the writer have a "track record" of expertise in the field ? As an example, entertainment figures such as singers or popular comedians may have expertise on entertainment and use their commentary to increase their popularity, but they usually have little knowledge or expertise about government policy. Similarly, professional writers are paid to produce articles that sell magazines, books or newspapers, but as far as we know, their pay does not generally depend on their truthfulness.

3. Potential for bias -- Kaper et al (2018) reported that only 5 of 1500 otolaryngology journals had a "low risk of bias".

• Is the writer being paid to support a particular opinion ? Writers who take money from commercial sources, are less trustworthy. As an example of this, articles about Migraine headache are often written by authors who take money from the migraine drug industry.
• Is the writer so passionate about the subject, that objectivity may be thrown under the bus ? One can spot this type of passion by a dependence on anecdotes to sustain an argument, and the lack of references to reputable sources. These types of articles are easy to find in social media sites.
• Is the journal being paid to publish this article ? Many "open access" journals, have publishing charges, and very little review. Older type print journals make their money by selling their publications to libraries, generally at nose-bleed prices. Open access journals in general greatly outnumber print journals, and also have lower quality scores (see SNIP below). Open access journals are paid to "publish" articles, which means they are posted on websites. Their review process is generally weak. These journals are often just paid websites. There are a gigantic number of open access journals that are "predatory" (see above). It is important to point out that not ALL open-access journals are predatory. PLOS and Hindawi are open but not predatory. In general though, we find articles in these journals to be difficult to follow -- probably reflecting a weak editorial process.

4. Is there a reasonable review process ?

• A reasonable review process includes agreement by at least 3 experts in the subject area that the work is valid. The difficult part is finding unpaid experts to give reasonable reviews.
• The older "print" type journals generally implement this process through some sort of "peer" system where individuals agree to review other scholar's work, hoping that they will also get the same courtesy when it is their turn.
• Newer "open" access journals often have little formal review, or implement criteria that omit the validity criteria, and substitute a review of the statistics and methodology, omitting any expertise. So one might have an article published saying that the moon was made out of green cheese, in an open access journal. An example of this non-quality based review process is PLOS, an online format, that has a review process that doesn't require the article to be significant. PLOS's impact has been decreasing steadily over the last few years. Some old style journals are going "down market", and becoming open access, evidently in an effort to cut costs (e.g. Neurology). Oddly enough, some giant print publishers such as Kluwer and Elsevier, also own "fake" open access journals.
• Of course, journals may choose to accept reviewers of an article chosen from friends of the author, previous collaborators, arch enemies, or similarly biased individuals. This is difficult to see, as the names of reviewers are rarely revealed. It is apparent only to the insiders.

5. Are the journals indexed in Pubmed ? Do the journals have any impact on the field ? Do they have a high quality score (Scopus SNIP). Through his academic resources, Dr. Hain has available some tools that provide reasonable data concerning the quality of journals. Here are 3 factors which he thinks are important. The last one (the "SNIP") score is the one that best reflects quality.

• Acceptance of a journal for indexing in Pubmed is a positive feature for a journal. Almost all of the citations on this cite will be from Pubmed. There are some journals listed in Pubmed, that have no impact factor (see below). This means that perhaps they were not a good pick for Pubmed.
• Having a high "impact factor" is a positive feature. This means that the articles in that journal are cited in other journals. A high impact does not mean that the journal is "high quality". It instead means that the journal, as well as the topic of the journal, is popular. There are several sources for impact factors. The oldest source is the Thomsen spinoff -- "InCites Journal Citation Reports". This is a subset of roughly 15,000 journals out of the universe of roughly 25-40,000 journals (see https://scienceintelligence.wordpress.com/2012/01/23/how-many-science-journals/) . Most of these journals (96%) are online. The highest impact factor is 187 (as of 2018), for CA-A, a cancer journal for clinicians. A very good impact factor is about 15 (in the top 100). A low impact factor is 1 (for example, "Corrosion Reviews" has an impact factor of 1.085). Impact factors can go even lower, for example "Linguistics" has an impact factor of only 0.378. Perhaps not many people are interested in citing linguistics papers. This doesn't mean the journals are bad, but rather they are not very popular.
• Quality - The Scopus "SNIP" score incoporates the general idea that quality is proportional to the citations for this journal/citations for all similar journals. This corrects for the audience factor - -Thus journals that are general in nature end up with lower relative scores than their impact factor. We think the Scopus SNIP score is a pretty good one for judging how believable a journal is. We have arbitrarily grouped SNIP scores between none and 1 as "low", between 1-2 as "medium", and above 2 as "high".

The following incorporates some of Dr. Hain's opinions, that may not necessarily reflect the opinions of the academic community at large. However, we have included Impact scores and SNIP scores, which are independent of Dr. Hain's ideas about quality.

Lowest quality journals related to dizziness or hearing:

• International journal of *, an open access journal. Nearly all journals beginning with the phrase "international journal of ...." are low credibility. A gigantic number of these journals start their title this way. Open access Journals based in India (see https://www.oajse.com/rioaj_a-z_list.html) lists more than 500 as of 2018. An interesting commentary about "fake" journals is here https://www.thehindu.com/sci-tech/fake-openaccess-journals-flourish-in-india-science/article5265331.ece. There are many lists of "predatory open access journals". For example, Beall's list. These journals generally have low or perhaps no, impact ratings as well (see above). We have grouped these as having SNIP scores that are either nonexistent or less than 1.
• Examples relevant to Dizziness and Hearing. --
  • Abstracts, such as of the Barany society. These are not reviewed.
  • American Journal of Otolaryngology (SNIP 0.908). This is a common journal to find on our pages in this site, but it has a low SNIP score.
  • Audiology today (no impact or SNIP). We would hesitate to call this a journal. Maybe a newsletter would be a better term.
  • Biomed Research International (impact factor 2.476 in 2016; SNIP score is 0.875) BRI's impact factor has been gradually rising since 2014. It's quality score, however, is low.
• Canadian Family Physician (impact factor of only 1.9, although it is presumably of general interest; SNIP is 0.711, not so good either).
• Clinical Otolaryngology and Allied Sciences (coverage discontinued in SCOPUS that provides SNIP scores). Impact factor was 2.377 in 2019.
• Experimental Brain Research (impact: 1.9, SNIP: .823) While we like EBR, it doesn't seem to be quoted much.
• ENT journal (no impact factor; No SNIP either). We would not rely on this journal.
• European Archives of Oto-Rhino-Laryngology. Snip score of 0.938.
• Hearing Journal (SNIP: 0.012) This journal has a very low SNIP score -- we would not rely on this journal either.
• The International tinnitus journal (This journal has no impact factor listed by Thomsen, although it is indexed in Pubmed; SNIP is 0.355 -- very low).
• The Journal of International Advanced Otology. Although indexed in Pubmed, the SNIP score is only 0.587, meaning that it is not considered very reliable.

Medium quality journals related to dizziness or hearing:

• These journals have SNIPs between 1 and 2. It includes almost all of the commonly referenced journals on this site. A few of these are open access.
• Examples:
  • Acta Otorhinolaryngologica Italica (SNIP 1.182). This is open access.
  • Audiology and Neuro-otology (Impact factor 1.791; Snip 1.49). Although the impact factor is low, the SNIP is relatively high.
  • Auris Nasus Larynx (Impact 1.1: SNIP 1.035)
  • Ear and Hearing (SNIP 1.571)
  • Frontiers in Neurology (SNIP: 1.096). Like most of the other "Frontier's" journals, this journal just barely makes into the "medium quality" group. It has an unusual review process, and it is open access as well.
  • International journal of audiology (Impact 1.733; SNIP 1.245)
  • JAMA otolarygology (Impact factor 2.9, SNIP 1.2).
  • JARO (Journal of association for research in otolaryngology, Impact: 2.4; SNIP 1.29).
  • J. ORL (Impact about 1.5, Snip 1.118)
  • Laryngoscope (impact factor 2.471; SNIP 1.487).
  • Medicine (Impact factor 1.8; we couldn't find this one in the Scopus database because of it's unsearchable name). Medicine's impact factor has been dropping rapidly -- it is now less than PLOS-1.
  • Neurology (SNIP 1.5 in 2019). Neurology's impact factor and SNIP wentdown, after they adopted an open-source format, and dumped their long print version. We don't think people will read their articles as much as they are no longer printed.
  • Otology/Neurotology (Impact factor is 2.024 -- less than PLOS-1; SNIP is 1.32). Although not open access, there must be some major problems here as it has such a low SNIP.
  • PLOS-1 (impact factor 2.8 SNIP 1.128). Plos-1's impact factor has been steadily declining since 2009. Perhaps this is because it is an open access journal, and it's review process is unusual.

High quality journals that publish articles relevant to dizziness or hearing:

• These are generally well established, print journals, such as "Brain". These journals are generally believable due to a strong review process, and being around long enough to have a large reader base. These journals have SNIPS above 2. That being said, some of these journals still publish terrible papers.
• There are also a single up/coming open source journal, namely PLOS-medicine. We think it is very difficult for an open-source journal to make it into the high quality group.
• Examples:
  • Brain (Impact factor 10.292; SNIP 2.732). Brain's impact factor has been increasing gradually since 1997. Brain does not have a wide readership, and it is amazing that it's impact factor is so high.
  • Cephalalgia (Impact, 3.609; SNIP 2.129). This journal about headache just barely gets into the high-quality group.
  • Cochrane Database Syst Rev (SNIP 3.489). This very structured general review journal is often cited.
  • JAMA (Impact factor 44.4; SNIP 9.160). JAMA's impact factor has been rising. JAMA doesn't publish much about Dizziness or Hearing.
  • JAMA neurology (Impact factor 10, SNIP 2.379). Not much here about Dizziness or Hearing either. Notice that the JAMA otolaryngology is back in the medium group.
  • Journal of Vestibular Research (Snip score 2.176)- The review process is not very strong for this journal.
  • New England Journal of Medicine (Impact factor: 72; SNIP 14.683). Almost none of the articles on this site are from NEJM, because NEJM does not deal much with dizziness. However, there are a few good ones.
  • PLOS-medicine (Impact of 11.8, and a good SNIP -- 3.612. ) This is very respectable. Not much here about dizziness/hearing however.

References:

• Kaper NM, Swart KMA, Grolman W, Van Der Heijden GJMG. Quality of reporting and risk of bias in therapeutic otolaryngology publications. J Laryngol Otol. 2018 Jan;132(1):22-28. doi: 10.1017/S0022215117002407. Epub 2017 Dec 12.