Theoretical Credibility Rankings
One Star () to Five Stars ()
Evaluating the credibility of one person's statements is difficult if not impossible, especially without knowing, for example, each person's background, training, affiliations, education, or experience. However, we feel that a guide to a person's theoretical credibility can be helpful, so we have built theoretical credibility ranking charts for each ProCon.org website to help differentiate the theoretical credibility of the various sources on our sites.
I. Our Rankings for Sources at Israeli-Palestinian ProCon.org:

Reports
Government statistics and reports.

Key Experts
We don't believe this issue has a Key Expert class. [Note: Key Experts definition varies by site]

Experts
Members of Congress, Ambassadors, Counsul Generals, heads of government, heads of major government organizations, members of legislative bodies, and Ph.D.'s with significant involvement in, or related to, the Palestinian - Israeli conflict. [Note: Experts definition varies by site]

Credible Publications
National and international mainstream publications which present unbiased reporting, such as media news (television, radio, internet) and non-profits whose reporting can generally be considered unbiased.

Organization/VIP/Other
Those individuals and organizations that do not fit into the other star categories.

II. Our Methodology:

Our theoretical credibility rankings are based upon the following premises:
  • The courts and many people equate a level of education and knowledge with a person's theoretical credibility.

  • Although ProCon.org doesn't have the resources to make complex evaluations of the credibility of each contributor to our website and such an evaluation would still contain a fair amount of subjectivity, we believe our theoretical ranking is more desirable than no ranking at all, and that it should be accurate at least 80% of the time.
We have therefore assigned a one through five star rating based on the theoretical credibility each of our sources should possess.

Additionally, we have customized the star categories to each of our sites' specific content because of their different subject matter.

III. Pros and Cons of Our Theoretical Credibility Ranking System:

Some people have praised our ranking system as a tool for differentiating the theoretical credibility of our sources. Others believe our star system should be eliminated because it is too subjective. The following chart sets out the main Pros and Cons about our theoretical credibility system:

Pro Theoretical Credibility System Con Theoretical Credibility System
Provides a quick guide to theoretical source quality Suggesting that one source is more theoretically credible than another reflects bias
Our star system is based on a person's theoretical education and knowledge - a concept used in the court system to try and distinguish reliable information The number of stars prejudges sources for readers rather than allowing readers to judge the sources' credibility for themselves
Stars add a simple and clean visual aid for a source's theoretical credibility Some 1 star individuals can be more credible than 3 or 4 starred sources. All organizations receive 1 or 2 stars regardless of their actual credibility
We estimate our star system is accurate roughly 80% of the time We estimate our star system is inaccurate roughly 20% of the time

IV. Some Flaws in Our Theoretical Credibility Ranking:

The following are examples of flaws or problems with our theoretical credibility rankings:
  1. Government Statistics - Some have questioned why we have chosen to give the credibility of government facts and statistics our highest theoretical rating of five stars.

    Our thinking is that government facts and statistics are generally reliable. However, what is less reliable, hence our lower rating, is when government personnel attempt to quote from such facts out of context, or worse when they misuse those facts on purpose or by accident.

    For example, we generally would give our highest rating--five stars, to a government report saying that there have been 52,850 killed in auto accidents in a given time period, but we would consider it less credible for a government employee to say in a speech, "Fifty-thousand people died last year in auto accidents." The government employee would probably receive one, three, or in some cases four stars, depending on the person's education and position.

  2. Star Ranking Challenges at Organizations - We rank most organizations as 1-star because they are often dynamic and subject to myriad influences, thereby making a ranking difficult if not impossible.

  3. Credibility Leap Between One-Star Organizations and Their Five-Star Reports - A government organization would be categorized as a one-star source, but a report issued by that organization would probably receive our five-star ranking.

  4. No Background Verifications - We do not verify the professional or educational background of our sources. For instance, if someone reports that they hold a Ph.D., M.D., or J.D. degree, then we presume that they hold that degree. We recognize that people may lie on their resumes, but we don't check resumes for accuracy.

  5. Each Site Has a Different Credibility Scale - Election officials are 3-star "Experts" on the Voting Machines website, but probably wouldn’t be on the Euthanasia website.

  6. Some Credibility Categories Do Not Exist on Other Sites - The Born Gay website lists people with Ph.D.s in Psychology as 4-star “Key Experts” while other sites may not even have a 4-star "Key Expert" category.

  7. Theoretical Credibility Can Change with Events - For example, leaders of Hamas were 1-star sources on the Israeli-Palestinian ProCon.org site until their party won the March 2006 Palestinian elections, and now those same people are considered 3-star sources as government officers.

  8. Determining What Is Unbiased Is Subjective - We classify "national and international mainstream publications" as 2-star sources that generally present "unbiased reporting." Many newspapers reach a national or international audience by publishing their articles on the internet. We've done our best to determine what is "mainstream" and generally "unbiased," however such distinctions are in themselves subjective.

  9. Doctors Get 4-Stars on Medical Marijuana Site Even if They Know Nothing About Medical Marijuana - On our medical marijuana site physicians receive a 4-star rating based only on their M.D. degree, with no consideration of their having had any experience with, or knowledge of, medical marijuana. A Ph.D. research scientist with extensive medical marijuana experience receives a 3-star rating.

  10. Many Sources on Sites Are 3-Stars and Lower - Our websites contain many 1-star or 3-star sources. For example, our ACLU and US-Iraq websites have 93% and 91% of the sources as 1- or 3-star sources (as of 05/18/07) respectively.

Please send us your criticisms or suggestions that would make our theoretical credibility ranking system better.