Evidence-Based Medicine Open Access
Copyright ©The Author(s) 2017. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Gastroenterol. Mar 7, 2017; 23(9): 1676-1696
Published online Mar 7, 2017. doi: 10.3748/wjg.v23.i9.1676
Inflammatory bowel disease: An evaluation of health information on the internet
Samy A Azer, Thekra I AlOlayan, Malak A AlGhamdi, Malak A AlSanea, Curriculum Development and Research Unit, Medical Education Department, College of Medicine, King Saud University, Riyadh 11461, Saudi Arabia
Author contributions: Azer SA participated in the study design and development and analysis and interpretation of data, construction of figures and tables and writing the manuscript; AlOlayan TI, AlGhamdi MA and AlSanea MA participated in the data collection, data analysis, interpretation of data, construction of tables, and review of the manuscript.
Supported by the College of Medicine Research Center, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia.
Institutional review board statement: The Institutional Review Boar, College of Medicine King Saud University, has approved the project and the approval number: F06/2014.
Conflict-of-interest statement: The authors declare that they have no conflict of interest.
Data sharing statement: No additional data are available.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Correspondence to: Dr. Samy A Azer, Professor, Curriculum Development and Research Unit, Medical Education Department, College of Medicine, King Saud University, PO Box 2925, Riyadh 11461, Saudi Arabia. azer2000@optusnet.com.au
Telephone: +966-11-4699178 Fax: +966-11-4699174
Received: October 28, 2016
Peer-review started: November 2, 2016
First decision: December 19, 2016
Revised: December 29, 2016
Accepted: February 16, 2017
Article in press: February 17, 2017
Published online: March 7, 2017

Abstract
AIM

To evaluate the quality and accuracy of websites written to the public on inflammatory bowel disease (IBD) (Crohn's disease and ulcerative colitis) and assess their readability level.

METHODS

Google™, Bing™, and Yahoo™ search engines were searched independently by three researchers in December 2014. Only English-language websites were selected on the basis of predetermined inclusion and exclusion criteria. Researchers independently evaluated the quality of each website by using the DISCERN and the HONcode instruments. The readability levels were calculated using two formulas; the Flesch-Kincaid Grade Level Index, and the Coleman-Liau Readability Index. The agreement between the evaluators was calculated using Cohen kappa coefficient.

RESULTS

Eighty-four websites were finally identified. Scores varied from a minimum DISCERN score of 18 to a maximum of 68 [mean ± SD, 42.2 ± 10.7; median = 41.5, interquartile range, interquartile range (IQR) = 15.8] and a minimum score of HONcode of 0.14 and a maximum of 0.95 (mean ± SD, 0.16 ± 0.19; median = 0.45, IQR = 0.29). Most of these websites were reviewed in 2014 and 2015 (n = 51). The creators of these websites were: universities and research centers (n = 25, 30%), foundations and associations (n = 15, 18%), commercial and pharmaceutical companies (n = 25, 30%), charities and volunteer work (n = 9, 10%), and non-university educational bodies (n = 10, 12%). The Flesch-Kincaid Grade Level readability score (mean ± SD) was 11.9 ± 2.4 and the Coleman-Liau Readability Index score was 12.6 ± 1.5. Significant correlation was found between the two readability scores (R2 = 0.509, P = 0.001). The overall agreement between evaluators measured by Cohen kappa coefficient was in the range of 0.804-0.876; rated as "Good".

CONCLUSION

The DISCERN and the HONcode scores of websites varied and the readability levels of most websites were above the public readability level. The study highlights the areas that need further improvement and development in patient education online materials about IBD.

Key Words: Inflammatory bowel disease, The internet, Patients' information, Evidence, Patients' education, online resources

Core tip: This is a comprehensive study analyzing the quality and accuracy of content and the readability level of websites in the English language on inflammatory bowel disease dedicated to the public. Two standardized instruments were used in assessing quality and accuracy and two methods were used in calculating readability level. The study showed variability in scores and the readability levels of most websites were above that for the public. Based on evidence, the study highlights the need for improving online patient education.



INTRODUCTION

Inflammatory bowel disease (IBD) refers to two chronic inflammatory disorders, Crohn's disease and ulcerative colitis. Both are life long, relapsing disorders of unknown etiology; possibly the result of interaction between genetic and environmental factors[1]. The diagnosis of these disorders is based on clinical features, endoscopy, and histological changes[2]. Crohn's disease may affect any part of the gastrointestinal tract but most commonly affects the distal ileum and proximal colon. The disease is characterized by inflammatory changes involving all the layers of the affected regions. In contrast, ulcerative colitis is characterized by continuous ulceration starting in the rectum and limited to the colonic mucosa[3]. IBD occurs worldwide with the highest incidence in developed countries mainly North America, United Kingdom and northern Europe. The incidence of ulcerative colitis in North America is approximately 19.5 per 100000 person years and 243 per 100000 person years in Europe while the incidence of Crohn's disease in North America is approximately 20.2 per 100000 person years and 12.7 per 100000 person years in Europe[4]. The aims of treatment are to induce remission in active disease and to maintain remission/prevent relapse. Therapeutic modalities include lifestyle modification, nutritional support, and medications. Surgery is reserved for the treatment of complications or when the medical therapy is ineffective. In addition to other complications, patients with IBD are at a higher risk of developing colorectal cancer. Therefore, patients have to undergo to regular checkup for early detection of the development of colon cancer[5].

With this information in mind, patients with IBD, as it is the case with other chronic diseases, usually seek information about the nature of the disease, its causes, investigations needed to diagnose the disease and therapeutic options. The advances in treatment modalities and options, and the increasing desire for patients to participate in decision-making about treatment choices necessitate the need for resources to support these decisions. Nowadays, patients have increasingly used the Internet as a source of health information because of its global accessibility, speed, and cost effectiveness[6]. Approximately 80% of the Internet users look for medical or health-related information through the Internet[7]. The topics most searched were information about specific disease or medical condition, treatment options, diet and nutrition, exercise and fitness and medications[7]. The increasing use of the Internet embraces a variety of aspects of topics searched, which gives the person an opportunity to investigate their questions from several resources. However, with the abundance of such information there is concern about the quality, accuracy, and readability level of the information available on the Internet about health care[8].

Therefore, the aims of this study were: to evaluate the quality, and accuracy of web-based information about IBD using two instruments, the DISCERN and the HONcode, as well as calculate the readability level by using two formulas, the Flesch-Kincaid Grade Level Index, and the Coleman-Liau Readability Index. The rationales for the study were to assess the educational usefulness of web-based information on IBD particularly their quality, accuracy and areas of deficiencies that need improvement. Also to assess whether these resources are easily read and understood by the public. Therefore our research questions are: (1) for the websites targeting the public and patients with IBD, what is the accuracy and the quality of these information resources? and (2) does the readability level of these online resources match with the recommended level for the public?

MATERIALS AND METHODS
Search design

In this study we assessed websites written for patients and the public on IBD by searching three search engines (Google™, Bing™ and Yahoo™), the selection of these three search engines was based on current statistical information that showed that these engines are the most searched by the public for health information[9]. The quality and accuracy of information provided on websites were assessed using two instruments: the DISCERN (http://www.discern.org.uk) and HONcode (http://www.hon.ch/honcode/) instrument. Details about these instruments and the justification for selecting them are discussed later. The readability of the websites was assessed using two methods: the Flesch-Kincaid Grade Readability Level and the Coleman-Liau Readability Index. After piloting the work and ensuring satisfactory use of these instruments by researchers, the work was carried out to assess the quality of websites. The Institutional Review Board, College of Medicine King Saud University, has approved the project and the approval number: F06/2014.

Searching the internet

Using the following key words: "inflammatory bowel disease", "Crohn's disease", "ulcerative colitis", "inflammatory bowel disease patient information", "Crohn's disease patient information", and "ulcerative colitis patient information", three search engines (Google™, Bing™ and Yahoo™) were searched. Researchers independently from 1 to 20 December 2014 conducted the search. Information for each website was recorded; these included: website title, website URL, name of creator, year of publication on the Internet, last date updated, and the objectives of the website. This information about each website was collected using the following online meter: http://whois.domaintools.com/. The data collected were evaluated on the bases of the inclusion and the exclusion criteria.

Inclusion and exclusion criteria

The inclusion criteria included: (1) websites covering public education about Crohn's disease, ulcerative colitis or IBD; and (2) websites focusing on patient education and in the English language. The excision criteria comprised: (1) websites addressing doctors or health professionals; (2) lectures, and advertisement on IBD; (3) websites in languages other than English; and (4) presentations at conferences.

Assessing accuracy and quality of information

Two instruments were used in assessing the quality and accuracy of information provided, namely the DISCERN instrument and the HONcode instrument. These two instruments have been widely used in the literature in assessing information on the Internet particularly health related issues and patients' education online resources[8,10-12]. More details about these two instruments can be summarized as follows:

DISCERN instrument: This instrument is a standardized set of criteria for judging the quality of health information and is written for the public to assess treatment options[8,10-12]. The DISCERN instrument was created by the University of Oxford, and the project was funded by the British Library and the National Health Service (NHS) Research & Development Programme[13] The instrument consists of 15 questions plus an overall quality rating question. The questions can be grouped under the three key topics as follows: Questions 1 to 8 addressing reliability, Questions 9 to 15 addressing specific detail about the information provided and treatment choices, and Q16 covering the overall quality rating[14]. The instrument has been used to assess healthcare-related websites and online resources. For example, the quality of patients' information on surgical treatment of haemorrhoids[12], and colorectal cancer information[11].

HONcode instrument: The Health on the Net (HON) Foundation, a non-profit, and non-government organization created this instrument in 1995. The instrument focuses on key questions on the provision of health information available on the Net, and provides a code of conduct addressing eight principles: (1) authoritative (indicates the qualifications of the authors); (2) complementarity (Information should support, not replace, the doctor-patient relationship); (3) privacy (Respect the privacy and confidentiality of personal data submitted to the site by the visitor); (4) attribution [cite the source(s) of published information, date medical and health pages]; (5) justifiability (site must back up claims relating to benefits and performance); (6) transparency (Accessible presentation, accurate email contact); (7) financial disclosure (Identify funding sources); and (8) advertising policy (Clearly distinguish advertising from editorial content)[15-17]. To earn HONcode certification, a website must conform to the eight principles of the HONcode of Conduct. An HONcode expert then assesses the candidate website using precise guidelines for each principle. Recently, the HON Foundation has developed an automated system to assist in detecting a website's HONcode conformity. Therefore, the automated assistance in conducting HONcode reviews can expedite the current time-consuming tasks of HONcode certification and ongoing surveillance. A recent study showed that there is concordance between automated and expert manual compliance detection for the criteria[18]. In this research we have used the electronic system available at: http://www.readabilityformulas.com/free-readability-formula-tests.php.

The HONcode has been widely used in the literature in assessing health-related websites[19]. The two instruments, the DISCERN and HONcode, do not exactly cover the same issues/topics, although there are some overlaps. Therefore, using these two instruments with these differences in mind could provide a better evaluation of the websites.

Piloting the study

The aims of piloting the study were: (1) to introduce the two instruments to the researchers and orient them on how to use each instrument in assessing the websites; and (2) identify difficulties facing the researchers on applying the two instruments and the sources of disagreements among them. Such exercise prior to the implementation of the two instruments was vital for ensuring optimal use of the instruments and maximizing the degree of agreement among evaluators when they apply these two instruments in the actual research. The piloting part was conducted as follows: (1) approximately 10 websites other than those identified for the research study were evaluated independently by three researchers using the two instruments; (2) the results of their evaluation were discussed with the aim to identify sources for difficulties/disagreements; (3) the identified differences were resolved after discussing them reaching to a solution; and (4) the same process was repeated on another 10 websites until the agreement between the researchers reached to an optimal level[20].

Conduction of the study

Along with the same approach described under piloting the study, the researchers evaluated the websites identified by applying the two instruments on each website and giving a score. The process was conducted by each researcher independently first by applying the DISCERN instrument then the HONcode instrument. The results of the assessment were placed on an Excel sheet for each researcher. The degree of agreement was measured using Cohen kappa coefficient[21].

Calculating website readability

The aims of calculating readability level of websites was to assess if they were written at the readability level of the general public and patients; should not exceed the 6th grade readability level[22,23]. Two methods were used to calculate readability: The Flesch-Kincaid Grade Level Index[24], and the Coleman-Liau Readability Index[25,26]. It was decided to use these two methods rather than one method so that we can compare the readability scores and examine if there were an agreement between the two methods, and hence strengthening the outcomes of our readability assessment and our conclusions. The two methods can be summarized as follows:

Flesch-Kincaid grade level index: This test helps in indicating how difficult a reading passage in the English language to understand. The test was developed by Rudolf Flesch and finalized by J Peter Kincaid for use by the United States Navy, hence the name of the test[24]. The test is based on the word length and the sentence length and is based on the following formula:

0.39 × [(total words)/(total sentences)] + 11.8 × [(total syllables)/(total words)] - 15.59

This method has been widely used in assessing the readability of websites and educational material[27].

Coleman-Liau readability index: This test differs from the above method in relying on characters instead of syllables per word. It enables the users however to grade the readability level. It has been widely used in assessing the readability of educational material[27].

We used a free online calculator (http://www.readabilityformulas.com) to calculate the readability level using the two readability methods. As per instructions provided by the website, the top, middle and bottom 150-200 words of each website were placed in the calculator and then the text readability was checked by calculating the number of sentences, words, syllables, and characters in the sample. A sufficient sample size of four to five full sentences; approximately 200-500 words in total were used. The scores recorded for each website were placed on an Excel sheet and reviewed by two other researchers before conducting final analysis for the means and standard deviations.

Grouping the websites under five categories

Assessment of the identified websites revealed variability in their creators. These can be grouped into 5 categories: (1) university, affiliated hospitals, and research centres; (2) foundations and associations; (3) commercial and pharmaceutical companies; (4) charities and volunteer works; and (5) non-university educational bodies such as colleges, academies, and councils. The grouping of websites under these five categories was carried out by researchers independently and was reviewed in a meeting for any disagreements.

Statistical analysis

The collected data were placed on an Excel Sheet (Microsoft Excel for Mac 2011, Microsoft Corporation, Redmond, WA, United States). All analysis was conducted by using SPSS software (SPSS Statistics version 22 for Mac, IBM Corporation, Armonk, NY, United States). For the data collected from measuring website accuracy, and the readability scores, the means, standard deviations, the median and interquartile range (IQR) were calculated. Pearson correlation studies and P-values for significance were calculated to examine if there were correlations between the scores obtained from the two readability methods[28]. A P-value of < 0.05 was considered significant. The agreement between the evaluators measured by the degree of inter-rater agreement using Cohen kappa coefficient was also carried out using SPSS software. This has been interpreted as "Poor", if the results in the range: 0.21-0.40; "Fair" 0.41-0.60; "Moderate" 0.61-0.80; "Good" 0.81-1.00.

RESULTS
General information about websites

The search of the three databases, Google™, Bing™ and Yahoo™, resulted in the identification of 300 websites. After the duplicates were removed we ended with 210 websites. On applying the inclusion and exclusion criteria, 84 websites were finally identified and included in the study (Figure 1).

Figure 1
Figure 1 PRISMA flowchart showing the websites on inflammatory bowel disease searched on the Internet and those finally included in the study.

Table 1 summarizes the general information about the 84 websites, including: website title, URL, author/ownership, year created, last updated, number of pages, number of tables, images and illustrations. The oldest two websites were created by the University of North Carolina (UNC), School of Medicine, North Carolina, United States and the Department of Surgery, University of California, California, United States, while the most recent was published in 2013 and created by New Health Guide, United States.

Table 1 Summarizes general information about websites on inflammatory bowel disease included in the study.
No.Website title, OrganisationURLAuthority/ownership, state, countryYear createdLast updatedNumber of pagesNumber of tablesNumber of imagesNumber of illustrations
1Inflammatory Bowel Disease (IBD), Mayo Clinichttp://www.mayoclinic.org/diseases-conditions/inflammatory-bowel-disease/basics/definition/con-20034908Mayo Foundation for Medical Education and Research, Arizona, United States199704 Feb 201411000
2Inflammatory Bowel Disease Health Center, WebMdhttp://www.webmd.com/ibd-crohns-disease/WebMD, Inc., Georgia, United States.199806 Sep 20132030
3Inflammatory Bowel Disease (IBD), Center for Disease Control and Prevention (CDC)http://www.cdc.gov/ibd/Centers for Disease Control and Prevention (CDC), Georgia, United States.199904 Sep 20143011
4Inflammatory Bowel Disease, NHS Choiceshttp://www.nhs.uk/conditions/inflammatory-bowel-disease/pages/introduction.aspxNHS England, Wakefield, United Kingdom.199629 Apr 20132000
5What are Crohn's and Colitis? Crohn's and Colitis Foundationhttp://www.ccfa.org/what-are-crohns-and-colitis/Crohns and Colitis Foundation of America, New York, United States.199614 Apr 20141010
6Inflammatory Bowel Disease, KidsHealthhttp://kidshealth.org/parent/medical/digestive/ibd.htmlKids Health Organisaion, The Nemours Foundation, Orlando, United States.199527 Jan 20154010
7Inflammatory Bowel Disease (IBD). FamilyDoctorhttp://familydoctor.org/familydoctor/en/diseases-conditions/inflammatory-bowel-disease.htmlAmerican Academy of Family Physicians, New Jersey, United States.199822 Jan 20157000
8Inflammatory Bowel Disease, Healthlinehttp://www.healthline.com/health/inflammatory-bowel-disease#Overview1Healthline, California, United States.200429 Nov 20116000
9Crohn's and Colitis, Australiahttps://www.crohnsandcolitis.com.au/about-crohns-colitis/inflammatory-bowel-disease/Crohn's and Colitis Australia, Victoria, Australia.200924 Jul 20143010
10Crohn's and Colitis UKhttp://www.crohnsandcolitis.org.uk/information-and-support/information-about-ibd/what-is-IBDCrohn's and Colitis UK, United Kingdom.201013 Nov 20142010
11Inflammatory Bowel Disease (IBD) (Intestinal Problems of IBD), MedicineNethttp://www.medicinenet.com/inflammatory_bowel_disease_intestinal_problems/article.htmMedicine.Net.com, WebMed Network, New York, United States.19956 Sep 2013120270
12Inflammatory Bowel Disease Center, Cedars-Sinaihttp://www.cedars-sinai.edu/Patients/Programs-and-Services/Inflammatory-Bowel-Disease-Center/Cedars-Sinai Medical Center, California, United States.19925 Jul 20132000
13Inflammatory Bowel Diseases Symptoms and Treatment: Livesciencehttp://www.livescience.com/39880-inflammatory-bowel-disease.htmlTanya Lewis, LiveScience Contributor, New York, United States.200110 Apr 201411010
14Inflammatory Bowel Diseases Program, Penn Medicine.orghttp://www.pennmedicine.org/gastroenterology/patient-care/gi-diseases/inflammatory-bowel-disease-ibd/Penn Medicine, Pennsylvania, United Stats.20036 Nov 20144000
15Inflammatory Bowel Diseases Support Groups, IBDsupport.orghttp://www.ibdsupport.org/IBD support.org, Utah, United States.201130 May 20142000
16Inflammatory Bowel Disease (IBD), ABC Health and wellbeinghttp://www.abc.net.au/health/library/stories/2012/02/22/3435688.htmAustralian Broadcasting Corporation, NSW, Australia.20013 Dec 20144010
17Inflammatory Bowel Disease (IBD), GIKidshttp://www.gikids.org/content/7/en/IBDGIKids and The NASPGHAN Foundation, Pennsylvania, United States.200911 Jun 20132010
18Inflammatory Bowel Disease, Vitamin D Councilhttps://www.vitamindcouncil.org/health-conditions/inflammatory-bowel-disease/The Vitamin D Council, California, United States.200717 Jun 20116000
19Inflammatory Bowel Disease Symptoms and Diagnosis, Seattle children'shttp://www.seattlechildrens.org/medical-conditions/digestive-gastrointestinal-conditions/ibd-symptoms/Children's Hospital and Regional Med. Ctr, Washington, United States.199323 Apr 20142000
20Crohn's Disease, Patient.co.ukhttp://www.patient.co.uk/health/crohns-disease-leafletPatient, Patient information Publications, Leeds, United Kingdom.199705 Mar 20138001
21Patient Information Crohn Disease (Beyond and the Basics), Uptodatehttp://www.uptodate.com/contents/crohn-disease-beyond-the-basicsUpToDate, Wolters Kluwer Health, Illinois, United States.199829 Jul 20145000
22Crohn's Disease, Centre for digestive diseaseshttp://www.cdd.com.au/pages/disease_info/crohns_disease.htmlThe Centre for Digestive Diseases, NSW, Australia.08 Jul 20134000
23Crohn's Disease, Patients: British Society for Gastroenterologyhttp://www.bsg.org.uk/patients/general/crohn-s-disease.htmlBritish Society of Gastroenterology, London, United Kingdom.199605 Aug 20146000
24Crohn's Disease, Bupahttp://www.bupa.co.uk/health-information/directory/c/crohns-diseaseThe British United Provident Association Ltd, London, United Kingdom199616 Jun 20146001
25Crohn's Disease, University of Maryland Medical Centerhttp://umm.edu/health/medical/reports/articles/crohns-diseaseUniversity of Maryland Medical Center, Maryland, United States199619 Sep 201320000
26Crohn's Disease, Symptoms, Diagnosis, Treatment, Southern Crosshttps://www.southerncross.co.nz/AboutTheGroup/HealthResources/MedicalLibrary/tabid/178/vw/1/ItemID/523/Crohns-disease-symptoms-diagnosis-treatment.aspxSouthern Cross Healthcare Group; Auckland, New Zealand.199801 Feb 20155000
27Crohn's Disease, American family physicianhttp://www.aafp.org/afp/2011/1215/p1379.htmlAmerican Academy of Family Physicians, Kansas, United States199502 Jul 20142000
28What is Crohn's Disease? What Causes Crohn's Disease? MNThttp://www.medicalnewstoday.com/articles/151620.phpChristian Nordquist, MNT, Sussex, United Kingdom.200302 Jan 20147000
29Crohn's Disease, Netdoctorhttp://www.netdoctor.co.uk/diseases/facts/crohnsdisease.htmNetDoctor.co. Ltd, London, United Kingdom.199824 Aug 20147010
30Crohn's Disease, UCSF medical centerhttp://www.ucsfhealth.org/conditions/crohns_disease/University of California San Francisco Medical Center, California, United States.200004 Jan 20122000
31Crohn's Disease Symptoms and Treatment, US.news Wellnesshttp://health.usnews.com/health-news/health-wellness/articles/2013/08/03/crohns-disease-symptoms-and-treatmentGuido Zanni, US News, New York, United States,199522 Jan 20153000
32What are the treatments for Crohn's disease? Beth Israel Deaconess Medical Centerhttp://www.bidmc.org/Centers-and-Departments/Departments/Digestive-Disease-Center/Inflammatory-Bowel-Disease-Program/Crohns-Disease/What-are-the-treatments-for-Crohns-disease.aspxBeth Israel Deaconess Medical Center, Massachusetts, United States.200216 Mar 200615041
33Diagnosing Crohn's, Crohn's and Me.http://www.crohnsandme.com/crohns-information/crohns-disease-diagnosis.aspxUCB Multinational Biopharmaceutical Company, Brussels, Belgium200529 Apr 20149111
34Understanding Crohn's Disease, Crohn's and Colitis.http://www.crohnsandcolitisinfo.com/Crohns/What-is-Crohns-DiseaseCrohn's and Colitis, Illinois, United States201102 Oct 20148011
35Learning About Crohn's Disease, National Human Genome Research Institutehttp://www.genome.gov/25521854National Human Genome Research Institute, Massachusetts, United States27 Sep 20112000
36Crohn's Disease, UPMC Life Changing Medicinehttp://www.upmc.com/services/digestive-disorders-center/services/ibd/conditions/pages/crohns-disease.aspxUPMC Digestive Disorders Center, UPMC Presbyterian, Pennsylvania, United States199904 Mar 20143000
37Crohn's Disease, Cincinnati Children'shttp://www.cincinnatichildrens.org/health/c/crohns/Cincinnati Children's Hospital Medical Center, Ohio, United States199815 may 20124010
38Crohn's Disease, Cleveland clinichttp://my.clevelandclinic.org/health/diseases_conditions/hic_Inflammatory_Bowel_Disease_IBD_QandA/hic_Crohns_DiseaseThe Cleveland Clinic Foundation, Ohio, United States199802 Jan 20152000
39Treatment of Crohn's Disease, UNC Multidisciplinary Center for IBD Research and Treatmenthttp://www.med.unc.edu/gi/specialties/ibd/about-ibd/treatment-of-ibd-1/treatment-of-crohns-diseaseUNC, School of Medicine, North Carolina, United States198607 Mar 201390910
40Crohn's Disease, Emedicine healthhttp://www.emedicinehealth.com/crohn_disease/article_em.htmEMedicine.com Inc, WebMD Network, Georgia, United States200306 Sep 201310001
41Understanding Crohn's Disease and Ulcerative Colitis. Australian Gastroenterology Institute.http://www.nevdgp.org.au/info/gastro/crohns.htmAustralian Gastroenterology Institute, Digestive Health Foundation, New South Wales, Australia02 Jun 20143000
42Crohn's Disease, Health Centershttp://www.drweil.com/drw/u/ART00339/Crohns-Disease.htmlWeil's Foundation, Arizona, United States199916 Jan 20123010
43Crohn's Disease- An Overview, the Royal Children's Hospital Melbournehttp://www.rch.org.au/kidsinfo/fact_sheets/Crohns_Disease_an_overview/The Royal Children's Hospital Melbourne, Victoria, Australia19 Jun 20143000
44Fighting Inflammatory Bowel Disease Together, the Irish Society for Colitis and Crohn's Diseasehttp://www.iscc.ie/page.php?id=18&title=What%20is%20IBDThe Irish Society for Colitis and Crohn's Disease, Dublin, United Kingdom200005 Jan 201440101
45Crohn's Disease Diagnosis, New health guidehttp://www.newhealthguide.org/Crohn%27s-Disease-Diagnosis.htmlNew Health Guide, United States201310 Feb 15.3010
46Crohn's Disease, HealthDay.http://consumer.healthday.com/encyclopedia/digestive-health-14/digestion-health-news-200/crohn-s-disease-644392.htmlHealthDay, New York, United States200215 Apr 20144000
47Ulcerative Colitis, Wikipediahttp://en.wikipedia.org/wiki/Ulcerative_colitisWikimedia Foundation, Inc, California, United States200108 May 201214661
48Living with UC, Do You Know Your Treatment Options?http://www.livingwithuc.ca/Janssen Inc., Canada2010,May 20148402
49What Is Ulcerative Colitis? Everyday Healthhttp://www.everydayhealth.com/conditions/ulcerative-colitisEveryday Health Media, LLC, New York, United States200410 Feb 201414000
50What Is Ulcerative Colitis? News Medicalhttp://www.news-medical.net/health/What-is-Ulcerative-Colitis.aspxThe AZO Network, New South Wales, Australia200426 Feb 20152000
51Crohn's Disease and Ulcerative Colitis, Better Health Channelhttp://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/Crohn%27s_disease_and_ulcerative_colitisCrohn's and Colitis, The State Government of Victoria, Victoria, Australia200728 Sep 20144000
52Ulcerative Colitis, Jackson Siegelbaum Gastroenterologyhttp://gicare.com/diseases/ulcerative-colitis/Jackson Gastroenterology Ltd, Central Pennsylvania, Pennsylvania, United States199707 April 20143003
53Ulcerative Colitis, Crohn's and Colitis Canadahttp://www.crohnsandcolitis.ca/site/c.dtJRL9NUJmL4H/b.9012449/k.C223/Ulcerative_Colitis.htmCrohn's and Colitis, Canada200819 Dec 201312001
54Ulcerative Colitis, Healthgradeshttp://www.healthgrades.com/conditions/ulcerative-colitisHealthGrades, Inc, Colorado, United States199908 May 20144000
55Information for Those with Ulcerative Colitis, Colitis UKhttp://www.ulcerativecolitis.org.uk/Colitis UK, Buckinghamshire, United Kingdom200520 Oct 201310000
56Colitis and Chronic Ulcerative Colitis, Virginia Masonhttps://www.virginiamason.org/ColitisandChronicUlcerativeColitisVirginia Mason Medical Center, Washington, United States199807 May 20083000
57Ulcerative Colitis, GastroNethttp://www.gastro.net.au/diseases/ulcerativecolitis.htmlGastroNet Australia Pty Ltd, Canberra, Australia200912 May 20145245
58Ulcerative Colitis, Ulcerative Colitis Net.http://www.ulcerativecolitis.net/Serovera, Florida, United States200028 Feb 20149310
59Inflammatory Bowel Disease, Lab Tests Onlinehttp://labtestsonline.org/understanding/conditions/inflammatory-bowelAmerican Association for Clinical Chemistry (AACC), Washington, DC, United States200108 Nov 20103302
60Inflammatory Bowel Disease Fact Sheet, Womenshealth.govhttp://www.womenshealth.gov/publications/our-publications/fact-sheet/inflammatory-bowel-disease.htmlWomenshealth.gov, the US Department of Health and Human Services199529 Nov 20143003
61Inflammatory Bowel Disease (IBD), Innerbodyhttp://www.innerbody.com/diseases-conditions/ibdInnerBody, California, United States199602 Oct 20125234
62Inflammatory Bowel Disease (IBD), Rightdiagnosishttp://www.rightdiagnosis.com/i/inflammatory_bowel_disease/intro.htmRightdiagnosis. com, United States200511 Jul 20132150
63Inflammatory Bowel Disease, Lifescript.comhttp://www.lifescript.com/health/centers/digestive/related_conditions/inflammatory_bowel_disease.aspxLifeScript, California, United States199905 May 20141521
64Inflammatory Bowel Disease (IBD), MUSC Healthhttp://www.ddc.musc.edu/public/symptomsDiseases/diseases/smallBowel/IBD.htmlDigestive Disease Center, The Medical University of South Carolina, South Carolina, United States199021 May 20131110
65Inflammatory Bowel Disease (IBD): Ulcerative Colitis, Crohn's Disease, New York-Presbyterian Digestive Diseases.http://nyp.org/services/digestive/ibd.htmlNewYork-Presbyterian Hospital, New York, United States199824 Aug 20071200
66Inflammatory Bowel Disease, Human Diseases and Conditions Forum.http://www.humanillnesses.com/original/Her-Kid/Inflammatory-Bowel-Disease.htmlHuman Diseases and Conditions Forum, United States200616 Oct 20132310
67Facts About Crohn's Disease, US. Food and Drug Administrationhttp://www.fda.gov/ForConsumers/ConsumerUpdates/ucm107358.htmUS Food and Drug Administration, The US Department of Health and Human Services, Maryland, United States199714 Oct 20141321
68Crohn's Disease Symptoms and warning Signs, SymptomFindhttp://www.symptomfind.com/diseases-conditions/crohns-disease-symptoms-warning-signs/Symptom.Find.Com. United States200821 Jun 20143212
69Crohn's Disease-At a Glance, SixPartsWater.Orghttp://www.sixpartswater.org/knowledge-centre/crohns-disease/glanceSixPartsWater.Org, United Kingdom200702 Oct 20122130
70Ulcerative Colitis, eMedTVhttp://colitis.emedtv.com/ulcerative-colitis/ulcerative-colitis.htmleMedTV, Washington, United States200507 May 20143052
71Crohn's Disease, Department of Surgery, University of California.http://colorectal.surgery.ucsf.edu/conditions--procedures/crohns-disease.aspxDepartment of Surgery, University of California, California, United States198610 Oct 20131141
72Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseaseshttp://www.niddk.nih.gov/health-information/health-topics/digestive-diseases/crohns-disease/Pages/facts.aspxThe National Institutes of Diabetes and Digestive and Kidney Diseases, NIDDK, Maryland, United States200220 Jul 20141120
73Crohn's Disease, Patient Education Centerhttp://www.patienteducationcenter.org/articles/crohns-disease/Patient Education Center, Harvard Medical School, Harvard Medical Publications, Massachusetts, United States200320 Jun 20142030
74Crohn's Disease Information, alot healthhttp://health.alot.com/conditions/crohns-disease-information--163Alot Health.Com, Arkansas, United States199415 Aug 20143111
75Crohn's Disease, Diagnose-me.Comhttp://www.diagnose-me.com/symptoms-of/crohns-disease.htmlDiagnose-me.Com, Hawaii, United States200221 Jan 20141341
76Crohns Disease Information: Is Colon Cleansing the Answer, Colon Cleanse Informationhttp://www.colon-cleanse-information.com/crohns-disease-information.htmlColon-Cleanse-Information.Com, MKR Concepts, Oregon, United States200711 Nov 20143050
77Crohn's Disease or Regional Enteritis, MD India.http://www.medindia.net/patients/patientinfo/Crohns-Disease.htmMedindia4u.com Pvt. Ltd, Chennai, India200021 Nov 20145010
78Crohn's Disease Information, Digestive Disordershttp://www.articleinsider.com/health-and-fitness/digestive-disorders/crohns-disease-informationDigestive Disorders, United States200303 Sep 20142121
79Inflammatory Bowel Disease, Patient Center, American College of Gastroenterologyhttp://patients.gi.org/topics/inflammatory-bowel-disease/Patient Center, American College of Gastroenterology, Maryland, United States199616 May 20134312
80Crohn's Disease:Symptoms, Diagnosis and Treatment, Disabled World.Comhttp://www.disabled-world.com/health/digestive/crohns-disease/Disabled World.Com, New York, United States200429 Oct 20121341
81Crohn's Disease, Nutritionist Resourcehttp://www.nutritionist-resource.org.uk/articles/crohns-disease.htmlNutritionist Resource, Surrey, United Kingdom20109 Feb 153130
82Crohn's Disease: Symptoms, Diagnosis and Treatment, verywell.comhttp://seniorhealth.about.com/cs/digestivetract/a/crohns_2.htmVerywell.com part of about.com, Inc., United States199910 Feb 146040
83Ulcerative Colitis, Halyard Surgical.http://www.ulcerative-colitis.org/Halyard Surgical, New South Wales, Australia20037 Nov 142061
84Ulcerative Colitis Overview, Health Communities.comhttp://www.healthcommunities.com/colitis/ulcerative-colitis-overview.shtmlHealthcommunities.com, New York, United States199810 May 133200

For other websites, four websites were published in 1987-1994, 44 were published in 1995-2002, and 29 were published in the years 2003-2011. Only four websites were difficult to identify the exact year of their publication. Websites were updated regularly, 51 websites were updated in 2015 and 2014, while 33 websites were updated earlier, including one website was updated in 2006.

Of the 84 websites, 60 websites comprised 1-5 pages, 16 websites had 6-10 pages, 8 had more than 11 pages. The website titled Crohn's disease by the University of Maryland medical center had the highest number of pages, 20 pages. The number of tables varied from zero to 6. Out of the 84 websites only 24 websites used tables to explain their content. The total number of tables in these websites was 55. The number of images varied from zero to 27. Out of the 84 websites only 42 websites had images to explain the content. The total number of images in these websites was 141. Again the number of illustrations varied from zero to 10. Out of the 84 websites, only 28 had illustrations to explain the content. The total number of illustrations was 53.

DISCERN and the HONcode scores of websites

In order to calculate the accuracy of the websites, we used two instruments, the DISCERN and the HONcode instruments. Table 2 summarizes the scores calculated from applying the DISCERN and the HONcode scores expressed as mean ± SD for each website. The DISCERN scores varied from a minimum of 18 to a maximum of 68 (mean ± SD, 42.2 ± 10.7; median = 41.5, IQR = 15.8). The lowest DISCERN score was scored by the website, Crohn's Disease Diagnosis, New health guide, while the highest DISCERN score was scored by the website, Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases. The HONcode trust worthy scores also varied from a minimum of 0.14 to a maximum of 0.95 (mean ± SD, 0.16 ± 0.19; median = 0.45, IQR = 0.29). The lowest HONcode score was scored by the website, Crohn's Disease, American family physician, while the maximum score was scored by the website, Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases. Along with the HONcode trust worthy scores, HONcode certificate was indicated for websites that have received such certificates, Table 2.

Table 2 Summarizes websites included in the study on inflammatory bowel disease included in the study: The accuracy scores (calculated using the DISCERN score and the HONcode score) and the readability scores.
No.Website title, OrganisationURLAccuracy scores
Readability scores
The DISCERN Score (mean ± SD)The HONcode score (Out of 100)*The Flesch-Kincaid Grade Level IndexThe Coleman-Liau Readability Index
1Inflammatory Bowel Disease (IBD), Mayo Clinichttp://www.mayoclinic.org/diseases-conditions/inflammatory-bowel-disease/basics/definition/con-2003490865.0 ± 1.00.86*13.1 ± 0.015.0 ± 0.0
2Inflammatory Bowel Disease Health Center, WebMdhttp://www.webmd.com/ibd-crohns-disease/41.3 ± 1.10.70*10.5 ± 3.213.3 ± 2.5
3Inflammatory Bowel Disease (IBD), Center for Disease Control and Prevention (CDC)http://www.cdc.gov/ibd/28.7 ± 0.60.9010.9 ± 2.713.3 ± 0.6
4Inflammatory Bowel Disease, NHS Choiceshttp://www.nhs.uk/conditions/inflammatory-bowel-disease/pages/introduction.aspx53.7 ± 0.60.6312.2 ± 1.612.3 ± 1.5
5What are Crohn's and Colitis? Crohn's and Colitis Foundationhttp://www.ccfa.org/what-are-crohns-and-colitis/50.3 ± 0.60.8116.2 ± 9.313.7 ± 0.6
6Inflammatory Bowel Disease, KidsHealthhttp://kidshealth.org/parent/medical/digestive/ibd.html50.7 ± 0.60.4512.0 ± 0.711.7 ± 0.6
7Inflammatory Bowel Disease (IBD). FamilyDoctorhttp://familydoctor.org/familydoctor/en/diseases-conditions/inflammatory-bowel-disease.html45.0 ± 1.00.63*9.6 ± 1.410.7 ± 0.6
8Inflammatory Bowel Disease, Healthlinehttp://www.healthline.com/health/inflammatory-bowel-disease#Overview141.0 ± 1.00.59*8.7 ± 0.611.7 ± 0.6
9Crohn's and Colitis, Australiahttps://www.crohnsandcolitis.com.au/about-crohns-colitis/inflammatory-bowel-disease/41.7 ± 0.60.3413.6 ± 3.213.0 ± 1.0
10Crohn's and Colitis UKhttp://www.crohnsandcolitis.org.uk/information-and-support/information-about-ibd/what-is-IBD60.7 ± 0.60.4510.4 ± 1.610.3 ± 3.1
11Inflammatory Bowel Disease (IBD) (Intestinal Problems of IBD), MedicineNethttp://www.medicinenet.com/inflammatory_bowel_disease_intestinal_problems/article.htm57.3 ± 1.50.75*13.3 ± 1.514.3 ± 1.5
12Inflammatory Bowel Disease Center, Cedars-Sinaihttp://www.cedars-sinai.edu/Patients/Programs-and-Services/Inflammatory-Bowel-Disease-Center/25.7 ± 1.10.2713.1 ± 2.914.0 ± 3.6
13Inflammatory Bowel Diseases Symptoms and Treatment: Livesciencehttp://www.livescience.com/39880-inflammatory-bowel-disease.html39.3 ± 0.60.2711.1 ± 1.411.3 ± 1.5
14Inflammatory Bowel Diseases Program, Penn Medicine.orghttp://www.pennmedicine.org/gastroenterology/patient-care/gi-diseases/inflammatory-bowel-disease-ibd/38.0 ± 0.00.5415.1 ± 2.613.0 ± 2.0
15Inflammatory Bowel Diseases Support Groups, IBDsupport.orghttp://www.ibdsupport.org/52.0 ± 1.70.61*12.8 ± 3.713.0 ± 3.6
16Inflammatory Bowel Disease (IBD), ABC Health and wellbeinghttp://www.abc.net.au/health/library/stories/2012/02/22/3435688.htm34.0 ± 0.00.4511.1 ± 0.311.3 ± 0.6
17Inflammatory Bowel Disease (IBD), GIKidshttp://www.gikids.org/content/7/en/IBD31.0 ± 0.00.5212.5 ± 0.312.7 ± 0.6
18Inflammatory Bowel Disease, Vitamin D Councilhttps://www.vitamindcouncil.org/health-conditions/inflammatory-bowel-disease/31.3 ± 1.10.4311.0 ± 10.410.7 ± 1.5
19Inflammatory Bowel Disease Symptoms and Diagnosis, Seattle children'shttp://www.seattlechildrens.org/medical-conditions/digestive-gastrointestinal-conditions/ibd-symptoms/37.3 ± 0.60.438.9 ± 1.011.0 ± 1.0
20Crohn's Disease, Patient.co.ukhttp://www.patient.co.uk/health/crohns-disease-leaflet55.7 ± 1.50.59*8.5 ± 2.010.0 ± 1.7
21Patient Information Crohn's Disease (Beyond and the Basics), Uptodatehttp://www.uptodate.com/contents/crohn-disease-beyond-the-basics36.7 ± 1.10.7011.2 ± 1.912.3 ± 0.6
22Crohn's Disease, Centre for digestive diseaseshttp://www.cdd.com.au/pages/disease_info/crohns_disease.html34.7 ± 0.60.5212.0 ± 0.614.3 ± 1.1
23Crohn's Disease, Patients: British Society for Gastroenterologyhttp://www.bsg.org.uk/patients/general/crohn-s-disease.html49.7 ± 0.60.5611.3 ± 1.710.7 ± 1.1
24Crohn's Disease, Bupahttp://www.bupa.co.uk/health-information/directory/c/crohns-disease52.0 ± 0.00.77*8.2 ± 0.59.0 ± 1.7
25Crohn's Disease, University of Maryland Medical Centerhttp://umm.edu/health/medical/reports/articles/crohns-disease64.7 ± 1.50.8115.3 ± 4.815.0 ± 1.0
26Crohn's Disease, Symptoms, Diagnosis, Treatment, Southern Crosshttps://www.southerncross.co.nz/AboutTheGroup/HealthResources/MedicalLibrary/tabid/178/vw/1/ItemID/523/Crohns-disease-symptoms-diagnosis-treatment.aspx42.7 ± 0.60.4013.8 ± 0.313.7 ± 0.6
27Crohn's Disease, American family physicianhttp://www.aafp.org/afp/2011/1215/p1379.html42.3 ± 1.10.189.0 ± 2.912.3 ± 4.2
28What is Crohn's Disease? What Causes Crohn's Disease? MNThttp://www.medicalnewstoday.com/articles/151620.php45.7 ± 1.50.50*11.2 ± 2.110.7 ± 1.5
29Crohn's Disease, Netdoctorhttp://www.netdoctor.co.uk/diseases/facts/crohnsdisease.htm43.0 ± 0.00.5910.0 ± 1.612.3 ± 1.5
30Crohn's Disease, UCSF medical centerhttp://www.ucsfhealth.org/conditions/crohns_disease/41.0 ± 1.00.4011.4 ± 0.511.3 ± 0.6
31Crohn's Disease Symptoms and Treatment, US.news Wellnesshttp://health.usnews.com/health-news/health-wellness/articles/2013/08/03/crohns-disease-symptoms-and-treatment48.3 ± 0.60.506.7 ± 0.812.3 ± 2.5
32What are the treatments for Crohn's disease? Beth Israel Deaconess Medical Centerhttp://www.bidmc.org/Centers-and-Departments/Departments/Digestive-Disease-Center/Inflammatory-Bowel-Disease-Program/Crohns-Disease/What-are-the-treatments-for-Crohns-disease.aspx41.7 ± 1.50.5914.3 ± 2.214.3 ± 1.1
33Diagnosing Crohn's, Crohn's and Me.http://www.crohnsandme.com/crohns-information/crohns-disease-diagnosis.aspx43.3 ± 1.10.2713.6 ± 1.112.7 ± 2.5
34Understanding Crohn's Disease, Crohn' and Colitis.http://www.crohnsandcolitisinfo.com/Crohns/What-is-Crohns-Disease41.7 ± 0.60.4512.4 ± 2.012.0 ± 2.6
35Learning About Crohn's Disease, National Human Genome Research Institutehttp://www.genome.gov/2552185429.0 ± 0.00.4010.8 ± 0.512.3 ± 1.5
36Crohn's Disease, UPMC Life Changing Medicinehttp://www.upmc.com/services/digestive-disorders-center/services/ibd/conditions/pages/crohns-disease.aspx44.3 ± 0.60.2720.3 ± 9.515.0 ± 2.0
37Crohn's Disease, Cincinnati Children'shttp://www.cincinnatichildrens.org/health/c/crohns/44.7 ± 0.60.409.3 ± 1.110.7 ± 0.6
38Crohn's Disease, Cleveland clinichttp://my.clevelandclinic.org/health/diseases_conditions/hic_Inflammatory_Bowel_Disease_IBD_QandA/hic_Crohns_Disease26.3 ± 1.10.50*12.9 ± 1.212.3 ± 2.3
39Treatment of Crohn's Disease, UNC Multidisciplinary Center for IBD Research and Treatmenthttp://www.med.unc.edu/gi/specialties/ibd/about-ibd/treatment-of-ibd-1/treatment-of-crohns-disease42.3 ± 0.60.4512.3 ± 0.513.3 ± 0.6
40Crohn's Disease, Emedicine healthhttp://www.emedicinehealth.com/crohn_disease/article_em.htm59.3 ± 1.20.86*13.0 ± 1.213.0 ± 1.7
41Understanding Crohn's Disease and Ulcerative Colitis. Australian Gastroenterology Institute.http://www.nevdgp.org.au/info/gastro/crohns.htm42.3 ± 1.10.2710.4 ± 0.711.3 ± 0.6
42Crohn's Disease, Health Centershttp://www.drweil.com/drw/u/ART00339/Crohns-Disease.html37.0 ± 0.00.5011.9 ± 1.112.7 ± 1.5
43Crohn's Disease- An Overview, the Royal Children's Hospital Melbournehttp://www.rch.org.au/kidsinfo/fact_sheets/Crohns_Disease_an_overview/33.7 ± 1.10.507.7 ± 2.011.3 ± 1.5
44Fighting Inflammatory Bowel Disease Together, the Irish Society for Colitis and Crohn's Diseasehttp://www.iscc.ie/page.php?id=18&title=What%20is%20IBD37.7 ± 0.60.4011.6 ± 1.410.7 ± 0.6
45Crohn's Disease Diagnosis, New health guidehttp://www.newhealthguide.org/Crohn%27s-Disease-Diagnosis.html18.3 ± 0.60.6513.5 ± 0.713.7 ± 1.1
46Crohn's Disease, HealthDay.http://consumer.healthday.com/encyclopedia/digestive-health-14/digestion-health-news-200/crohn-s-disease-644392.html61.0 ± 1.00.86*10.9 ± 1.412.0 ± 0.0
47Ulcerative Colitis, Wikipediahttp://en.wikipedia.org/wiki/Ulcerative_colitis54.0 ± 0.00.6315.2 ± 2.114.0 ± 2.6
48Living with UC, Do You Know Your Treatment Options?http://www.livingwithuc.ca/44.3 ± 1.10.6812.2 ± 1.611.3 ± 2.3
49What Is Ulcerative Colitis? Everyday Healthhttp://www.everydayhealth.com/conditions/ulcerative-colitis40.0 ± 0.00.54*14.1 ± 0.713.0 ± 1.0
50What Is Ulcerative Colitis? News Medicalhttp://www.news-medical.net/health/What-is-Ulcerative-Colitis.aspx46.0 ± 1.00.5911.9 ± 0.213.0 ± 1.0
51Crohn's Disease and Ulcerative Colitis, Better Health Channelhttp://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/Crohn%27s_disease_and_ulcerative_colitis34.3 ± 1.10.319.8 ± 2.411.3 ± 1.5
52Ulcerative Colitis, Jackson Siegelbaum Gastroenterologyhttp://gicare.com/diseases/ulcerative-colitis/52.3 ± 0.60.3610.6 ± 1.411.7 ± 2.1
53Ulcerative Colitis, Crohn's and Colitis Canadahttp://www.crohnsandcolitis.ca/site/c.dtJRL9NUJmL4H/b.9012449/k.C223/Ulcerative_Colitis.htm19.7 ± 0.60.319.2 ± 1.810.0 ± 1.0
54Ulcerative Colitis, Healthgradeshttp://www.healthgrades.com/conditions/ulcerative-colitis52.7 ± 1.10.6814.5 ± 3.514.0 ± 1.0
55Information for Those with Ulcerative Colitis, Colitis UKhttp://www.ulcerativecolitis.org.uk/49.0 ± 1.70.4514.5 ± 2.011.7 ± 1.1
56Colitis and Chronic Ulcerative Colitis, Virginia Masonhttps://www.virginiamason.org/ColitisandChronicUlcerativeColitis40.7 ± 0.60.3116.7 ± 3.915.0 ± 2.0
57Ulcerative Colitis, GastroNethttp://www.gastro.net.au/diseases/ulcerativecolitis.html26.7 ± 0.60.6013.1 ± 1.113.3 ± 2.3
58Ulcerative Colitis, Ulcerative Colitis Net.http://www.ulcerativecolitis.net/41.3 ± 0.60.3011.4 ± 3.312.0 ± 2.0
59Inflammatory Bowel Disease, Lab Tests Onlinehttp://labtestsonline.org/understanding/conditions/inflammatory-bowel38.3 ± 0.60.32*13.0 ± 0.712.0 ± 1.0
60Inflammatory Bowel Disease Fact Sheet, Womenshealth.govhttp://www.womenshealth.gov/publications/our-publications/fact-sheet/inflammatory-bowel-disease.html55.7 ± 1.50.408.8 ± 0.710.0 ± 1.0
61Inflammatory Bowel Disease (IBD), Innerbodyhttp://www.innerbody.com/diseases-conditions/ibd39.0 ± 1.00.2517.2 ± 5.716.7 ± 1.5
62Inflammatory Bowel Disease (IBD), Rightdiagnosishttp://www.rightdiagnosis.com/i/inflammatory_bowel_disease/intro.htm38.0 ± 1.00.3315.9 ± 4.115.7 ± 2.1
63Inflammatory Bowel Disease, Lifescript.comhttp://www.lifescript.com/health/centers/digestive/related_conditions/inflammatory_bowel_disease.aspx32.0 ± 1.00.5015.1 ± 2.812.7 ± 1.5
64Inflammatory Bowel Disease (IBD), MUSC Healthhttp://www.ddc.musc.edu/public/symptomsDiseases/diseases/smallBowel/IBD.html22.7 ± 1.10.2714.2 ± 2.215.7 ± 2.9
65Inflammatory Bowel Disease (IBD): Ulcerative Colitis, Crohn's Disease, New York-Presbyterian Digestive Diseaseshttp://nyp.org/services/digestive/ibd.html32.3 ± 0.60.2713.2 ± 0.512.7 ± 0.6
66Inflammatory Bowel Disease, Human Diseases and Conditions Forum.http://www.humanillnesses.com/original/Her-Kid/Inflammatory-Bowel-Disease.html35.3 ± 1.50.1410.7 ± 0.913.3 ± 2.3
67Facts About Crohn's Disease, US. Food and Drug Administrationhttp://www.fda.gov/ForConsumers/ConsumerUpdates/ucm107358.htm34.7 ± 2.10.3610.6 ± 2.613.7 ± 2.1
68Crohn's Disease Symptoms and warning Signs, SymptomFindhttp://www.symptomfind.com/diseases-conditions/crohns-disease-symptoms-warning-signs/26.0 ± 1.00.26*10.0 ± 1.913.7 ± 2.3
69Crohn's Disease-At a Glance, SixPartsWater.Orghttp://www.sixpartswater.org/knowledge-centre/crohns-disease/glance39.7 ± 0.60.1518.3 ± 3.412.7 ± 0.6
70Ulcerative Colitis, eMedTVhttp://colitis.emedtv.com/ulcerative-colitis/ulcerative-colitis.html55.3 ± 0.60.32*12.1 ± 0.811.0 ± 2.0
71Crohn's Disease, Department of Surgery, University of California.http://colorectal.surgery.ucsf.edu/conditions--procedures/ulcerative-colitis.aspx51.0 ± 1.00.7312.4 ± 0.812.3 ± 2.5
72Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseaseshttp://www.niddk.nih.gov/health-information/health-topics/digestive-diseases/crohns-disease/Pages/facts.aspx68.3 ± 1.20.9513.6 ± 3.913.7 ± 5.5
73Crohn's Disease, Patient Education Centerhttp://www.patienteducationcenter.org/articles/crohns-disease/38.0 ± 1.00.308.9 ± 0.812.3 ± 1.5
74Crohn's Disease Information, alot healthhttp://health.alot.com/conditions/crohns-disease-information--16332.3 ± 0.60.2210.4 ± 1.512.0 ± 2.6
75Crohn's Disease, Diagnose-me.Comhttp://www.diagnose-me.com/symptoms-of/crohns-disease.html32.0 ± 1.00.2812.5 ± 1.614.3 ± 1.1
76Crohns Disease Information: Is Colon Cleansing the Answer, Colon Cleanse Informationhttp://www.colon-cleanse-information.com/crohns-disease-information.html35.7 ± 2.50.3613.2 ± 0.514.7 ± 1.5
77Crohn's Disease or Regional Enteritis, MD India.http://www.medindia.net/patients/patientinfo/Crohns-Disease.htm46.3 ± 1.10.5411.8 ± 1.314.0 ± 2.6
78Crohn's Disease Information, Digestive Disordershttp://www.articleinsider.com/health-and-fitness/digestive-disorders/crohns-disease-information33.3 ± 0.60.6010.2 ± 0.612.3 ± 0.6
79Inflammatory Bowel Disease, Patient Center, American College of Gastroenterologyhttp://patients.gi.org/topics/inflammatory-bowel-disease/58.3 ± 1.20.8012.7 ± 1.313.0 ± 1.0
80Crohn's Disease:Symptoms, Diagnosis and Treatment, Disabled World.Comhttp://www.disabled-world.com/health/digestive/crohns-disease/26.3 ± 2.30.198.6 ± 0.612.00 ± 1.73
81Crohn's Disease, Nutritionist Resourcehttp://www.nutritionist-resource.org.uk/articles/crohns-disease.html39.3 ± 0.60.5710.2 ± 2.611.3 ± 1.1
82Crohn's Disease: Symptoms, Diagnosis and Treatment, verywell.comhttp://seniorhealth.about.com/cs/digestivetract/a/crohns_2.htm44.3 ± 1.50.3712.6 ± 0.711.0 ± 1.0
83Ulcerative Colitis, Halyard Surgical.http://www.ulcerative-colitis.org/51.7 ± 1.50.2512.1 ± 0.612.7 ± 2.5
84Ulcerative Colitis Overview, Health Communities.comhttp://www.healthcommunities.com/colitis/ulcerative-colitis-overview.shtml52.7 ± 1.50.23*12.6 ± 1.212.7 ± 1.1

The top ten websites on IBD as per the DISCREN scores were in the following order: The Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases (scored 68), Inflammatory Bowel Disease, MayoClinic (Scored 65), Crohn's Disease, University of Maryland Medical Center (scored 64), Crohn's Disease, HealthDay (scored 61), Crohn's Disease and Colitis UK (scored 60), Crohn's Disease, eMedicine health (scored 59), Inflammatory Bowel Disease, Patient Center, American College of Gastroenterology (scored 58), Inflammatory Bowel Disease, MedicineNet (scored 57), Inflammatory Bowel Disease, Fact Sheet, Womenshealth.gov (scored 55), Ulcerative Colitis, eMedTV (scored 55). The top ten websites as per the HONcode tool were in the following order: Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases (scored 0.95), Inflammatory Bowel Disease, Center for Disease Control and Prevention (scored 0.90), Inflammatory Bowel Disease, MayoClinic (scored 0.86), Crohn's Disease, eMedicine health (scored 0.86), Crohn's Disease, HealthDay (scored 0.86), What are Crohn's & Colitis? Crohn's & Colitis Foundation (scored 0.81), Crohn's Disease, University of Maryland Medical Center (scored 0.81), Inflammatory Bowel Disease, Patient Center, American College of Gastroenterology (scored 0.80), Crohn's Disease, Bupa (scored 0.77), and Inflammatory Bowel Disease, MedicineNet (scored 0.75). It is interesting to note that the website, Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases was ranked number one as per the two instruments. Seven websites in total were among the top ten websites as per both the DISCERN and the HONcode scores. Nine out of the ten websites were created in United State.

Grouping the websites under five categories

Table 3 summarizes the grouping of the 84 websites under five categories on the basis of the website creators. Universities and research centers created 25 (%), professional foundations and associations created 15 (%), commercial and pharmaceutical companies created 25 (%), charities and volunteers contributed to 9 (%), non-university educational bodies such as colleges, academies, councils, WebMed contributed to 10 (%). Further analysis revealed that there was no significant differences in the DISCERN scores between the groups (P = 0.472) but the HONcode scores were different (P = 0.041). Examples of content deficiencies or scientific content inaccuracies and suggestions for improvement are shown on Table 4.

Table 3 Grouping the websites on inflammatory bowel disease included in the study under five categories.
CategoryNumberDISCERN score
HONcode score
mean ± SD695%CI for meansmean ± SD795%CI for means
Universities and Research Centers12541.3 ± 11.336.6-45.90.46 ± 0.200.37-0.54
Foundations and Associations21544.5 ± 10.538.6-50.20.53 ± 0.160.44-0.62
Commercial and Pharmaceutical Companies32540.4 ± 10.136.3-44.60.44 ± 0.180.36-0.52
Charities and Volunteer work4940.2 ± 11.931.0-49.40.39 ± 0.170.26-0.52
Non-university Educational Bodies51046.7 ± 9.739.7-53-60.63 ± 0.200.48-0.76
Total8442.1 ± 10.739.8-44.50.48 ± 0.190.44-0.52
Table 4 Examples of assessment of the content of some websites on inflammatory bowel disease.
Website NumberTitleAreas of deficienciesSuggestions for improvement
45Crohn's Disease Diagnosis, New health guideSymptoms of Crohn's disease are briefly mentioned. Some details are needed to explain the common presenting symptoms. No mention of differential diagnosis. No mention of investigations needed to confirm the diagnosis. Nothing is mentioned about treatment of Crohn's disease.Symptoms may include abdominal pain, typically in the right lower quadrant, diarrhoea, some blood may be present in stools, fatigue. In more severe disease fever, and weight loss may be present. Some patients may have nausea, and abdominal distention together with abdominal pain. It is worth to mention that Crohn's disease is a lifelong illness (chronic disease). People who have Crohn's will experience periods of flare-ups, when their symptoms are active, and other times when their symptoms go into remission. Up to 30% of patients may have changes in the area around the anus including anal fistulas (internal tracts connecting the anal lumen with the skin around the anus), abscess, skin tags, and anal fissures. About 10%-20% of patients also have joint pains, lower back pain, skin rash known as erythema nodosum, and eye changes. (images showing some of these changes will enhance this part). A section discussing investigations should be added. In addition to detailed medical history, the treating doctor will initiate the evaluation by testing for infectious conditions that can cause inflammation of the colon, screen for endocrine-metabolic disorders such as excessive activity of the thyroid gland. Therefore biochemical tests and stool tests are needed. Endoscopic evaluation (colonoscopy) should be carried out in patients who have symptoms suggestive of inflammatory bowel disease and no evidence for an infection to explain symptoms. Small bowel images, computed tomography (CT) enterography may also be needed. Nutritional changes, medical and surgical treatment should be briefly discussed.
53Ulcerative Colitis, Crohn's and Colitis CanadaSymptoms are briefly stated. No mention of differential diagnosis, investigations and no discussion of medical and surgical treatment.
64Inflammatory Bowel Disease (IBD), MUSC HealthSymptoms of inflammatory bowel disease are not clearly written. One would wonder, are "bowel sores" and "intestinal bleeding" symptoms? Differential diagnosis is not mentioned. The approach for diagnosing inflammatory bowel disease is not mentioned and the treatment of IBD is not explained.As discussed earlier.
68Crohn's Disease Symptoms and Warning Signs, SymptomsFindAlthough symptoms of Crohn's disease are mentioned briefly, they are not explained. Mild, moderate and severe inflammatory bowel disease are stated but not explained. This should be explained in a simple language. Complications are mentioned but there was no mention how the disease is diagnosed, and what investigations are needed. Nothing is mentioned about nutritional changes, medical and surgical treatment of inflammatory bowel disease.Patients are described to have mild ulcerative colitis when they have: -Fewer than four bowel motions (stools) per day. -No bleeding or small amounts of bleeding in their stools. -Normal erythrocyte sedimentation rate (ESR) -No fever, no anaemia and no increases in their heart rate, Patients are described to have moderate ulcerative colitis when they have: -More than four stools per day. -Mild elevation in ESR. Patients are described to have severe ulcerative colitis when they have: -More than six stools a day (loose stools). -Fever, rapid heartbeat, and anaemia. -Elevated ESR. The website may also mention changes that necessitate hospital admission and medical attention. Websites may provide key questions that patients may use when they review their treating doctors. Examples of these questions: •I wonder what's causing these symptoms? •What type of tests do I need? Do these tests require any special preparation? •What treatments are available, and which do you recommend? •Are there any medications that I should avoid? •Do I need to follow any dietary restrictions? •Are there any risks if I become pregnant? The MayoClinic website has listed a number of useful questions that patients can use.
12Inflammatory Bowel Disease Center, cedars-Sinai.Under symptoms of Crohn's disease, it is written, "The most common signs are pain in the stomach area (usually on the right side) and diarrhea", it is not clear what is meant by pain in the stomach area on the right side? Complications are provided but no signs are stated. No mention of differential diagnosis, no mention of investigations and possible findings. Nutritional changes, medical and surgical treatment are not explained.The authors should differentiate between symptoms and signs. Scientific errors are noted in the website and common presenting symptoms should be stated. It may be useful to explain the symptoms under two main headings: symptoms in children, and symptoms in adults. Investigations needed to diagnose the disease should be discussed. Patients are usually interested to know more detail about these investigations. Information provided should answer questions such as -Name of the test -Why the test is need? -What is the test about? -Nature of the investigation (invasive vs non-invasive) -Are there special preparations needed prior to the test? -Any possible complications related to the investigation? -What can the results of the test tell the patient and the treating doctor? It is also important to state that IBD is an ongoing condition (chronic disease), so some of the tests may need to be repeated from time to time, or extra tests may be needed. These investigations may include: (1) Blood tests including full blood count, inflammatory markers tests including erythrocyte sedimentation rate (ESR), c-reactive protein (CRP), liver function tests, urea and electrolytes, and other biochemical tests, (2) stool tests including stool microscopy, stool culture and sensitivity, fecal markers such as fecal calprotectin, fecal lactoferrin, (3) Endoscopy including colonoscopy, sigmoidoscopy, proctoscopy, with biopsies for histological studies (4) radiological studies such as barium studies, CT scans, MRI scans and PET scans.
Readability level of websites

Table 2 summarizes the readability scores calculated by using two methods, the Flesch-Kincaid Grade Level Index and Coleman-Liau Readability Index. The minimum score for the Flesch-Kincaid Grade Level Index was 6.7 for the website Crohn's Disease Symptoms and Treatment, United States news Wellness, while the maximum score was 20.3 for the website Crohn's Disease, UPMC Life Changing Medicine. Out of the 84 websites, 28 received a mean of 6.7 to 10.9, forty-six received a mean of 11.0 to 14.5, and ten websites received a mean of 15.7 to 20.3. The overall mean score for the 84 websites was 11.9 ± 2.4.

For the Coleman-Liau Readability Index the minimum score was 9.0 for the website Crohn's Disease, Bupa, while the maximum score was 16 for the website Inflammatory Bowel Disease, Fact Sheet, Womenshealth. Out of the 84 websites, eleven received a mean score of 9.0 to 10.9, thirty-nine received a score of 11.0 to 12.7, and thirty-four received a score of 13.0 to 16.0. The overall mean score for the 84 websites was 12.6 ± 1.5.

Significant correlation was found between the Flesch-Kincaid Grade Level index scores and the Coleman-Liau Index scores (R² = 0.509, P < 0.001) (Figure 2).

Figure 2
Figure 2 Correlation between the scores of the two readability methods: Coleman-Liau Readability Index and Flesch-Kincaid Grade Level Index.
The agreement between the evaluators

Table 5 summarizes the inter-rater agreement between evaluators for the DISCERN instrument items. The overall Cohen kappa scores were in the range of 0.804-0.876.

Table 5 Summarizes the inter-rater agreement between evaluators calculated using Cohen kappa coefficient scores.
DISCERN itemsMean score (95%CI of the difference)
Reviewer variability (κrange)
Evaluator 1Evaluator 2Evaluator3
1. Are the aims clear?1.7 (1.5-2.0)1.7 (1.5-2.0)1.8 (1.5-2.0)0.885-0.953
2. Does it achieve its aims?1.4 (0.9-1.8)1.5 (1.0-1.9)1.5 (1.0-1.9)0.790-0.904
3. Is it relevant?3.8 (3.6-4.0)3.8 (3.6-4.0)3.8 (3.6-4.0)0.792-0.887
4. Is it clear what sources of information were used to compile the publication (other than the author or producer)?2.5 (2.2-2.9)2.5 (2.2-2.8)2.5 (2.2-2.8)0.879-0.880
5. Is it clear when the information used or reported in the publication was produced?2.7 (2.4-3.1)2.7 (2.3-3.0)2.7 (2.3-3.0)0.793-0.875
6. Is it balanced and unbiased?3.7 (3.5-4.0)3.8 (3.6-4.0)3.7 (3.5-3.9)0.796-0.890
7. Does it provide details of additional sources of support and information?2.2 (1.9-2.5)2.2 (1.9-2.5)2.2 (1.9-2.5)0.792-0.827
8. Does it refer to areas of uncertainty?3.2 (3.0-3.4)3.1 (3.0-3.3)3.2 (3.0-3.3)0.764-0.832
9. Does it describe how each treatment works?3.1 (2.8-3.4)3.1 (2.8-3.4)3.1 (2.8-3.4)0.859-0.874
10. Does it describe the benefits of each treatment?2.4 (2.2-2.7)2.4 (2.2-2.7)2.5 (2.2-2.8)0.782-0.841
11. Does it describe the risks of each treatment?2.3 (2.0-2.6)2.3 (2.0-2.6)2.3 (2.0-2.7)0.772-0.902
12. Does it describe what would happen if no treatment is used?1.5 (1.3-1.7)1.5 (1.3-1.7)1.5 (1.3-1.8)0.870-0.923
13. Does it describe how the treatment choices affect overall quality of life?2.0 (1.8-2.2)2.0 (1.8-2.2)2.1 (1.9-2.3)0.859-0.906
14. Is it clear that there may be more than one possible treatment choice?3.6 (3.4-3.9)3.6 (3.4-3.9)3.6 (3.4-3.8)0.773-0.849
15. Does it provide support for shared decision-making?2.7 (2.4-2.9)2.8 (2.6-3.0)2.8 (2.5-3.0)0.767-0.854
16. Based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices?2.9 (2.7-3.2)3.0 (2.7-3.2)2.9 (2.6-3.2)0.900-0.959
DISCUSSION

Several studies pointed to continuous progress from paper to electronic and online-based patient education[29,30]. The aims of the study were to evaluate the quality and accuracy of information available on IBD websites and calculate the readability level using two methods. To maximize the yield of the search, we searched three search engines commonly used by the public seeking information related to healthcare. The study showed that the 84 websites identified were created by universities, affiliated hospitals and research centers, professional foundations and associations, commercial and pharmaceutical companies created, charities and volunteers, as well as non-university educational bodies (such as colleges, academies, councils, and WebMed). The involvement of universities, affiliated hospitals, and research centers is directed at health information exchange as well as public and patient education with the aim to improve the quality of care, engage the patient in the decision-making processes and the journey of treatment as well as enhance patient's awareness about the nature of their illness. Such educational approaches while having multiple impacts on the patients' healthcare; it can also help in reducing the costs of treatment[31]. The current move from paper-based to online health care education may be related to the progressive increases in the use of the Internet by the public and patients[32]. Furthermore, Morgan et al[33] showed that patients with genetic and chronic diseases have great interest in participating in clinical studies and a desire to understand information discussed during reviewing their healthcare provider. These patients may have more questions after they leave the doctor's clinic and usually tend to search the Internet for answers[33]. Compared to paper-based health education, the Internet appears to provide a wider range of answers and options. However, the quality of information provided and the readability level remain as areas of concern[8,34].

As per this study, the DISCERN and the HONcode scores varied. However, no significant differences in the DISCERN scores were found between the groups but when the groups were compared on the basis of the HONcode scores, the difference was significant. A weak correlation was found between the DISCERN scores and the HONcode scores (R2 = 0.217). The results are consistent with the variability of the DISCERN scores of websites in each group and the fact that the two instruments are not measuring the same characteristics[25]. Interestingly, seven out of the top 10 websites on IBD scored higher on both the DISCERN and the HONcode scales. Looking into the readability levels of these seven websites, the readability using the Flesch-Kincaid Grade level was in the range 11 to 15, while for the Coleman-Liau Readability Index the range was 12 to 15. This indicates that even the top 7 websites had a readability level not adjusted to the public level.

Out of the 84 websites, only 17 displayed the HONcode certificate. A recent study found that only three websites out of 78 showed HONcode certificates[35]. Although the number of websites granted a HONcode certificate is small yet there is no correlation between the calculated HONcode scores and having a certificate on the website. Absence of the HONcode certificate from a website doesn't necessarily indicate poor quality of the website. This is because the process of issuing the HONcode is based on a voluntary application for the certificate. Therefore, it is possible that the owners/authority responsible for these websites did not apply for the HONcode certificate.

The readability scores were calculated by using two methods, the Flesch-Kincaid Grade Level Index and Coleman-Liau Readability Index. The moderate correlation between the Flesch-Kincaid Grade Level index scores and the Coleman-Liau Index scores is consistent with other work[20] and indicates that the results from the two calculations are consistent. The findings show that the majority of the studies had a readability level equivalent to year 11 and 12. However, the national reading grade level average has been estimated to be about the 6th-grade[36] and the general agreement is that the reading level for patient information materials should not exceed this level and be no less than what a 4th-grade is capable of reading[37]. With these findings in mind, there is a need for editing the content of most websites identified and adjusting the reading levels to meet the recommended reading levels for the public.

This study has a number of strengths; first, we searched three different search engines commonly used by the public seeking health-related information with the aim to maximize the yield of the search. Second, we used two instruments the DISCERN and the HONcode to measure the accuracy of contents. Both instruments have been widely used in assessing online health information material. Third, three evaluators independently conducted the evaluation and the inter-rater agreement among the assessors was within the accepted limits. Finally, the readability was measured by using two different methods. However, this study is not without limitations; the study is just a representation of websites identified at the time of the search. Only websites in the English language were included, and there is the possibility that there are other websites in other languages that match with our inclusion criteria and were not included. A multinational study may be needed to identify any differences if any and resolve gaps in this area. Therefore, despite all efforts and the plans considered, we may have missed some websites.

The results of this study may be of value to general practitioners, physicians, gastroenterologists, nurses, and allied health professionals, the public and medical students interested in online education material on IBD. The top 10 websites with the highest DISCERN and the HONcode scores identified from this study provide examples of educationally useful websites that can be recommended by treating physicians to their patients. However, their readability level was above the recommended level for the public and they may be suitable for educated patients only.

Future directions

This study highlights a number of future directions in research in the area of Internet-based patient education particularly patients with IBD. These can be summarized as follows: First, planning for creating online educational material for the public and patients with IBD necessitates more care for innovation, content accuracy and readability level to match the recommended needs of the public. Second, more work is needed to enhance the use of images, illustrations, and videos in improving the educational usefulness of websites on IBD and engage the patients and the public using such online resources. The use of these educational tools should aim at explaining difficult concepts, and enhancing understanding of the message given. As shown from this study the use of these educational tools was deficient in most websites. Third, future research should aim at assessing the impact of using Internet education and health literacy in patients with IBD and whether such resources have made impacts on number of hospital admissions, costs associated with poor health literacy, effective health education techniques, and how poor health literacy influences management outcome in these patients.

In conclusion, health literacy about IBD and the use of Internet as a medium for education appears to be increasing. Universities, research centers, commercial and pharmaceutical companies, professional foundations and associations were the major contributors to online resources written for the public and patients. Several deficiencies in the content were observed and most websites failed to meet the recommendations set by the National Institute of Health and American Medical Association that patients resources should be written about the 6th-grade level. Effective use of diagrams, illustrations, videos, and tables to explain difficult concepts should be encouraged. Revising the websites and resolving the gap between the readability of written health information and the literacy skills of the public will improve the purpose of these websites and make them a useful healthcare resource to patients with IBD.

ACKNOWLEDGMENTS

The authors would like to thank Dr. Sarah Azer of Box Hill hospital for her kind review of the manuscript.

COMMENTS
Background

Patients with inflammatory bowel disease (IBD), as it is the case with other chronic diseases, seek information about the nature of their disease. The increasing use of the Internet embraces the significance of online resources educating patients and the public.

Research frontiers

With the abundance of information there is concern about the quality, accuracy and readability level of information available on the web, thus it might be useful to assess the quality of these resources, identify specific deficiencies and examine whether these websites meet the recommendations of national bodies.

Innovations and breakthrough

The goal of this paper is to use comprehensive analysis to assess the quality of websites, accuracy of content and readability levels of websites on IBD dedicated to patients and the public.

Applications

The study highlights a number of future directions in the area of Internet-based patient education particularly patients with IBD and raises the need for improving such resources particularly in relation to specific areas identified in the study.

Terminology

Scientific accuracy and quality of content were evaluated using two standardised instruments widely used in research. The readability level was calculated on the bases of word length, sentence length and syllables.

Peer-review

This paper is an interesting evaluation of quality, accuracy, and readability of websites dedicated to the public. It is a novelty and represents a beginning point for judging and improving websites dedicated to IBD.

Footnotes

Manuscript source: Invited manuscript

Specialty type: Gastroenterology and hepatology

Country of origin: Saudi Arabia

Peer-review report classification

Grade A (Excellent): A

Grade B (Very good): 0

Grade C (Good): 0

Grade D (Fair): 0

Grade E (Poor): 0

P- Reviewer: Manguso F S- Editor: Gong ZM L- Editor: A E- Editor: Liu WX

References
1.  Stenke E, Hussey S. Ulcerative colitis: management in adults, children and young people (NICE Clinical Guideline CG166). Arch Dis Child Educ Pract Ed. 2014;99:194-197.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 14]  [Article Influence: 1.4]  [Reference Citation Analysis (0)]
2.  Mowat C, Cole A, Windsor A, Ahmad T, Arnott I, Driscoll R, Mitton S, Orchard T, Rutter M, Younge L. Guidelines for the management of inflammatory bowel disease in adults. Gut. 2011;60:571-607.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 933]  [Cited by in F6Publishing: 893]  [Article Influence: 68.7]  [Reference Citation Analysis (0)]
3.  Lichtenstein GR, Hanauer SB, Sandborn WJ; Practice Parameters Committee of American College of Gastroenterology. Management of Crohn's disease in adults. Am J Gastroenterol. 2009;104:465-483; quiz 464, 484.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 619]  [Cited by in F6Publishing: 618]  [Article Influence: 41.2]  [Reference Citation Analysis (0)]
4.  Burisch J, Munkholm P. The epidemiology of inflammatory bowel disease. Scand J Gastroenterol. 2015;50:942-951.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 225]  [Cited by in F6Publishing: 227]  [Article Influence: 25.2]  [Reference Citation Analysis (0)]
5.  Azer SA. Overview of molecular pathways in inflammatory bowel disease associated with colorectal cancer development. Eur J Gastroenterol Hepatol. 2013;25:271-281.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 57]  [Cited by in F6Publishing: 61]  [Article Influence: 5.5]  [Reference Citation Analysis (0)]
6.  Wasserman M, Baxter NN, Rosen B, Burnstein M, Halverson AL. Systematic review of internet patient information on colorectal cancer surgery. Dis Colon Rectum. 2014;57:64-69.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 62]  [Cited by in F6Publishing: 62]  [Article Influence: 6.2]  [Reference Citation Analysis (0)]
7.  Fox S Health Information Online. Washington, DC. Pew Internet & American Life Project. 2005;May 17 Available from: http://www.pewinternet.org/2005/05/17/health-information-online/.  [PubMed]  [DOI]  [Cited in This Article: ]
8.  Azer SA. Evaluation of gastroenterology and hepatology articles on Wikipedia: are they suitable as learning resources for medical students? Eur J Gastroenterol Hepatol. 2014;26:155-163.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 38]  [Cited by in F6Publishing: 40]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
9.   Available from: https://www.ebizmba.com/articles/search-engines.  [PubMed]  [DOI]  [Cited in This Article: ]
10.  Schreuders EH, Grobbee EJ, Kuipers EJ, Spaander MC, Veldhuyzen van Zanten SJ. Variable Quality and Readability of Patient-oriented Websites on Colorectal Cancer Screening. Clin Gastroenterol Hepatol. 2017;15:79-85.e3.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 21]  [Article Influence: 3.0]  [Reference Citation Analysis (0)]
11.  Grewal P, Alagaratnam S. The quality and readability of colorectal cancer information on the internet. Int J Surg. 2013;11:410-413.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 64]  [Cited by in F6Publishing: 73]  [Article Influence: 6.6]  [Reference Citation Analysis (0)]
12.  Yeung TM, D'Souza ND. Quality analysis of patient information on surgical treatment of haemorrhoids on the internet. Ann R Coll Surg Engl. 2013;95:341-344.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 19]  [Cited by in F6Publishing: 20]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
13.  Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53:105-111.  [PubMed]  [DOI]  [Cited in This Article: ]
14.  Charnock D. The DISCERN handbook. Quality criteria for consumer health information on treatment choices. Radcliffe Medical Press, Ltd, Oxon, United Kingdom.  Available from: http://www.discern.org.uk/discern.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
15.  Health on the net foundation (HON). Nongovernment organization.  Available from: https://www.healthonnet.org/HONcode/Conduct.html.  [PubMed]  [DOI]  [Cited in This Article: ]
16.  Commission of the European Communities. eEurope 2002 quality criteria for health related websites.  Available from: https://www.hon.ch/HONcode/Webmasters/HON_CCE_en.htm.  [PubMed]  [DOI]  [Cited in This Article: ]
17.  Certificate of compliance. Healthline network. HONcode.  Available from: http://www.healthonnet.org/HONcode/Conduct.html?HONConduct694855.  [PubMed]  [DOI]  [Cited in This Article: ]
18.  Boyer C, Dolamic L. Automated Detection of HONcode Website Conformity Compared to Manual Detection: An Evaluation. J Med Internet Res. 2015;17:e135.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 29]  [Cited by in F6Publishing: 25]  [Article Influence: 2.8]  [Reference Citation Analysis (0)]
19.  Nghiem AZ, Mahmoud Y, Som R. Evaluating the quality of internet information for breast cancer. Breast. 2016;25:34-37.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 44]  [Cited by in F6Publishing: 38]  [Article Influence: 4.2]  [Reference Citation Analysis (0)]
20.  Azer SA. Is Wikipedia a reliable learning resource for medical students? Evaluating respiratory topics. Adv Physiol Educ. 2015;39:5-14.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 34]  [Cited by in F6Publishing: 17]  [Article Influence: 1.9]  [Reference Citation Analysis (0)]
21.  Azer SA, Azer S. Bibliometric analysis of the top-cited gastroenterology and hepatology articles. BMJ Open. 2016;6:e009889.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 33]  [Cited by in F6Publishing: 32]  [Article Influence: 4.0]  [Reference Citation Analysis (0)]
22.  National Institutes of Health. How to Write Easy to Read Health Materials. National Library of Medicine Web site.  Available from: http://www.nlm.nih.gov/medlineplus/etr.html.  [PubMed]  [DOI]  [Cited in This Article: ]
23.  Kutner M, Greenberg E, Jin Y, Paulsen C.  The health literacy of America's adults: results from the 2003 National Assessment of Adult Literacy. Washington: Department of Education (US); 2006;  Available from: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006483.  [PubMed]  [DOI]  [Cited in This Article: ]
24.  Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006;33:352-373.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 305]  [Cited by in F6Publishing: 301]  [Article Influence: 16.7]  [Reference Citation Analysis (0)]
25.  Eltorai AE, Naqvi SS, Ghanian S, Eberson CP, Weiss AP, Born CT, Daniels AH. Readability of Invasive Procedure Consent Forms. Clin Transl Sci. 2015;8:830-833.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 45]  [Cited by in F6Publishing: 47]  [Article Influence: 5.2]  [Reference Citation Analysis (0)]
26.  Tian C, Champlin S, Mackert M, Lazard A, Agrawal D. Readability, suitability, and health content assessment of web-based patient education materials on colorectal cancer screening. Gastrointest Endosc. 2014;80:284-290.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 60]  [Cited by in F6Publishing: 67]  [Article Influence: 6.7]  [Reference Citation Analysis (0)]
27.  Azer SA, AlSwaidan NM, Alshwairikh LA, AlShammari JM. Accuracy and readability of cardiovascular entries on Wikipedia: are they reliable learning resources for medical students? BMJ Open. 2015;5:e008187.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 30]  [Cited by in F6Publishing: 29]  [Article Influence: 3.2]  [Reference Citation Analysis (0)]
28.  Powell JA, Darvell M, Gray JA. The doctor, the patient and the world-wide web: how the internet is changing healthcare. J R Soc Med. 2003;96:74-76.  [PubMed]  [DOI]  [Cited in This Article: ]
29.  Cline RJ, Haynes KM. Consumer health information seeking on the Internet: the state of the art. Health Educ Res. 2001;16:671-692.  [PubMed]  [DOI]  [Cited in This Article: ]
30.  Patient information forum Making the case for information. The evidence for investing in high quality health information for patients and the public. The full report. The patient information forum. May. 2013; Available from: http://www.pifonline.org.uk/wp-content/uploads/2013/05/PiF-full-report-FINAL-new.pdf.  [PubMed]  [DOI]  [Cited in This Article: ]
31.  Diaz JA, Griffith RA, Ng JJ, Reinert SE, Friedmann PD, Moulton AW. Patients' use of the Internet for medical information. J Gen Intern Med. 2002;17:180-185.  [PubMed]  [DOI]  [Cited in This Article: ]
32.  Morgan T, Schmidt J, Haakonsen C, Lewis J, Della Rocca M, Morrison S, Biesecker B, Kaphingst KA. Using the internet to seek information about genetic and rare diseases: a case study comparing data from 2006 and 2011. JMIR Res Protoc. 2014;3:e10.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 24]  [Cited by in F6Publishing: 25]  [Article Influence: 2.5]  [Reference Citation Analysis (0)]
33.  Shahar S, Shirley N, Noah SA. Quality and accuracy assessment of nutrition information on the Web for cancer prevention. Inform Health Soc Care. 2013;38:15-26.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 11]  [Cited by in F6Publishing: 12]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
34.  Wong LM, Yan H, Margel D, Fleshner NE. Urologists in cyberspace: A review of the quality of health information from American urologists' websites using three validated tools. Can Urol Assoc J. 2013;7:100-107.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 17]  [Cited by in F6Publishing: 19]  [Article Influence: 1.7]  [Reference Citation Analysis (0)]
35.  Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care. 2008;53:1310-1315.  [PubMed]  [DOI]  [Cited in This Article: ]
36.  Paasche-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med. 2003;348:721-726.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 447]  [Cited by in F6Publishing: 389]  [Article Influence: 18.5]  [Reference Citation Analysis (0)]