Mark Bell: Analysis of Wikipedia and Britannica
I’ve been talking with Mark and his work is fascinating, so I invited him to write a few guest posts for Using Wiki in Education. Here’s a bit about him: Mark Bell is a researcher, computer programmer and father. He specializes in social networks in virtual worlds and Wikipedia. He has just finished his Masters in Telecommunications at Ball State University and will be starting at Indiana University in the fall as part of the telecommunications PhD program.
Here’s his post:
A few folks have asked me to write something on my the thesis I just completed for my Masters in Telecommunication. It is entitled, “The Transformation of the Encyclopedia: A Textual Analysis and Comparison of The Encyclopædia Britannica and Wikipedia” This is a long title that really means, I did a textual analysis Encyclopædia Britannica (EB) and Wikipedia (WP). This essentially compares the expert-driven system of EB with the collectively-written WP.
Methods
For my thesis, I took four articles and compared them; two from EB and two from Wikipedia. With the help of my committee, I choose “Communism” and “Dwight D. Eisenhower”. Since I wanted to compare similar things, I used the online version of the EB articles. After downloading the articles, I removed any auxiliary text that was not part of the article and compared the EB article to the WP article. I wanted to see what differences there were between EB and WP at the basic building block levels. Once I had the articles cleaned up, I ran a number of readability tests, word count tests and grammatical comparisons. Readability tests count things like the number of words in sentences and syllables per word to get an idea of the grade level needed to read the text. I also compared numbers of nouns and verbs in each article and noun and verb phrases. I wanted to see if either encyclopedia had a more nominal (definitional) or verbal (function) nature to the language. I was also interested in how EB and WP constructed statements of facts and values. I thought in these areas I may find significant differences.
Results
After comparing EB and WP with these measures, I really found no significant difference. The readability scores were less than a grade level apart. The numbers of nouns and verbs were roughly the same and the articles had similar statements of facts and values. Now, I would like to say this shows the collective system of WP writes in the same way as the expert-driven EB but looking at so few articles my results are a bit tainted. For these particular articles, I can say the collective writes in a similar way as the EB authors an editors.
More Research
When the Nature (Giles, J. Internet encyclopedias go head to head. Nature, 438(7070), 900-901) article compared EB an WP science articles and found no significant difference in number of errors, EB replied with a whitepaper pointing out all the flaws they saw in the study. One of the things EB said that Nature didn’t look at was readability. Later this summer I want to center on a readability comparison of EB and WP. This time I want to compare 100 articles and have a more statistically sound set of results.
Acknowledgments
I want to thank Dr. James Chesebro, Dr. John Dailey and Dr. Michael Holmes for their invaluable help and guidance with my thesis.
