Alethiometer: a Framework for Assessing Trustworthiness and Content Validity in Social Media

 

@SNOW/WWW, 2014, by Eva Jaho, Efstratios Tzoannos, Aris Papadopoulos, Nikos Sarrisalethiometer1

Social media refers to the interaction among people in virtual communities and networks, powered by Web 2.0 technologies, during which they exchange news, ideas, and generally information. However, it is not easy to digest the massive amounts of information that the community is offering. Hundreds of new blogs are appearing every day, hundreds of thousands of pictures and videos are uploaded and millions of tweets are posted every minute. Validation of content or presentation of it in an objective manner is a crucial challenge, in order to avoid manipulations and guarantee the democratic role of the media.

We have developed the framework of a platform for assessing trustworthiness in one of the most popular social media, Twitter. “Alethiometer” derives from the Greek word Αλήθεια, which means truth. The principle of Alethiometer is determined around three axes: Contributor, Content and Context. The analysis of the validity of Contributor concerns parameters such as trust, reputation and influence of an information source. Content validity is expressed through parameters such as the language used, the history and possible manipulations performed on the content. And finally, Context analysis examines whether the ’what’, ’when’ and ’where’ of an online publication concur with each other. Joint analysis of the validity of Contributor, Content and Context provides a more thorough approach for revealing trustworthiness.

Existing methods on the veracity of social media content has focused on validating either the source of content or the content itself, but not these two aspects simultaneously. Furthermore, the analysis of the context of a post or article (publication date, place, etc.) and its coherence to the content itself can reveal mistakes that are often hidden in a well written text. Joint analysis of the validity of Contributor, Content and Context provides a more thorough approach for revealing truthfulness.

For the analysis of each framework category, we have defined a set of related parameters, which we term as modalities. Modalities concerning a contributor include the reputation, history of valid contributions, popularity, influence, and account validity. Modalities referring to the posted content include the importance and reputation of the contained web links, the content popularity, influence, its originality, authenticity and objectivity. Finally, analysis of context refers to cross-checking for similar reports in different social media, the coherence between the content and tags, attached links and multimedia, and the coherence between reference location/time and publication location/time.

For each item (post, tweet, etc.), modality parameters are rated on a discrete 5-point scale, from 0 to 4. A score for the significance of each parameter is also derived by comparing its value with the value of parameters of all other similar items. By combining the parameter scores with the significance of modality parameters, a single score is derived for each modality, which characterizes the quality of a contributor and of the content provided by that contributor.

A preliminary statistical analysis on a large corpus of Twitter data has been conducted, which showed that different parameters describing a user (no. of followers, no. of tweets, account age) exhibit a different behavior and are highly uncorrelated. For example, a ‘new’ user can have a large number of tweets/followers and vice-versa. The highest correlation found is between friends and followers, whereas the lowest between followers and tweets. All correlation values were however quite small, which means that these parameters are relatively independent from one another and have to be considered individually. These findings support our approach to examine many different parameters in order to evaluate social media content and contributors, and decide on their trustworthiness and validity.

This work has been supported by the European Commission under EU projects SocialSensor (FP7-ICT-2011-7-287975, http://www.socialsensor.eu) and REVEAL (FP7-ICT-2013-10-610928, http://www.revealproject.eu).

For more details, please consult our position paper:

– Eva Jaho, Efstratios Tzoannos, Aris Papadopoulos, Nikos Sarris, “Alethiometer: a framework for assessing trustworthiness and content validity in social media”, Second Workshop on Social News on the Web @ WWW ’14 (SNOW 2014), Seoul, Korea, April 2014.