Drivers, enablers and risk mitigation: how does context influence the spread of disinformation, how can we better monitor, anticipate and prepare?

Hello, and a very warm welcome to this UNDP-UNESCO consultation on the globally important topic of disinformation.  I'm Louise Shaxson, Director of the Digital Societies Programme at ODI, a London-based think tank, and I'm delighted to be moderating the consultation with Simon Finley from UNDP for this week.  Next week we'll hand over the moderating job to other colleagues, but we will still remain very engaged.  As a reminder, the consultation runs for three weeks, so there's plenty of time to get involved in some fascinating and in-depth discussions with a truly worldwide group of people!  

When you comment, please let us know which particular question or questions you are responding to; it helps other people anchor themselves in the conversation. 

And now it's over to you - how would you answer the questions set out at the top of this page?

We look forward very much to hearing from you all,

Louise & Simon

Qn1: In my country, Uganda, the primary sources of disinformation are politicians and some people who are closely connected to the powers that be - both in political and technocratic circles. Often, the disinformation comes as a result of rumours of issues that are being discussed at the high levels of decision making but have not been concluded or sanctioned for dissemination. Insiders leak the half-baked information and as it does its rounds on various social media platforms, particularly WhatsApp and Facebook, it becomes distorted and sensationalised. And by implication, the motivations driving especially the sharing of disinformation include the apparent fun that comes from sensationalism, the desire to tarnish names of key personalities such as opposition politicians (by those in government) or high level technocrats (by their workplace rivalries) among others. It would be difficult to pin point the so called “super spreaders” of disinformation. It is important to note however that redundancy is a major problem. When may people, especially the youth lack employment, they have ample time to engage in unhelpful communication, including the spread of disinformation both on and offline. The extent to which disinformation becomes influential is dependent on the subject matter. Political propaganda against opposition politicians tends to negatively influence public opinions and often creates hostile sentiments and responses.

Disinformation has become quite a big issue in Ghana. With experience in social policy and development space there are about 3 main drivers of disinformation (i) Politics - politicians and their surrogates intentionally creates fake news to discredit their opponents or government interventions. Citizens then believe some of this fake news and its affect their confidence in government or otherwise. (ii) Bloggers - Many bloggers depends on followers to earn money and they believe fake and sensational news attracts followers. So they create fake news to gain followers and they get paid based on the number of followers they have. (iii) The advent of social media has also become a primary driver of disinformation. Posting and sharing information of social media is not that much regulated. And also the Ministry of Communication and the Ministry of Information have warned the public about fake news but they are unable to effectively regulate the use of social media and production of social media content.

Hello, everyone.
Disinformation and its implication are crucial for all societies, and their discussion should involve as many stakeholders as possible. This consultation is a concrete way to make the debate open and broad. Congratulations to UNDP/UNESCO for this initiative!.

I am the Human Development, Multidimensional Progress, and SDG Center Manager in UNDP-Venezuela. 

Regarding the QN1, the primary source of disinformation in Venezuela is the political confrontation. In our cases, political hyperpolarization affects almost every topic in social discussions and deepens social cohesion deterioration.  The traditional national media are more prudent about pollution information. But the social networks are the preferred field for disinformation disseminators. Hyperpartisan behavior has a strong correlation with disinformation in social networks.

It is not easy to think that the volumes of misinformation could reduce in this context. But  I believe a way to address de information pollution is to identifying and supporting trusted sources. 

Thanks Daniel! The political motivations and drivers for disinformation are extremely important and can sometimes get lost as people focus on how social media spreads polarizing information. If the political climate is a driver, does anyone have positive experiences in addressing this in their own context?

Hi Everyone, I'm wondering if you've tapped into the fact checking community? They have so much experience in this area.... I recommend you approach them if you can.

Thanks for your suggestion. We have identified the main communities, and this is very informative for disinformation purposes. We find out in this context that all users' communities have some kind of pollution information. But the super spreaders" of disinformation almost always belong to the hyperpartisan communities. The hyperpolarization drives disinformation.

1.En Colombia proviene principalmente de figuras políticas, estrategias organizadas con cuentas falsas que posicionan tendencias en Twitter, y estrategias organizadas en grupos de Whatsapp que difunden contenidos a grandes velocidades.

2. Medios de comunicación tradicionales o con larga trayectoria.

3. El desconocimiento de los métodos de verificación de información y la falta de casos ejemplares sobre las implicaciones de compartir desinformación.

6. El documento de UNESCO sobre desinfodemia  reúne varios ejemplos locales de casos e iniciativas: https://en.unesco.org/covid19/disinfodemic

Las organizaciones de fact-checking son fundamentales para que la sociedad civil y medios de comunicación tengan herramientas para combatir las acciones de desinformación.

Week One Summary

Hi all

Thanks for all your thoughtful comments - this is developing into a very rich discussion.  For my own benefit I tried to summarise what I think has been said, and thought I would share it with you - please let me know whether I'm on the right track!

Where does it come from?  Several factors give rise to it: political partisanship came through quite strongly in Rebecca, Edem and Daniel's points.  Disinformation helps this partisanship to turn into hyperpolarisation, as Ludwwin pointed out, when people actively strategise to spread disinformation at high speed.  But it's not just partisanship: the desire to achieve political influence is obviously important, but there's also a desire for sensationalism and excitement, particularly from people who are simply bored and who have time to 'stoke the rumour mill'.  Edem made the point that bloggers who rely on numbers of eyeballs for their revenue are more likely to write sensationalist content - we shouldn't forget that Facebook already knows that our brains are hardwired to be attracted to divisiveness and immediate experiences INSERT LINK.  And even as far back as 2009, we knew that social media was leading to 'an inability to emphathise and a shaky sense of identity" (see  https://www.theguardian.com/uk/2009/feb/24/social-networking-site-chang… - who remembers Bebo??).  

What is disinformation trying to achieve?   Polarisation, disempowerment, confusion and discrediting others all came through from the contributions - all of which came through in Yves' writing about the situation in DRC.  If the aim is disempowerment and confusion then it seems to me that what might be driving it is the combination of a 'shaky sense of identity' and an inability to emphathise with others.  Obviously the question of how strong your sense of identity is isn't just about being online - there are long histories of questioning the identity of marginalised groups in order to disempower them.  But the way that social media changes our ability to emphathise does seem to be a particular facet of the online world and I wonder whether there's a sort of flow of lack of ability to emphathise --> shaky sense of identity --> feeling of disempowerment.  And boredom probably doesn't help with our sense of identity either.  

Then I started looking for the factors that don't help slow the spread of disinfo (or actively encourage it to spread).  Politicians owning traditional media, existing tensions & conflicts, and the perception of being overlooked in decision making processes all stood out for me, and it was interesting that they all seemed to come from the offline space.  Which took me to Ben's point, and how he highlighted that this online/offline question hasn't been very well researched.

A lot for me to think about!  Do let me know what you think about my summary - it's just what I highlighted as I was reading the above.  And please bring your friends, colleagues, Twitter followers etc into the discussions - it would be great to hear from different parts of the world.  

From Chile, we would want to contribute with this reflections:

1.- In Chile, sharing false or biased information is associated with digital activism. One reason is that these groups tend to share more information on social networks. Other reason is the way information pollution works on social networks. When a certain information coincides with our worldviews, we tend to believe in it more easily since it confirms the visions and values that we previously hold. This is called confirmation bias.

3.- At least two factors can be mentioned. First, the lack of awareness about how social networks work and how the algorithms used by platforms such as twitter or Facebook filter and bias the information to which we are exposed based on our values, political views and interests. Second, there is no culture about checking the information to which we are exposed and there is little awareness of the magnitude of the circulating information that could be false.

4.- One of the main problems with information pollution in social networks is the way in which the algorithms of platforms such as Twitter segregate users. In general, users on social networks interact with other users based on common interests and worldviews, which results in ideologically and socially closed social networks. The existence of these informative "bubbles" makes it difficult to contrast information and views on different topics.

7.- Awareness and pedagogy campaigns. Build alliances that allow breaking the information bubble.

Hello! Welcome to Week 2 of the UNDP-UNESCO consultation on disinformation.  I'm Rachel Pollack, and I work in UNESCO's Section for Freedom of Expression and Safety of Journalists.

I'll be serving as moderator of Room 2 this week, taking over from Louise Shaxson and Simon Finley.

This Room covers topics surrounding the drivers, enablers and mitigation mechanisms of disinformation such as: the primary sources of disinformation; the factors that make the public vulnerable to it; the extent to which internet companies' algorithms push disinformation; and the role of various stakeholders in monitoring disinformation. 

Please answer any of the questions at the top of the page, whether just one or all seven. It will help if you can please indicate the question numbers in your response. Feel free to introduce yourself if you wish. 

Your contributions will help shape UNESCO and UNDP's actions to counter disinformation around the world.

Looking forward to your input!

Rachel

What are the factors that make the public vulnerable to disinformation? What are the key normative, technology, governance and social cohesion enablers of disinformation?
 

An interesting finding from MIT is that fake political news, aside from just getting consumed quickly, gets shared up to 3x as quickly as factual news. Moreover, "controlling for many factors, false news was 70% more likely to be retweeted than the truth." This suggests to me two things: 1) there's something very intrinsic to the content itself (wording/language, "novelty" in ideas, formatting and headlines) that draw people in; and/or 2) that the content has been targeted well to the individual, whether it's through alignment in socio-political beliefs or being involved in an active social network that drives the spread of misinformation via information cascades. Regarding Point 1, I would be super keen to see the use of Natural Language Processing via sentiment analysis to explore if similar linguistic patterns emerge in identifying misinformation. Regarding Point 2, this is what Busara has been largely occupied by in recent projects. 

At the center is the capacity of fake news to override System 1/deliberate thinking by drawing upon reflexive human biases. This includes limited attention, the need for group belonging/identification (which occurs on a spectrum but is quite tied to Louise's idea on a shaky sense of identity that needs to be validated), and the comparative advantage that mis/disinformation is oftentimes more evocative and/or novel than run-of-the-mill news (well, depending on what country we're talking about these days). 

A very interesting follow-up to the earlier discussion. I would say nowadays, it is very difficult to pinpoint one primary source or specific motivations driving the creation and sharing of disinformation (Q1). It very much depends not only on the country/community, but also very much on the topic, the medium and people most influential in relation to that particular topic. That said, however, I believe that social media is by most accounts a powerful 'super spreader' of disinformation, because quite simply anyone can share and sell information as facts that then grows and mushrooms into new shapes and forms. 

That leads me to Q2: nowadays, I would never trust any news or information shared on social media. Unfortunately, even the most neutral information sources today, such as BBC etc., do get influenced by the super spreading of false information. I therefore only trust the 'law of 3', i.e. if in neutral, more traditional mediums of news sharing I see the same piece of information or news being shared for more than 3x then I do classify it as credible. I do not, however, think there is one source of news or information sharing that can be trust nowadays as information has become increasingly fluid.

In relation to Q3, I wouldn't say that the public is vulnerable to disinformation in general. It very much depends on who is sharing that information - i.e. if you are a supporter of a particular political figure and he/she share a particular piece of news or information as a fact, you are simply more likely to believe it. In a sense, the name, credibility and your interest in some public figures can make you vulnerable to disinformation. In addition, you could also be more likely to believe disinformation if it supports your position – it is simply human nature, especially when you are not sufficiently informed about the topic. We have witnessed this quite strongly with the coronavirus pandemic. Ample disinformation has been shared as ‘facts’ that support different positions on the virus and hysteric super spreading of these different – and sometimes even contradictory ‘facts’ – has been shared around. This leads me to Q4.

In relation to the virus, algorithms did indeed play a role in the spread of disinformation. I do not know, however, to what extent. I think this differs by topic, company, community and country. But it no doubt amplifies the problem. I am, however, conflicted over Q5. In relation to this question, I am not entirely sure if monitoring works. Monitoring can be and is many cases bias or follows algorithms that might censor information that may not be necessarily inaccurate. So personally, I am not sure if monitoring is an effective solution – self-censorship combined with awareness raising tools/campaigns might be more effective in my opinion. In relation to the coronavirus, FB, for example, has been actively monitoring content for a while now and its policy hasn’t been effective. If anything, it has been counter-productive, as the more content is deleted and banned the more those who didn’t believe in the severity of virus, were convinced that there is some sort of conspiracy happening. I have witnessed this first hand within my own network – friends whose content has been deleted on FB ended up believing in some sort of conspiracy around the coronavirus.

I am not sure about Q6, but I definitely think that CSOs, governments and media and internet companies have a serious role to play in controlling the spread of disinformation (Q7). But this shouldn’t be about banning content, but rather about monitoring the flow of information and identifying drivers and enablers of disinformation and sharing more accurate and informed knowledge and information to combat disinformation. Banning content on one platform adds to the drivers of disinformation – as we have seen with the coronavirus pandemic. Also banned content finds an audience elsewhere. I think educating people, sharing accurate and informed information and re-directing drivers and enablers would help and might prove to be more effective.

Evening, everyone!

Just a few reflection from me on fake news in Ukraine.

In Ukraine, enablers of disinformation from Russia and other sources such as domestic TV channels and newspapers with ties to oligarchs include many factors, including exposure, political attitudes and receptivity to foreign media influence. While there was some development of independent media ecosystem, Ukraine and many other FSU countries continued consuming (and producing ) content from and for Russian (and Russian-speaking ) audiences. Such was (and still is) the case in many parts of Ukraine. In many cases people will continue consuming such content despite being aware that its biased. When confronted about false or misleading facts in the news sources some also dismiss it as false allegations by someone from the other end of political spectrum. I would add that understanding politics of disinformation in Ukraine requires getting acquainted with the interaction of domestic political and media landscapes, as media companies are owned by oligarchs and are generally perceived to be biased in favour/ against whoever that in-person seeks to support/ undermine. 

In Ukraine, civil society plays a crucial role in monitoring disinformation and providing training to journalists and other representatives of civil society to enhance early warning capacity. However, as is the case in many countries,  media literacy of the population is generally quite low and fact checking is limited to a small community of civil society organisations and engaged citizens. I should add that raising media literacy alone won't help tackle disinformation. In case of Ukraine (as is the case in many post-Soviet countries) trust in government and political institutions is weak, and in many cases people will trust official sources no more than other media channels. In the post-Soviet era, many countries of the former Soviet bloc, including Ukraine struggled with nation-building (precarious economic conditions, lack of unifying political leadership and ethnic divisions are some of the reasons for that). Lack of national unity and absence of impartial news sources, is also driving people to engage with media channels which often spreads false or misleading information creating siloed communities where polarising opinions can flourish. 

A few brief reflections from an analyst who works on fragility and conflict in South and Southeast Asia, but who lives in the United States.

• In your country/community, what are the primary sources and motivations driving the creation and sharing of disinformation? Who are the “super spreaders” of disinformation, those who have sufficient influence and following to amplify on and offline?

For the angry and aggrieved, social media are an immediate, unfiltered, and powerful tool of expression. Their views go unchallenged by facts, laws, or historical complexity. They blend into the daily flood of information online. It can be sometime before the extreme and often ill-informed viewpoints are seen and assessed as a threat by those who are in a position to counter or check them. This ability to be challenged by peers or members of their own community is an important first step towards compromise, developing a common understanding, or narrowing the gap between extreme viewpoints.

• What are considered trusted news sources among different groups? What are the criteria for trusting these sources?

I am not sure the traditional "fact-checking by media" model is working. There will never be enough resources. It is too slow and often deliberately discredited by those with a political interest in spreading disinformation.

My sense is that countering deliberate or misinformed disinformation, particularly that that prompts violent conflict, needs to be done at a much more local level by activating allies that have their own sources of community legitimacy. Community by community, however we define this, we need to understand who are the trusted sources of information, which is not always the same as a "news source".

• What are the factors that make the public vulnerable to disinformation? What are the key normative, technology, governance and social cohesion enablers of disinformation?

In some places, I see the distrust of and dysfunction in representative government as a distinct problem, especially when it is not representative. Weak relationships between communities and those who govern them aid disinformation. A lack of transparency in decision making does not help. Corruption undermines legitimacy. A lack of understanding of how and why decisions are made is part of this problem. The way decisions are made involving communities and communicated to them is important in countering those inclined to use disinformation as an unconscious or deliberate tool. These weak relationships undermine the ability of concerned or active citizens, who are a small group, to quickly go to the source to cross-check disinformation and potentially counter it. These active citizens are the go-to people for many others in a community as sources of information when a "hot" issue appears. Making direct source information more easily available or increasing its prominence online, including on social media, could be a way to counter-disinformation. These should not just be understood as mass tools, but tools for the key active citizens or activists to use.

• To what extent do Internet companies’ algorithms actively push disinformation, and what is the role of closed communications networks in amplifying the problem?

The algorithms reward anger, passion, and frenetic levels of activity. They are easily gamed. How do we add speed bumps into these social media tools to slow things down or slow down reaction time? How do we create a culture of "stop, think, check, before you respond" in communities of concern? How do we create or reinforce trusted people or places to check-in before responding? How do we reinforce or encourage placing more value on information from named, certified, or known sources versus the anonymous, pseudonymous, and unverified?

Hello all, and thanks for joining us for the third and final week of the UNDP-UNESCO consultation on disinformation. I’m Ruth Canagarajah and I currently work as a Senior Associate at the Busara Center for Behavioral Economics in Nairobi. 

I’ll be your moderator for Room 2 and given that it is our final week, I am especially keen to hear your remaining insights and experiences in relation to disinformation and its drivers, enablers, and risk mitigation approaches. Along with still fleshing out ideas on the guiding 7 questions found at the top of this page, I encourage you to interact with the other commenters on this forum. This is a wonderful time in our consultation process for more dynamic conversations and informal back-and-forths to take place on this hub for idea exchange. Additionally, I also encourage you to explore Rooms 1 and 3 for comments, questions, and cross-pollination.

I look forward to the last and final round of contributions, no matter how brief or in-depth the inputs!

Warm regards,

Ruth

Week Two Summary

Dear All,

Thank you for your thought-provoking contributions to the discussion in Room 2.

For those just joining, and as the moderation of Room 2 is skillfully taken up by Ruth, here are a few highlights from last week and over the weekend.

Ruth kicked off the conversation by sharing a study by MIT that found that false political news gets shared up to 3x as quickly as factual news. She pointed out that this reveals information about both the content itself and the way it targets specific audiences.

The sources and motivations driving the creation of disinformation vary significantly depending on the country/community, topic and medium, pointed out Dina. She stated that CSOs, government, media and internet companies have an important role to play in monitoring and sharing accurate and informed knowledge, and she warned that banning content has proven ineffective.

We received an insightful perspective on the situation in Ukraine from Olena. Civil society plays a crucial role in monitoring disinformation and providing training to journalists and others, Olena pointed out. She underlined the need for greater media and information literacy, while also pointing to challenges related to wider social and political divisions.

Jim reflected on the importance of community in countering disinformation, starting with challenging the false information spread by peers. It is also at the community level that trust in sources and greater transparency in political decision-making can happen, he noted. Jim raised a series of questions, including on how we can “create a culture of ‘stop, think, check, before you respond”?

A common thread across these contributions was the need for sharing reliable information and for fostering greater media and information literacy.

With great thanks again to all for these insightful contributions, I wish you an engaging discussion up ahead! Please do feel free to comment again and invite your colleagues and partners to join the discussion as well.

Every contribution—no matter the length, no matter if it addresses one question or all—will be valuable for shaping our collective work on countering disinformation.

Rachel

--

Week One Summary (Room 2) by Moderator Louise Shaxson.

Hello everyone, if you haven't seen this report in today's news, it is very relevant to what we are discussing here - how disinformation is combined with other methods of cyber warfare to destabilise.  It makes chilling reading. 

Separately, I've commented in Room 3 on a report that has come out in the UK that advocates for setting up an independent agency to monitor platforms (do go over there and have a look).  I wonder, though, how much that would achieve without a supporting structure that specifically looks at the issues of identity and community that we are discussing here.  And does it all come back to governance?  If, as Jim says, the traditional fact-checking model isn't working, is part of the problem that we haven't been nearly as imaginative as we need to be about how our governance structures need to change to handle our very rapidly shifting sense of identity and community?  Has anyone who's posted here had any experience of working with citizen's juries, for example?  Or with anything similar or more innovative?

I wanted to share a few thoughts based on my experience researching, primarily the use of WhatsApp, during recent elections in West Africa (Sierra Leone, Nigeria and The Gambia).

1. Given the nature of my work, it is perhaps not surprising that I see political actors as the main sharers of falsehoods. This extends to government officials and those who are affiliated with political actors and parties, that are not always formally part of the party, but who work with it to advance its electoral chances (with the aim of a political appointment if the party/candidate is successful).
2. Yes, a strong, existing, presence on all social media platforms is key (content is often shared on one platform - say Facebook - and then copy and pasted to Twitter and WhatsApp for example. In northern Nigeria several social commentators who had built up a following over several years online and offline were able to use, sell that captive audience to political bidders. I think here the idea of offline credibility can help with how information you share online is received (and vice-versa). Pastors and other religious/traditional leaders in Nigeria can be super-spreaders, as they are trusted arbiters of information among their congregation so if they take a piece of false information from online (deliberately or not) and share it with followers (offline), people will be much more inclined to believe it because of their standing in society.
3. The source [the person sharing it] of news, matters more in some cases, than the content itself on platforms like WhatsApp. I think there is some good Afrobarometer data on this for multiple African countries, which puts radio as the most trusted source of mainstream media information (and the most listened to) and this was backed up my some of the small survey work done in Nigeria, where radio remains a source to verify content. 
4. In Nigeria, there is a lack of trust in the government's ability to provide credible information and so that provides a space for falsehoods to flourish, and to be more easily believed, that align with existing biases and divisions. And the most effective electoral disinformation in Nigeria draws on/out these divisions. There is also a lack of digital/civic literacy among, particular older users, but also more generally, that means people struggle to discern what is true and what is false. And this is also limited by people's access to the internet. In Sierra Leone many young people I spoke with 'managed 25MB a day' which allowed them to use WhatsApp but not to download videos or PDFs or to fact-check away from the application. In many more rural parts of Nigeria this also applies. This also links to Facebook Basics and its potential issues in making its platforms the only source of online 'news' for some users. Others have researched/written about this more extensively - http://democracyinafrica.org/facebook_scramble_africa/
5. Certainly closed communication networks like WhatsApp can amplify the risks of disinformation. First by being more private they make it easier to people to hide their identity when creating content (people in Nigeria create WhatsApp groups of 2 or 3 people and share into it first, so that it comes out of the group as labelled 'forwarded' making it hard to know the original source). Even if you do know the source, all you know is their phone number, names and a profile are not required like on Facebook. Instead users often rely on who shares the information with them on WhatsApp as the intimacy of the platform lends itself to this. So they don't always look to fact-check or verify online (though many do), but will judge its veracity based on what they know and who shares the content with them/how many times they receive it.
6. This is difficult. A story can go viral on social media in 1-2 hours and its not always clear what that story will be, so early warning is quite difficult to do. Catching up with disinformation once it starts circulating is also difficult, something that limits the impact fact-checking can have (as by the time you do a through fact-check and present the findings, the disinformation can be everywhere online and already feeding in to people's bias. Efforts to flag accounts or groups that are known to have previously spread disinformation is one possibility, but when it comes to politics for example this will always lead to accusations of taking sides politically, for things like health disinformation its perhaps more plausible to identify accounts where disinformation is shared regularly and then communicate this information to citizens. 

Overall I think understanding the context in which disinformation is shared is fundamental to providing tailored responses that have shared principles but that are tailored to meet the specific reality of a country, or even states within a country. One thing that our Nigeria research (https://www.researchgate.net/publication/334736880_WHATSAPP_AND_NIGERIA…) showed was that localised pieces of disinformation (those focused on a state level event, rather than a national happening) were more likely to have been seen/engaged with. 

Just to add on the question of how government institutions can participate and monitor in the information landscape. In politically polarised environments its hard to establish and sustain credible independent bodies (how will they be appointed etc) and that's even before you discuss whether they should be. You can perhaps look at the model used for journalism as one possible solution, whereby members are represented by a body that sets standards they must adhere too, but this only can apply to bloggers or online influencers, as otherwise you'd have every citizen being a part of this association. I think the more viable shorter term solution is around building codes of conduct for online engagement that are enforced by other users. So with WhatsApp you empower group admins to be better able to control what is discussed and shared in a group (perhaps here the platforms can also do more to support this). Though this might be a lot of work for the group admins!

Generally I think social media platforms need to do a lot more in the African context in terms of getting content moderators who speak local languages. Hausa has c.50 million speakers, so even if 2% online that's 1 million people who might use the language to converse (Hausa Facebook very vibrant in Nigeria). But I don't know if Facebook or Twitter have many, if any. There are also questions about whether the user terms are available in these languages (https://www.vice.com/en/article/xg897a/hate-speech-on-facebook-is-pushi…) and if that's the case, users can argue they didn't sign up to the platforms rules for use (hypothetically). 

I think platforms, working with media and civil society, can also do more to educate users about features of their applications that might not be well known. For example on WhatsApp how to change your group settings so you aren't automatically added to groups but that you receive an invite and then decide if you want to join. 

HI Jamie, thanks so much for your detailed contribution.  I particularly like your comments about the inability of platforms to do effective content moderation in local languages, and the important point about whether user terms are available in those languages.  Something we haven't really talked about in this conversation yet is platform's business models, which came through in your point that "several social commentators who had built up a following over several years online and offline were able to use, sell that captive audience to political bidders."  The advertising industry says that 'money follows eyeballs', which combines with what we well know - that platforms encourage us to share content that generates strong emotions.  This isn't just an online thing - tabloid newspapers sell copy with their front page headlines that are designed to outrage - but as has been noted above it's the speed and spread on social media that's the problem.   But I wonder if anyone has a deeper analysis to share on platforms' business models?   

One of the things I was thinking about was that we need to be involving young people in this conversation (not just in this consultation but more generally) - or at least to be aware of their efforts.  Has anyone else come across this by MSc student Abbie Richards?  I've invited her to take part, it would be great to have her energy and enthusiasm.  Does anyone know of any other similar initiatives happening around the world?    

Hello everyone, I would like to answer questions 1, 2, 3, and 7 on behalf of the Global Disinformation Index (GDI),  a global not-for-profit organisation whose mission is to reduce disinformation and it’s harms. We do this by providing neutral, independent risk assessments of news sites based on their risk of carrying disinformation. Our risk assessments use a combination of human review and artificial intelligence.

One of the motivations for actors to create disinformation is financial: content that triggers strong negative emotions (hatred, greed, envy, etc.) tends to generate the most clicks. We have documented significant advertising from well-known brands–from Amazon to Volvo–which are giving a funding stream to sites peddling disinformation. Weekly updates of popular brands advertising beside disinformation can also be found at https://disinformationindex.org/research/

• GDI estimates that advertisers will unwittingly provide US$25 million to nearly 500 English-language coronavirus disinformation sites in 2020.
• The GDI has estimates that disinformation sites generate more than a quarter billion dollars per year in ad revenues. See: https://disinformationindex.org/wp-content/uploads/2019/09/GDI_Ad-tech_Report_Screen_AW16.pdf
• A recent study found nearly one in every three ad dollars could not be traced once they entered the ad tech system. https://www.isba.org.uk/media/2424/executive-summary-programmatic-supply-chain-transparency-study.pdf 

GDI has developed a methodology for assessing the disinformation-risk of news sites with the criteria of a domain’s content, operations, and context. The content score is based on an anonymised review of 10 of the top-shared articles on a domain that have been randomly selected. The review is done by a researcher and the source of the articles is not disclosed to them. The operations score is based upon the underlying policies, standards and rules that domains abide by to establish trust and reliability. The context score assesses the reputational practices, reliability, and trustworthiness of a news domain. These disinformation flags are assessed by an independent expert survey of respondents from across the political spectrum. More information can be found at: https://disinformationindex.org/wp-content/uploads/2019/12/GDI_Index-Methodology_Report_Dec2019.pdf

This criteria for trusting news sources has already been piloted in South Africa and the UK, with the findings available at: https://disinformationindex.org/research/

GDI’s technology has also uncovered some of the worst offenders in disinformation-519 sites in the English language which publish the highest volumes of divisive, polarising content. Over one-third of those 519 sites carry ads: 189 domains. Among this group GDI has identified a few sites that have “high narrative density”—these sites could be considered “super spreaders” because they carry the highest amount of disinforming content as a proportion of all their output. For more information on this see: https://disinformationindex.org/2020/10/how-can-advertisers-disrupt-disinformation-dont-fund-it/

Solutions:

The solution is to track and block advertising on disinformation sites to remove the financial incentives of spreading disinformation. Brands, ad tech companies and platforms can reduce traffic on coronavirus disinformation sites and help shift the focus to credible journalism by: 

1. Adopting a common disinformation risk rating for news sites in order to guarantee transparency for consumers and ad placement companies. 
2. Using real-time lists of high-risk disinformation sites to steer ads from disinformation content and to quality, trusted news sites. For example, GDI produces real-time lists of risk-rated domains for use by brands, ad exchanges, ad networks and brand safety companies.
3. Aligning brand and platform policies for high-risk content. There is a notable policy gap between content policies and ad policies for platforms and brands. This needs to be closed now — only they can do it.
4. Ensuring high-risk disinformation sites are not being promoted via algorithms on platforms. GDI’s risk ratings offer the ability for platforms to set ‘tolerance thresholds’ for which domains to promote in feeds and searches.

What are considered trusted news sources among different groups? What are the criteria for trusting these sources?

+ Recent research in Nairobi, Kenya by Busarai identified that trust in a particular platform is often driven by the level of trust in the source of information behind that platform. The communities we engaged with, in predominantly low-income, underserved neighbourhoods, perceive information delivered through TVs as more trustworthy than other platforms because of the perception that it's well researched, especially COVID-19 information. Additionally, participants that demonstrated low levels of trust in the Kenyan government as an entity, preferred not accessing information from sources that they believe the government uses to pass information to its citizens, including television

In your country/community, what are the primary sources and motivations driving the creation and sharing of disinformation? Who are the “super spreaders” of disinformation, those who have sufficient influence and following to amplify on and offline?

+ Older populations are less likely to regularly use digital platforms to access news, compared to the youth (<35 years old). Therefore in the instances where this demographic comes across information online, they’re more likely to not only strongly believe that piece of information, they’re more likely to widely disseminate within their networks. To a significant segment of this population, trust in information is driven by their high levels of trust in digital platforms such as Facebook, Twitter, online news sites, but especially WhatsApp. 

What are the factors that make the public vulnerable to disinformation? What are the key normative, technology, governance and social cohesion enablers of disinformation?

+ Any successful efforts to identify and flag misinformation depends on audiences’ ability to effectively and critically evaluate information they come across. It is however well established in behavioral science research that this process of critically evaluating information is “cognitively expensive", particularly on digital information platforms such as social media tools, where users are often bombarded by a barrage of information every time they refresh their home pages. This explains why it’s easier for most people to fall for false information, because the human mind can only process, hold and critically evaluate so much information in a short period. By understanding how different types of audiences evaluate information, stakeholders keen to address mis- and disinformation can begin to identify effective strategies and interventions to encourage audiences to consume and critically evaluate information better.

+ Finally, Busara recently conducted a small experimental study to answer the question: 

Are people better able to classify news as true or false if the subject takes place in their country of residence?

Find out more through this Medium post:

https://medium.com/busara-center-blog/off-the-record-15-warning-fake-ne…

One of the problems we have identified from Chequeado, based in Argentina, in the disinformation landscape is that there is very little research on who the actors behind it are, specially outside of the USA and Europe. This is why we are working with LatamChequea, a network of latinoamerican fact checkers, to do in depth investigation on who the actors are and what are the dynamics by which disinformation circulates in the countries and among different countries. We believe this will help better understand the phenomenon and allow us to better expose it and stop it. 

Monitoring social media platforms to identify disinformation and know how viral they are before we act on them takes effort and time. In Chequeado we work to make this work more efficient, and we, as other fact checkers, could greatly benefit from more tools that are specifically designed for fact checkers (since a lot of what we use is designed for marketing or other uses), and specifically tools that would give us more insight on specific groups that share a certain disinformation so we can focus on them when we debunk it, so the correct information can reach as many people who saw the disinformation in the first place. 

Week Three Summary

Dear all,

We had a week full of insightful comments and discussions, thanks to your inputs. Themes that emerged this week is in relation to the multiple “Hyra-headed” avenues in which disinformation is driven forward (i.e. the tension between offline versus online resources, and the difficulty of early warning systems to identify them before they gain traction); organizational methodologies and research insights from GDI, Busara, Farm Radio, and Chequeado in navigating the social, financial, and psychological drivers of disinformation, as well as some experiences with innovative fact-checking models; and youth-led typologies in defining the complex boundaries and modes of disinformation.

Louise kicked off last week’s content by sharing an article on disinformation + cyber warfare, and posing the need for more imaginative structures (especially those inclusive of issues of identity/community) to monitor platforms. Aligned with this, Aaron shared the work of Global Disinformation Index (GDI) which combines artificial intelligence and human review to reduce disinformation. One can learn more about their index scoring here and their broader methodology here but the GDI approach offers insights into both the financial drivers of disinformation and a process that starts from identifying disinformation all the way o down-ranking misinformation (in other words, promoting a closed cycle of activity). Moreover, Aaron’s insights give specific insight to platform business models and the financial “benefits” that divisive, polarising sources of disinformation accrue. 

In the theme of insights into super spreaders and how disinformation gets consumed, Jamie Hitchen gave context to his work in West Africa, specifically the offline, on-the-ground super spreaders of disinformation (pastors, religious leaders, social commenters), and how the the lack of digital/civic literacy, especially amongst older users, means a more pronounced struggle to discern what is true and false. Katie Burnham also related Farm Africa’s experiences with “trusted” sources of information that spread disinformation, specifically religious leaders, and how Farm Africa trains its radio broadcasters to identify and manage situations of on-the-air/live sharing of disinformation through trainings. Finally, Gideon Too of Busara also contributed the Kenyan dynamics of misinformation, showing that trust in the source of information differs by segments of society (i.e. poorer, under-served neighbourhoods think messages shared through TV is most trustworthy) and echoing that older generations with low digital/civic literacy are especially likely to strongly believe and widely share disinformation. Gideon’s inputs highlight the need to understand populations’ consumptions and behaviours on disinformation through a segmented lens (rural vs urban; young vs old; educated vs uneducated; perhaps even male vs female), instead of assuming the general population will react and engage with disinformation in the same way. Finally, Olivia and Matias of Chequeado explain how its group of fact checkers and use of specific tools is being used to identify disinformation before it spreads and increase the research base on who the primary spreaders of disinformation are. In the theme of early warnings, Jamie mentioned how difficult it can be for independent bodies to take on this position, especially in contexts of political polarisation, and the need to highlight more localised processes first. (i.e. group admins of WhatsApp groups and social media platforms). Language is an important consideration for effective content moderation.

Finally, Louise introduced an interesting “pyramid” typology of disinformation created by Abbie Richards, and another was found by some Harvard students in mapping disinformation on the axes of targets and motivations—both interesting ways of conceptualising “infodemics”.

With the extension of this consultation for one more week, I leave the moderator position in the hands of UNDP’s own Niamh Hanafin. We look forward to reading your remaining contributions!

Best,

Ruth

--

Week Two Summary (Room 2) by Moderator Rachel Pollack.
Week One Summary (Room 2) by Moderator Louise Shaxson.

Qs1. In my community, the main motivations that drive the potential creation and sharing of disinformation is the lack of verification by users, mainly in the field of social networks. Without a doubt, social networks have proven to be both a great disseminator of information and disinformation. The context of the health crisis has favored the spread of false news. Therefore, and as part of a proactive transparency strategy, strategies have been promoted for the generation, verification, publication and dissemination of quality public information, which allows citizens to reduce information asymmetries and optimize decision-making. In this sense, deepening the launch of similar actions could anticipate and better prepare ourselves for the spread of disinformation, since having timely information, in the current health emergency framework, counteracts the spread of disinformation.

hi everyone

Just a few things on specific questions: 

In your country/community, what are the primary sources and motivations driving the creation and sharing of disinformation? Who are the “super spreaders” of disinformation, those who have sufficient influence and following to amplify on and offline?
While we have not identified local sources of disinformation (ie. malintent), notable local politicians and celebrities have uncritically been content to repeat debunked misinformation. As evidenced elsewhere globally, it remains a challenge to persuade people to think critically in some cases of loyalty to an individual or a group.

What roles of civil society organisations, government, media and Internet companies play in terms of monitoring drivers and enablers of disinformation as a contribution towards social preparedness?
We had a positive experience working with a public health oriented CSO. In this partnership, we supplied fact checks and explainer articles that were disseminated to the CSO’s network of users—a segment of the community we may not have otherwise reached. The partnership maintained our editorial independence, while bringing mutual benefits.
 

This feedback is submitted by 10 organizations and civil society initiatives (GLOBSEC, nelez.cz, PSSI, CSD, Res Publica, Semantic Visions, Global Focus, Political Capital, Eastern Europe Studies Centre, DISI) from 6 European countries (Bulgaria, Hungary, Czechia, Lithuania, Slovakia and Romania) joined in the Alliance for Healthy Infosphere.

1. In your country/community, what are the primary sources and motivations driving the creation and sharing of disinformation? Who are the “super spreaders” of disinformation, those who have sufficient influence and following to amplify on and offline?

• Well-established disinformation outlets & influencers whose content is re-shared on social media
• Growing number of non-state actors motivated by either economic or ideological reasons use disinformation to achieve their goals
• Growing number of domestic political representatives, including representatives of far right or far left political parties, have been spreading various disinformation and hoaxes and voicing them during political debates. Thus, fringe disinformation has increasingly become part of the mainstream public debate
• Disinformation is peddled by websites imitating good quality journalistic outlets, often with unclear ownership and editorial board
• Disinformation is spread through manipulation of algorithmic processes on digital platforms. This represents a serious problem given that Facebook has become primary source of information for many young people in the V4 countries, according to latest NDI research (https://www.ndi.org/sites/default/files/NDI_Youth2020_V4.pdf )
• Economically motivated actors who profit from disinformation ecosystems generate vast incomes through advertising
• Medical disinformation channels, once a fringe aspect of the disinformation scene, have been catapulted to centre of information space and thus public debate
• Anti-mask movements and 5G conspiracy theories proponents cause radicalization and polarization, particularly at the time of the pandemic
• Report documenting political disinformation: Slovak Parliamentary Election 2020: Liberalism as a Threat, Facebook as a Battlefield report (https://www.globsec.org/publications/slovak-parliamentary-election-2020/ )
   

2. What are considered trusted news sources among different groups? What are the criteria for trusting these sources?

• This differs across different groups because social media have completely overhauled the traditional media model as news becomes accessed from digital platforms. Not only are reliable sources of information packaged in the same way as personal content from users and disinformation sources, the income generated from ad revenues no longer goes toward financing quality journalism – an essential pillar for any functional democracy.
• However, the following criteria would help: Easy access to information about media ownership to help the consumers identify bias of the given media and increase the transparency of the media landscape. Also, the obligation to publish the ownership information would help uncover the owners of the outlets and websites promoting harmful content, including disinformation and conspiracy theories, who often hide their identity. Revelation about media-ownership would help prevent the attempts to monopolise the media landscape.
• Good ranking would motivate the outlets to promote their evaluation on their websites / platforms. These labels should be enforced at search engines and social media platforms.
   

3. What are the factors that make the public vulnerable to disinformation? What are the key normative, technology, governance and social cohesion enablers of disinformation?

• Consumption of information from social media: Exposure to disinformation spread by manipulating algorithmic processes on online platforms
• Political actors deploying disinformation as part of their communication strategy with their key audiences
• Lack of digital skills and no notion of digital citizenship
• Lack of efficient strategic communication at all levels of the state
• Unregulated digital space
• Disinformation economy which turns disinformation into a profitable business

4. To what extent do Internet companies’ algorithms actively push disinformation, and what is the role of closed communications networks in amplifying the problem?
   • Disinformation is spread by manipulating algorithmic processes on online platforms to a great extent, as algorithms are designed to motivate the user to stay on the platform for as long as possible. Polarizing, hateful and manipulative content ensures this better than any other content.
   • Due to lack of transparency, it is difficult to evaluate the extent to which closed groups are responsible for disinformation dissemination. However, the likely probability is that it is another important factor contributing to information chaos.
      
5. What kind of monitoring can provide effective early warning of risks of potentially harmful disinformation?
   • Strengthened sharing of information and monitoring activity across borders and between authorities
   • Independent oversight bodies for digital platforms to investigate reporting irregularities
   • Enhanced reporting obligations on online advertising during electoral campaigns
   • Increased transparency measures adopted by digital platforms in the context of electoral campaigns and algorithms’ deployment
   • Monitoring of accounts which repeatedly violate community standards
      
6. What examples have you seen of disinformation mapping feeding effectively into national or local policy decisions, including institutional codes of conduct, and/or programmes?

• European Commission has addressed the spread of disinformation through a self-regulatory approach, which in a Code of Practice on Disinformation being subscribed by major online platforms and trade associations representing the advertising industry.
• However, Alliance for Healthy Infosphere’s stance is that even though Code of Practice on Disinformation (CoP) represents an important step in the right direction, number of evaluations conclude that it does not go far enough. Without common European approach to digital environment, member states are creating their own rules (NetzDG in Germany for example) which leads to further fragmentation of the Single market, while the problem with illegal content and disinformation online persists because of lack of regulatory oversight and transparency.
• Digital platforms operate internationally and hence best approach to form policy would be based on international alliances as well.
   

7. What roles of civil society organisations, government, media and Internet companies play in terms of monitoring drivers and enablers of disinformation as a contribution towards social preparedness?

• Social media platforms should regularly publish comprehensive databases about political advertisements, as well as reported and deleted content, including bots and fake accounts taken down in a particular country to provide governments and civil society with accurate information based on which policy recommendations and measures could be adopted
• National authorities often lack resources, expertise or access to tools which would enable them to effectively monitor in real time implementation of electoral legislation and take corrective measures. By pooling resources on EU level or internationally, sharing access to information, and adopting obligations for advertising service providers, national authorities would have a much better chance to actually implement these rules in the digital space. 
• Civil society plays an important role in shaping agenda in fact-checking, research, trainings, and media literacy programmes, activities which should be further supported. Resources needs to be dedicated to encouraging multi-stakeholder discussions to bridge the divide between citizens and elites and address the lack of trust thereof.
• Good quality journalism plays an important role in acting as an antidote to disinformation campaigns. However, journalists face intimidation and sometimes even bodily harm (investigative journalist Jan Kuciak and his fiancé were brutally murdered in Slovakia in 2018). Comments below the public profiles of journalists or media that publish the journalists’ articles on online platforms are full of hate speech directed at the journalists concerned. Whether they are trolls or real people, they often shape the discussion and divert the attention from the issue discussed in the article. Hate speech, threats and bullying continues in the private messages on online platforms or mailbox. Physical threats are present in the online space as well as offline space, as the case of Slovakia suggests.
• Social media have completely overhauled the traditional media model as news becomes accessed from digital platforms. Not only are reliable sources of information packaged in the same way as personal content from users and disinformation sources, the income generated from ad revenues no longer goes toward financing quality journalism – an essential pillar for any functional democracy. Digital platforms need to be meaningfully regulated and taxed. Income from such taxation should go towards new models of quality journalism funding. Media outlets ranked as good quality journalism by independent ranking institutions should not be packaged on digital platforms in the same way as all other content.

Dear all,

Thank you for this brilliant initiative and for developing this really engaging platform. I work at the International Committee of the Red Cross in Geneva and I conduct research in the area of violence against healthcare. Please find below my contribution to some questions:

3. What are the factors that make the public vulnerable to disinformation? What are the key normative, technology, governance and social cohesion enablers of disinformation?

Confusing, discordant and even conflicting directives from public authorities generate puzzlement among the public, who may therefore seek clarity through other sources, including unreliable ones. We are currently witnessing this phenomenon with regard to Covid-19 and health disinformation. Social media fulfill very well the public’s need for answers, as they speak the language of “emotions” and “gut feelings”, rather than of scientific evidence. This language appeals to the average social media user much more than scientific content featuring medical information that s/he could hardly grasp.

6. What kind of monitoring can provide effective early warning of risks of potentially harmful disinformation?

An “early-warning process” should be put in place. In particular, monitoring comments and posts on social media before disinformation spreads could help identify, understand and respond to people’s concerns and questions before a rumor develops to answer these questions. Social media offer a great opportunity to “take the temperature” among people on social, economic, political issues, but it is crucial to do it an early stage, preventing the rumor to develop or limiting its outreach.   

Hi there, I am the founder of Factmata, a startup developing tools to detect misinformation using the linguistic markers that define it. We think about this in 3 ways.

1. Algorithms to score how untrustworthy a piece of content is based on known signals that are highly correlated e.g. detecting clickbait language, hyperpartisan language, hateful content, one-sided content, agenda-driven content as well.
2. Algorithms to score how trustworthy a piece of content is via signals that denote credibility e.g. balanced arguments, reasoned tone, claims which do not contain farfetched connections between people, places and things (different to what is out there on the internet).

These two systems can then be combined to produce credibility scores for content, that then can aggregate to the website level or journalist level if needed.

Note, these algorithms may be imperfect but do not use anything apart from the content of what is being said to judge credibility (rather than judging the author or site, which we believe introduces a lack of fairness).

The way we want to implement this "credibility score for any piece of content" is the following:

1. Change how newsfeeds are ranked. Embed this signal into any newsfeeds which rank information for users, and deprioritise the focus on clicks/shares/likes
2. Remove harmful content via more effective content moderation of the grey area.

By using this supply ranking system you can improve how advertising networks are kept clean (preventing monetisation of misinformation by bad actors), improve social networks, publishing networks, news aggregators and much more.

The system needs to be built not by one company like us but many different players all providing signals (no one entity will have the capital to build such a sophisticated system well). These signals need to have open training and test data, be fair, and have clear visibility into who judged the credibility and their backgrounds and potential expertise in the subject they are labelling.