DESCRIPTORS AND MEASUREMENTS OF VERBAL VIOLENCE IN TWEETS Collective Action and Social Media Lab Illinois Institute of Technology http://www.casmlab.org APA (6th Edition) CITATION: Guberman, J. and Hemphill, L. (2016) Descriptors and measurements of verbal violence in tweets [data file and codebook]. doi: 10.6084/m9.figshare.3179368. SPONSOR: This dataset was developed with support from the National Science Foundation Award # 1525662. SUMMARY: The purpose of this data collection was to test a scale for detecting verbal violence in Tweets. Workers at Mechanical Turk were first asked to complete a qualification test and then invited to code additional Tweets according to our scale. The qualification test involved a detailed explanation of each item of the scale, a walkthrough of a tweet that we had coded according to all 14 scale-items, a practice exercise, and a test. In the practice exercise, potential coders attempted to code a tweet on their own using our scale. After submitting their ratings, they were shown our own ratings for the same tweet and explanations for each of our ratings. The test component consisted of another coding task, in which coders were asked to code another tweet that we had already coded ourselves. The workers who, on test, with our ratings of that tweet on at least 11 out of the 14 items “passed” the test, earning the qualification that allowed them to participate in future coding tasks. Variables in the data include the ID of the Tweet (so that you may find it on Twitter; Twitter Terms of Service prohibit us from sharing the Tweets), the ID number we assigned to the coder, the rating that coder provided for each of the 14 items on our scale, the gender and age of the coder, and any comments the coder provided. DATA FORMAT: Excel CODER INSTRUCTIONS: Here are the instructions coders received: The purpose of this HIT is to aid in the development of a scale to measure aggression on Twitter. Each assignment will require you to look at a Twitter post (a tweet) and rate the tweet according to a 14 item scale. Each scale item will be rated on a 3-point Likert style scale, where 1 = “Uncharacteristic of this Tweet” and 3 = “Characteristic of this tweet." It should take no more than 5 minutes to complete each tweet. If and when you encounter a tweet containing a link, it is advised that you copy and paste that link into a separate tab or window of your web browser. Exploring these links will help add context to the tweet and may in If you are unsure how to interpret a tweet, it may help to view it in its original online form. In the event that a given tweet is part of a larger Twitter conversation, viewing it online will expose this conversation. To do this, click the link directly below the tweet. Again, in the event of a dead link, please rate the tweet to the best of your ability based on the text in the tweet alone. If after doing this you are still unsure how to code an item, you may select "I'm Not Sure." SCALE VALUES: Each tweet was coded as one of the following: 1 = Not Characteristic 2 = I'm not sure 3 = Characteristic SCALE ITEMS: The 14 scale items and the instructions coders received for each item are below: # Item 1 User threatens physical violence as a means of protecting the user's rights. An example of such a threat might appear as follows: "If they try to take away my guns/stop me from speaking my mind/tell me I can’t pray to [insert deity]/etcetera, then I’ll shoot them." 2 User threatens other people or groups of people with physical harm and/or sexual violence. With no indication that the threats are a means to protect the user's rights. If it appears that a threat is made and that the threat is either (a) unprovoked or (b) not an attempt to protect the user's rights (i.e. right to bear arms, free-speech, religious freedom), mark this as 'Characteristic'. 3 User openly expresses disagreement. Expressed disagreement can be with another person, a group of people, an institution, or an idea. If you’re unsure whether a user is expressing disagreement, it might be helpful to follow the provided link and see if the tweet is part of a larger conversation. If the link is dead and you’re unsure whether the user is expressing disagreement, please answer ‘Not Sure’. 4 User shares personal opinions of people, groups, or institutions that the user disfavors. If you believe an opinion shared in this tweet was intended to be negative, please answer 'Characteristic'. 5 User engages in or attempts to start arguments with people, groups, or social movements that the user disagrees with. If it is not obvious whether or not the user is expressing disagreement, it may be inferred by whether the user is engaging in an argument. To determine whether the user is engaging in an argument, it may be helpful to follow the provided link and see if the tweet is part of a larger conversation. To determine whether the user is trying to start an argument, consider whether the post seems to be intentionally inflammatory. If the link is dead and you’re unsure how to respond to this item, you may answer ‘Not Sure’. 6 User tweets potentially reputation damaging rumors about something else If anything in the post seems libelous, answer this as 'Characteristic'. 7 User tweets non-physical threats or threatening reactions to or about someone. This item concerns threats that are unrelated to bodily harm or sexual violence. Qualifying threats include (but are not limited too) threatening to disclose another person's personally identifiable information, threatening someone with a lawsuit, and threatening to fire someone or to get someone fired. 8 User shares potentially embarrassing photos or videos of someone else. Use your best judgment on this one. If the post contains a link, check the link to see if it leads to potentially embarrassing material. 9 User shares someone else's personally identifiable information. Examples of personally identifiable information include (but are not limited to) drivers licenses, passports, home addresses, work addresses, email addresses, phone numbers, social security numbers, information about someone's children or significant other. 10 User attacks the credibility of another person or group of people in an attempt to invalidate the other party's stance or argument. 11 User writes retaliatory comments in response to another person or group's words or actions When responding to this item, it may be helpful to follow the provided link to find out whether this tweet is part of a conversation or thread. 12 User expresses feelings that user or a group that user belongs to is being treated unfairly Expressed disagreement does not necessarily indicate feelings of unfair treatment. Look for words like "low blow," "unfair," "unjust," "biased," "discriminatory," or any variations thereof. 13 User expresses feelings of being misrepresented and/or under-represented by other people, groups of people, the media, etcetera. Look for whether or not the user is disagreeing with how others are representing the user. If the user is responding to and disagreeing with a statement that casts the user in a certain light, mark this as 'Characteristic'. 14 User defends user's self or user's image, or the image of a group that the user belongs to or associates with. Regardless of whether or not the user's image has actually been questioned, trivialized, or threatened. For example, if the user asserts that the user or a group the user belongs to is something or someway, thus reinforcing the user's preferred self/group-image, mark this as 'Characteristic'. You may be able to infer belongingness or association from the hashtags used, if not from the text alone.