Mapping Twitter Data

The following script queries Twitter and then produces a map of the results. It is adapted from https://opensource.com/article/17/6/collecting-and-mapping-twitter-data-using-r

Basics of searching Twitter with R

Searching Twitter with R is a straightforward process, but there are some considerations that can make things more difficult.

  • Twitter imposes rate limits on searches which impact the number of searches that can be conducted. Twitter can (and does) modify their terms which can change the rate limits. If you make too many requests, Twitter can blacklist your app which will mean that there won't be any search results. For the latest on the rate limits and strategies to avoid running afoul of them visit: https://developer.twitter.com/en/docs/basics/rate-limiting.html
  • There are multiple packages that can be used to search Twitter but the most popular one is twitteR and it is generally the easiest to use.

The first step is to load the required packages. We will be using:

  • twitteR which will do the Twitter searching
  • leaflet which will help make the map by providing background tiles
  • maps to help do the processing of the data
  • magrittr is a package that facilitates data cleaning and munging

It is also important to define variables for key paramaters from the Twitter API. When a developer account is set up, several tokens and keys will be generated.

These should be considered secret and not included in code that is shared!

Replace the parameters in the script with the keys and tokens for your developer account.

In [ ]:
#Install the required packages
install.packages("twitteR")
install.packages("leaflet") 
install.packages("maps")
install.packages("magrittr")
library(leaflet)
library(maps)
library (magrittr)

require(twitteR)

#Parameters for the request to Twitter
reqURL <- "https://api.twitter.com/oauth/request_token"
accessURL <- "https://api.twitter.com/oauth/access_token"
accessToken <- "YOUR ACCESS TOKEN GOES HERE"
accessTokenSecret <-"YOUR ACCESS TOKEN SECRET GOES HERE"
authURL <- "https://api.twitter.com/oauth/authorize"
apiKey <- "YOUR API KEY GOES HERE"
apiSecret <- "YOUR API SECRET GOES HERE"
CUSTOMER_KEY <-"YOUR CUSTOMER KEY GOES HERE"
CUSTOMER_SECRET <-"YOUR CUSTOMER SECRET GOES HERE"

setup_twitter_oauth(CUSTOMER_KEY, CUSTOMER_SECRET, accessToken, accessTokenSecret)

The next step is to query Twitter using the searchTwitter function and then produce the map. A full description of the searchTwitter function including the available arguments can be found at: https://www.rdocumentation.org/packages/twitteR/versions/1.1.9/topics/searchTwitter

In [35]:
# Search Twitter
tweets <- searchTwitter("#AIDS OR #HIV", n = 500, lang = "en")
#Put the results in a data frame
tweets.df <-twListToDF(tweets)

There is a lot of information returned from the search including things such as how many followers the sender has, how frequently it was favorited and lat/long. Before being able to map the tweets, it is necessary to convert them from string values to numeric.
Note: Only a small percentage of users include geotags in their tweets so there may be many results which can't be mapped.

In [43]:
# Change the lat/long values in the twitter results to numeric
tweets.map <- transform(tweets.df, 
                        latitude=as.numeric(latitude), 
                        longitude=as.numeric(longitude))

# Eliminate the ones without a geocode
tweets.map <- tweets.map[-which(is.na(tweets.map$longitude)),]

If you'd like you can output the results into a .csv file

In [33]:
#Uncomment the following line to write them into a CSV file for additional analysis
#write.csv(tweets.df, file ="YOUR PATH HERE")

The next step is to use leaflet to produce a map.
Leaflet is a set of open-source JavaScript Libraries to produce interactive maps. A full overview can be found at: https://rstudio.github.io/leaflet/

In [ ]:
m <- leaflet(tweets.map) %>% addTiles() 
m %>% addCircles(lng = ~longitude, lat = ~latitude, 
                 popup = m$type, weight = 8, radius = 40, 
                 color = "#fb3004", stroke = TRUE, fillOpacity = 0.8)

Here is an image of a map showing one of the results from the search. The leaflet code can be adapted to produce an html version of the map that can be included in a website.

alt text