Getting Tweeters

Introduction

Why get tweeters?
- Get network characteristics
- Get user demographics
- Linked survey designs

Introduction

Before that: getting data outside the API
- Datasets shared online, e.g.,:

Hydrating

tweet_IDs <- readRDS("data/wm_IDs_samp.rds")
head(tweet_IDs)

[1] "823360655835168640" "822727580923154432" "822953780127862784"
[4] "823136522199253120" "821584199949942784" "822868286979342208"

library(academictwitteR)

hydrated_tweets <- hydrate_tweets(tweet_IDs, errors = T,
                                  data_path = "data/hydrated_tweets/")

Batch 1 out of 10 : ids 823360655835168640 to 822916951878103040 
Total  78  tweet(s) can't be retrieved.
Total of  22  out of  1000  tweet(s) retrieved.
Batch 2 out of 10 : ids 823501383282241408 to 822817264915378048 
Total  154  tweet(s) can't be retrieved.
Total of  46  out of  1000  tweet(s) retrieved.
Batch 3 out of 10 : ids 823087701117337600 to 823227815374061440 
Total  225  tweet(s) can't be retrieved.
Total of  75  out of  1000  tweet(s) retrieved.
Batch 4 out of 10 : ids 823045907264499712 to 825034383769767808 
Total  301  tweet(s) can't be retrieved.
Total of  99  out of  1000  tweet(s) retrieved.
Batch 5 out of 10 : ids 822984540494950400 to 822996642618101760 
Error in make_query(url = endpoint_url, params = params, bearer_token = bearer_token,  : 
  something went wrong. Status code: 400

Hydrating

Alternatively:

Use the hydrator tool from DocTheNow

Getting tweeters

library(academictwitteR)
library(dplyr)
library(lubridate)
library(ggplot2)


cjb_ID <- get_user_id("cbarrie")
cjb_ID

   cbarrie 
"95226101"

Getting whom I follow (friends)

IDs
User data

userfwing <- get_user_following(cjb_ID)

ids <- userfwing$id

head(ids)

[1] "1451947528333762562" "326311946"           "2458351124"         
[4] "29943199"            "124852105"           "111456333"

id	url	created_at	profile_image_url	public_metrics	verified	username	description	entities	name	location	protected	pinned_tweet_id	from_id
2772818466	http://t.co/dcXO3BFFKi	2014-08-27T09:56:27.000Z	https://pbs.twimg.com/profile_images/530648659751211009/njUakWs1_normal.png	5535	FALSE	ResearchProfes	Provides a database of funding opportunities, news (@ResFortnight @ResearchEurope @Funding_Insight @HE_Analyst) & insight to researchers Part of @exlibrisgroup	0, 22, http://t.co/dcXO3BFFKi, http://www.researchprofessional.com, researchprofessional.com	ResearchProfessional	London, England	FALSE	1260177908083109888	95226101
2777000013	https://t.co/x9nsgWxDpd	2014-09-19T20:33:18.000Z	https://pbs.twimg.com/profile_images/1309528585703747585/C-rQRr3W_normal.jpg	12252	FALSE	funnylad5	Ladhood, Pls Like, Sheeps. (he/him) PLEASE buy my book: https://t.co/5pcyN0h8N6…	0, 23, https://t.co/x9nsgWxDpd, http://liamwilliams.uk, liamwilliams.uk	Liam Williams	NA	FALSE	1400417623347712002	95226101
281440041	https://t.co/4QrjaFCVui	2011-04-13T09:00:23.000Z	https://pbs.twimg.com/profile_images/1390702571812184064/v6s0uN5n_normal.jpg	31273	FALSE	_alice_evans	📚 THE GREAT GENDER DIVERGENCE ⚖ Why have all societies become more gender equal? 🌏 Why are some more gender equal than others? 💼 @KingsCollegeLon @Harvard	0, 23, https://t.co/4QrjaFCVui, http://www.draliceevans.com, draliceevans.com	Alice Evans	Podcast: “ROCKING OUR PRIORS”	FALSE	1467170791930449931	95226101
76970560	https://t.co/Z9EExQKt3M	2009-09-24T15:49:49.000Z	https://pbs.twimg.com/profile_images/1280821897253486593/brwCk9nw_normal.jpg	175580	FALSE	WisdenCricket	The independent voice of cricket. For more from Wisden check out @WisdenAlmanack and @WisdenIndia.	0, 23, https://t.co/Z9EExQKt3M, http://wisden.com, wisden.com	Wisden	The Oval, London	FALSE	1508514302478663685	95226101
134174402		2010-04-17T17:09:10.000Z	https://pbs.twimg.com/profile_images/1268527041458118656/5DQvd6HC_normal.jpg	1471	FALSE	Ew_Luger	Director of Research Innovation @eca_edinburgh; Deputy Director of Research @UoE_EFI; Fellow @turinginst; Into HDI, Policy & Ethics (views own, obv)	NULL	Dr. Ewa Luger	Edinburgh, Scotland	FALSE	1493614270201872385	95226101
1961901686		2013-10-15T04:37:29.000Z	https://pbs.twimg.com/profile_images/378800000598885799/97638c2cb18b2471207e15471c7adc8d_normal.jpeg	3075	FALSE	ECSR_Soc	European Consortium for Sociological Research. Conferences, summer schools, and the European Sociological Review https://t.co/pRHBNIJmm1	NULL	ECSR Sociology	NA	FALSE	NA	95226101
634782060	https://t.co/VTi07uBQPx	2012-07-13T16:44:46.000Z	https://pbs.twimg.com/profile_images/987824505010634753/lNAot9ww_normal.jpg	4271	FALSE	heshan_cheri	Coder, Data artist, Architect at large. Director of Engineering @Foursquare, Co-Founder of @Unfoldedinc, Creator of #kepergl, Ex-@Uber DataViz	0, 23, https://t.co/VTi07uBQPx, http://unfolded.ai, unfolded.ai	Shan He	San Francisco	FALSE	1349438469995601922	95226101
774568999	https://t.co/sRpEAaKdl3	2012-08-22T21:50:35.000Z	https://pbs.twimg.com/profile_images/1453111572759465990/T4h4HHIZ_normal.jpg	378	FALSE	DiegoMartin979		0, 23, https://t.co/sRpEAaKdl3, https://sites.google.com/view/diegoamartin/home, sites.google.com/view/diegoamar…	Diego Martin	NA	FALSE	NA	95226101
253791979	https://t.co/P8bW0X8ciW	2011-02-17T23:58:19.000Z	https://pbs.twimg.com/profile_images/507322253776650240/TyAxiMi6_normal.jpeg	5366	FALSE	daattali	Founder & Lead R-Shiny Consultant @ AttaliTech Ltd. #rstats blogger, tool builder, extreme traveller. MSc in Bioinformatics, Bachelor of Computer Science.	0, 23, https://t.co/P8bW0X8ciW, https://AttaliTech.com, AttaliTech.com	Dean Attali	Toronto	FALSE	969595742657511424	95226101
3029938271	https://t.co/71mKBmqsuo	2015-02-11T09:38:48.000Z	https://pbs.twimg.com/profile_images/1109143993588289536/EZk66Mb-_normal.png	941	FALSE	AvitalLivny	Asst Professor @IllinoisPolSci @Illinois_Alma (she/her/hers) Turkey & MENA; ethnicity & religion; measurement "Trust & the Islamic Advantage" (CUP) now out!	0, 23, https://t.co/71mKBmqsuo, http://www.alivny.com, alivny.com	Avital Livny	Urbana, IL	FALSE	1435613559518543874	95226101
988588884	https://t.co/W4XSwsC9bb	2012-12-04T11:37:54.000Z	https://pbs.twimg.com/profile_images/3785200321/e37c72b211bbd05524acf45316c84f85_normal.jpeg	4087	FALSE	McEwen_Nicola	Prof of Territorial Politics @EdinburghUni, @CCC_Research @Edinburgh_PIR. Senior Fellow @ukandeu. Retweets etc probably just my way of finding stuff later	0, 23, https://t.co/W4XSwsC9bb, http://centreonconstitutionalchange.ac.uk, centreonconstitutionalchange.ac.uk	Nicola McEwen	Edinburgh and Falkirk	FALSE	NA	95226101
2592605892		2014-06-28T06:46:12.000Z	https://pbs.twimg.com/profile_images/1509512361899696131/xtzjFVzI_normal.jpg	1525	FALSE	AndiHaxhiu1	PhD candidate and Tutor at @EdinburghUni. Research focus on the Museum’s role in divided societies.	NULL	Andi Haxhiu 🏀⚽️📚🇽🇰🏴󠁧󠁢󠁳󠁣󠁴󠁿	Edinburgh, Scotland	FALSE	1507142652579528707	95226101
132039766		2010-04-12T03:23:46.000Z	https://pbs.twimg.com/profile_images/3741515492/520509a30f638362be75ef1e884c9d99_normal.jpeg	390	FALSE	meganicka	UX Researcher \| Passionate about tech and human rights, data security and privacy	NULL	Megan Metzger	Palo Alto, CA	FALSE	NA	95226101
606433108	https://t.co/6TD3gfChZC	2012-06-12T16:08:06.000Z	https://pbs.twimg.com/profile_images/1249385075298500616/5Ph1f1HZ_normal.jpg	774	FALSE	ChantalBerman	I write about protest, mining, political economy, and state violence in SWANA. Asst. Prof @GUGovt, fellow @HarvardWCFIA. 🇫🇷/🇺🇸. She/her. Rare tweets.	0, 23, https://t.co/6TD3gfChZC, http://www.chantalberman.com/, chantalberman.com	Dr. Chantal	Washington DC	FALSE	NA	95226101
274991683		2011-03-31T12:29:00.000Z	https://pbs.twimg.com/profile_images/1478359914301038592/vnFus_WU_normal.jpg	76524	FALSE	josephfcox	journalist on hackers/crime/privacy for motherboard. Signal: +44 20 8133 5190. Wickr: josephcox. Email: joseph.cox@vice.com	NULL	Joseph Cox	NA	FALSE	1319270206347173888	95226101
585482512	https://t.co/28Rc5hhGfI	2012-05-20T08:21:29.000Z	https://pbs.twimg.com/profile_images/1221578584827015168/Hkyy8pna_normal.jpg	25331	FALSE	RivkahBrown	Commissioning Editor & Reporter @novaramedia // Editor @_VashtiMedia //	0, 23, https://t.co/28Rc5hhGfI, http://rivkahbrown.com/, rivkahbrown.com	Rivkah Brown	rivkah@novaramedia.com	FALSE	NA	95226101
2359610046	https://t.co/PFP2VaZgx1	2014-02-24T14:48:35.000Z	https://pbs.twimg.com/profile_images/1225726255238172672/tC1cdGsu_normal.jpg	897	FALSE	ArunFrey	Postdoc @OxfordDemSci @NuffieldCollege researching migration, conflict, and inequality. Causal inference and computational social science \| Here for the tl;drs.	0, 23, https://t.co/PFP2VaZgx1, https://arunfrey.github.io, arunfrey.github.io	Arun Frey	Oxford, England	FALSE	1504742742592827394	95226101
3096613518		2015-03-18T19:15:38.000Z	https://pbs.twimg.com/profile_images/1435628618504355848/0ZJZPq2N_normal.jpg	2631	FALSE	PKellstedt	Professor of Political Science @TAMU; Editor in Chief of @PSRMJournal. Views are my own.	NULL	Paul Kellstedt	College Station, TX	FALSE	NA	95226101
3252834682	https://t.co/M4daUI190f	2015-05-13T21:56:27.000Z	https://pbs.twimg.com/profile_images/1385176789153161219/GqY0v3IX_normal.jpg	520	FALSE	MatthiasDilling	Politics faculty @PCS_Swansea & associate member @Politics_Oxford. PhD @NuffieldCollege. Tweeting on German and European politics and political parties	0, 23, https://t.co/M4daUI190f, https://matthiasdilling.com/, matthiasdilling.com	Matthias Dilling	NA	FALSE	NA	95226101
2905142649	https://t.co/7mHXdwyffc	2014-12-04T15:11:31.000Z	https://pbs.twimg.com/profile_images/1460178909811589121/LURc_CTg_normal.jpg	12115	FALSE	FawazGerges	Professor of International Relations at the London School of Economics. @LSEIRDept	0, 23, https://t.co/7mHXdwyffc, http://fgerges.net/, fgerges.net	Fawaz Gerges	London, UK	FALSE	NA	95226101

User-level inference

Example from here

from m3inference import M3Inference
import pprint
m3 = M3Inference() # see docstring for details
pred = m3.infer('./test/data_resized.jsonl') # also see docstring for details
pprint.pprint(pred)

OrderedDict([('720389270335135745',
              {'age': {'19-29': 0.1546,
                       '30-39': 0.114,
                       '<=18': 0.0481,
                       '>=40': 0.6833},
               'gender': {'female': 0.0066, 'male': 0.9934},
               'org': {'is-org': 0.7508, 'non-org': 0.2492}}),
             ('21447363',
              {'age': {'19-29': 0.0157,
                       '30-39': 0.9837,
                       '<=18': 0.0004,
                       '>=40': 0.0002},
               'gender': {'female': 0.9866, 'male': 0.0134},
               'org': {'is-org': 0.0002, 'non-org': 0.9998}}),
    ...
  ...

User-level inference

devtools::install_github("pablobarbera/twitter_ideology/pkg/tweetscores")
library(tweetscores)

User-level inference

results <- estimateIdeology("cbarrie", ids)
plot(results)

User-level geolocation

userfwing %>%
  group_by(location) %>%
  summarise(count = n()) %>%
  top_n(count)

# A tibble: 12 × 2
   location            count
   <chr>               <int>
 1 Cambridge, MA          15
 2 Edinburgh              22
 3 Edinburgh, Scotland    39
 4 London                 60
 5 London, England        46
 6 New York, NY           24
 7 Oxford                 15
 8 Oxford, England        40
 9 San Francisco, CA      18
10 United Kingdom         13
11 Washington, DC         29
12 <NA>                  384