Data-Informed Thinking + Doing

Sentiment Analysis Using Lexicons

Extracting the polarity and emotions from tweets during the 2020 NBA Draft—using WordNet and VADER in Julia, Python, and R.

WordNet is a lexical database that provides information about the relationships between words, such as synonyms, antonyms, and hypernyms. It can be used to identify the sentiment of a text based on the sentiment of its individual words. However, WordNet does not have any built-in knowledge of sentiment or emotion, so its accuracy for sentiment analysis tasks may be limited.

Valence Aware Dictionary and sEntiment Reasoner (or VADER), on the other hand, is a rule-based sentiment analysis tool that is specifically designed to handle social media texts, such as tweets, that often contain informal language, slang, and emojis. It uses a combination of linguistic rules and heuristics to determine the sentiment of a text, and has been shown to perform well on social media texts. However, its performance on other types of text may be limited.

Getting Started

If you are interested in reproducing this work, here are the versions of Julia, Python, and R used (as well as the respective packages for each). Additionally, Leland Wilkinson’s approach to data visualization (Grammar of Graphics) has been adopted in this work. Finally, my coding style here is verbose, in order to trace back where functions/methods and variables are originating from.

VERSION
v"1.5.0"
import Pkg
Pkg.add(name="JSON", version="0.21.4")
Pkg.add(name="DataFrames", version="1.3.0")
Pkg.add(name="Colors", version="0.12.8")
Pkg.add(name="Cairo", version="1.0.5")
Pkg.add(name="Gadfly", version="1.3.4")
using JSON
using DataFrames
using Colors
using Cairo
using Gadfly
import sys
print(sys.version)
3.9.6 (v3.9.6:db3ff76da1, Jun 28 2021, 11:49:53) 
[Clang 6.0 (clang-600.0.57)]
!pip install pandas==2.0.0
!pip install plotnine==0.10.1
!pip install nltk==3.6.7
import random
import json
import pandas
import plotnine
import nltk

# from nltk.classify import NaiveBayesClassifier
# from nltk.corpus import subjectivity
# from nltk.sentiment import SentimentAnalyzer
# from nltk.sentiment.util import *
R.version.string
[1] "R version 4.1.1 (2021-08-10)"
require(devtools)
devtools::install_version("jsonlite", version="1.8.4", repos="http://cran.us.r-project.org")
devtools::install_version("dplyr", version="1.0.4", repos="http://cran.us.r-project.org")
devtools::install_version("ggplot2", version="3.3.3", repos="http://cran.us.r-project.org")
devtools::install_version("vader", version="0.2.1", repos="http://cran.us.r-project.org")
library(jsonlite)
library(dplyr)
library(ggplot2)
library(vader)

Importing and Examining Dataset

tweets_jl = read("../../dataset/twitter-jalensmith-nbadraft.json", String) |>
    JSON.parse |>
    DataFrames.DataFrame
100×2 DataFrame
 Row │ id                   text
     │ String               String
─────┼────────────────────────────────────────────────────────
   1 │ 1329246787270414338  @Suns @thejalen_smith This is th…
   2 │ 1329246786578472960  @JamesCBundy @AytonFan @Suns @th…
   3 │ 1329246785336791041  Welcome to the @Suns @thejalen_s…
   4 │ 1329246784091082752  @valdez_dolfo @Suns @thejalen_sm…
   5 │ 1329246782904299525  RT @MattLevine__: Maryland’s Jal…
   6 │ 1329246780723179520  RT @NBA: With the 10th pick of t…
   7 │ 1329246776478621698  RT @NBA: With the 10th pick of t…
   8 │ 1329246772787634176  @TerrapinHoops @thejalen_smith @…
   9 │ 1329246770883424264  @BrunoFernandoMV @thejalen_smith…
  10 │ 1329246761265868805  RT @NBA: With the 10th pick of t…
  11 │ 1329246760724803585  RT @MSJStudents: Jalen Smith ‘18…
  12 │ 1329246757876871168  RT @Suns: Our guy @thejalen_smit…
  13 │ 1329246755615956992  RT @Suns: Our guy @thejalen_smit…
  14 │ 1329246755100258306  RT @TerrapinHoops: STIX TO THE V…
  15 │ 1329246753388949504  RT @TerrapinHoops: STIX TO THE V…
  16 │ 1329246753162465288  @Suns @thejalen_smith Yeah let’s…
  17 │ 1329246752386510849  RT @Suns: Our guy @thejalen_smit…
  18 │ 1329246752382267392  RT @NBA: With the 10th pick of t…
  19 │ 1329246752139079682  @DatskoNathan @Suns @thejalen_sm…
  20 │ 1329246750423609346  @CoachTahjHolden @thejalen_smith…
  21 │ 1329246749375008768  RT @NBA: With the 10th pick of t…
  22 │ 1329246744518025217  RT @TerrapinHoops: STIX TO THE V…
  23 │ 1329246742710259713  RT @NBADraft: With the 10th pick…
  24 │ 1329246742429261830  RT @NBA: With the 10th pick of t…
  25 │ 1329246740592005121  @AytonFan @Suns @thejalen_smith …
  26 │ 1329246732597809152  RT @NBA: With the 10th pick of t…
  27 │ 1329246732509712384  @thejalen_smith congratulations …
  28 │ 1329246731335311362  RT @NBADraft: With the 10th pick…
  29 │ 1329246728332054529  RT @Suns: Our guy @thejalen_smit…
  30 │ 1329246727300210689  RT @N8veJay: @Suns @thejalen_smi…
  31 │ 1329246725920354305  RT @Suns: Our guy @thejalen_smit…
  32 │ 1329246722564837377  @thejalen_smith Welcome 🙏🏼
  33 │ 1329246718739746817  RT @TerrapinHoops: STIX TO THE V…
  34 │ 1329246716873216000  @Suns @thejalen_smith Sticks
  35 │ 1329246715619266561  RT @Suns: Our guy @thejalen_smit…
  36 │ 1329246714444832770  Congrats @thejalen_smith. Make m…
  37 │ 1329246714138603523  RT @TerrapinHoops: STIX TO THE V…
  38 │ 1329246712024625154  @thejalen_smith welcome to the v…
  39 │ 1329246710346878977  @spimpojr @thejalen_smith @Suns …
  40 │ 1329246709164113920  RT @NBA: With the 10th pick of t…
  41 │ 1329246700100218882  RT @NBA: With the 10th pick of t…
  42 │ 1329246695461449728  RT @NBA: With the 10th pick of t…
  43 │ 1329246689228537856  RT @Suns: Our guy @thejalen_smit…
  44 │ 1329246686850535425  @MikalBridgesSZN @Suns @thejalen…
  45 │ 1329246680768679939  @Suns @thejalen_smith Welcome to…
  46 │ 1329246679531458560  RT @TerrapinHoops: STIX TO THE V…
  ⋮  │          ⋮                           ⋮
  56 │ 1329246663177854976  @Suns @thejalen_smith QUIEN?
  57 │ 1329246660946440194  RT @TerrapinHoops: STIX TO THE V…
  58 │ 1329246660472475659  RT @TerrapinHoops: STIX TO THE V…
  59 │ 1329246659147038722  @thisisnotjoshua @OubreFanClub @…
  60 │ 1329246658165682176  RT @lilabbromberg: Love to see t…
  61 │ 1329246655510581248  @Suns @thejalen_smith I trust JJ
  62 │ 1329246654176911360  RT @TerrapinHoops: STIX TO THE V…
  63 │ 1329246651941269511  RT @TerrapinHoops: STIX TO THE V…
  64 │ 1329246647830806530  RT @TerrapinHoops: STIX TO THE V…
  65 │ 1329246645976961025  RT @TerrapinHoops: STIX TO THE V…
  66 │ 1329246644538388480  RT @TerrapinHoops: STIX TO THE V…
  67 │ 1329246639073144832  @MattLevine__ @thejalen_smith @T…
  68 │ 1329246633486217223  RT @Suns: Our guy @thejalen_smit…
  69 │ 1329246633180213255  @Suns @thejalen_smith Is it just…
  70 │ 1329246631242379266  @NBA @NBADraft @Suns @thejalen_s…
  71 │ 1329246628717400064  RT @Suns: Our guy @thejalen_smit…
  72 │ 1329246624099528705  RT @lilabbromberg: Love to see t…
  73 │ 1329246622421696515  @thejalen_smith welcome to Arizo…
  74 │ 1329246619519225856  RT @NBA: With the 10th pick of t…
  75 │ 1329246618080727040  @Suns @thejalen_smith yall out h…
  76 │ 1329246615379587072  RT @TerrapinHoops: STIX TO THE V…
  77 │ 1329246607255040002  @Suns @thejalen_smith CBS Sports…
  78 │ 1329246606873432068  RT @NBA: With the 10th pick of t…
  79 │ 1329246606735126529  RT @TerrapinHoops: STIX TO THE V…
  80 │ 1329246605485215746  RT @NBA: With the 10th pick of t…
  81 │ 1329246602041683968  @BigTenNetwork @thejalen_smith @…
  82 │ 1329246601932644352  RT @Suns: Our guy @thejalen_smit…
  83 │ 1329246600368107526  RT @TerrapinHoops: STIX TO THE V…
  84 │ 1329246597641809922  RT @TerrapinHoops: STIX TO THE V…
  85 │ 1329246597532717057  @thejalen_smith Welcome to the v…
  86 │ 1329246596370980864  RT @davidaldridgedc: Happy for @…
  87 │ 1329246596157087746  @thejalen_smith will do well in …
  88 │ 1329246594516992000  @VegasDogGuy1 @NBA @NBADraft @Su…
  89 │ 1329246591983742982  RT @Suns: Our guy @thejalen_smit…
  90 │ 1329246590058553344  Love to see this for @thejalen_s…
  91 │ 1329246587021787137  Welcome to AZ @thejalen_smith ☀️
  92 │ 1329246583926484994  @valdez_dolfo @Suns @thejalen_sm…
  93 │ 1329246580994502656  @Suns @thejalen_smith Lot of bot…
  94 │ 1329246580759674882  @thejalen_smith Welcome to the s…
  95 │ 1329246577408548869  RT @Suns: Our guy @thejalen_smit…
  96 │ 1329246576343216128  Congratulations to UMD Alum @the…
  97 │ 1329246571469303813  RT @Suns: Our guy @thejalen_smit…
  98 │ 1329246568084606977  Welcome to Phoenix Sticks! @thej…
  99 │ 1329246567564537856  THE TOP 10 TURTLE! @thejalen_smi…
 100 │ 1329246567363211272  Congrats @thejalen_smith !!!! ht…
                                                9 rows omitted
tweets_py = pandas.read_json("../../dataset/twitter-jalensmith-nbadraft.json")
tweets_py.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 100 entries, 0 to 99
Data columns (total 2 columns):
 #   Column  Non-Null Count  Dtype 
---  ------  --------------  ----- 
 0   id      100 non-null    int64 
 1   text    100 non-null    object
dtypes: int64(1), object(1)
memory usage: 1.7+ KB
tweets_py.head(n=8)
                    id                                                                                                                                                                                     text
0  1329246787270414336  @Suns @thejalen_smith This is the biggest waste of a draft pick I have ever seen... Haliburton is legit right there but no we go for the dude who would probably have been better at 20
1  1329246786578472960                                                 @JamesCBundy @AytonFan @Suns @thejalen_smith Cause CP is like 73 years old and you’d need a successor especially with Rubio out the door
2  1329246785336791040                                                                                                                                                     Welcome to the @Suns @thejalen_smith
3  1329246784091082752                                             @valdez_dolfo @Suns @thejalen_smith Your talents of sitting on Twitter and watching basketball from a couch? Bye. Dallas is perfect for you.
4  1329246782904299520                                           RT @MattLevine__: Maryland’s Jalen Smith has been selected by the Phoenix Suns with the 10th overall pick in the 2020 #NBADraft.\n\n@thejalen…
5  1329246780723179520                                           RT @NBA: With the 10th pick of the @NBADraft, the @Suns select Jalen Smith (@thejalen_smith)!\n\n2020 #NBADraft presented by State Farm on ES…
6  1329246776478621696                                           RT @NBA: With the 10th pick of the @NBADraft, the @Suns select Jalen Smith (@thejalen_smith)!\n\n2020 #NBADraft presented by State Farm on ES…
7  1329246772787634176                                                                   @TerrapinHoops @thejalen_smith @Suns @NBADraft Man was really hoping he’d end up with the @celtics ... good luck Stix!
tweets_py.tail(n=8)
                     id                                                                                                                                                                                                                           text
92  1329246580994502656                                                                                                                                                                               @Suns @thejalen_smith Lot of bots on here. Chill
93  1329246580759674880                                                                                                                                                                @thejalen_smith Welcome to the suns man https://t.co/nrLMhkYD54
94  1329246577408548864                                                                                                                                                                                             RT @Suns: Our guy @thejalen_smith!
95  1329246576343216128                                                                                Congratulations to UMD Alum @thejalen_smith on being drafted by @Suns !! Great young talent added to a team looking to win now #Terps #NBADraft
96  1329246571469303808                                                                                                                                                                                             RT @Suns: Our guy @thejalen_smith!
97  1329246568084606976                                                                                                                                                                           Welcome to Phoenix Sticks! @thejalen_smith #NBADraft
98  1329246567564537856  THE TOP 10 TURTLE! @thejalen_smith goes to the @Suns at #10 in the #NBADraft 💪💪The desert better be ready for 10 CCs of dunks, blocks, smooth jumpers &amp; of course...\n\nTURTLE POWER 🐢🐢🐢 #HeATerp https://t.co/9HhEkV7KI1
99  1329246567363211264                                                                                                                                                                          Congrats @thejalen_smith !!!! https://t.co/gXagWn77OT
tweets_r <- jsonlite::fromJSON("../../dataset/twitter-jalensmith-nbadraft.json") %>%
    as.data.frame()
str(object=tweets_r)
'data.frame':	100 obs. of  2 variables:
 $ id  : chr  "1329246787270414338" "1329246786578472960" "1329246785336791041" "1329246784091082752" ...
 $ text: chr  "@Suns @thejalen_smith This is the biggest waste of a draft pick I have ever seen... Haliburton is legit right t"| __truncated__ "@JamesCBundy @AytonFan @Suns @thejalen_smith Cause CP is like 73 years old and you’d need a successor especiall"| __truncated__ "Welcome to the @Suns @thejalen_smith" "@valdez_dolfo @Suns @thejalen_smith Your talents of sitting on Twitter and watching basketball from a couch? By"| __truncated__ ...
head(x=tweets_r, n=8)
                   id                                                                                                                                                                                    text
1 1329246787270414338 @Suns @thejalen_smith This is the biggest waste of a draft pick I have ever seen... Haliburton is legit right there but no we go for the dude who would probably have been better at 20
2 1329246786578472960                                                @JamesCBundy @AytonFan @Suns @thejalen_smith Cause CP is like 73 years old and you’d need a successor especially with Rubio out the door
3 1329246785336791041                                                                                                                                                    Welcome to the @Suns @thejalen_smith
4 1329246784091082752                                            @valdez_dolfo @Suns @thejalen_smith Your talents of sitting on Twitter and watching basketball from a couch? Bye. Dallas is perfect for you.
5 1329246782904299525                                          RT @MattLevine__: Maryland’s Jalen Smith has been selected by the Phoenix Suns with the 10th overall pick in the 2020 #NBADraft.\n\n@thejalen…
6 1329246780723179520                                          RT @NBA: With the 10th pick of the @NBADraft, the @Suns select Jalen Smith (@thejalen_smith)!\n\n2020 #NBADraft presented by State Farm on ES…
7 1329246776478621698                                          RT @NBA: With the 10th pick of the @NBADraft, the @Suns select Jalen Smith (@thejalen_smith)!\n\n2020 #NBADraft presented by State Farm on ES…
8 1329246772787634176                                                                  @TerrapinHoops @thejalen_smith @Suns @NBADraft Man was really hoping he’d end up with the @celtics ... good luck Stix!
tail(x=tweets_r, n=8)
                     id                                                                                                                                                                                                                               text
93  1329246580994502656                                                                                                                                                                                   @Suns @thejalen_smith Lot of bots on here. Chill
94  1329246580759674882                                                                                                                                                                    @thejalen_smith Welcome to the suns man https://t.co/nrLMhkYD54
95  1329246577408548869                                                                                                                                                                                                 RT @Suns: Our guy @thejalen_smith!
96  1329246576343216128                                                                                    Congratulations to UMD Alum @thejalen_smith on being drafted by @Suns !! Great young talent added to a team looking to win now #Terps #NBADraft
97  1329246571469303813                                                                                                                                                                                                 RT @Suns: Our guy @thejalen_smith!
98  1329246568084606977                                                                                                                                                                               Welcome to Phoenix Sticks! @thejalen_smith #NBADraft
99  1329246567564537856 THE TOP 10 TURTLE! @thejalen_smith goes to the @Suns at #10 in the #NBADraft 💪💪The desert better be ready for 10 CCs of dunks, blocks, smooth jumpers &amp; of course...\n\nTURTLE POWER 🐢🐢🐢 #HeATerp https://t.co/9HhEkV7KI1
100 1329246567363211272                                                                                                                                                                              Congrats @thejalen_smith !!!! https://t.co/gXagWn77OT

Wrangling Data

tweets_clean_jl = tweets_jl
100×2 DataFrame
 Row │ id                   text
     │ String               String
─────┼────────────────────────────────────────────────────────
   1 │ 1329246787270414338  @Suns @thejalen_smith This is th…
   2 │ 1329246786578472960  @JamesCBundy @AytonFan @Suns @th…
   3 │ 1329246785336791041  Welcome to the @Suns @thejalen_s…
   4 │ 1329246784091082752  @valdez_dolfo @Suns @thejalen_sm…
   5 │ 1329246782904299525  RT @MattLevine__: Maryland’s Jal…
   6 │ 1329246780723179520  RT @NBA: With the 10th pick of t…
   7 │ 1329246776478621698  RT @NBA: With the 10th pick of t…
   8 │ 1329246772787634176  @TerrapinHoops @thejalen_smith @…
   9 │ 1329246770883424264  @BrunoFernandoMV @thejalen_smith…
  10 │ 1329246761265868805  RT @NBA: With the 10th pick of t…
  11 │ 1329246760724803585  RT @MSJStudents: Jalen Smith ‘18…
  12 │ 1329246757876871168  RT @Suns: Our guy @thejalen_smit…
  13 │ 1329246755615956992  RT @Suns: Our guy @thejalen_smit…
  14 │ 1329246755100258306  RT @TerrapinHoops: STIX TO THE V…
  15 │ 1329246753388949504  RT @TerrapinHoops: STIX TO THE V…
  16 │ 1329246753162465288  @Suns @thejalen_smith Yeah let’s…
  17 │ 1329246752386510849  RT @Suns: Our guy @thejalen_smit…
  18 │ 1329246752382267392  RT @NBA: With the 10th pick of t…
  19 │ 1329246752139079682  @DatskoNathan @Suns @thejalen_sm…
  20 │ 1329246750423609346  @CoachTahjHolden @thejalen_smith…
  21 │ 1329246749375008768  RT @NBA: With the 10th pick of t…
  22 │ 1329246744518025217  RT @TerrapinHoops: STIX TO THE V…
  23 │ 1329246742710259713  RT @NBADraft: With the 10th pick…
  24 │ 1329246742429261830  RT @NBA: With the 10th pick of t…
  25 │ 1329246740592005121  @AytonFan @Suns @thejalen_smith …
  26 │ 1329246732597809152  RT @NBA: With the 10th pick of t…
  27 │ 1329246732509712384  @thejalen_smith congratulations …
  28 │ 1329246731335311362  RT @NBADraft: With the 10th pick…
  29 │ 1329246728332054529  RT @Suns: Our guy @thejalen_smit…
  30 │ 1329246727300210689  RT @N8veJay: @Suns @thejalen_smi…
  31 │ 1329246725920354305  RT @Suns: Our guy @thejalen_smit…
  32 │ 1329246722564837377  @thejalen_smith Welcome 🙏🏼
  33 │ 1329246718739746817  RT @TerrapinHoops: STIX TO THE V…
  34 │ 1329246716873216000  @Suns @thejalen_smith Sticks
  35 │ 1329246715619266561  RT @Suns: Our guy @thejalen_smit…
  36 │ 1329246714444832770  Congrats @thejalen_smith. Make m…
  37 │ 1329246714138603523  RT @TerrapinHoops: STIX TO THE V…
  38 │ 1329246712024625154  @thejalen_smith welcome to the v…
  39 │ 1329246710346878977  @spimpojr @thejalen_smith @Suns …
  40 │ 1329246709164113920  RT @NBA: With the 10th pick of t…
  41 │ 1329246700100218882  RT @NBA: With the 10th pick of t…
  42 │ 1329246695461449728  RT @NBA: With the 10th pick of t…
  43 │ 1329246689228537856  RT @Suns: Our guy @thejalen_smit…
  44 │ 1329246686850535425  @MikalBridgesSZN @Suns @thejalen…
  45 │ 1329246680768679939  @Suns @thejalen_smith Welcome to…
  46 │ 1329246679531458560  RT @TerrapinHoops: STIX TO THE V…
  ⋮  │          ⋮                           ⋮
  56 │ 1329246663177854976  @Suns @thejalen_smith QUIEN?
  57 │ 1329246660946440194  RT @TerrapinHoops: STIX TO THE V…
  58 │ 1329246660472475659  RT @TerrapinHoops: STIX TO THE V…
  59 │ 1329246659147038722  @thisisnotjoshua @OubreFanClub @…
  60 │ 1329246658165682176  RT @lilabbromberg: Love to see t…
  61 │ 1329246655510581248  @Suns @thejalen_smith I trust JJ
  62 │ 1329246654176911360  RT @TerrapinHoops: STIX TO THE V…
  63 │ 1329246651941269511  RT @TerrapinHoops: STIX TO THE V…
  64 │ 1329246647830806530  RT @TerrapinHoops: STIX TO THE V…
  65 │ 1329246645976961025  RT @TerrapinHoops: STIX TO THE V…
  66 │ 1329246644538388480  RT @TerrapinHoops: STIX TO THE V…
  67 │ 1329246639073144832  @MattLevine__ @thejalen_smith @T…
  68 │ 1329246633486217223  RT @Suns: Our guy @thejalen_smit…
  69 │ 1329246633180213255  @Suns @thejalen_smith Is it just…
  70 │ 1329246631242379266  @NBA @NBADraft @Suns @thejalen_s…
  71 │ 1329246628717400064  RT @Suns: Our guy @thejalen_smit…
  72 │ 1329246624099528705  RT @lilabbromberg: Love to see t…
  73 │ 1329246622421696515  @thejalen_smith welcome to Arizo…
  74 │ 1329246619519225856  RT @NBA: With the 10th pick of t…
  75 │ 1329246618080727040  @Suns @thejalen_smith yall out h…
  76 │ 1329246615379587072  RT @TerrapinHoops: STIX TO THE V…
  77 │ 1329246607255040002  @Suns @thejalen_smith CBS Sports…
  78 │ 1329246606873432068  RT @NBA: With the 10th pick of t…
  79 │ 1329246606735126529  RT @TerrapinHoops: STIX TO THE V…
  80 │ 1329246605485215746  RT @NBA: With the 10th pick of t…
  81 │ 1329246602041683968  @BigTenNetwork @thejalen_smith @…
  82 │ 1329246601932644352  RT @Suns: Our guy @thejalen_smit…
  83 │ 1329246600368107526  RT @TerrapinHoops: STIX TO THE V…
  84 │ 1329246597641809922  RT @TerrapinHoops: STIX TO THE V…
  85 │ 1329246597532717057  @thejalen_smith Welcome to the v…
  86 │ 1329246596370980864  RT @davidaldridgedc: Happy for @…
  87 │ 1329246596157087746  @thejalen_smith will do well in …
  88 │ 1329246594516992000  @VegasDogGuy1 @NBA @NBADraft @Su…
  89 │ 1329246591983742982  RT @Suns: Our guy @thejalen_smit…
  90 │ 1329246590058553344  Love to see this for @thejalen_s…
  91 │ 1329246587021787137  Welcome to AZ @thejalen_smith ☀️
  92 │ 1329246583926484994  @valdez_dolfo @Suns @thejalen_sm…
  93 │ 1329246580994502656  @Suns @thejalen_smith Lot of bot…
  94 │ 1329246580759674882  @thejalen_smith Welcome to the s…
  95 │ 1329246577408548869  RT @Suns: Our guy @thejalen_smit…
  96 │ 1329246576343216128  Congratulations to UMD Alum @the…
  97 │ 1329246571469303813  RT @Suns: Our guy @thejalen_smit…
  98 │ 1329246568084606977  Welcome to Phoenix Sticks! @thej…
  99 │ 1329246567564537856  THE TOP 10 TURTLE! @thejalen_smi…
 100 │ 1329246567363211272  Congrats @thejalen_smith !!!! ht…
                                                9 rows omitted
tweets_clean_py = tweets_py
tweets_clean_r <- tweets_r

Performing Sentiment Analysis

# sentiment_analysis = nltk.sentiment.SentimentIntensityAnalyzer()
# sentiment_polarity_scores = [sentiment_analysis.polarity_scores(tweet["text"]) for tweet in tweets_json_py]
tweets_clean_r <- vader::vader_df(text=tweets_clean_r$text)
head(x=tweets_clean_r, n=8)
                                                                                                                                                                                     text                                                                                                        word_scores compound   pos  neu  neg but_count
1 @Suns @thejalen_smith This is the biggest waste of a draft pick I have ever seen... Haliburton is legit right there but no we go for the dude who would probably have been better at 20 {0, 0, 0, 0, 0, 0, -0.9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2.85, 0, 0}    0.039 0.095 0.79 0.12         1
2                                                @JamesCBundy @AytonFan @Suns @thejalen_smith Cause CP is like 73 years old and you’d need a successor especially with Rubio out the door                                             {0, 0, 0, 0, 0, 0, 0, 1.5, 0, 0, 0, 0, 0, 0, 0, 0.9, 0, 0, 0, 0, 0, 0}    0.527 0.180 0.82 0.00         0
3                                                                                                                                                    Welcome to the @Suns @thejalen_smith                                                                                                    {2, 0, 0, 0, 0}    0.459 0.429 0.57 0.00         0
4                                            @valdez_dolfo @Suns @thejalen_smith Your talents of sitting on Twitter and watching basketball from a couch? Bye. Dallas is perfect for you.                                                  {0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2.7, 0, 0}    0.772 0.261 0.74 0.00         0
5                                          RT @MattLevine__: Maryland’s Jalen Smith has been selected by the Phoenix Suns with the 10th overall pick in the 2020 #NBADraft.\n\n@thejalen…                                                 {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}    0.000 0.000 1.00 0.00         0
6                                          RT @NBA: With the 10th pick of the @NBADraft, the @Suns select Jalen Smith (@thejalen_smith)!\n\n2020 #NBADraft presented by State Farm on ES…                                              {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}    0.000 0.000 1.00 0.00         0
7                                          RT @NBA: With the 10th pick of the @NBADraft, the @Suns select Jalen Smith (@thejalen_smith)!\n\n2020 #NBADraft presented by State Farm on ES…                                              {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}    0.000 0.000 1.00 0.00         0
8                                                                  @TerrapinHoops @thejalen_smith @Suns @NBADraft Man was really hoping he’d end up with the @celtics ... good luck Stix!                                                       {0, 0, 0, 0, 0, 0, 0, 2.093, 0, 0, 0, 0, 0, 0, 0, 1.9, 2, 0}    0.851 0.382 0.62 0.00         0
tail(x=tweets_clean_r, n=8)
                                                                                                                                                                                                                                  text                                                                                                          word_scores compound  pos  neu   neg but_count
93                                                                                                                                                                                    @Suns @thejalen_smith Lot of bots on here. Chill                                                                                             {0, 0, 0, 0, 0, 0, 0, 0}     0.00 0.00 1.00 0.000         0
94                                                                                                                                                                     @thejalen_smith Welcome to the suns man https://t.co/nrLMhkYD54                                                                                                {0, 2, 0, 0, 0, 0, 0}     0.46 0.33 0.67 0.000         0
95                                                                                                                                                                                                  RT @Suns: Our guy @thejalen_smith!                                                                                                      {0, 0, 0, 0, 0}     0.00 0.00 1.00 0.000         0
96                                                                                     Congratulations to UMD Alum @thejalen_smith on being drafted by @Suns !! Great young talent added to a team looking to win now #Terps #NBADraft                                     {2.9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3.1, 0, 1.8, 0, 0, 0, 0, 0, 0, 2.8, 0, 0, 0}     0.94 0.43 0.57 0.000         0
97                                                                                                                                                                                                  RT @Suns: Our guy @thejalen_smith!                                                                                                      {0, 0, 0, 0, 0}     0.00 0.00 1.00 0.000         0
98                                                                                                                                                                                Welcome to Phoenix Sticks! @thejalen_smith #NBADraft                                                                                                   {2, 0, 0, 0, 0, 0}     0.51 0.40 0.60 0.000         0
99  THE TOP 10 TURTLE! @thejalen_smith goes to the @Suns at #10 in the #NBADraft 💪💪The desert better be ready for 10 CCs of dunks, blocks, smooth jumpers &amp; of course...\n\nTURTLE POWER 🐢🐢🐢 #HeATerp https://t.co/9HhEkV7KI1 {0, 1.533, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.9, 0, 1.5, 0, 0, 0, 0, 0, -0.9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}     0.74 0.20 0.75 0.046         0
100                                                                                                                                                                              Congrats @thejalen_smith !!!! https://t.co/gXagWn77OT                                                                                                       {2.4, 0, 0, 0}     0.68 0.60 0.40 0.000         0
Applied Advanced Analytics & AI in Sports