§7 Sentiment Analysis 😊😐😢

Welcome to the world of sentiment analysis in Digital Humanities! In this chapter, we’ll explore how to analyze the emotional content of texts using computational methods.

Learning Objectives
  • 📊 Understand the concept of sentiment analysis and its applications in Digital Humanities
  • 🧮 Learn about different approaches to sentiment analysis
  • 💻 Implement basic sentiment analysis using R
  • 📈 Visualize sentiment patterns in literary texts

1 What is Sentiment Analysis? 🤔

Sentiment analysis, also known as opinion mining, is a technique used to determine the emotional tone behind a body of text. It’s a powerful tool in Digital Humanities for understanding the emotional content of literary works, historical documents, social media posts, and more.

1.1 Sentiment Analysis in Digital Humanities

In Digital Humanities, sentiment analysis offers several benefits:

  1. Scale: Analyze emotional patterns across large corpora of texts.
  2. Objectivity: Provide a systematic approach to studying emotions in text.
  3. Comparative Analysis: Compare emotional tones across different authors, time periods, or genres.
  4. Narrative Arc Analysis: Study how emotions change throughout a narrative.
  5. Cultural Insights: Explore emotional expressions in different cultural contexts.

1.2 Approaches to Sentiment Analysis

There are several approaches to sentiment analysis:

  1. Lexicon-based: Uses pre-defined dictionaries of words associated with positive or negative sentiments.
  2. Machine Learning: Trains models on labeled data to predict sentiment.
  3. Rule-based: Uses a set of manually crafted rules to determine sentiment.
  4. Hybrid: Combines multiple approaches for more accurate results.

In this chapter, we’ll focus on the lexicon-based approach using the tidytext package in R.

2 Understanding Lexicon-Based Sentiment Analysis

Lexicon-based sentiment analysis is like having an emotional dictionary that helps us understand the feelings expressed in a text. Let’s explore this concept through hands-on examples!

2.1 Try It Yourself: Basic Sentiment Tagging 🏷️

Before computers can analyze sentiment, they need a dictionary that tells them which words express positive or negative emotions. But creating such a dictionary isn’t straightforward! Even humans often disagree about whether a word is positive, negative, or neutral.

Key Terms
  • Sentiment Dictionary (or Lexicon): A pre-defined list of words with their associated emotional values
  • Tagging: The process of labeling words with their emotional values (positive, negative, or neutral)
  • Sentiment Value: The emotional weight assigned to a word (e.g., “happy” = positive, “sad” = negative)

Let’s experience this challenge firsthand. In the exercise below, you’ll act as a “human lexicon creator 🕵️” tagging words just like we do when building sentiment dictionaries. Work with a partner to discover why this fundamental step of sentiment analysis is more complex than it might seem.

If humans disagree on word sentiment, imagine how challenging it is for computers! Let’s look at some examples:

  1. Multiple Meanings & Context
    • “dark”: Could be negative (mood) or neutral (description)
    • “cold”: Could be negative (feeling) or neutral (temperature)
    • “bright”: Could be positive (mood) or neutral (light level)
  2. Slang & Modern Usage
    • “killing”: Could be negative (harm) or positive slang (“killing it” = doing well)
    • “wild”: Could be positive (exciting) or negative (dangerous)
  3. Subjective Interpretation
    • “simple”: Could be positive (clear, elegant) or negative (unsophisticated)
    • “quiet”: Could be positive (peaceful) or negative (lonely)
    • “deep”: Could be neutral (measurement) or positive (profound)

These ambiguities demonstrate why:

  1. Context is crucial for understanding meaning
  2. Cultural and temporal differences affect interpretation
  3. Even “simple” words can have complex meanings
  4. Automated sentiment analysis needs sophisticated approaches to handle these nuances

2.2 Common Sentiment Lexicons 📖

After seeing how challenging it is to create sentiment dictionaries, let’s look at some widely-used lexicons in Digital Humanities research:

The Big Three
  1. AFINN: A simple lexicon that rates words from -5 (very negative) to +5 (very positive)
    • Example: “superb” = +5, “catastrophic” = -4
    • Good for: Quick analysis, clear numerical scores
    • Limitation: Misses nuanced emotions
  2. Bing: Categorizes words as either positive or negative
    • Example: “good” = positive, “bad” = negative
    • Good for: Binary sentiment analysis
    • Limitation: No neutral category or intensity levels
  3. NRC: Classifies words into eight basic emotions plus positive/negative
    • Emotions: anger, fear, anticipation, trust, surprise, sadness, joy, disgust
    • Example: “birthday” → joy, anticipation, positive
    • Good for: Detailed emotional analysis
    • Limitation: More complex to interpret

2.2.1 Try It Yourself: Comparing Lexicons 🔍

Let’s explore how each lexicon works through a collaborative exercise:

  1. Individual Tagging
    • Work independently to tag the words below using each lexicon
    • Keep your answers private until everyone is done
    • Try to justify your choices as you go
  2. Group Discussion
    • Form groups of 2-3 students
    • Compare your tags with your groupmates
    • Discuss where and why your tags differed
    • What factors influenced your decisions?

Let’s begin with the interactive tagging exercise below:

Comparing Lexicons
  • Compare how the same word might be tagged differently in each lexicon
  • Notice how AFINN’s numerical scale captures intensity
  • See how Bing’s simplicity makes quick classification easier
  • Explore how NRC reveals emotional complexity
Reflection Questions

  1. Which lexicon would you choose for analyzing:

    • Movie reviews?
    • Historical documents?
    • Social media posts?

  2. What are the trade-offs between simplicity and detail?

  3. How might cultural or temporal differences affect these lexicons?

  1. Choosing Lexicons for Different Contexts:

    • Movie Reviews: AFINN might be most suitable because:

      • Its -5 to +5 scale captures intensity of opinions
      • Reviews often use strong emotional language
      • Numerical scores align well with star ratings

    • Historical Documents: Bing might work better because:

      • Binary classification is less likely to misinterpret archaic language
      • Simpler positive/negative distinction suits formal writing
      • Less affected by changes in emotional intensity over time

    • Social Media Posts: NRC would be ideal because:

      • Captures nuanced emotions like joy, anger, surprise
      • Suits informal, emotionally expressive language
      • Better at handling modern slang and emoji contexts

  2. Trade-offs Between Simplicity and Detail:

    • Simple Lexicons (e.g., Bing)
      • ✅ Easier to implement and interpret
      • ✅ More robust across different contexts
      • ❌ Miss nuanced emotional expressions
      • ❌ Can’t capture intensity of sentiment
    • Complex Lexicons (e.g., NRC)
      • ✅ Richer emotional analysis
      • ✅ Better for nuanced interpretation
      • ❌ More computationally intensive
      • ❌ Higher risk of misclassification

  3. Cultural and Temporal Impacts:

    • Cultural Differences
      • Words may have different emotional connotations across cultures
      • Metaphors and idioms might not translate emotionally
      • Cultural context affects interpretation of neutral vs. emotional language
    • Temporal Changes
      • Word meanings and emotional associations evolve over time
      • Modern lexicons might misinterpret historical usage
      • Slang and informal language constantly create new emotional expressions

3 Implementing Sentiment Analysis in R

Let’s start with a practical example using Jane Austen’s novels. We’ll use the janeaustenr package for the text data and tidytext for sentiment analysis.

library(pacman)
p_load(janeaustenr, tidyverse, tidytext, hrbrthemes)

# Combine Austen's novels
austen_books <- austen_books() %>%
  group_by(book) %>%
  mutate(
    linenumber = row_number(),
    chapter = cumsum(str_detect(text, regex("^chapter [\\divxlc]", ignore_case = TRUE)))
  ) %>%
  ungroup()

Now, let’s perform sentiment analysis:

# Perform sentiment analysis
austen_sentiment <- austen_books %>%
  unnest_tokens(word, text) %>%
  inner_join(get_sentiments("bing"), multiple="all") %>%
  count(book, index = linenumber %/% 100, sentiment) %>%
  pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) %>% 
  mutate(sentiment = positive - negative)

# Plot sentiment over narrative time
ggplot(austen_sentiment, aes(index, sentiment, fill = book)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~book, ncol = 2, scales = "free_x") +
  labs(
    title = "Sentiment Analysis of Jane Austen's Novels",
    subtitle = "Emotional trajectory throughout the narrative",
    x = "Narrative Time",
    y = "Sentiment Score",
    caption = "Data: janeaustenr package | Analysis: tidytext"
  ) +
  theme_minimal() +
  theme(
    text = element_text(size = 12),
    strip.text = element_text(size = 14, face = "bold"),
    panel.grid.minor = element_blank(),
    plot.title = element_text(size = 18, face = "bold"),
    plot.subtitle = element_text(size = 14, color = "gray50"),
    plot.caption = element_text(size = 10, color = "gray50"),
    axis.title = element_text(size = 12, face = "bold"),
    plot.margin = margin(20, 20, 20, 20)
  ) +
  scale_fill_brewer(palette = "Set2") +
  scale_y_continuous(expand = expansion(mult = c(0.1, 0.1)))

# Display summary statistics
austen_summary <- austen_sentiment %>%
  group_by(book) %>%
  summarize(
    mean_sentiment = mean(sentiment),
    max_sentiment = max(sentiment),
    min_sentiment = min(sentiment)
  )

knitr::kable(austen_summary, caption = "Summary Statistics of Sentiment Analysis", digits = 2)
Summary Statistics of Sentiment Analysis
book mean_sentiment max_sentiment min_sentiment
Sense & Sensibility 9.94 51 -26
Pride & Prejudice 10.69 42 -31
Mansfield Park 12.47 61 -45
Emma 14.40 48 -31
Northanger Abbey 9.19 47 -33
Persuasion 15.14 59 -13
  1. What patterns do you notice in the emotional arcs of Austen’s novels?
  2. How might these sentiment patterns relate to the plot structures of the novels?
  3. What limitations might this sentiment analysis approach have when applied to historical texts like Austen’s novels?

4 Learning Check 🏁

5 Conclusion

Key Takeaways

In this chapter, we’ve covered:

  • The concept of sentiment analysis and its applications in Digital Humanities
  • Implementing basic sentiment analysis using R and the tidytext package
  • Visualizing sentiment patterns in literary texts
  • Comparing different sentiment lexicons and their implications

Sentiment analysis is a powerful tool for exploring emotional patterns in texts, offering new perspectives on literary works, historical documents, and cultural artifacts. As we’ve seen, it can reveal interesting patterns and raise new questions for humanities scholars to explore. However, it’s important to remember that sentiment analysis tools have limitations, especially when applied to historical texts or complex literary works. They should be used as part of a broader analytical toolkit, complementing rather than replacing close reading and traditional literary analysis.

Sentiment Analysis Basics

Implementation in R

Visualization Techniques

Advanced Techniques