Add readme
This commit is contained in:
Gaston Sanchez
GitHub
parent
3e704a9d9e
commit
a9ef33f01e
196
README.Rmd
Normal file
196
README.Rmd
Normal file
@ -0,0 +1,196 @@
|
|||||||
|
---
|
||||||
|
title: "Harry Potter Data"
|
||||||
|
output:
|
||||||
|
md_document:
|
||||||
|
variant: markdown_github
|
||||||
|
---
|
||||||
|
|
||||||
|
<!-- README.md is generated from README.Rmd. Please edit that file -->
|
||||||
|
|
||||||
|
```{r include = FALSE}
|
||||||
|
knitr::opts_chunk$set(
|
||||||
|
collapse = TRUE,
|
||||||
|
comment = "#>"
|
||||||
|
)
|
||||||
|
|
||||||
|
library(tidyverse)
|
||||||
|
library(tidytext)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## About
|
||||||
|
|
||||||
|
This repository contains various data files that can be used to perform a
|
||||||
|
text analysis of [Harry Potter](https://en.wikipedia.org/wiki/Harry_Potter)
|
||||||
|
books, written by Joanne Kathleen Rowling:
|
||||||
|
|
||||||
|
1) Harry Potter and the Philosopher's Stone
|
||||||
|
|
||||||
|
2) Harry Potter and the Chamber of Secrets
|
||||||
|
|
||||||
|
3) Harry Potter and the Prisoner of Azkaban
|
||||||
|
|
||||||
|
4) Harry Potter and the Goblet of Fire
|
||||||
|
|
||||||
|
5) Harry Potter and the Order of the Phoenix
|
||||||
|
|
||||||
|
6) Harry Potter and the Half-Blood Prince
|
||||||
|
|
||||||
|
7) Harry Potter and the Deathly Hallows
|
||||||
|
|
||||||
|
|
||||||
|
To perform the text analysis, we recommend using _tidyverse_ tools (see
|
||||||
|
packages below) and getting inspiration from the book
|
||||||
|
[Text Mining with R: A Tidy Approach](https://www.tidytextmining.com/index.html)
|
||||||
|
(by Silge & Robinson):
|
||||||
|
|
||||||
|
```r
|
||||||
|
library(tidyverse)
|
||||||
|
library(tidytext)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Content
|
||||||
|
|
||||||
|
The content of this repo is divided in three directories, each one containing
|
||||||
|
different types of files.
|
||||||
|
|
||||||
|
- [csv-data-file/](csv-data-file) contains the text of all Harry Potter books
|
||||||
|
in a single CSV file.
|
||||||
|
|
||||||
|
- [rda-data-files/](rda-data-files) contains the seven Harry Potter books
|
||||||
|
stored in R-Data (binary) files---one filer per book.
|
||||||
|
|
||||||
|
- [sentiment-lexicons/](sentiment-lexicons) contains a handful of sentiment
|
||||||
|
lexicons, also stored in R-Data (binary) files---one file per lexicon.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Harry Potter CSV file
|
||||||
|
|
||||||
|
The data of all the books is available in `csv` format:
|
||||||
|
`harry_potter_books.csv`.
|
||||||
|
|
||||||
|
Assuming that this file is in your working directory, you can import it---via
|
||||||
|
tidyverse's `readr()`---as follows:
|
||||||
|
|
||||||
|
```r
|
||||||
|
# requires package tidyverse
|
||||||
|
hp_books = read_csv("harry_potter_books.csv", col_types = "ccc")
|
||||||
|
```
|
||||||
|
|
||||||
|
```{r echo = FALSE}
|
||||||
|
hp_books = read_csv("csv-data-file/harry_potter_books.csv", col_types = "ccc")
|
||||||
|
```
|
||||||
|
|
||||||
|
This data set is fairly simple---in terms of its structure---although the
|
||||||
|
text content is far from being tidy. The dataset has `r nrow(hp_books)` rows
|
||||||
|
and `r ncol(hp_books)` columns:
|
||||||
|
|
||||||
|
1) `text`: text content
|
||||||
|
|
||||||
|
2) `book`: title of associated book
|
||||||
|
|
||||||
|
3) `chapter`: associated chapter number
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Harry Potter R-Data Files
|
||||||
|
|
||||||
|
The data of each book is also available in its own R-Data `rda` file
|
||||||
|
(see [rda-data-files/](rda-data-files)):
|
||||||
|
|
||||||
|
- `"philosophers_stone.rda"`
|
||||||
|
- `"chamber_of_secrets.rda"`
|
||||||
|
- `"prisoner_of_azkaban.rda"`
|
||||||
|
- `"goblet_of_fire.rda"`
|
||||||
|
- `"order_of_the_phoenix.rda"`
|
||||||
|
- `"half_blood_prince.rda"`
|
||||||
|
- `"deathly_hallows.rda"`
|
||||||
|
|
||||||
|
These files come from the R package `"harrypotter"` by Bradley Boehmke
|
||||||
|
|
||||||
|
[https://github.com/bradleyboehmke/harrypotter](https://github.com/bradleyboehmke/harrypotter)
|
||||||
|
|
||||||
|
To import these files use the `load()` function. For example, consider the
|
||||||
|
first book "Harry Potter and the Philosopher's Stone"; here's how to `load()`
|
||||||
|
it in R:
|
||||||
|
|
||||||
|
```r
|
||||||
|
# assuming that the rda file is in your working directory
|
||||||
|
load("philosophers_stone.rda")
|
||||||
|
```
|
||||||
|
|
||||||
|
```{r echo = FALSE}
|
||||||
|
load("rda-data-files/philosophers_stone.rda")
|
||||||
|
```
|
||||||
|
|
||||||
|
Assuming that `"philosophers_stone.rda"` has been loaded, the text of this
|
||||||
|
book is available in the homonym character vector `philosophers_stone`
|
||||||
|
|
||||||
|
```{r}
|
||||||
|
# text is in a character vector
|
||||||
|
# (with as many elements as chapters in the book)
|
||||||
|
length(philosophers_stone)
|
||||||
|
```
|
||||||
|
|
||||||
|
The number of elements in `philosophers_stone` corresponds
|
||||||
|
to the number of chapters in this book: 17 chapters.
|
||||||
|
|
||||||
|
You may want to use these files to perform bigram analysis (or other type of
|
||||||
|
n-gram analysis).
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Sentiment Lexicons
|
||||||
|
|
||||||
|
In addition to the Harry Potter text, you can also find data for a handful of
|
||||||
|
sentiment lexicons from the R package `"textdata"` (by Hvitfeldt and Silge):
|
||||||
|
|
||||||
|
- `"bing"`: Bing Liu's General purpose English sentiment lexicon that
|
||||||
|
categorizes words in a binary fashion, either positive or negative
|
||||||
|
|
||||||
|
- `"afinn"`: AFINN is a lexicon of English words rated for valence with an
|
||||||
|
integer between minus five (negative) and plus five (positive). The words have
|
||||||
|
been manually labeled by Finn Årup Nielsen in 2009-2011.
|
||||||
|
|
||||||
|
- `"nrc"`: General purpose English sentiment/emotion lexicon. This lexicon
|
||||||
|
labels words with six possible sentiments or emotions: "negative", "positive",
|
||||||
|
"anger", "anticipation", "disgust", "fear", "joy", "sadness", "surprise", or
|
||||||
|
"trust". The annotations were manually done through Amazon's Mechanical Turk.
|
||||||
|
|
||||||
|
- `"loughran"`: English sentiment lexicon created for use with financial
|
||||||
|
documents. This lexicon labels words with six possible sentiments important in
|
||||||
|
financial contexts: "negative", "positive", "litigious", "uncertainty",
|
||||||
|
"constraining", or "superfluous".
|
||||||
|
|
||||||
|
|
||||||
|
These lexicons come in `rda` data files (see
|
||||||
|
[sentiment-lexicons/](sentiment-lexicons)):
|
||||||
|
|
||||||
|
- `bing.rda`
|
||||||
|
- `afinn.rda`
|
||||||
|
- `bing.rda`
|
||||||
|
- `loughran.rda`
|
||||||
|
|
||||||
|
To import them in R, use the `load()` function. For example, here's how to
|
||||||
|
import the Bing lexicon:
|
||||||
|
|
||||||
|
```r
|
||||||
|
# assuming that the rda files are in your working directory
|
||||||
|
load("bing.rda")
|
||||||
|
```
|
||||||
|
|
||||||
|
```{r echo = FALSE}
|
||||||
|
# assuming that the rda file is in your working directory
|
||||||
|
load("sentiment-lexicons/bing.rda")
|
||||||
|
```
|
||||||
|
|
||||||
|
Assuming that you've loaded the file `"bing.rda"`, the associated lexicon is
|
||||||
|
available in the homonym tibble `bing`
|
||||||
|
|
||||||
|
```{r}
|
||||||
|
bing
|
||||||
|
```
|
||||||
|
|
||||||
175
README.md
Normal file
175
README.md
Normal file
@ -0,0 +1,175 @@
|
|||||||
|
<!-- README.md is generated from README.Rmd. Please edit that file -->
|
||||||
|
|
||||||
|
## About
|
||||||
|
|
||||||
|
This repository contains various data files that can be used to perform
|
||||||
|
a text analysis of [Harry
|
||||||
|
Potter](https://en.wikipedia.org/wiki/Harry_Potter) books, written by
|
||||||
|
Joanne Kathleen Rowling:
|
||||||
|
|
||||||
|
1. Harry Potter and the Philosopher’s Stone
|
||||||
|
|
||||||
|
2. Harry Potter and the Chamber of Secrets
|
||||||
|
|
||||||
|
3. Harry Potter and the Prisoner of Azkaban
|
||||||
|
|
||||||
|
4. Harry Potter and the Goblet of Fire
|
||||||
|
|
||||||
|
5. Harry Potter and the Order of the Phoenix
|
||||||
|
|
||||||
|
6. Harry Potter and the Half-Blood Prince
|
||||||
|
|
||||||
|
7. Harry Potter and the Deathly Hallows
|
||||||
|
|
||||||
|
To perform the text analysis, we recommend using *tidyverse* tools (see
|
||||||
|
packages below) and getting inspiration from the book [Text Mining with
|
||||||
|
R: A Tidy Approach](https://www.tidytextmining.com/index.html) (by Silge
|
||||||
|
& Robinson):
|
||||||
|
|
||||||
|
``` r
|
||||||
|
library(tidyverse)
|
||||||
|
library(tidytext)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Content
|
||||||
|
|
||||||
|
The content of this repo is divided in three directories, each one
|
||||||
|
containing different types of files.
|
||||||
|
|
||||||
|
- [csv-data-file/](csv-data-file) contains the text of all Harry
|
||||||
|
Potter books in a single CSV file.
|
||||||
|
|
||||||
|
- [rda-data-files/](rda-data-files) contains the seven Harry Potter
|
||||||
|
books stored in R-Data (binary) files—one filer per book.
|
||||||
|
|
||||||
|
- [sentiment-lexicons/](sentiment-lexicons) contains a handful of
|
||||||
|
sentiment lexicons, also stored in R-Data (binary) files—one file
|
||||||
|
per lexicon.
|
||||||
|
|
||||||
|
### Harry Potter CSV file
|
||||||
|
|
||||||
|
The data of all the books is available in `csv` format:
|
||||||
|
`harry_potter_books.csv`.
|
||||||
|
|
||||||
|
Assuming that this file is in your working directory, you can import
|
||||||
|
it—via tidyverse’s `readr()`—as follows:
|
||||||
|
|
||||||
|
``` r
|
||||||
|
# requires package tidyverse
|
||||||
|
hp_books = read_csv("harry_potter_books.csv", col_types = "ccc")
|
||||||
|
```
|
||||||
|
|
||||||
|
This data set is fairly simple—in terms of its structure—although the
|
||||||
|
text content is far from being tidy. The dataset has 95085 rows and 3
|
||||||
|
columns:
|
||||||
|
|
||||||
|
1. `text`: text content
|
||||||
|
|
||||||
|
2. `book`: title of associated book
|
||||||
|
|
||||||
|
3. `chapter`: associated chapter number
|
||||||
|
|
||||||
|
### Harry Potter R-Data Files
|
||||||
|
|
||||||
|
The data of each book is also available in its own R-Data `rda` file
|
||||||
|
(see [rda-data-files/](rda-data-files)):
|
||||||
|
|
||||||
|
- `"philosophers_stone.rda"`
|
||||||
|
- `"chamber_of_secrets.rda"`
|
||||||
|
- `"prisoner_of_azkaban.rda"`
|
||||||
|
- `"goblet_of_fire.rda"`
|
||||||
|
- `"order_of_the_phoenix.rda"`
|
||||||
|
- `"half_blood_prince.rda"`
|
||||||
|
- `"deathly_hallows.rda"`
|
||||||
|
|
||||||
|
These files come from the R package `"harrypotter"` by Bradley Boehmke
|
||||||
|
|
||||||
|
<https://github.com/bradleyboehmke/harrypotter>
|
||||||
|
|
||||||
|
To import these files use the `load()` function. For example, consider
|
||||||
|
the first book “Harry Potter and the Philosopher’s Stone”; here’s how to
|
||||||
|
`load()` it in R:
|
||||||
|
|
||||||
|
``` r
|
||||||
|
# assuming that the rda file is in your working directory
|
||||||
|
load("philosophers_stone.rda")
|
||||||
|
```
|
||||||
|
|
||||||
|
Assuming that `"philosophers_stone.rda"` has been loaded, the text of
|
||||||
|
this book is available in the homonym character vector
|
||||||
|
`philosophers_stone`
|
||||||
|
|
||||||
|
``` r
|
||||||
|
# text is in a character vector
|
||||||
|
# (with as many elements as chapters in the book)
|
||||||
|
length(philosophers_stone)
|
||||||
|
#> [1] 17
|
||||||
|
```
|
||||||
|
|
||||||
|
The number of elements in `philosophers_stone` corresponds to the number
|
||||||
|
of chapters in this book: 17 chapters.
|
||||||
|
|
||||||
|
You may want to use these files to perform bigram analysis (or other
|
||||||
|
type of n-gram analysis).
|
||||||
|
|
||||||
|
### Sentiment Lexicons
|
||||||
|
|
||||||
|
In addition to the Harry Potter text, you can also find data for a
|
||||||
|
handful of sentiment lexicons from the R package `"textdata"` (by
|
||||||
|
Hvitfeldt and Silge):
|
||||||
|
|
||||||
|
- `"bing"`: Bing Liu’s General purpose English sentiment lexicon that
|
||||||
|
categorizes words in a binary fashion, either positive or negative
|
||||||
|
|
||||||
|
- `"afinn"`: AFINN is a lexicon of English words rated for valence
|
||||||
|
with an integer between minus five (negative) and plus five
|
||||||
|
(positive). The words have been manually labeled by Finn Årup
|
||||||
|
Nielsen in 2009-2011.
|
||||||
|
|
||||||
|
- `"nrc"`: General purpose English sentiment/emotion lexicon. This
|
||||||
|
lexicon labels words with six possible sentiments or emotions:
|
||||||
|
“negative”, “positive”, “anger”, “anticipation”, “disgust”, “fear”,
|
||||||
|
“joy”, “sadness”, “surprise”, or “trust”. The annotations were
|
||||||
|
manually done through Amazon’s Mechanical Turk.
|
||||||
|
|
||||||
|
- `"loughran"`: English sentiment lexicon created for use with
|
||||||
|
financial documents. This lexicon labels words with six possible
|
||||||
|
sentiments important in financial contexts: “negative”, “positive”,
|
||||||
|
“litigious”, “uncertainty”, “constraining”, or “superfluous”.
|
||||||
|
|
||||||
|
These lexicons come in `rda` data files (see
|
||||||
|
[sentiment-lexicons/](sentiment-lexicons)):
|
||||||
|
|
||||||
|
- `bing.rda`
|
||||||
|
- `afinn.rda`
|
||||||
|
- `bing.rda`
|
||||||
|
- `loughran.rda`
|
||||||
|
|
||||||
|
To import them in R, use the `load()` function. For example, here’s how
|
||||||
|
to import the Bing lexicon:
|
||||||
|
|
||||||
|
``` r
|
||||||
|
# assuming that the rda files are in your working directory
|
||||||
|
load("bing.rda")
|
||||||
|
```
|
||||||
|
|
||||||
|
Assuming that you’ve loaded the file `"bing.rda"`, the associated
|
||||||
|
lexicon is available in the homonym tibble `bing`
|
||||||
|
|
||||||
|
``` r
|
||||||
|
bing
|
||||||
|
#> # A tibble: 6,786 × 2
|
||||||
|
#> word sentiment
|
||||||
|
#> <chr> <chr>
|
||||||
|
#> 1 2-faces negative
|
||||||
|
#> 2 abnormal negative
|
||||||
|
#> 3 abolish negative
|
||||||
|
#> 4 abominable negative
|
||||||
|
#> 5 abominably negative
|
||||||
|
#> 6 abominate negative
|
||||||
|
#> 7 abomination negative
|
||||||
|
#> 8 abort negative
|
||||||
|
#> 9 aborted negative
|
||||||
|
#> 10 aborts negative
|
||||||
|
#> # … with 6,776 more rows
|
||||||
|
```
|
||||||
Loading…
x
Reference in New Issue
Block a user