Working on the front-end
Some checks failed
Build / build (push) Has been cancelled

This commit is contained in:
Sebastiaan de Schaetzen 2025-03-13 06:24:41 +01:00
parent 6fd1850e20
commit 995fc6ceea
8 changed files with 30 additions and 25 deletions

View File

@ -1,18 +1,13 @@
# vivaplusdl # Viva++
Viva+DL is a tool that scrapes Viva+ and stores videos and metadata in the same format as [https://github.com/jmbannon/ytdl-sub](ytdl-sub). Viva++ is a tool that scrapes Viva+ and provides a TV-friendly user-interface with some additional features not present on Viva+.
This ensures one can transition from ytdl-sub (with which you can only get the free content from the VLDL Youtube channel) to Viva+.
![Screenshot of Season 2014 in Jellyfin](screenshot.png)
## Environment Variables ## Environment Variables
The following environment variables can be used to configure vivaplusdl: The following environment variables can be used to configure viva++:
- `VIVAPLUS_USER`: The username for logging into Viva+ - `VIVAPLUS_USER`: The username for logging into Viva+
- `VIVAPLUS_PASS`: The password for logging into Viva+ - `VIVAPLUS_PASS`: The password for logging into Viva+
- `VIVAPLUS_SLEEPTIME`: Amount of time to sleep between scrapes in minutes. Defaults to 15. - `VIVAPLUS_SLEEPTIME`: Amount of time to sleep between scrapes in minutes. Defaults to 15.
- `VIVAPLUS_DATABASE`: Path to SQLite database holding metadata, scrape links, and scrape statuses for videos. Defautls to `videos.db3` - `VIVAPLUS_DATABASE`: Path to SQLite database holding metadata, scrape links, and scrape statuses for videos. Defautls to `videos.db3`
- `VIVAPLUS_DOWNLOAD`: Directory in which temporary files will be stored. **This directory and all its contents are removed on startup**
- `VIVAPLUS_DESTINATION`: Directory that converted videos and NFO metadata files will be stored. The directory structure and metadata is compatible with Jellyfin, and probably only Jellyfin.
## Database Structure ## Database Structure
The database contains a single table called `videos` with the following columns: The database contains a single table called `videos` with the following columns:
@ -25,22 +20,12 @@ The database contains a single table called `videos` with the following columns:
- `cast`: The URL to a stream mux which can be downloaded directly using ytdlp. Note that these containing identifying information, and are time restricted. If one needs to redownload a video, set this column to `NULL` before starting vivaplusdl. - `cast`: The URL to a stream mux which can be downloaded directly using ytdlp. Note that these containing identifying information, and are time restricted. If one needs to redownload a video, set this column to `NULL` before starting vivaplusdl.
- `description`: The description added to the video. Not always present. - `description`: The description added to the video. Not always present.
- `year`: The year part of `upload_date`. Added as a separate column to make some queries a little easier. - `year`: The year part of `upload_date`. Added as a separate column to make some queries a little easier.
- `episode`: The episode number. See the section about episode numbering. - `episode`: A number that orders different videos on a single day. E.g., when two videos are uploaded on 13/03/2025, the first one will be episode 1, the second will be episode 2.
- `run`: The run during which this episode was scraped. Every time a run starts that finds at least one new video, the run will be increment by one. This field is necessary to properly calculate the episode numbers across runs. - `run`: The run during which this episode was scraped. Every time a run starts that finds at least one new video, the run will be increment by one. This field is necessary to properly calculate the episode numbers across runs.
- `state`: The download state of the video. `done` indicates that the video is already imported into Jellyfin. `pending` means that no attempt to import it has occurred as yet, or that all imports attempts have resulted in errors. - `state`: The download state of the video. `done` indicates that the video is already imported. `pending` means that no attempt to import it has occurred as yet, or that all imports attempts have resulted in errors. `local` is similar to `done`, but with the additional implication that the video is available locally.
- `thumbnail`: Link to the video thumbnail - `thumbnail`: Link to the video thumbnail
## Episode Numbering ## Fetching Process
The episodes are numbered in (almost) the exact same way as ytdl-sub numbers videos. This makes transitioning easier.
Episodes numbers are in the format of `MMDDEE`, where:
- `MM` is the number month the episode was uploaded on. For the months 1 through 9, no leading zero is present
- `DD` is the day the episode was uploaded on. For single digit days, a leading zero is added.
- `EE` is the episode number for the database. The first video on a day gets the number `01`, the second gets `02`, etc. The next day the episode resets back to 1. This ensure that multi-part episodes that are uploaded on the same day (such as the *Which Star Wars movie is better?*-trilogy) are shown by Jellyfin in the proper order.
Additionally, each episode needs a season. The season number used is simply the year at which it was uploaded.
## Download Process
This tools makes use of Playwright to interact with the website (as there is no API to find this information). This tools makes use of Playwright to interact with the website (as there is no API to find this information).
It goes through these steps to download episodes: It goes through these steps to download episodes:
@ -48,7 +33,6 @@ It goes through these steps to download episodes:
2) Go to the all videos pages sorted from newest to oldest and press the *End* key until we find a video that is already present in our database. During the one-time seeding process, the oldest video is manually added to the database using a SQL migration. For each video the `url` and `run` are stored in the database. The database state for this episode is now set to `pending`. 2) Go to the all videos pages sorted from newest to oldest and press the *End* key until we find a video that is already present in our database. During the one-time seeding process, the oldest video is manually added to the database using a SQL migration. For each video the `url` and `run` are stored in the database. The database state for this episode is now set to `pending`.
3) Each video that does not have metadata (the `cast` column is set to null), we fetch the video page and extract the title, upload date, description, and cast url. The record is updated to contain this information. 3) Each video that does not have metadata (the `cast` column is set to null), we fetch the video page and extract the title, upload date, description, and cast url. The record is updated to contain this information.
4) The proper episode numbers (or at least, the `EE` part of it) is calculated. This steps performs no network requests. 4) The proper episode numbers (or at least, the `EE` part of it) is calculated. This steps performs no network requests.
5) Download the episodes into the temporary directory using ytdlp. Once the video is downloaded successfully, the video is moved into Jellyfin, and the XML sidecar is written containing the proper description, title, year, and air date, and finally the thumbnail is written into a file with a Jellyfin-compatible name. The database state for this episode is now set to `done`.
If any errors occur during the process, the program will log an error and quit. If any errors occur during the process, the program will log an error and quit.
When running it as a Docker container, Docker will automatically start it again. When running it as a Docker container, Docker will automatically start it again.

View File

@ -11,10 +11,10 @@ func staticFileServer() http.Handler {
return http.FileServer(http.Dir("./static")) return http.FileServer(http.Dir("./static"))
} }
func renderTemplate(wr io.Writer) error { func renderTemplate(wr io.Writer, templateName string, viewModel any) error {
templates, err := template.ParseGlob("web/*.gohtml") templates, err := template.ParseGlob("web/*.gohtml")
if err != nil { if err != nil {
return err return err
} }
return templates.Execute(wr, nil) return templates.Execute(wr, viewModel)
} }

View File

@ -0,0 +1 @@
update videos set state = 'local' where state = 'done';

View File

@ -0,0 +1 @@
alter table videos rename state to fetch_state;

View File

@ -7,3 +7,7 @@ body {
h1 { h1 {
text-transform: uppercase; text-transform: uppercase;
} }
.video-box {
border: 1px solid white;
}

6
viewmodels.go Normal file
View File

@ -0,0 +1,6 @@
package main
type VideoInfoVM struct {
Title string
Description string
}

View File

@ -1,3 +1,4 @@
{{- /*gotype: vivaplusdl.VideoInfoVM*/}}
<!DOCTYPE html> <!DOCTYPE html>
<html lang="en"> <html lang="en">
<head> <head>
@ -7,5 +8,9 @@
</head> </head>
<body> <body>
<h1>Viva++</h1> <h1>Viva++</h1>
<div class="video-box">
<h2>{{.Title}}</h2>
<div>{{.Description}}</div>
</div>
</body> </body>
</html> </html>

View File

@ -11,7 +11,11 @@ func serveWebview() {
log.Printf("Listening on %s", addr) log.Printf("Listening on %s", addr)
http.Handle("/static/", http.StripPrefix("/static/", staticFileServer())) http.Handle("/static/", http.StripPrefix("/static/", staticFileServer()))
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
err := renderTemplate(w) vm := VideoInfoVM{
Title: "Foo",
Description: "Lorem Ipsum sed dolor...",
}
err := renderTemplate(w, "index", vm)
if err != nil { if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError) http.Error(w, err.Error(), http.StatusInternalServerError)
} }