Compare commits

...

13 Commits

Author SHA1 Message Date
Kieran Eglin 2dfdb31aa4 Fixed release tag hopefully 2024-05-14 11:03:36 -07:00
Kieran Eglin c27e1dba96 Updated GH actions release template 2024-05-14 10:56:55 -07:00
Kieran Eglin 65b8790a35 Bumped version 2024-05-14 10:35:25 -07:00
Kieran Eglin b7c6f272a2 Updated README 2024-05-13 14:43:20 -07:00
Kieran a38ffbc55b [Enhancement] Allow redownloading of files for existing media items (#239)
* Added ability to specify overwrite behaviour when downloading media

* Added helper for redownloading media items

* renamed media redownload worker to disambiguate it from similarly named methods

* Added new redownload option to source actions dropdown

* Refactored MediaQuery to use a __using__ macro

* docs
2024-05-13 14:25:39 -07:00
Kieran Eglin 5c86e7192e Removed 'v' prefix from sidebar 2024-05-13 09:39:46 -07:00
Kieran 77a9ff4d9f Made tz data location user-configurable (#238) 2024-05-13 09:27:16 -07:00
Kieran Eglin d093799801 Bumped version 2024-05-10 10:05:47 -07:00
Kieran 90f709dbc8 Updated modals and README (#235) 2024-05-10 10:05:22 -07:00
Kieran c5975e08da Removed query params to get rid of flicker (#233) 2024-05-08 10:14:25 -07:00
Kieran 933daf8c78 [Housekeeping] Improve test stability (#232)
* Ensured all tests use at least the DataCase module

* addressed one flakey test

* More flakey tests

* Consolidated Mox usage
2024-05-08 09:57:36 -07:00
Kieran b5d2ba4191 [Housekeeping] Reduce Docker image size (#231)
* First pass at cutting down image size

* Made selfhosted docker work multiplatform

* Updated compile-time value to be runtime

* Fixed issue with tzdata file permissions

* Applied better fix for tz issue
2024-05-08 09:01:21 -07:00
Kieran a1a568b7f7 [Enhancement] Add history table to homepage (And TZ support) (#229)
* Added basic history table to homepage

* Improved homepage history query

* Set table to use 24h time

* Added timezone support

* Updated README to reflect new TZ stuff
2024-05-07 12:16:57 -07:00
79 changed files with 587 additions and 254 deletions
+1 -3
View File
@@ -40,9 +40,7 @@ jobs:
ghcr.io/${{ github.repository }}
# All non-release actions will be tagged as `dev` (ie: push, workflow_dispatch)
tags: |
type=semver,pattern={{version}},prefix=v
type=semver,pattern={{major}}.{{minor}},prefix=v
type=semver,pattern={{major}},prefix=v
type=ref,event=tag
type=raw,value=dev,enable=${{ github.event_name != 'release' }}
flavor: |
latest=auto
+10 -3
View File
@@ -40,9 +40,9 @@
## What it does
Pinchflat is a self-hosted app for downloading YouTube content built using [yt-dlp](https://github.com/yt-dlp/yt-dlp). It's designed to be lightweight, self-contained, and easy to use. You set up rules for how to download content from YouTube channels or playlists and it'll do the rest, checking periodically for new content. It's perfect for people who want to download content for use in with a media center app (Plex, Jellyfin, Kodi) or for those who want to archive media!
Pinchflat is a self-hosted app for downloading YouTube content built using [yt-dlp](https://github.com/yt-dlp/yt-dlp). It's designed to be lightweight, self-contained, and easy to use. You set up rules for how to download content from YouTube channels or playlists and it'll do the rest, periodically checking for new content. It's perfect for people who want to download content for use in with a media center app (Plex, Jellyfin, Kodi) or for those who want to archive media!
It's _not_ great for downloading one-off videos - it's built to download large amounts of content and keep it up to date. It's also not meant for consuming content in-app - Pinchflat downloads content to disk where you can then watch it with a media center app or VLC.
While you can [download individual videos](https://github.com/kieraneglin/pinchflat/wiki/Frequently-Asked-Questions#how-do-i-download-one-off-videos), Pinchflat is best suited for downloading content from channels or playlists. It's also not meant for consuming content in-app - Pinchflat downloads content to disk where you can then watch it with a media center app or VLC.
If it doesn't work for your use case, please make a feature request! You can also check out these great alternatives: [Tube Archivist](https://github.com/tubearchivist/tubearchivist), [ytdl-sub](https://github.com/jmbannon/ytdl-sub), and [TubeSync](https://github.com/meeb/tubesync)
@@ -91,6 +91,9 @@ version: '3'
services:
pinchflat:
image: keglin/pinchflat:latest
environment:
# Set the timezone to your local timezone
- TZ=America/New_York
ports:
- '8945:8945'
volumes:
@@ -110,7 +113,9 @@ services:
```bash
# Be sure to replace /host/path/to/config and /host/path/to/downloads below with
# the paths to the directories you created in step 1
# Be sure to replace America/New_York with your local timezone
docker run \
-e TZ=America/New_York \
-p 8945:8945 \
-v /host/path/to/config:/config \
-v /host/path/to/downloads:/downloads \
@@ -138,7 +143,9 @@ If you change this setting and it works well for you, please leave a comment on
## EFF donations
A portion of all donations to Pinchflat will be donated to the [Electronic Frontier Foundation](https://www.eff.org/). The EFF defends your online liberties and [backed](https://github.com/github/dmca/blob/9a85e0f021f7967af80e186b890776a50443f06c/2020/11/2020-11-16-RIAA-reversal-effletter.pdf) `youtube-dl` when Google took them down. [See here](https://github.com/kieraneglin/pinchflat/wiki/EFF-Donation-Receipts) for a list of donation receipts.
Prior to 2024-05-10, a portion of all donations were given to the [Electronic Frontier Foundation](https://www.eff.org/). Now, the app doesn't accept donations that go to me personally and instead directs you straight to the EFF. [Here](https://github.com/kieraneglin/pinchflat/issues/234) are some people that have generously donated.
The EFF defends your online liberties and [backed](https://github.com/github/dmca/blob/9a85e0f021f7967af80e186b890776a50443f06c/2020/11/2020-11-16-RIAA-reversal-effletter.pdf) `youtube-dl` when Google took them down.
## Pre-release disclaimer
+3 -2
View File
@@ -25,7 +25,8 @@ config :pinchflat,
basic_auth_username: "",
basic_auth_password: "",
expose_feed_endpoints: false,
file_watcher_poll_interval: 1000
file_watcher_poll_interval: 1000,
timezone: "UTC"
config :pinchflat, Pinchflat.Repo,
journal_mode: :wal,
@@ -53,7 +54,7 @@ config :pinchflat, Oban,
{Oban.Plugins.Cron,
crontab: [
{"0 1 * * *", Pinchflat.Downloading.MediaRetentionWorker},
{"0 2 * * *", Pinchflat.Downloading.MediaRedownloadWorker}
{"0 2 * * *", Pinchflat.Downloading.MediaQualityUpgradeWorker}
]}
],
# TODO: consider making this an env var or something?
+4 -1
View File
@@ -63,7 +63,10 @@ if config_env() == :prod do
extras_directory: extras_path,
tmpfile_directory: Path.join([System.tmp_dir!(), "pinchflat", "data"]),
dns_cluster_query: System.get_env("DNS_CLUSTER_QUERY"),
expose_feed_endpoints: expose_feed_endpoints
expose_feed_endpoints: expose_feed_endpoints,
timezone: System.get_env("TIMEZONE") || System.get_env("TZ") || "UTC"
config :tzdata, :data_dir, System.get_env("TZ_DATA_DIR", "/etc/elixir_tzdata_data")
config :pinchflat, Pinchflat.Repo,
database: db_path,
+19 -8
View File
@@ -5,10 +5,23 @@ ARG DEV_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEB
FROM ${DEV_IMAGE}
ARG TARGETPLATFORM
RUN echo "Building for ${TARGETPLATFORM:?}"
# Install debian packages
RUN apt-get update -qq
RUN apt-get install -y inotify-tools ffmpeg curl git openssh-client jq \
python3 python3-pip python3-setuptools python3-wheel python3-dev locales procps
RUN apt-get install -y inotify-tools curl git openssh-client jq \
python3 python3-setuptools python3-wheel python3-dev pipx \
python3-mutagen locales procps build-essential
# Install ffmpeg
RUN export FFMPEG_DOWNLOAD=$(case ${TARGETPLATFORM:-linux/amd64} in \
"linux/amd64") echo "https://github.com/yt-dlp/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz" ;; \
"linux/arm64") echo "https://github.com/yt-dlp/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linuxarm64-gpl.tar.xz" ;; \
*) echo "" ;; esac) && \
curl -L ${FFMPEG_DOWNLOAD} --output /tmp/ffmpeg.tar.xz && \
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffmpeg" && \
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffprobe"
# Install nodejs
RUN curl -sL https://deb.nodesource.com/setup_20.x -o nodesource_setup.sh
@@ -21,16 +34,14 @@ RUN mix local.hex --force
RUN mix local.rebar --force
# Download and update YT-DLP
# NOTE: If you're seeing weird issues, consider using the FFMPEG released by yt-dlp
RUN curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp
RUN chmod a+rx /usr/local/bin/yt-dlp
RUN yt-dlp -U
# Download Apprise
RUN python3 -m pip install -U apprise --break-system-packages
# Download Mutagen for music thumbnail generation
RUN python3 -m pip install -U mutagen --break-system-packages
# Install Apprise
RUN export PIPX_HOME=/opt/pipx && \
export PIPX_BIN_DIR=/usr/local/bin && \
pipx install apprise
# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
@@ -15,11 +15,11 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilder do
Returns {:ok, [Keyword.t()]}
"""
def build(%MediaItem{} = media_item_with_preloads) do
def build(%MediaItem{} = media_item_with_preloads, override_opts \\ []) do
media_profile = media_item_with_preloads.source.media_profile
built_options =
default_options() ++
default_options(override_opts) ++
subtitle_options(media_profile) ++
thumbnail_options(media_item_with_preloads) ++
metadata_options(media_profile) ++
@@ -50,11 +50,12 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilder do
build_output_path_for(%MediaItem{source: source_with_preloads})
end
defp default_options do
defp default_options(override_opts) do
overwrite_behaviour = Keyword.get(override_opts, :overwrite_behaviour, :force_overwrites)
[
:no_progress,
# Add force-overwrites to make sure redownloading works
:force_overwrites,
overwrite_behaviour,
# This makes the date metadata conform to what jellyfin expects
parse_metadata: "%(upload_date>%Y-%m-%d)s:(?P<meta_date>.+)"
]
@@ -7,6 +7,8 @@ defmodule Pinchflat.Downloading.DownloadingHelpers do
require Logger
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Media
alias Pinchflat.Tasks
@@ -64,4 +66,30 @@ defmodule Pinchflat.Downloading.DownloadingHelpers do
{:error, :should_not_download}
end
end
@doc """
For a given source, enqueues download jobs for all media items _that have already been downloaded_.
This is useful for when a source's download settings have changed and you want to run through all
existing media and retry the download. For instance, if the source didn't originally download thumbnails
and you've changed the source to download them, you can use this to download all the thumbnails for
existing media items.
NOTE: does not delete existing files whatsoever. Does not overwrite the existing media file if it exists
at the location it expects. Will cause a full redownload of everything if the output template has changed
NOTE: unrelated to the MediaQualityUpgradeWorker, which is for redownloading media items for quality upgrades
or improved sponsorblock segments
Returns [{:ok, %Task{}} | {:error, any()}]
"""
def kickoff_redownload_for_existing_media(%Source{} = source) do
MediaQuery.new()
|> MediaQuery.for_source(source)
|> MediaQuery.with_media_downloaded_at()
|> MediaQuery.where_download_not_prevented()
|> MediaQuery.where_not_culled()
|> Repo.all()
|> Enum.map(&MediaDownloadWorker.kickoff_with_task/1)
end
end
@@ -33,12 +33,18 @@ defmodule Pinchflat.Downloading.MediaDownloadWorker do
Does not download media if its source is set to not download media
(unless forced).
Options:
- `force`: force download even if the source is set to not download media. Fully
re-downloads media, including the video
- `quality_upgrade?`: re-downloads media, including the video. Does not force download
if the source is set to not download media
Returns :ok | {:ok, %MediaItem{}} | {:error, any, ...any}
"""
@impl Oban.Worker
def perform(%Oban.Job{args: %{"id" => media_item_id} = args}) do
should_force = Map.get(args, "force", false)
is_redownload = Map.get(args, "redownload?", false)
is_quality_upgrade = Map.get(args, "quality_upgrade?", false)
media_item =
media_item_id
@@ -47,7 +53,7 @@ defmodule Pinchflat.Downloading.MediaDownloadWorker do
# If the source or media item is set to not download media, perform a no-op unless forced
if (media_item.source.download_media && !media_item.prevent_download) || should_force do
download_media_and_schedule_jobs(media_item, is_redownload)
download_media_and_schedule_jobs(media_item, is_quality_upgrade, should_force)
else
:ok
end
@@ -56,13 +62,16 @@ defmodule Pinchflat.Downloading.MediaDownloadWorker do
Ecto.StaleEntryError -> Logger.info("#{__MODULE__} discarded: media item #{media_item_id} stale")
end
defp download_media_and_schedule_jobs(media_item, is_redownload) do
case MediaDownloader.download_for_media_item(media_item) do
defp download_media_and_schedule_jobs(media_item, is_quality_upgrade, should_force) do
overwrite_behaviour = if should_force || is_quality_upgrade, do: :force_overwrites, else: :no_force_overwrites
override_opts = [overwrite_behaviour: overwrite_behaviour]
case MediaDownloader.download_for_media_item(media_item, override_opts) do
{:ok, downloaded_media_item} ->
{:ok, updated_media_item} =
Media.update_media_item(downloaded_media_item, %{
media_size_bytes: compute_media_filesize(downloaded_media_item),
media_redownloaded_at: get_redownloaded_at(is_redownload)
media_redownloaded_at: get_redownloaded_at(is_quality_upgrade)
})
:ok = run_user_script(updated_media_item)
@@ -29,11 +29,11 @@ defmodule Pinchflat.Downloading.MediaDownloader do
Returns {:ok, %MediaItem{}} | {:error, any, ...any}
"""
def download_for_media_item(%MediaItem{} = media_item) do
def download_for_media_item(%MediaItem{} = media_item, override_opts \\ []) do
output_filepath = FilesystemUtils.generate_metadata_tmpfile(:json)
media_with_preloads = Repo.preload(media_item, [:metadata, source: :media_profile])
case download_with_options(media_item.original_url, media_with_preloads, output_filepath) do
case download_with_options(media_item.original_url, media_with_preloads, output_filepath, override_opts) do
{:ok, parsed_json} ->
update_media_item_from_parsed_json(media_with_preloads, parsed_json)
@@ -103,8 +103,8 @@ defmodule Pinchflat.Downloading.MediaDownloader do
end
end
defp download_with_options(url, item_with_preloads, output_filepath) do
{:ok, options} = DownloadOptionBuilder.build(item_with_preloads)
defp download_with_options(url, item_with_preloads, output_filepath, override_opts) do
{:ok, options} = DownloadOptionBuilder.build(item_with_preloads, override_opts)
YtDlpMedia.download(url, options, output_filepath: output_filepath)
end
@@ -1,4 +1,4 @@
defmodule Pinchflat.Downloading.MediaRedownloadWorker do
defmodule Pinchflat.Downloading.MediaQualityUpgradeWorker do
@moduledoc false
use Oban.Worker,
@@ -12,7 +12,9 @@ defmodule Pinchflat.Downloading.MediaRedownloadWorker do
alias Pinchflat.Downloading.MediaDownloadWorker
@doc """
Redownloads media items that are eligible for redownload.
Redownloads media items that are eligible for redownload for the purpose
of upgrading the quality of the media or improving things like sponsorblock
segments.
This worker is scheduled to run daily via the Oban Cron plugin
and it should run _after_ the retention worker.
@@ -25,7 +27,7 @@ defmodule Pinchflat.Downloading.MediaRedownloadWorker do
Logger.info("Redownloading #{length(redownloadable_media)} media items")
Enum.each(redownloadable_media, fn media_item ->
MediaDownloadWorker.kickoff_with_task(media_item, %{redownload?: true})
MediaDownloadWorker.kickoff_with_task(media_item, %{quality_upgrade?: true})
end)
end
end
@@ -7,10 +7,11 @@ defmodule Pinchflat.FastIndexing.FastIndexingHelpers do
require Logger
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Media
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaQuery
alias Pinchflat.FastIndexing.YoutubeRss
alias Pinchflat.Downloading.DownloadingHelpers
@@ -5,8 +5,9 @@ defmodule Pinchflat.Lifecycle.Notifications.SourceNotifications do
require Logger
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Media.MediaQuery
@doc """
Wraps a function that may change the number of pending or downloaded
+1 -1
View File
@@ -4,12 +4,12 @@ defmodule Pinchflat.Media do
"""
import Ecto.Query, warn: false
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Tasks
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
alias Pinchflat.Media.MediaQuery
alias Pinchflat.Utils.FilesystemUtils
alias Pinchflat.Metadata.MediaMetadata
+2 -1
View File
@@ -4,6 +4,8 @@ defmodule Pinchflat.Media.MediaItem do
"""
use Ecto.Schema
use Pinchflat.Media.MediaQuery
import Ecto.Changeset
import Pinchflat.Utils.ChangesetUtils
@@ -12,7 +14,6 @@ defmodule Pinchflat.Media.MediaItem do
alias Pinchflat.Sources
alias Pinchflat.Tasks.Task
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaQuery
alias Pinchflat.Metadata.MediaMetadata
alias Pinchflat.Media.MediaItemsSearchIndex
+19
View File
@@ -12,6 +12,17 @@ defmodule Pinchflat.Media.MediaQuery do
alias Pinchflat.Media.MediaItem
# This allows the module to be aliased and query methods to be used
# all in one go
# usage: use Pinchflat.Media.MediaQuery
defmacro __using__(_opts) do
quote do
import Ecto.Query, warn: false
alias unquote(__MODULE__)
end
end
# Prefixes:
# - for_* - belonging to a certain record
# - join_* - for joining on a certain record
@@ -21,6 +32,8 @@ defmodule Pinchflat.Media.MediaQuery do
# Suffixes:
# - _for - the arg passed is an association record
# NOTE: that dyanmic query approach kinda rocked - should refactor in future
def new do
MediaItem
end
@@ -172,6 +185,12 @@ defmodule Pinchflat.Media.MediaQuery do
|> matching_source_title_regex()
end
def where_pending_or_downloaded(query) do
query
|> where_pending_download()
|> or_where([mi], not is_nil(mi.media_downloaded_at))
end
defp require_assoc(query, identifier) do
if has_named_binding?(query, identifier) do
query
+2 -1
View File
@@ -4,8 +4,9 @@ defmodule Pinchflat.Podcasts.PodcastHelpers do
or its media items
"""
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Media.MediaQuery
alias Pinchflat.Metadata.MediaMetadata
alias Pinchflat.Metadata.SourceMetadata
+6 -3
View File
@@ -26,14 +26,17 @@ defmodule Pinchflat.Release do
load_app()
directories =
Enum.uniq([
[
"/config",
"/downloads",
Application.get_env(:pinchflat, :media_directory),
Application.get_env(:pinchflat, :tmpfile_directory),
Application.get_env(:pinchflat, :extras_directory),
Application.get_env(:pinchflat, :metadata_directory)
])
Application.get_env(:pinchflat, :metadata_directory),
Application.get_env(:tzdata, :data_dir)
]
|> Enum.uniq()
|> Enum.filter(&(&1 != nil))
Enum.each(directories, fn dir ->
Logger.info("Checking permissions for #{dir}")
+2 -2
View File
@@ -4,12 +4,12 @@ defmodule Pinchflat.Sources do
"""
import Ecto.Query, warn: false
alias Pinchflat.Repo
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Media
alias Pinchflat.Tasks
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaQuery
alias Pinchflat.Profiles.MediaProfile
alias Pinchflat.YtDlp.MediaCollection
alias Pinchflat.Metadata.SourceMetadata
@@ -27,21 +27,20 @@ defmodule PinchflatWeb.CustomComponents.TableComponents do
def table(assigns) do
~H"""
<table class={["w-full table-auto", @table_class]}>
<table class={["w-full table-auto bg-boxdark", @table_class]}>
<thead>
<tr class="bg-gray-2 text-left dark:bg-meta-4">
<th :for={col <- @col} class="px-4 py-4 font-medium text-black dark:text-white xl:pl-11">
<tr class="text-left bg-meta-4">
<th :for={col <- @col} class="px-4 py-4 font-medium text-white xl:pl-11">
<%= col[:label] %>
</th>
</tr>
</thead>
<tbody>
<tr :for={{row, i} <- Enum.with_index(@rows)}>
<tr :for={row <- @rows} class="border-b border-boxdark hover:border-strokedark">
<td
:for={col <- @col}
class={[
"px-4 py-5 pl-9 dark:border-strokedark xl:pl-11",
i + 1 > length(@rows) && "border-b border-[#eee] dark:border-π",
"px-4 py-5 pl-9 xl:pl-11",
col[:class]
]}
>
@@ -78,4 +78,20 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
<span><%= @text %></span>
"""
end
@doc """
Renders a UTC datetime in the specified format and timezone
"""
attr :datetime, :any, required: true
attr :format, :string, default: "%Y-%m-%d %H:%M:%S"
attr :timezone, :string, default: nil
def datetime_in_zone(assigns) do
timezone = assigns.timezone || Application.get_env(:pinchflat, :timezone)
assigns = Map.put(assigns, :timezone, timezone)
~H"""
<time><%= Calendar.strftime(Timex.Timezone.convert(@datetime, @timezone), @format) %></time>
"""
end
end
@@ -3,7 +3,7 @@
<header class="sticky top-0 z-999 flex w-full bg-white drop-shadow-1 dark:bg-boxdark dark:drop-shadow-none">
<div class="w-65 px-4 py-2 shadow-2 md:px-6">
<div class="flex items-center gap-2 py-2">
<img src={~p"/images/logo.png?cachebust=2024-03-20"} alt="Pinchflat" class="w-auto" />
<img src={~p"/images/logo-2024-03-20.png"} alt="Pinchflat" class="w-auto" />
</div>
</div>
</header>
@@ -3,21 +3,22 @@
<h3 class="text-2xl text-white">Donate</h3>
<p class="text-sm">Thank you for your support :&rpar;</p>
<p class="mt-4">
If you find the project valuable and want to support its development, a
<.inline_link href="https://www.paypal.me/kieraneglin">donation</.inline_link>
would be greatly appreciated.
If you find the project valuable and want to say thanks, a
<.inline_link href="https://supporters.eff.org/donate">
donation to the Electronic Frontier Foundation
</.inline_link>
would be greatly appreciated! You can optionally post proof of donation on
<.inline_link href="https://github.com/kieraneglin/pinchflat/issues/234">this issue</.inline_link>
to hopefully inspire others to donate as well.
</p>
<p class="mt-4">
Plus, $5 USD from any donation over $10 USD will be donated to
<.inline_link href="https://www.eff.org/">The Electronic Frontier Foundation</.inline_link>
who defend your online liberties and backed
The Electronic Frontier Foundation is a nonprofit organization who defend your online liberties and backed
<.inline_code>youtube-dl</.inline_code>
when Google took them down<.inline_link href="https://github.com/github/dmca/blob/9a85e0f021f7967af80e186b890776a50443f06c/2020/11/2020-11-16-RIAA-reversal-effletter.pdf">
<.icon name="hero-arrow-top-right-on-square" class="h-3 w-3" />
</.inline_link>.
</.inline_link>
</p>
<.link href="https://www.paypal.me/kieraneglin" target="_blank">
<.link href="https://supporters.eff.org/donate" target="_blank">
<.button color="bg-primary" class="w-full mt-8">
Donate
</.button>
@@ -2,7 +2,7 @@
<div class="flex flex-grow items-center justify-between lg:justify-end px-4 py-4 shadow-2 md:px-6 2xl:px-11">
<div class="flex items-center gap-2 sm:gap-4 lg:hidden w-2/6">
<section class="pr-1">
<img src={~p"/images/icon.png?cachebust=2024-03-20"} alt="Pinchflat" class="w-10" />
<img src={~p"/images/icon-2024-03-20.png"} alt="Pinchflat" class="w-10" />
</section>
<button
@@ -9,7 +9,7 @@
<section>
<div class="flex items-center justify-between gap-2 px-6 py-4">
<a href="/" class="flex items-center">
<img src={~p"/images/logo.png?cachebust=2024-03-20"} alt="Pinchflat" class="w-auto" />
<img src={~p"/images/logo-2024-03-20.png"} alt="Pinchflat" class="w-auto" />
</a>
<button class="block mt-3 lg:hidden" @click.stop="sidebarVisible = !sidebarVisible">
@@ -18,7 +18,9 @@
</div>
<div class="no-scrollbar flex flex-col overflow-y-auto duration-300 ease-linear">
<nav class="mt-3 px-4 py-4 lg:px-6">
<h3 class="mb-4 ml-4 text-sm font-medium text-bodydark2">MENU</h3>
<h3 class="mb-4 ml-4 text-sm font-medium text-bodydark2">
<span>MENU</span>
</h3>
<div class="flex flex-col justify-between">
<ul class="mb-6 flex flex-col gap-1.5">
<.sidebar_item icon="hero-home" text="Home" href={~p"/"} />
@@ -58,7 +60,7 @@
class="group relative flex items-center gap-2.5 px-4 pt-2 text-sm"
x-on:click={"markVersionAsSeen('#{Application.spec(:pinchflat)[:vsn]}')"}
>
<span>Pinchflat v<%= Application.spec(:pinchflat)[:vsn] %></span>
<span>Pinchflat <%= Application.spec(:pinchflat)[:vsn] %></span>
<a
href="https://github.com/kieraneglin/pinchflat/releases"
target="_blank"
@@ -30,7 +30,7 @@ defmodule Pinchflat.UpgradeButtonLive do
|> String.trim()
|> String.downcase()
if normalized_text == "got it!" do
if normalized_text == "got it" do
Settings.set(pro_enabled: true)
{:noreply, update(socket, :button_disabled, fn _ -> false end)}
@@ -3,18 +3,20 @@
<h3 class="text-2xl text-white">Pro Mode</h3>
<p class="text-sm">Don't worry - Pinchflat is completely free :&rpar;</p>
<p class="mt-4">
If you find the project valuable and want to support its development, a
<.inline_link href="https://www.paypal.me/kieraneglin">donation</.inline_link>
would be greatly appreciated.
If you find the project valuable and want to say thanks, a
<.inline_link href="https://supporters.eff.org/donate">
donation to the Electronic Frontier Foundation
</.inline_link>
would be greatly appreciated! You can optionally post proof of donation on
<.inline_link href="https://github.com/kieraneglin/pinchflat/issues/234">this issue</.inline_link>
to hopefully inspire others to donate as well.
</p>
<p class="mt-4">
Plus, $5 USD from any donation over $10 USD will be donated to
<.inline_link href="https://www.eff.org/">The Electronic Frontier Foundation</.inline_link>
who defend your online liberties and backed
The Electronic Frontier Foundation is a nonprofit organization who defend your online liberties and backed
<.inline_code>youtube-dl</.inline_code>
when Google took them down<.inline_link href="https://github.com/github/dmca/blob/9a85e0f021f7967af80e186b890776a50443f06c/2020/11/2020-11-16-RIAA-reversal-effletter.pdf">
<.icon name="hero-arrow-top-right-on-square" class="h-3 w-3" />
</.inline_link>. <strong>You do not need to donate to unlock Pro</strong>. It's just a way to say thanks!
</.inline_link>. <strong>You do not need to donate to unlock Pro</strong>. It's just a way to show support!
</p>
<p class="mt-4">
@@ -1,10 +1,9 @@
defmodule PinchflatWeb.Pages.PageController do
alias Pinchflat.Media.MediaItem
use PinchflatWeb, :controller
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
alias Pinchflat.Profiles.MediaProfile
def home(conn, params) do
@@ -25,7 +24,10 @@ defmodule PinchflatWeb.Pages.PageController do
|> render(:home,
media_profile_count: Repo.aggregate(MediaProfile, :count, :id),
source_count: Repo.aggregate(Source, :count, :id),
media_item_count: Repo.aggregate(MediaItem, :count, :id)
media_item_count:
MediaQuery.new()
|> MediaQuery.with_media_downloaded_at()
|> Repo.aggregate(:count, :id)
)
end
@@ -0,0 +1,109 @@
defmodule Pinchflat.Pages.HistoryTableLive do
use PinchflatWeb, :live_view
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Utils.NumberUtils
alias PinchflatWeb.CustomComponents.TextComponents
@limit 10
def render(%{records: []} = assigns) do
~H"""
<div class="mb-4 flex items-center">
<.icon_button icon_name="hero-arrow-path" class="h-10 w-10" phx-click="reload_page" />
<p class="ml-2">Nothing Here!</p>
</div>
"""
end
def render(assigns) do
~H"""
<div>
<span class="mb-4 flex items-center">
<.icon_button icon_name="hero-arrow-path" class="h-10 w-10" phx-click="reload_page" tooltip="Refresh" />
<span class="ml-2">Showing <%= length(@records) %> of <%= @total_record_count %></span>
</span>
<div class="max-w-full overflow-x-auto">
<.table rows={@records} table_class="text-white">
<:col :let={media_item} label="Title">
<.subtle_link href={~p"/sources/#{media_item.source_id}/media/#{media_item}"}>
<%= StringUtils.truncate(media_item.title, 35) %>
</.subtle_link>
</:col>
<:col :let={media_item} label="Upload Date">
<%= media_item.upload_date %>
</:col>
<:col :let={media_item} label="Indexed At">
<%= format_datetime(media_item.inserted_at) %>
</:col>
<:col :let={media_item} label="Downloaded At">
<%= format_datetime(media_item.media_downloaded_at) %>
</:col>
<:col :let={media_item} label="Source">
<.subtle_link href={~p"/sources/#{media_item.source_id}"}>
<%= StringUtils.truncate(media_item.source.custom_name, 35) %>
</.subtle_link>
</:col>
</.table>
</div>
<section class="flex justify-center mt-5">
<.live_pagination_controls page_number={@page} total_pages={@total_pages} />
</section>
</div>
"""
end
def mount(_params, _session, socket) do
page = 1
base_query = generate_base_query()
pagination_attrs = fetch_pagination_attributes(base_query, page)
{:ok, assign(socket, Map.merge(pagination_attrs, %{base_query: base_query}))}
end
def handle_event("page_change", %{"direction" => direction}, %{assigns: assigns} = socket) do
direction = if direction == "inc", do: 1, else: -1
new_page = assigns.page + direction
new_assigns = fetch_pagination_attributes(assigns.base_query, new_page)
{:noreply, assign(socket, new_assigns)}
end
def handle_event("reload_page", _params, %{assigns: assigns} = socket) do
new_assigns = fetch_pagination_attributes(assigns.base_query, assigns.page)
{:noreply, assign(socket, new_assigns)}
end
defp fetch_pagination_attributes(base_query, page) do
total_record_count = Repo.aggregate(base_query, :count, :id)
total_pages = max(ceil(total_record_count / @limit), 1)
page = NumberUtils.clamp(page, 1, total_pages)
records = fetch_records(base_query, page)
%{page: page, total_pages: total_pages, records: records, total_record_count: total_record_count}
end
defp fetch_records(base_query, page) do
offset = (page - 1) * @limit
base_query
|> limit(^@limit)
|> offset(^offset)
|> Repo.all()
|> Repo.preload(:source)
end
defp generate_base_query do
MediaQuery.new()
|> MediaQuery.where_pending_or_downloaded()
|> order_by(desc: :id)
end
defp format_datetime(nil), do: ""
defp format_datetime(datetime) do
TextComponents.datetime_in_zone(%{datetime: datetime, format: "%Y-%m-%d %H:%M"})
end
end
@@ -17,11 +17,17 @@
</div>
<div class="rounded-sm border px-7.5 py-6 shadow-default border-strokedark bg-boxdark">
<span class="mt-4 flex flex-col items-center justify-center">
<span class="text-md font-medium">Media Item(s)</span>
<span class="text-md font-medium">Downloaded Media</span>
<h4 class="text-title-md font-bold text-white">
<%= @media_item_count %>
</h4>
</span>
</div>
<span class="text-strokedark">I know this page isn't super useful yet, but give it time :&rpar;</span>
</div>
<div class="rounded-sm border shadow-default border-strokedark bg-boxdark mt-4 p-5">
<span class="text-2xl font-medium mb-4">History</span>
<section class="mt-6">
<%= live_render(@conn, Pinchflat.Pages.HistoryTableLive) %>
</section>
</div>
@@ -1,10 +1,10 @@
defmodule PinchflatWeb.Podcasts.PodcastController do
use PinchflatWeb, :controller
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
alias Pinchflat.Media.MediaQuery
alias Pinchflat.Podcasts.RssFeedBuilder
alias Pinchflat.Podcasts.PodcastHelpers
@@ -104,7 +104,7 @@ defmodule PinchflatWeb.Sources.SourceController do
|> redirect(to: ~p"/sources")
end
def force_download(conn, %{"source_id" => id}) do
def force_download_pending(conn, %{"source_id" => id}) do
wrap_forced_action(
conn,
id,
@@ -113,6 +113,15 @@ defmodule PinchflatWeb.Sources.SourceController do
)
end
def force_redownload(conn, %{"source_id" => id}) do
wrap_forced_action(
conn,
id,
"Forcing re-download of downloaded media items.",
&DownloadingHelpers.kickoff_redownload_for_existing_media/1
)
end
def force_index(conn, %{"source_id" => id}) do
wrap_forced_action(
conn,
@@ -31,11 +31,20 @@
</:option>
<:option :if={@source.download_media}>
<.link
href={~p"/sources/#{@source}/force_download"}
href={~p"/sources/#{@source}/force_download_pending"}
method="post"
data-confirm="Are you sure you want to force a download of all *pending* media items? This isn't normally needed."
>
Force Download
Download Pending
</.link>
</:option>
<:option :if={@source.download_media}>
<.link
href={~p"/sources/#{@source}/force_redownload"}
method="post"
data-confirm="Are you sure you want to re-download all currently downloaded media items? This isn't normally needed and won't change anything if the files already exist."
>
Redownload Existing
</.link>
</:option>
<:option>
@@ -1,10 +1,9 @@
defmodule Pinchflat.Sources.MediaItemTableLive do
use PinchflatWeb, :live_view
import Ecto.Query, warn: false
use Pinchflat.Media.MediaQuery
alias Pinchflat.Repo
alias Pinchflat.Sources
alias Pinchflat.Media.MediaQuery
alias Pinchflat.Utils.NumberUtils
@limit 10
+2 -1
View File
@@ -33,7 +33,8 @@ defmodule PinchflatWeb.Router do
resources "/settings", Settings.SettingController, only: [:show, :update], singleton: true
resources "/sources", Sources.SourceController do
post "/force_download", Sources.SourceController, :force_download
post "/force_download_pending", Sources.SourceController, :force_download_pending
post "/force_redownload", Sources.SourceController, :force_redownload
post "/force_index", Sources.SourceController, :force_index
post "/force_metadata_refresh", Sources.SourceController, :force_metadata_refresh
+2 -1
View File
@@ -4,7 +4,7 @@ defmodule Pinchflat.MixProject do
def project do
[
app: :pinchflat,
version: "0.1.17",
version: "2024.5.14",
elixir: "~> 1.16",
elixirc_paths: elixirc_paths(Mix.env()),
start_permanent: Mix.env() == :prod,
@@ -68,6 +68,7 @@ defmodule Pinchflat.MixProject do
{:plug_cowboy, "~> 2.5"},
{:oban, "~> 2.16"},
{:nimble_parsec, "~> 1.4"},
{:timex, "~> 3.0"},
{:mox, "~> 1.0", only: :test},
{:credo, "~> 1.7", only: [:dev, :test], runtime: false},
{:credo_naming, "~> 2.1", only: [:dev, :test], runtime: false},
+11
View File
@@ -2,6 +2,8 @@
"bunt": {:hex, :bunt, "1.0.0", "081c2c665f086849e6d57900292b3a161727ab40431219529f13c4ddcf3e7a44", [:mix], [], "hexpm", "dc5f86aa08a5f6fa6b8096f0735c4e76d54ae5c9fa2c143e5a1fc7c1cd9bb6b5"},
"castore": {:hex, :castore, "1.0.5", "9eeebb394cc9a0f3ae56b813459f990abb0a3dedee1be6b27fdb50301930502f", [:mix], [], "hexpm", "8d7c597c3e4a64c395980882d4bca3cebb8d74197c590dc272cfd3b6a6310578"},
"cc_precompiler": {:hex, :cc_precompiler, "0.1.9", "e8d3364f310da6ce6463c3dd20cf90ae7bbecbf6c5203b98bf9b48035592649b", [:mix], [{:elixir_make, "~> 0.7", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "9dcab3d0f3038621f1601f13539e7a9ee99843862e66ad62827b0c42b2f58a54"},
"certifi": {:hex, :certifi, "2.12.0", "2d1cca2ec95f59643862af91f001478c9863c2ac9cb6e2f89780bfd8de987329", [:rebar3], [], "hexpm", "ee68d85df22e554040cdb4be100f33873ac6051387baf6a8f6ce82272340ff1c"},
"combine": {:hex, :combine, "0.10.0", "eff8224eeb56498a2af13011d142c5e7997a80c8f5b97c499f84c841032e429f", [:mix], [], "hexpm", "1b1dbc1790073076580d0d1d64e42eae2366583e7aecd455d1215b0d16f2451b"},
"cowboy": {:hex, :cowboy, "2.10.0", "ff9ffeff91dae4ae270dd975642997afe2a1179d94b1887863e43f681a203e26", [:make, :rebar3], [{:cowlib, "2.12.1", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, "1.8.0", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm", "3afdccb7183cc6f143cb14d3cf51fa00e53db9ec80cdcd525482f5e99bc41d6b"},
"cowboy_telemetry": {:hex, :cowboy_telemetry, "0.4.0", "f239f68b588efa7707abce16a84d0d2acf3a0f50571f8bb7f56a15865aae820c", [:rebar3], [{:cowboy, "~> 2.7", [hex: :cowboy, repo: "hexpm", optional: false]}, {:telemetry, "~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "7d98bac1ee4565d31b62d59f8823dfd8356a169e7fcbb83831b8a5397404c9de"},
"cowlib": {:hex, :cowlib, "2.12.1", "a9fa9a625f1d2025fe6b462cb865881329b5caff8f1854d1cbc9f9533f00e1e1", [:make, :rebar3], [], "hexpm", "163b73f6367a7341b33c794c4e88e7dbfe6498ac42dcd69ef44c5bc5507c8db0"},
@@ -24,15 +26,20 @@
"finch": {:hex, :finch, "0.17.0", "17d06e1d44d891d20dbd437335eebe844e2426a0cd7e3a3e220b461127c73f70", [:mix], [{:castore, "~> 0.1 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: false]}, {:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.3", [hex: :mint, repo: "hexpm", optional: false]}, {:nimble_options, "~> 0.4 or ~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:nimble_pool, "~> 0.2.6 or ~> 1.0", [hex: :nimble_pool, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "8d014a661bb6a437263d4b5abf0bcbd3cf0deb26b1e8596f2a271d22e48934c7"},
"floki": {:hex, :floki, "0.35.2", "87f8c75ed8654b9635b311774308b2760b47e9a579dabf2e4d5f1e1d42c39e0b", [:mix], [], "hexpm", "6b05289a8e9eac475f644f09c2e4ba7e19201fd002b89c28c1293e7bd16773d9"},
"gettext": {:hex, :gettext, "0.24.0", "6f4d90ac5f3111673cbefc4ebee96fe5f37a114861ab8c7b7d5b30a1108ce6d8", [:mix], [{:expo, "~> 0.5.1", [hex: :expo, repo: "hexpm", optional: false]}], "hexpm", "bdf75cdfcbe9e4622dd18e034b227d77dd17f0f133853a1c73b97b3d6c770e8b"},
"hackney": {:hex, :hackney, "1.20.1", "8d97aec62ddddd757d128bfd1df6c5861093419f8f7a4223823537bad5d064e2", [:rebar3], [{:certifi, "~> 2.12.0", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~> 6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~> 1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~> 1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.4.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~> 1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "fe9094e5f1a2a2c0a7d10918fee36bfec0ec2a979994cff8cfe8058cd9af38e3"},
"hpax": {:hex, :hpax, "0.1.2", "09a75600d9d8bbd064cdd741f21fc06fc1f4cf3d0fcc335e5aa19be1a7235c84", [:mix], [], "hexpm", "2c87843d5a23f5f16748ebe77969880e29809580efdaccd615cd3bed628a8c13"},
"idna": {:hex, :idna, "6.1.1", "8a63070e9f7d0c62eb9d9fcb360a7de382448200fbbd1b106cc96d3d8099df8d", [:rebar3], [{:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "92376eb7894412ed19ac475e4a86f7b413c1b9fbb5bd16dccd57934157944cea"},
"jason": {:hex, :jason, "1.4.1", "af1504e35f629ddcdd6addb3513c3853991f694921b1b9368b0bd32beb9f1b63", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "fbb01ecdfd565b56261302f7e1fcc27c4fb8f32d56eab74db621fc154604a7a1"},
"metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm", "69b09adddc4f74a40716ae54d140f93beb0fb8978d8636eaded0c31b6f099f16"},
"mime": {:hex, :mime, "2.0.5", "dc34c8efd439abe6ae0343edbb8556f4d63f178594894720607772a041b04b02", [:mix], [], "hexpm", "da0d64a365c45bc9935cc5c8a7fc5e49a0e0f9932a761c55d6c52b142780a05c"},
"mimerl": {:hex, :mimerl, "1.3.0", "d0cd9fc04b9061f82490f6581e0128379830e78535e017f7780f37fea7545726", [:rebar3], [], "hexpm", "a1e15a50d1887217de95f0b9b0793e32853f7c258a5cd227650889b38839fe9d"},
"mint": {:hex, :mint, "1.5.2", "4805e059f96028948870d23d7783613b7e6b0e2fb4e98d720383852a760067fd", [:mix], [{:castore, "~> 0.1.0 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:hpax, "~> 0.1.1", [hex: :hpax, repo: "hexpm", optional: false]}], "hexpm", "d77d9e9ce4eb35941907f1d3df38d8f750c357865353e21d335bdcdf6d892a02"},
"mox": {:hex, :mox, "1.1.0", "0f5e399649ce9ab7602f72e718305c0f9cdc351190f72844599545e4996af73c", [:mix], [], "hexpm", "d44474c50be02d5b72131070281a5d3895c0e7a95c780e90bc0cfe712f633a13"},
"nimble_options": {:hex, :nimble_options, "1.1.0", "3b31a57ede9cb1502071fade751ab0c7b8dbe75a9a4c2b5bbb0943a690b63172", [:mix], [], "hexpm", "8bbbb3941af3ca9acc7835f5655ea062111c9c27bcac53e004460dfd19008a99"},
"nimble_parsec": {:hex, :nimble_parsec, "1.4.0", "51f9b613ea62cfa97b25ccc2c1b4216e81df970acd8e16e8d1bdc58fef21370d", [:mix], [], "hexpm", "9c565862810fb383e9838c1dd2d7d2c437b3d13b267414ba6af33e50d2d1cf28"},
"nimble_pool": {:hex, :nimble_pool, "1.0.0", "5eb82705d138f4dd4423f69ceb19ac667b3b492ae570c9f5c900bb3d2f50a847", [:mix], [], "hexpm", "80be3b882d2d351882256087078e1b1952a28bf98d0a287be87e4a24a710b67a"},
"oban": {:hex, :oban, "2.17.3", "ddfd5710aadcd550d2e174c8d73ce5f1865601418cf54a91775f20443fb832b7", [:mix], [{:ecto_sql, "~> 3.6", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:ecto_sqlite3, "~> 0.9", [hex: :ecto_sqlite3, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: false]}, {:postgrex, "~> 0.16", [hex: :postgrex, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "452eada8bfe0d0fefd0740ab5fa8cf3ef6c375df0b4a3c3805d179022a04738a"},
"parse_trans": {:hex, :parse_trans, "3.4.1", "6e6aa8167cb44cc8f39441d05193be6e6f4e7c2946cb2759f015f8c56b76e5ff", [:rebar3], [], "hexpm", "620a406ce75dada827b82e453c19cf06776be266f5a67cff34e1ef2cbb60e49a"},
"phoenix": {:hex, :phoenix, "1.7.10", "02189140a61b2ce85bb633a9b6fd02dff705a5f1596869547aeb2b2b95edd729", [:mix], [{:castore, ">= 0.0.0", [hex: :castore, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.6", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:plug_crypto, "~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:websock_adapter, "~> 0.5.3", [hex: :websock_adapter, repo: "hexpm", optional: false]}], "hexpm", "cf784932e010fd736d656d7fead6a584a4498efefe5b8227e9f383bf15bb79d0"},
"phoenix_ecto": {:hex, :phoenix_ecto, "4.4.3", "86e9878f833829c3f66da03d75254c155d91d72a201eb56ae83482328dc7ca93", [:mix], [{:ecto, "~> 3.5", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.14.2 or ~> 3.0 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "d36c401206f3011fefd63d04e8ef626ec8791975d9d107f9a0817d426f61ac07"},
"phoenix_html": {:hex, :phoenix_html, "3.3.3", "380b8fb45912b5638d2f1d925a3771b4516b9a78587249cabe394e0a5d579dc9", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: true]}], "hexpm", "923ebe6fec6e2e3b3e569dfbdc6560de932cd54b000ada0208b5f45024bdd76c"},
@@ -46,12 +53,16 @@
"plug_crypto": {:hex, :plug_crypto, "2.0.0", "77515cc10af06645abbfb5e6ad7a3e9714f805ae118fa1a70205f80d2d70fe73", [:mix], [], "hexpm", "53695bae57cc4e54566d993eb01074e4d894b65a3766f1c43e2c61a1b0f45ea9"},
"ranch": {:hex, :ranch, "1.8.0", "8c7a100a139fd57f17327b6413e4167ac559fbc04ca7448e9be9057311597a1d", [:make, :rebar3], [], "hexpm", "49fbcfd3682fab1f5d109351b61257676da1a2fdbe295904176d5e521a2ddfe5"},
"sobelow": {:hex, :sobelow, "0.13.0", "218afe9075904793f5c64b8837cc356e493d88fddde126a463839351870b8d1e", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "cd6e9026b85fc35d7529da14f95e85a078d9dd1907a9097b3ba6ac7ebbe34a0d"},
"ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.7", "354c321cf377240c7b8716899e182ce4890c5938111a1296add3ec74cf1715df", [:make, :mix, :rebar3], [], "hexpm", "fe4c190e8f37401d30167c8c405eda19469f34577987c76dde613e838bbc67f8"},
"swoosh": {:hex, :swoosh, "1.14.4", "94e9dba91f7695a10f49b0172c4a4cb658ef24abef7e8140394521b7f3bbb2d4", [:mix], [{:cowboy, "~> 1.1 or ~> 2.4", [hex: :cowboy, repo: "hexpm", optional: true]}, {:ex_aws, "~> 2.1", [hex: :ex_aws, repo: "hexpm", optional: true]}, {:finch, "~> 0.6", [hex: :finch, repo: "hexpm", optional: true]}, {:gen_smtp, "~> 0.13 or ~> 1.0", [hex: :gen_smtp, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:mail, "~> 0.2", [hex: :mail, repo: "hexpm", optional: true]}, {:mime, "~> 1.1 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug, "~> 1.9", [hex: :plug, repo: "hexpm", optional: true]}, {:plug_cowboy, ">= 1.0.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:req, "~> 0.4 or ~> 1.0", [hex: :req, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.2 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "081c5a590e4ba85cc89baddf7b2beecf6c13f7f84a958f1cd969290815f0f026"},
"table_rex": {:hex, :table_rex, "4.0.0", "3c613a68ebdc6d4d1e731bc973c233500974ec3993c99fcdabb210407b90959b", [:mix], [], "hexpm", "c35c4d5612ca49ebb0344ea10387da4d2afe278387d4019e4d8111e815df8f55"},
"tailwind": {:hex, :tailwind, "0.2.2", "9e27288b568ede1d88517e8c61259bc214a12d7eed271e102db4c93fcca9b2cd", [:mix], [{:castore, ">= 0.0.0", [hex: :castore, repo: "hexpm", optional: false]}], "hexpm", "ccfb5025179ea307f7f899d1bb3905cd0ac9f687ed77feebc8f67bdca78565c4"},
"telemetry": {:hex, :telemetry, "1.2.1", "68fdfe8d8f05a8428483a97d7aab2f268aaff24b49e0f599faa091f1d4e7f61c", [:rebar3], [], "hexpm", "dad9ce9d8effc621708f99eac538ef1cbe05d6a874dd741de2e689c47feafed5"},
"telemetry_metrics": {:hex, :telemetry_metrics, "0.6.2", "2caabe9344ec17eafe5403304771c3539f3b6e2f7fb6a6f602558c825d0d0bfb", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "9b43db0dc33863930b9ef9d27137e78974756f5f198cae18409970ed6fa5b561"},
"telemetry_poller": {:hex, :telemetry_poller, "1.0.0", "db91bb424e07f2bb6e73926fcafbfcbcb295f0193e0a00e825e589a0a47e8453", [:rebar3], [{:telemetry, "~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "b3a24eafd66c3f42da30fc3ca7dda1e9d546c12250a2d60d7b81d264fbec4f6e"},
"timex": {:hex, :timex, "3.7.11", "bb95cb4eb1d06e27346325de506bcc6c30f9c6dea40d1ebe390b262fad1862d1", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.20", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 1.1", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm", "8b9024f7efbabaf9bd7aa04f65cf8dcd7c9818ca5737677c7b76acbc6a94d1aa"},
"tzdata": {:hex, :tzdata, "1.1.1", "20c8043476dfda8504952d00adac41c6eda23912278add38edc140ae0c5bcc46", [:mix], [{:hackney, "~> 1.17", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "a69cec8352eafcd2e198dea28a34113b60fdc6cb57eb5ad65c10292a6ba89787"},
"unicode_util_compat": {:hex, :unicode_util_compat, "0.7.0", "bc84380c9ab48177092f43ac89e4dfa2c6d62b40b8bd132b1059ecc7232f9a78", [:rebar3], [], "hexpm", "25eee6d67df61960cf6a794239566599b09e17e668d3700247bc498638152521"},
"websock": {:hex, :websock, "0.5.3", "2f69a6ebe810328555b6fe5c831a851f485e303a7c8ce6c5f675abeb20ebdadc", [:mix], [], "hexpm", "6105453d7fac22c712ad66fab1d45abdf049868f253cf719b625151460b8b453"},
"websock_adapter": {:hex, :websock_adapter, "0.5.5", "9dfeee8269b27e958a65b3e235b7e447769f66b5b5925385f5a569269164a210", [:mix], [{:bandit, ">= 0.6.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.6", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "4b977ba4a01918acbf77045ff88de7f6972c2a009213c515a445c48f224ffce9"},
}

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 19 KiB

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 14 KiB

+66 -57
View File
@@ -1,16 +1,5 @@
# Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian
# instead of Alpine to avoid DNS resolution issues in production.
#
# https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=ubuntu
# https://hub.docker.com/_/ubuntu?tab=tags
#
# This file is based on these images:
#
# - https://hub.docker.com/r/hexpm/elixir/tags - for the build image
# - https://hub.docker.com/_/debian?tab=tags&page=1&name=bullseye-20231009-slim - for the release image
# - https://pkgs.org/ - resource for finding needed packages
# - Ex: hexpm/elixir:1.16.0-erlang-26.2.1-debian-bullseye-20231009-slim
#
ARG ELIXIR_VERSION=1.16.2
ARG OTP_VERSION=26.2.2
ARG DEBIAN_VERSION=bookworm-20240130-slim
@@ -20,31 +9,46 @@ ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"
FROM ${BUILDER_IMAGE} as builder
ARG TARGETPLATFORM
RUN echo "Building for ${TARGETPLATFORM:?}"
# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git curl \
&& apt-get clean && rm -f /var/lib/apt/lists/*_*
RUN apt-get update -y && \
# System packages
apt-get install -y \
build-essential \
git \
curl && \
# Node.js and Yarn
curl -sL https://deb.nodesource.com/setup_20.x -o nodesource_setup.sh && \
bash nodesource_setup.sh && \
apt-get install -y nodejs && \
npm install -g yarn && \
# Hex and Rebar
mix local.hex --force && \
mix local.rebar --force && \
# FFmpeg
export FFMPEG_DOWNLOAD=$(case ${TARGETPLATFORM:-linux/amd64} in \
"linux/amd64") echo "https://github.com/yt-dlp/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz" ;; \
"linux/arm64") echo "https://github.com/yt-dlp/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linuxarm64-gpl.tar.xz" ;; \
*) echo "" ;; esac) && \
curl -L ${FFMPEG_DOWNLOAD} --output /tmp/ffmpeg.tar.xz && \
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/local/bin/ "ffmpeg" && \
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/local/bin/ "ffprobe" && \
# Cleanup
apt-get clean && \
rm -f /var/lib/apt/lists/*_*
# prepare build dir
WORKDIR /app
# Install nodejs
RUN curl -sL https://deb.nodesource.com/setup_20.x -o nodesource_setup.sh
RUN bash nodesource_setup.sh
RUN apt-get install -y nodejs
RUN npm install -g yarn
# install hex + rebar
RUN mix local.hex --force && \
mix local.rebar --force
# set build ENV
ENV MIX_ENV="prod"
ENV ERL_FLAGS="+JPperf true"
# install mix dependencies
COPY mix.exs mix.lock ./
RUN mix deps.get --only $MIX_ENV
RUN mkdir config
RUN mix deps.get --only $MIX_ENV && mkdir config
# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
@@ -53,17 +57,11 @@ COPY config/config.exs config/${MIX_ENV}.exs config/
RUN mix deps.compile
COPY priv priv
COPY lib lib
COPY assets assets
# compile assets
RUN yarn --cwd assets install
RUN mix assets.deploy
# Compile the release
RUN mix compile
# Compile assets
RUN yarn --cwd assets install && mix assets.deploy && mix compile
# Changes to config/runtime.exs don't require recompiling the code
COPY config/runtime.exs config/
@@ -71,30 +69,46 @@ COPY config/runtime.exs config/
COPY rel rel
RUN mix release
# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
## -- Release Stage --
FROM ${RUNNER_IMAGE}
ARG PORT=8945
RUN apt-get update -y
RUN apt-get install -y libstdc++6 openssl libncurses5 locales ca-certificates \
ffmpeg curl git openssh-client nano python3 python3-pip jq procps
RUN apt-get clean && rm -f /var/lib/apt/lists/*_*
COPY --from=builder ./usr/local/bin/ffmpeg /usr/bin/ffmpeg
COPY --from=builder ./usr/local/bin/ffprobe /usr/bin/ffprobe
# Download and update YT-DLP
RUN curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp
RUN chmod a+rx /usr/local/bin/yt-dlp
RUN yt-dlp -U
RUN apt-get update -y && \
# System packages
apt-get install -y \
libstdc++6 \
openssl \
libncurses5 \
locales \
ca-certificates \
python3-mutagen \
curl \
openssh-client \
nano \
python3 \
pipx \
jq \
procps && \
# Apprise
export PIPX_HOME=/opt/pipx && \
export PIPX_BIN_DIR=/usr/local/bin && \
pipx install apprise && \
# yt-dlp
curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp && \
chmod a+rx /usr/local/bin/yt-dlp && \
yt-dlp -U && \
# Set the locale
sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen && \
# Clean up
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Download Apprise
RUN python3 -m pip install -U apprise --break-system-packages
# Download Mutagen for music thumbnail generation
RUN python3 -m pip install -U mutagen --break-system-packages
# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
# More locale setup
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
@@ -102,7 +116,7 @@ ENV LC_ALL en_US.UTF-8
WORKDIR "/app"
# Set up data volumes
RUN mkdir /config /downloads
RUN mkdir /config /downloads /etc/elixir_tzdata_data && chmod ugo+rw /etc/elixir_tzdata_data
# set runner ENV
ENV MIX_ENV="prod"
@@ -124,11 +138,6 @@ COPY --from=builder /app/_build/${MIX_ENV}/rel/pinchflat ./
# root, use --user 0:0 or something.
RUN passwd -d root
# If using an environment that doesn't automatically reap zombie processes, it is
# advised to add an init process such as tini via `apt-get install`
# above and adding an entrypoint. See https://github.com/krallin/tini for details
# ENTRYPOINT ["/tini", "--"]
HEALTHCHECK --interval=120s --start-period=10s \
CMD curl --fail http://localhost:${PORT}/healthcheck || exit 1
@@ -1,7 +1,6 @@
defmodule Pinchflat.Boot.PreJobStartupTasksTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.JobFixtures
alias Pinchflat.Settings
@@ -65,6 +65,13 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilderTest do
assert :force_overwrites in res
assert {:parse_metadata, "%(upload_date>%Y-%m-%d)s:(?P<meta_date>.+)"} in res
end
test "includes override options if specified", %{media_item: media_item} do
assert {:ok, res} = DownloadOptionBuilder.build(media_item, overwrite_behaviour: :no_force_overwrites)
refute :force_overwrites in res
assert :no_force_overwrites in res
end
end
describe "build/1 when testing subtitle options" do
@@ -1,7 +1,6 @@
defmodule Pinchflat.Downloading.DownloadingHelpersTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
import Pinchflat.ProfilesFixtures
@@ -10,8 +9,6 @@ defmodule Pinchflat.Downloading.DownloadingHelpersTest do
alias Pinchflat.Downloading.DownloadingHelpers
alias Pinchflat.Downloading.MediaDownloadWorker
setup :verify_on_exit!
describe "enqueue_pending_download_tasks/1" do
test "it enqueues a job for each pending media item" do
source = source_fixture()
@@ -113,4 +110,32 @@ defmodule Pinchflat.Downloading.DownloadingHelpersTest do
refute_enqueued(worker: MediaDownloadWorker)
end
end
describe "kickoff_redownload_for_existing_media/1" do
test "enqueues a download job for each downloaded media item" do
source = source_fixture()
media_item = media_item_fixture(source_id: source.id, media_downloaded_at: now())
assert [{:ok, _}] = DownloadingHelpers.kickoff_redownload_for_existing_media(source)
assert_enqueued(worker: MediaDownloadWorker, args: %{"id" => media_item.id})
end
test "doesn't enqueue jobs for media that should be ignored" do
source = source_fixture()
other_source = source_fixture()
_not_downloaded = media_item_fixture(source_id: source.id, media_downloaded_at: nil)
_other_source = media_item_fixture(source_id: other_source.id, media_downloaded_at: now())
_download_prevented =
media_item_fixture(source_id: source.id, media_downloaded_at: now(), prevent_download: true)
_culled =
media_item_fixture(source_id: source.id, media_downloaded_at: now(), culled_at: now())
assert [] = DownloadingHelpers.kickoff_redownload_for_existing_media(source)
refute_enqueued(worker: MediaDownloadWorker)
end
end
end
@@ -1,7 +1,6 @@
defmodule Pinchflat.Downloading.MediaDownloadWorkerTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
alias Pinchflat.Media
@@ -9,8 +8,6 @@ defmodule Pinchflat.Downloading.MediaDownloadWorkerTest do
alias Pinchflat.Utils.FilesystemUtils
alias Pinchflat.Downloading.MediaDownloadWorker
setup :verify_on_exit!
setup do
stub(UserScriptRunnerMock, :run, fn _event_type, _data -> :ok end)
stub(HTTPClientMock, :get, fn _url, _headers, _opts -> {:ok, ""} end)
@@ -111,7 +108,7 @@ defmodule Pinchflat.Downloading.MediaDownloadWorkerTest do
end)
Oban.Testing.with_testing_mode(:inline, fn ->
{:ok, job} = Oban.insert(MediaDownloadWorker.new(%{id: media_item.id, redownload?: true}))
{:ok, job} = Oban.insert(MediaDownloadWorker.new(%{id: media_item.id, quality_upgrade?: true}))
assert job.state == "completed"
end)
@@ -139,15 +136,6 @@ defmodule Pinchflat.Downloading.MediaDownloadWorkerTest do
perform_job(MediaDownloadWorker, %{id: media_item.id})
end
test "downloads anyway if forced", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, _opts, _ot, _addl -> :ok end)
Sources.update_source(media_item.source, %{download_media: false})
Media.update_media_item(media_item, %{prevent_download: true})
perform_job(MediaDownloadWorker, %{id: media_item.id, force: true})
end
test "it saves the file's size to the database", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, _opts, _ot, _addl ->
metadata = render_parsed_metadata(:media_metadata)
@@ -162,18 +150,7 @@ defmodule Pinchflat.Downloading.MediaDownloadWorkerTest do
assert media_item.media_size_bytes > 0
end
test "saves redownloaded_at if this is for a redownload", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, _opts, _ot, _addl ->
{:ok, render_metadata(:media_metadata)}
end)
perform_job(MediaDownloadWorker, %{id: media_item.id, redownload?: true})
media_item = Repo.reload(media_item)
assert media_item.media_redownloaded_at != nil
end
test "doesn't save redownloaded_at if this is not for a redownload", %{media_item: media_item} do
test "does not set redownloaded_at by default", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, _opts, _ot, _addl ->
{:ok, render_metadata(:media_metadata)}
end)
@@ -201,5 +178,62 @@ defmodule Pinchflat.Downloading.MediaDownloadWorkerTest do
test "does not blow up if the record doesn't exist" do
assert :ok = perform_job(MediaDownloadWorker, %{id: 0})
end
test "sets the no_force_overwrites runner option", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, opts, _ot, _addl ->
assert :no_force_overwrites in opts
refute :force_overwrites in opts
{:ok, render_metadata(:media_metadata)}
end)
perform_job(MediaDownloadWorker, %{id: media_item.id})
end
end
describe "perform/1 when testing forced downloads" do
test "ignores 'prevent_download' if forced", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, _opts, _ot, _addl -> :ok end)
Sources.update_source(media_item.source, %{download_media: false})
Media.update_media_item(media_item, %{prevent_download: true})
perform_job(MediaDownloadWorker, %{id: media_item.id, force: true})
end
test "sets force_overwrites runner option", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, opts, _ot, _addl ->
assert :force_overwrites in opts
refute :no_force_overwrites in opts
{:ok, render_metadata(:media_metadata)}
end)
perform_job(MediaDownloadWorker, %{id: media_item.id, force: true})
end
end
describe "perform/1 when testing re-downloads" do
test "sets redownloaded_at on the media_item", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, _opts, _ot, _addl ->
{:ok, render_metadata(:media_metadata)}
end)
perform_job(MediaDownloadWorker, %{id: media_item.id, quality_upgrade?: true})
media_item = Repo.reload(media_item)
assert media_item.media_redownloaded_at != nil
end
test "sets force_overwrites runner option", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, opts, _ot, _addl ->
assert :force_overwrites in opts
refute :no_force_overwrites in opts
{:ok, render_metadata(:media_metadata)}
end)
perform_job(MediaDownloadWorker, %{id: media_item.id, force: true})
end
end
end
@@ -1,14 +1,12 @@
defmodule Pinchflat.Downloading.MediaDownloaderTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
import Pinchflat.ProfilesFixtures
alias Pinchflat.Downloading.MediaDownloader
setup :verify_on_exit!
setup do
media_item =
Repo.preload(
@@ -67,6 +65,21 @@ defmodule Pinchflat.Downloading.MediaDownloaderTest do
end
end
describe "download_for_media_item/3 when testing override options" do
test "includes override opts if specified", %{media_item: media_item} do
expect(YtDlpRunnerMock, :run, fn _url, opts, _ot, _addl ->
refute :force_overwrites in opts
assert :no_force_overwrites in opts
{:ok, render_metadata(:media_metadata)}
end)
override_opts = [overwrite_behaviour: :no_force_overwrites]
assert {:ok, _} = MediaDownloader.download_for_media_item(media_item, override_opts)
end
end
describe "download_for_media_item/3 when testing retries" do
test "returns a recovered tuple on recoverable errors", %{media_item: media_item} do
message = "Unable to communicate with SponsorBlock"
@@ -1,4 +1,4 @@
defmodule Pinchflat.Downloading.MediaRedownloadWorkerTest do
defmodule Pinchflat.Downloading.MediaQualityUpgradeWorkerTest do
use Pinchflat.DataCase
import Pinchflat.MediaFixtures
@@ -6,7 +6,7 @@ defmodule Pinchflat.Downloading.MediaRedownloadWorkerTest do
import Pinchflat.ProfilesFixtures
alias Pinchflat.Downloading.MediaDownloadWorker
alias Pinchflat.Downloading.MediaRedownloadWorker
alias Pinchflat.Downloading.MediaQualityUpgradeWorker
describe "perform/1" do
test "kicks off a task for redownloadable media items" do
@@ -20,9 +20,9 @@ defmodule Pinchflat.Downloading.MediaRedownloadWorkerTest do
media_downloaded_at: now_minus(5, :days)
})
perform_job(MediaRedownloadWorker, %{})
perform_job(MediaQualityUpgradeWorker, %{})
assert [_] = all_enqueued(worker: MediaDownloadWorker, args: %{id: media_item.id, redownload?: true})
assert [_] = all_enqueued(worker: MediaDownloadWorker, args: %{id: media_item.id, quality_upgrade?: true})
end
test "does not kickoff a task for non-redownloadable media items" do
@@ -36,7 +36,7 @@ defmodule Pinchflat.Downloading.MediaRedownloadWorkerTest do
media_downloaded_at: now_minus(1, :day)
})
perform_job(MediaRedownloadWorker, %{})
perform_job(MediaQualityUpgradeWorker, %{})
assert [] = all_enqueued(worker: MediaDownloadWorker)
end
@@ -1,15 +1,12 @@
defmodule Pinchflat.Downloading.MediaRetentionWorkerTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
alias Pinchflat.Media
alias Pinchflat.Downloading.MediaRetentionWorker
setup :verify_on_exit!
describe "perform/1" do
setup do
stub(UserScriptRunnerMock, :run, fn _event_type, _data -> :ok end)
@@ -1,5 +1,5 @@
defmodule Pinchflat.Downloading.OutputPath.ParserTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Downloading.OutputPath.Parser
@@ -1,7 +1,6 @@
defmodule Pinchflat.FastIndexing.FastIndexingHelpersTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
import Pinchflat.ProfilesFixtures
@@ -11,8 +10,6 @@ defmodule Pinchflat.FastIndexing.FastIndexingHelpersTest do
alias Pinchflat.Downloading.MediaDownloadWorker
alias Pinchflat.FastIndexing.FastIndexingHelpers
setup :verify_on_exit!
describe "kickoff_download_tasks_from_youtube_rss_feed/1" do
setup do
stub(YtDlpRunnerMock, :run, fn _url, _opts, _ot ->
@@ -1,15 +1,12 @@
defmodule Pinchflat.FastIndexing.FastIndexingWorkerTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.SourcesFixtures
alias Pinchflat.Settings
alias Pinchflat.Sources.Source
alias Pinchflat.FastIndexing.FastIndexingWorker
setup :verify_on_exit!
describe "kickoff_with_task/2" do
test "starts the worker" do
source = source_fixture(fast_index: true)
@@ -1,12 +1,10 @@
defmodule Pinchflat.FastIndexing.YoutubeRssTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.SourcesFixtures
alias Pinchflat.FastIndexing.YoutubeRss
setup :verify_on_exit!
setup do
source = source_fixture()
@@ -1,5 +1,5 @@
defmodule Pinchflat.Lifecycle.Notifications.CommandRunnerTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Lifecycle.Notifications.CommandRunner, as: Runner
@@ -1,7 +1,6 @@
defmodule Pinchflat.Lifecycle.Notifications.SourceNotificationsTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
@@ -9,8 +8,6 @@ defmodule Pinchflat.Lifecycle.Notifications.SourceNotificationsTest do
@apprise_servers ["server_1", "server_2"]
setup :verify_on_exit!
describe "wrap_new_media_notification/3" do
test "sends a notification when the pending count changes" do
source = source_fixture()
@@ -1,5 +1,5 @@
defmodule Pinchflat.Lifecycle.UserScripts.CommandRunnerTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.FilesystemUtils
alias Pinchflat.Lifecycle.UserScripts.CommandRunner, as: Runner
@@ -15,11 +15,12 @@ defmodule Pinchflat.Lifecycle.UserScripts.CommandRunnerTest do
test "runs the provided lifecycle file if present" do
# We *love* indirectly testing side effects
tmp_dir = Application.get_env(:pinchflat, :tmpfile_directory)
File.write(filepath(), "#!/bin/bash\ntouch #{tmp_dir}/test_file\n")
filename = "#{tmp_dir}/test_file-#{Enum.random(1..1000)}"
File.write(filepath(), "#!/bin/bash\ntouch #{filename}\n")
refute File.exists?("#{tmp_dir}/test_file")
refute File.exists?(filename)
assert :ok = Runner.run(:media_downloaded, %{})
assert File.exists?("#{tmp_dir}/test_file")
assert File.exists?(filename)
end
test "passes the event name to the script" do
-3
View File
@@ -1,7 +1,6 @@
defmodule Pinchflat.MediaTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.TasksFixtures
import Pinchflat.MediaFixtures
import Pinchflat.ProfilesFixtures
@@ -13,8 +12,6 @@ defmodule Pinchflat.MediaTest do
alias Pinchflat.YtDlp.Media, as: YtDlpMedia
setup :verify_on_exit!
@invalid_attrs %{title: nil, media_id: nil, media_filepath: nil}
describe "schema" do
@@ -1,6 +1,6 @@
defmodule Pinchflat.Metadata.MetadataFileHelpersTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
alias Pinchflat.Metadata.MetadataFileHelpers, as: Helpers
@@ -11,8 +11,6 @@ defmodule Pinchflat.Metadata.MetadataFileHelpersTest do
{:ok, %{media_item: media_item}}
end
setup :verify_on_exit!
describe "compress_and_store_metadata_for/2" do
test "returns the filepath", %{media_item: media_item} do
metadata_map = %{"foo" => "bar"}
@@ -1,6 +1,6 @@
defmodule Pinchflat.Metadata.SourceMetadataStorageWorkerTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.SourcesFixtures
import Pinchflat.ProfilesFixtures
@@ -11,8 +11,6 @@ defmodule Pinchflat.Metadata.SourceMetadataStorageWorkerTest do
@source_details_ot "%(.{channel,channel_id,playlist_id,playlist_title,filename})j"
@metadata_ot "playlist:%()j"
setup :verify_on_exit!
describe "kickoff_with_task/1" do
test "enqueues a new worker for the source" do
source = source_fixture()
-3
View File
@@ -1,7 +1,6 @@
defmodule Pinchflat.ProfilesTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
import Pinchflat.ProfilesFixtures
@@ -11,8 +10,6 @@ defmodule Pinchflat.ProfilesTest do
@invalid_attrs %{name: nil, output_path_template: nil}
setup :verify_on_exit!
describe "schema" do
test "can be JSON encoded without error" do
profile = media_profile_fixture()
@@ -1,5 +1,5 @@
defmodule Pinchflat.SlowIndexing.FileFollowerServerTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias alias Pinchflat.Utils.FilesystemUtils
alias Pinchflat.SlowIndexing.FileFollowerServer
@@ -1,7 +1,6 @@
defmodule Pinchflat.SlowIndexing.MediaCollectionIndexingWorkerTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.TasksFixtures
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
@@ -13,8 +12,6 @@ defmodule Pinchflat.SlowIndexing.MediaCollectionIndexingWorkerTest do
alias Pinchflat.Downloading.MediaDownloadWorker
alias Pinchflat.SlowIndexing.MediaCollectionIndexingWorker
setup :verify_on_exit!
describe "kickoff_with_task/3" do
setup do
source = source_fixture(index_frequency_minutes: 10)
@@ -1,7 +1,6 @@
defmodule Pinchflat.SlowIndexing.SlowIndexingHelpersTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.TasksFixtures
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
@@ -15,8 +14,6 @@ defmodule Pinchflat.SlowIndexing.SlowIndexingHelpersTest do
alias Pinchflat.SlowIndexing.SlowIndexingHelpers
alias Pinchflat.SlowIndexing.MediaCollectionIndexingWorker
setup :verify_on_exit!
describe "kickoff_indexing_task/3" do
test "it schedules a job" do
source = source_fixture(index_frequency_minutes: 1)
+1 -3
View File
@@ -1,6 +1,6 @@
defmodule Pinchflat.SourcesTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.TasksFixtures
import Pinchflat.MediaFixtures
import Pinchflat.ProfilesFixtures
@@ -18,8 +18,6 @@ defmodule Pinchflat.SourcesTest do
@invalid_source_attrs %{name: nil, collection_id: nil}
setup :verify_on_exit!
describe "schema" do
test "source_metadata is deleted when the source is deleted" do
source =
@@ -1,5 +1,5 @@
defmodule Pinchflat.Utils.ChangesetUtilsTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
defmodule MockSchema do
use Ecto.Schema
+1 -1
View File
@@ -1,5 +1,5 @@
defmodule Pinchflat.Utils.CliUtilsTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.CliUtils
+1 -1
View File
@@ -1,5 +1,5 @@
defmodule Pinchflat.Utils.DatetimeUtilsTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.DatetimeUtils
+1 -1
View File
@@ -1,5 +1,5 @@
defmodule Pinchflat.Utils.FunctionUtilsTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.FunctionUtils
+1 -1
View File
@@ -1,5 +1,5 @@
defmodule Pinchflat.Utils.NumberUtilsTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.NumberUtils
+1 -1
View File
@@ -1,5 +1,5 @@
defmodule Pinchflat.Utils.StringUtilsTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.StringUtils
+1 -1
View File
@@ -1,5 +1,5 @@
defmodule Pinchflat.Utils.XmlUtilsTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.XmlUtils
@@ -1,5 +1,5 @@
defmodule Pinchflat.YtDlp.CommandRunnerTest do
use ExUnit.Case, async: false
use Pinchflat.DataCase
alias Pinchflat.Utils.FilesystemUtils
@@ -1,6 +1,6 @@
defmodule Pinchflat.YtDlp.MediaCollectionTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.SourcesFixtures
alias Pinchflat.YtDlp.Media
@@ -8,8 +8,6 @@ defmodule Pinchflat.YtDlp.MediaCollectionTest do
@channel_url "https://www.youtube.com/c/PinchflatTestChannel"
setup :verify_on_exit!
describe "get_media_attributes_for_collection/2" do
test "returns a list of video attributes with no blank elements" do
expect(YtDlpRunnerMock, :run, fn _url, _opts, _ot, _addl_opts ->
+1 -3
View File
@@ -1,14 +1,12 @@
defmodule Pinchflat.YtDlp.MediaTest do
use Pinchflat.DataCase
import Mox
import Pinchflat.MediaFixtures
alias Pinchflat.YtDlp.Media
@media_url "https://www.youtube.com/watch?v=TiZPUDkDYbk"
setup :verify_on_exit!
describe "download/2" do
test "it calls the backend runner with the expected arguments" do
expect(YtDlpRunnerMock, :run, fn @media_url, opts, ot, addl ->
@@ -1,14 +1,11 @@
defmodule PinchflatWeb.MediaItemControllerTest do
use PinchflatWeb.ConnCase
import Mox
import Pinchflat.MediaFixtures
alias Pinchflat.Repo
alias Pinchflat.Downloading.MediaDownloadWorker
setup :verify_on_exit!
describe "show media" do
setup [:create_media_item]
@@ -1,7 +1,6 @@
defmodule PinchflatWeb.MediaProfileControllerTest do
use PinchflatWeb.ConnCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
import Pinchflat.ProfilesFixtures
@@ -16,8 +15,6 @@ defmodule PinchflatWeb.MediaProfileControllerTest do
}
@invalid_attrs %{name: nil, output_path_template: nil}
setup :verify_on_exit!
setup do
Settings.set(onboarding: false)
@@ -41,7 +38,7 @@ defmodule PinchflatWeb.MediaProfileControllerTest do
Settings.set(onboarding: true)
conn = get(conn, ~p"/media_profiles/new")
refute html_response(conn, 200) =~ "MENU"
refute html_response(conn, 200) =~ "<span>MENU</span>"
end
end
@@ -1,13 +1,10 @@
defmodule PinchflatWeb.Settings.AppriseServerLiveTest do
use PinchflatWeb.ConnCase
import Mox
import Phoenix.LiveViewTest
alias Pinchflat.Settings.AppriseServerLive
setup :verify_on_exit!
describe "initial rendering" do
test "renders the input", %{conn: conn} do
{:ok, _view, html} = live_isolated(conn, AppriseServerLive, session: create_session(""))
@@ -1,6 +1,5 @@
defmodule PinchflatWeb.SourceControllerTest do
use PinchflatWeb.ConnCase
import Mox
import Pinchflat.MediaFixtures
import Pinchflat.SourcesFixtures
@@ -32,8 +31,6 @@ defmodule PinchflatWeb.SourceControllerTest do
}
end
setup :verify_on_exit!
describe "index" do
test "lists all sources", %{conn: conn} do
conn = get(conn, ~p"/sources")
@@ -169,20 +166,38 @@ defmodule PinchflatWeb.SourceControllerTest do
end
end
describe "force_download" do
describe "force_download_pending" do
test "enqueues pending download tasks", %{conn: conn} do
source = source_fixture()
_media_item = media_item_fixture(%{source_id: source.id, media_filepath: nil})
assert [] = all_enqueued(worker: MediaDownloadWorker)
post(conn, ~p"/sources/#{source.id}/force_download")
post(conn, ~p"/sources/#{source.id}/force_download_pending")
assert [_] = all_enqueued(worker: MediaDownloadWorker)
end
test "redirects to the source page", %{conn: conn} do
source = source_fixture()
conn = post(conn, ~p"/sources/#{source.id}/force_download")
conn = post(conn, ~p"/sources/#{source.id}/force_download_pending")
assert redirected_to(conn) == ~p"/sources/#{source.id}"
end
end
describe "force_redownload" do
test "enqueues re-download tasks", %{conn: conn} do
source = source_fixture()
_media_item = media_item_fixture(source_id: source.id, media_downloaded_at: now())
assert [] = all_enqueued(worker: MediaDownloadWorker)
post(conn, ~p"/sources/#{source.id}/force_redownload")
assert [_] = all_enqueued(worker: MediaDownloadWorker)
end
test "redirects to the source page", %{conn: conn} do
source = source_fixture()
conn = post(conn, ~p"/sources/#{source.id}/force_redownload")
assert redirected_to(conn) == ~p"/sources/#{source.id}"
end
end
@@ -34,8 +34,10 @@ defmodule PinchflatWeb.Sources.MediaItemTableLiveTest do
describe "media_state" do
test "shows pending media when pending", %{conn: conn, source: source} do
downloaded_media_item = media_item_fixture(source_id: source.id)
pending_media_item = media_item_fixture(source_id: source.id, media_filepath: nil)
downloaded_media_item = media_item_fixture(source_id: source.id, title: "DL-#{Enum.random(0..9999)}")
pending_media_item =
media_item_fixture(source_id: source.id, media_filepath: nil, title: "P-#{Enum.random(0..9999)}")
{:ok, _view, html} = live_isolated(conn, MediaItemTableLive, session: create_session(source, "pending"))
+7
View File
@@ -17,6 +17,8 @@ defmodule PinchflatWeb.ConnCase do
use ExUnit.CaseTemplate
alias Pinchflat.TestingHelperMethods
using do
quote do
# The default endpoint for testing
@@ -27,14 +29,19 @@ defmodule PinchflatWeb.ConnCase do
use Oban.Testing, repo: Repo
# Import conveniences for testing with connections
import Mox
import Plug.Conn
import Phoenix.ConnTest
import PinchflatWeb.ConnCase
import Pinchflat.TestingHelperMethods
setup :verify_on_exit!
end
end
setup tags do
TestingHelperMethods.create_platform_directories()
Pinchflat.DataCase.setup_sandbox(tags)
conn = Phoenix.ConnTest.build_conn()
+6
View File
@@ -15,6 +15,7 @@ defmodule Pinchflat.DataCase do
"""
use ExUnit.CaseTemplate
alias Pinchflat.TestingHelperMethods
using do
quote do
@@ -22,16 +23,21 @@ defmodule Pinchflat.DataCase do
use Oban.Testing, repo: Repo
import Mox
import Ecto
import Ecto.Changeset
import Ecto.Query
import Pinchflat.DataCase
import Pinchflat.TestingHelperMethods
setup :verify_on_exit!
end
end
setup tags do
TestingHelperMethods.create_platform_directories()
Pinchflat.DataCase.setup_sandbox(tags)
:ok
end
+7
View File
@@ -54,4 +54,11 @@ defmodule Pinchflat.TestingHelperMethods do
|> render_metadata()
|> Phoenix.json_library().decode!()
end
def create_platform_directories do
File.mkdir_p!(Application.get_env(:pinchflat, :media_directory))
File.mkdir_p!(Application.get_env(:pinchflat, :metadata_directory))
File.mkdir_p!(Application.get_env(:pinchflat, :extras_directory))
File.mkdir_p!(Application.get_env(:pinchflat, :tmpfile_directory))
end
end