Kieran
dc0313d875
* Made method to getting singular media details; Renamed other related method * Takes a fun and flirty digression to remove abstractions around yt-dlp since I'm 100% committed to using it exclusively * Removed commented test code * Lays the groundwork for fast indexing * Added module for working with youtube RSS feed * Added methods to kick off indexing workers from RSS response * Improve short detection (#59) * Made media attribute-related yt-dlp calls return a struct * Added shorts attribute to media items * Added ability to discern a short from yt-dlp response * Updated search to use new shorts attribute * Fast index UI (#63) * Added fast_index field and adds it to source form * Added fast indexing to source changeset operations * Added fast indexing worker and updated other modules to start using it * Handled fast index worker on source update * Add support modals (#65) * Added fast indexing upgrade modal * Improved modal on smaller screens * Updated links to work again * Added donation modal * Reverted source fast index to 15 minutes * Removed unneeded HTML attributes from old alpine approach
54 lines
2.2 KiB
Elixir
54 lines
2.2 KiB
Elixir
defmodule Pinchflat.Workers.MediaIndexingWorker do
|
|
@moduledoc false
|
|
|
|
use Oban.Worker,
|
|
queue: :media_indexing,
|
|
unique: [period: :infinity, states: [:available, :scheduled, :retryable]],
|
|
tags: ["media_source", "media_indexing"]
|
|
|
|
require Logger
|
|
|
|
alias Pinchflat.Sources
|
|
alias Pinchflat.Tasks.MediaItemTasks
|
|
|
|
@impl Oban.Worker
|
|
@doc """
|
|
Similar to `MediaCollectionIndexingWorker`, but for individual media items.
|
|
Does not reschedule or check anything to do with a source's indexing
|
|
frequency - only collects initial metadata then kicks off a download.
|
|
`MediaCollectionIndexingWorker` should be preferred in general, but this is
|
|
useful for downloading one-off media items based on a URL (like for fast indexing).
|
|
|
|
Only downloads media that _should_ be downloaded (ie: the source is set to download
|
|
and the media matches the profile's format preferences)
|
|
|
|
Order of operations:
|
|
1. SourceTasks.kickoff_indexing_tasks_from_youtube_rss_feed/1 (which is running
|
|
in its own worker) periodically checks the YouTube RSS feed for new media
|
|
2. If new media is found, it enqueues a MediaIndexingWorker (this module) for each new media
|
|
item
|
|
3. This worker fetches the media metadata and uses that to determine if it should be
|
|
downloaded. If so, it enqueues a MediaDownloadWorker
|
|
|
|
Each is a worker because they all either need to be scheduled periodically or call out to
|
|
an external service and will be long-running. They're split into different jobs to separate
|
|
retry logic for each step and allow us to better optimize various queues (eg: the indexing
|
|
steps can keep running while the slow download steps are worked through).
|
|
|
|
Returns :ok
|
|
"""
|
|
def perform(%Oban.Job{args: %{"id" => source_id, "media_url" => media_url}}) do
|
|
source = Sources.get_source!(source_id)
|
|
|
|
case MediaItemTasks.index_and_enqueue_download_for_media_item(source, media_url) do
|
|
{:ok, media_item} ->
|
|
Logger.debug("Indexed and enqueued download for url: #{media_url} (media item: #{media_item.id})")
|
|
|
|
{:error, reason} ->
|
|
Logger.debug("Failed to index and enqueue download for url: #{media_url} (reason: #{inspect(reason)})")
|
|
end
|
|
|
|
:ok
|
|
end
|
|
end
|