hermes-agent/pinchflat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Redo indexing mechanism (#16)

Browse Source 
* Bumped up the line length because I fear no man

* Refactored indexing

Previously, indexing worked by collecting the video IDs of only videos
that matched indexing criteria. This new model instead stores ALL videos
for a given source, but will only _download_ videos that meet that criteria.
This lets us backfill without indexing, makes it easier to add in other
backends, lets us download one-off videos for a source that don't quite
meet criteria, you name it.

* Updated media finders to respect format filters; Added credo file
This commit is contained in:
Kieran
2024-02-09 18:23:37 -08:00
committed by GitHub
parent e1565ad22f
commit e0637331bb
31 changed files with 646 additions and 385 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/pinchflat_web/controllers/media_sources/source_html/edit.html.heex
+1 -5
View File
@@ -8,11 +8,7 @@
<div class="rounded-sm border border-stroke bg-white px-5 pb-2.5 pt-6 shadow-default dark:border-strokedark dark:bg-boxdark sm:px-7.5 xl:pb-1">
<div class="max-w-full overflow-x-auto">
<div class="flex flex-col gap-10">
<.source_form
changeset={@changeset}
media_profiles={@media_profiles}
action={~p"/sources/#{@source}"}
/>
<.source_form changeset={@changeset} media_profiles={@media_profiles} action={~p"/sources/#{@source}"} />
</div>
</div>
</div>
lib/pinchflat_web/controllers/media_sources/source_html/index.html.heex
+2 -8
View File
@@ -29,16 +29,10 @@
</.link>
</:col>
<:col :let={source} label="" class="flex place-content-evenly">
<.link
navigate={~p"/sources/#{source.id}"}
class="hover:text-secondary duration-200 ease-in-out mx-0.5"
>
<.link navigate={~p"/sources/#{source.id}"} class="hover:text-secondary duration-200 ease-in-out mx-0.5">
<.icon name="hero-eye" />
</.link>
<.link
navigate={~p"/sources/#{source.id}/edit"}
class="hover:text-secondary duration-200 ease-in-out mx-0.5"
>
<.link navigate={~p"/sources/#{source.id}/edit"} class="hover:text-secondary duration-200 ease-in-out mx-0.5">
<.icon name="hero-pencil-square" />
</.link>
</:col>
lib/pinchflat_web/controllers/media_sources/source_html/source_form.html.heex
+2 -12
View File
@@ -3,12 +3,7 @@
Oops, something went wrong! Please check the errors below.
</.error>
<.input
field={f[:original_url]}
type="text"
label="Source URL"
help="URL of a channel or playlist (required)"
/>
<.input field={f[:original_url]} type="text" label="Source URL" help="URL of a channel or playlist (required)" />
<.input field={f[:friendly_name]} type="text" label="Friendly Name" />
@@ -19,12 +14,7 @@
label="Media Profile"
/>
<.input
field={f[:collection_type]}
options={friendly_collection_types()}
type="select"
label="Source Type"
/>
<.input field={f[:collection_type]} options={friendly_collection_types()} type="select" label="Source Type" />
<.input
field={f[:index_frequency_minutes]}
lib/pinchflat_web/controllers/page_html/home.html.heex
+5 -25
View File
@@ -136,11 +136,7 @@
href="https://twitter.com/elixirphoenix"
class="group -mx-2 -my-0.5 inline-flex items-center gap-3 rounded-lg px-2 py-0.5 hover:bg-zinc-50 hover:text-zinc-900"
>
<svg
viewBox="0 0 16 16"
aria-hidden="true"
class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600"
>
<svg viewBox="0 0 16 16" aria-hidden="true" class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600">
<path d="M5.403 14c5.283 0 8.172-4.617 8.172-8.62 0-.131 0-.262-.008-.391A6.033 6.033 0 0 0 15 3.419a5.503 5.503 0 0 1-1.65.477 3.018 3.018 0 0 0 1.263-1.676 5.579 5.579 0 0 1-1.824.736 2.832 2.832 0 0 0-1.63-.916 2.746 2.746 0 0 0-1.821.319A2.973 2.973 0 0 0 8.076 3.78a3.185 3.185 0 0 0-.182 1.938 7.826 7.826 0 0 1-3.279-.918 8.253 8.253 0 0 1-2.64-2.247 3.176 3.176 0 0 0-.315 2.208 3.037 3.037 0 0 0 1.203 1.836A2.739 2.739 0 0 1 1.56 6.22v.038c0 .7.23 1.377.65 1.919.42.54 1.004.912 1.654 1.05-.423.122-.866.14-1.297.052.184.602.541 1.129 1.022 1.506a2.78 2.78 0 0 0 1.662.598 5.656 5.656 0 0 1-2.007 1.074A5.475 5.475 0 0 1 1 12.64a7.827 7.827 0 0 0 4.403 1.358" />
</svg>
Follow on Twitter
@@ -151,11 +147,7 @@
href="https://elixirforum.com"
class="group -mx-2 -my-0.5 inline-flex items-center gap-3 rounded-lg px-2 py-0.5 hover:bg-zinc-50 hover:text-zinc-900"
>
<svg
viewBox="0 0 16 16"
aria-hidden="true"
class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600"
>
<svg viewBox="0 0 16 16" aria-hidden="true" class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600">
<path d="M8 13.833c3.866 0 7-2.873 7-6.416C15 3.873 11.866 1 8 1S1 3.873 1 7.417c0 1.081.292 2.1.808 2.995.606 1.05.806 2.399.086 3.375l-.208.283c-.285.386-.01.905.465.85.852-.098 2.048-.318 3.137-.81a3.717 3.717 0 0 1 1.91-.318c.263.027.53.041.802.041Z" />
</svg>
Discuss on the Elixir Forum
@@ -166,11 +158,7 @@
href="https://web.libera.chat/#elixir"
class="group -mx-2 -my-0.5 inline-flex items-center gap-3 rounded-lg px-2 py-0.5 hover:bg-zinc-50 hover:text-zinc-900"
>
<svg
viewBox="0 0 16 16"
aria-hidden="true"
class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600"
>
<svg viewBox="0 0 16 16" aria-hidden="true" class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600">
<path
fill-rule="evenodd"
clip-rule="evenodd"
@@ -190,11 +178,7 @@
href="https://discord.gg/elixir"
class="group -mx-2 -my-0.5 inline-flex items-center gap-3 rounded-lg px-2 py-0.5 hover:bg-zinc-50 hover:text-zinc-900"
>
<svg
viewBox="0 0 16 16"
aria-hidden="true"
class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600"
>
<svg viewBox="0 0 16 16" aria-hidden="true" class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600">
<path d="M13.545 2.995c-1.02-.46-2.114-.8-3.257-.994a.05.05 0 0 0-.052.024c-.141.246-.297.567-.406.82a12.377 12.377 0 0 0-3.658 0 8.238 8.238 0 0 0-.412-.82.052.052 0 0 0-.052-.024 13.315 13.315 0 0 0-3.257.994.046.046 0 0 0-.021.018C.356 6.063-.213 9.036.066 11.973c.001.015.01.029.02.038a13.353 13.353 0 0 0 3.996 1.987.052.052 0 0 0 .056-.018c.308-.414.582-.85.818-1.309a.05.05 0 0 0-.028-.069 8.808 8.808 0 0 1-1.248-.585.05.05 0 0 1-.005-.084c.084-.062.168-.126.248-.191a.05.05 0 0 1 .051-.007c2.619 1.176 5.454 1.176 8.041 0a.05.05 0 0 1 .053.006c.08.065.164.13.248.192a.05.05 0 0 1-.004.084c-.399.23-.813.423-1.249.585a.05.05 0 0 0-.027.07c.24.457.514.893.817 1.307a.051.051 0 0 0 .056.019 13.31 13.31 0 0 0 4.001-1.987.05.05 0 0 0 .021-.037c.334-3.396-.559-6.345-2.365-8.96a.04.04 0 0 0-.021-.02Zm-8.198 7.19c-.789 0-1.438-.712-1.438-1.587 0-.874.637-1.586 1.438-1.586.807 0 1.45.718 1.438 1.586 0 .875-.637 1.587-1.438 1.587Zm5.316 0c-.788 0-1.438-.712-1.438-1.587 0-.874.637-1.586 1.438-1.586.807 0 1.45.718 1.438 1.586 0 .875-.63 1.587-1.438 1.587Z" />
</svg>
Join our Discord server
@@ -205,11 +189,7 @@
href="https://fly.io/docs/elixir/getting-started/"
class="group -mx-2 -my-0.5 inline-flex items-center gap-3 rounded-lg px-2 py-0.5 hover:bg-zinc-50 hover:text-zinc-900"
>
<svg
viewBox="0 0 20 20"
aria-hidden="true"
class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600"
>
<svg viewBox="0 0 20 20" aria-hidden="true" class="h-4 w-4 fill-zinc-400 group-hover:fill-zinc-600">
<path d="M1 12.5A4.5 4.5 0 005.5 17H15a4 4 0 001.866-7.539 3.504 3.504 0 00-4.504-4.272A4.5 4.5 0 004.06 8.235 4.502 4.502 0 001 12.5z" />
</svg>
Deploy your application