Compare commits

..

51 Commits

Author SHA1 Message Date
Kieran Eglin 61ae50735f Bumped version 2025-01-14 13:13:32 -08:00
Kieran d8d7353228 [Enhancement] Add Discord link (#565)
* Add a discord link in sidebar

* Added discord link to README
2025-01-14 13:12:43 -08:00
Kieran Eglin 03a0afd657 Add blurb about websockets 2025-01-14 12:54:43 -08:00
Kieran ca90da49f5 Add simple icons (#564) 2025-01-14 12:53:35 -08:00
Kieran Eglin 40cde43be1 Added grafana dashboards 2025-01-14 12:04:33 -08:00
Kieran e9f6b45953 [Enhancement] Add rate limiting to yt-dlp requests; prevent saving Media Items when throttled by YouTube (#559)
* Added sleep interval to settings

* Added new sleep setting to yt-dlp runner and added tests

* Added setting for form; updated setting name

* Updated form label

* Prevented saving/updating of media items if being throttled by youtube

* Added the bot message to the list of non-retryable errors

* Fixed typo
2025-01-14 11:38:40 -08:00
Kieran fb27988963 [Enhancement] Add Prometheus support (#556)
* Added prometheus to deps list

* WIP - screwing around with Prometheus and grafana

* Added basic prometheus config

* Updated docs in prom_ex module

* Updated README
2025-01-09 12:38:17 -08:00
Kieran 8a40d296c4 Updated healthcheck to run every 30s (#555) 2025-01-09 11:17:55 -08:00
Kieran Eglin d8fdba8f6d Bumped version 2025-01-03 15:40:36 -08:00
Kieran e4c186eacb Added IDs as an option for output templates (#543) 2025-01-03 10:50:43 -08:00
Kieran e150355874 Added the ability to mark chapters via sponsorblock (#542) 2025-01-03 09:53:38 -08:00
Kieran Eglin 967e21a8a1 Removed double alias 2025-01-03 09:13:58 -08:00
Kieran 9185f075ca [Enhancement] Overhaul indexing to be more efficient (#540)
* WIP - created methods for breaking on existing media

* WIP - got everything hooked up for POC

* Add some docs, tests

* Refactors

* Updated TODO
2025-01-02 15:48:18 -08:00
Kieran Eglin 09d1653f4b Removed testing route from router 2024-12-30 17:44:45 -08:00
Kieran Eglin 115f675ae3 Bumped version 2024-12-30 17:42:18 -08:00
Kieran f51b219860 [Bugfix] Improve OPML route security (#535)
* WIP - moved plugs; set up a new token-protected route plug

* Added a route_token column to settings model

* Hooked up token_protected_route plug to database

* Hooked up new OPML route to UI; turned RSS and OPML feed buttons into links

* Docs, tests

* Added a note about the phoenix bug
2024-12-30 17:40:23 -08:00
Kieran Eglin 246ca3b299 Bumped version 2024-12-29 10:14:03 -08:00
Robert Kleinschuster c9bd1ea7bd Added OPML Endpoint for podcast rss feeds (#512)
* Added OPML Endpoint for podcast rss feeds

* changed opml route and added controller test for opml endpoint

* add copy opml feed button

* add copy opml feed button - correct url

* fix html indenting

* add indentation to opml

Co-authored-by: Kieran <kieran.eglin@gmail.com>

* use convention for unused controller params

Co-authored-by: Kieran <kieran.eglin@gmail.com>

* add test for opml_sources helper function

* change opml endpoint to be more inline with the other routes

---------

Co-authored-by: robs <git@robs.social>
Co-authored-by: Kieran <kieran.eglin@gmail.com>
2024-12-20 11:47:03 -08:00
Kieran a2a70fcce2 [Bugfix] Respect cookies preference when performing download pre-check (#517)
* Updated get_downloadable_status to pass yt cookies

* Updated tests
2024-12-17 11:18:47 -08:00
Kieran Eglin 0be469dcb0 Added --warnings-as-errors to mix test but only when called by ex_check 2024-12-13 12:41:06 -08:00
Kieran 023f449dbe [Housekeeping] Pass the current action when calling the yt-dlp runner (#514)
* Updated yt-dlp runner to take an action type

* Added actions to all callers of the yt-dlp runner

* [SQUASH] updated test files to use new mocking strategy

* Removed unneeded alias
2024-12-13 12:29:05 -08:00
Robert Kleinschuster e9d365ee9e use css truncate everywhere and remove StringUtils.truncate function (#513)
Co-authored-by: robs <git@robs.social>
2024-12-13 12:23:12 -08:00
Kieran 53e106dac2 [Enhancement] Add sorting, pagination, and new attributes to sources index table (#510)
* WIP - started improving handling of sorting for sources index table

* WIP - Added UI to table to indicate sort column and direction

* Refactored toggle liveview into a livecomponent

* Added sorting for all table attrs

* Added pagination to the sources table

* Added tests for updated liveviews and live components

* Add tests for new helper methods

* Added fancy new CSS to my sources table

* Added size to sources table

* Adds relative div to ensure that sorting arrow doesn't run away

* Fixed da tests
2024-12-13 09:49:00 -08:00
Robert Kleinschuster e56f39a158 Separate tabs for pending and downloaded in media history (#508)
* Separate tabs for pending and downloaded in media history
 - closes #504

* correct query for pending state and remove unecessary assign

* correct pending where

---------

Co-authored-by: robs <git@robs.social>
2024-12-13 08:58:42 -08:00
Robert Kleinschuster 01c9afa1c3 truncate table columns via css (#507)
- closes #506

Co-authored-by: robs <git@robs.social>
2024-12-13 08:57:32 -08:00
Kieran Eglin 5371b5d236 Bumped version 2024-12-10 14:32:20 -08:00
Kieran 83b1fa23d0 [Housekeeping] Update config options to enhance NixOS compatibility (#493)
* Make more paths configurable

* Updated tzdata default path to use the specified 'extras' path
2024-12-10 14:19:34 -08:00
Kieran f8e7b2e42d Added support for 1440p (#498) 2024-12-06 11:13:22 -08:00
Kieran a5b65061f0 [Housekeeping] Bump Phoenix LiveView to 1.0.0 (#495)
* bumped liveview to 1.0.0

* Converted interpolation to new syntax
2024-12-06 10:37:15 -08:00
Kieran 0fb971dd0a Added presets to cutoff date selector (#494) 2024-12-04 12:32:47 -08:00
Kieran Eglin 75fb1a6ab9 Bumped version 2024-11-27 10:43:07 -08:00
Kieran 652fcccb4a [Enhancement] Add audio track language selection to Media Profile (#487)
* Moved quality options to their own module

* Added language and format selection to quality option builder

* [WIP] migrating tests

* Added audio_lang to media_profile table

* Renamed column; added format options and tests

* Adds UI for audio_track to the media profile form

* Adds a version string to in-app streams to help with cache busting
2024-11-27 10:39:29 -08:00
Kieran bfb27427ce [Bugfix] Ensure livestreams aren't downloaded until they're finished processing (#485)
* Added logic to ignore downloads that aren't in the right live state

* Added tests for get_downloadable_status method

* Added tests for media downloader module

* Added tests to download worker modeule
2024-11-26 11:56:33 -08:00
Kieran d9c48370df [Enhancement] Adds ability to enable/disable sources (#481)
* [Unrelated] updated module name for existing liveview module

* Updated toggle component and moved MP index table to a liveview

* [WIP] reverted MP index table; added source count to MP index

* Moved new live table to sources index

* Added 'enabled' boolean to sources

* Got 'enabled' logic working re: downloading pending media

* Updated sources context to do the right thing when a source is updated

* Docs and tests

* Updated slow indexing to maintain its old schedule if re-enabled

* Hooked up the enabled toggle to the sources page

* [Unrelated] added direct links to various tabs on the sources table

* More tests

* Removed unneeded guard in

* Removed outdated comment
2024-11-21 14:38:37 -08:00
Duong Nguyen 4c8c0461be [Enhancement] Add option to use existing Media Profile as template for new profile (#466)
* Add option to use existing Media Profile as template for new profile

* Forgot to commit the edit form too

* Reset deletion mark on Source controller

* Add test for new preload profile feature

* mix check
2024-11-20 10:25:53 -08:00
Kieran a02f71f304 [Enhancement] Add support for yt-dlp plugins + add lifecycle script event for app boot (#465)
* Added new script type to pre-job startup tasks

* Updated Dockerfile to create the needful directories

* added tests
2024-11-08 15:38:00 -08:00
Kieran 83c10b2b00 [Enhancement] Track the predicted final filepath for indexed media items (#461)
* Added ability to pass additional yt-dlp options to indexing step

* Added predicted_filename to media struct

* WIP added ability to predict filepath to source indexing

* renamed predicted_filepath

* Added the ability to predict filepath when fast indexing

* Add predicted_media_filepath to media items table

* Addressed TODOs
2024-11-08 09:42:59 -08:00
Kieran Eglin 8c0dd0bb6b Bumped version 2024-10-30 14:13:08 -07:00
Kieran Eglin 7a374f1162 Updated README 2024-10-30 14:12:29 -07:00
Kieran a5afcf168b Went back to using original_url over webpage_url to improve short detection (#447) 2024-10-30 14:09:28 -07:00
Kieran Eglin 2f83fe03d2 Updated default tzdata location to prevent needless pulls every time I push a new version 2024-10-30 10:18:03 -07:00
Kieran Eglin 3c10d78e78 Updated timezone logic to prevent boot crash 2024-10-30 10:14:04 -07:00
Kieran 8c0df65c0c Took the easy way out and removed playlist index explicitly from the update method (#442) 2024-10-28 10:26:31 -07:00
Kieran 47bb1cb98d Updated timezone logic to not prevent booting (#441) 2024-10-28 09:15:45 -07:00
Kieran Eglin 20547e60e4 Bump version 2024-10-25 11:42:16 -07:00
Kieran 0fd3106cb8 Removed root pw workaround (#437) 2024-10-25 11:17:16 -07:00
Kieran 108a141c65 Improve error messages when adding a non-valid source URL (#436) 2024-10-25 11:01:46 -07:00
Kieran 6a1b7b0160 [Enhancement] Misc. UI updates (#434)
* Format 'Scheduled At' in task table according to user's TZ

* Changed video preview layout to 1 column on all displays

* Made links in descriptions clickable

* Updated Copy JSON button to properly escape newline characters
2024-10-24 12:47:48 -07:00
Kieran 3c8d99196a [Bugfix] Fix off-by-1 error for retention date logic (#432)
* Added a sanity check test to the media context

* Improves logic for handing media item culling dates
2024-10-24 10:26:30 -07:00
Jonas Bergler cae86953a0 document that BASE_ROUTE_PATH requires the prefix to be stripped (#413) 2024-10-22 12:24:34 -07:00
FibreTTP f661747a0c Flip settings submenu chevron (#405) 2024-10-10 14:19:03 -07:00
146 changed files with 13988 additions and 1126 deletions
+24 -20
View File
@@ -15,7 +15,8 @@
<div align="center">
[![](https://img.shields.io/github/license/kieraneglin/pinchflat?style=for-the-badge&color=ee512b)](LICENSE)
[![](https://img.shields.io/github/v/release/kieraneglin/pinchflat?style=for-the-badge)](https://github.com/kieraneglin/pinchflat/releases)
[![](https://img.shields.io/github/v/release/kieraneglin/pinchflat?style=for-the-badge&color=purple)](https://github.com/kieraneglin/pinchflat/releases)
[![](https://img.shields.io/static/v1?style=for-the-badge&logo=discord&message=Chat&color=5865F2&label=Discord)](https://discord.gg/j7T6dCuwU4)
[![](https://img.shields.io/github/actions/workflow/status/kieraneglin/pinchflat/lint_and_test.yml?style=for-the-badge)](#)
[![](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode&style=for-the-badge)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/kieraneglin/pinchflat)
@@ -33,6 +34,7 @@
- [Portainer](#portainer)
- [Docker](#docker)
- [Environment Variables](#environment-variables)
- [A note on reverse proxies](#reverse-proxies)
- [Username and Password (authentication)](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password)
- [Frequently asked questions](https://github.com/kieraneglin/pinchflat/wiki/Frequently-Asked-Questions)
- [Documentation](https://github.com/kieraneglin/pinchflat/wiki)
@@ -53,7 +55,7 @@ If it doesn't work for your use case, please make a feature request! You can als
- Self-contained - just one Docker container with no external dependencies
- Powerful naming system so content is stored where and how you want it
- Easy-to-use web interface with presets to get you started right away
- First-class support for media center apps like Plex, Jellyfin, and Kodi
- First-class support for media center apps like Plex, Jellyfin, and Kodi ([docs](https://github.com/kieraneglin/pinchflat/wiki/Frequently-Asked-Questions#how-do-i-get-media-into-plexjellyfinkodi))
- Supports serving RSS feeds to your favourite podcast app ([docs](https://github.com/kieraneglin/pinchflat/wiki/Podcast-RSS-Feeds))
- Automatically downloads new content from channels and playlists
- Uses a novel approach to download new content more quickly than other apps
@@ -63,7 +65,7 @@ If it doesn't work for your use case, please make a feature request! You can als
- Allows automatically redownloading new media after a set period
- This can help improve the download quality of new content or improve SponsorBlock tags
- Optionally automatically delete old content ([docs](https://github.com/kieraneglin/pinchflat/wiki/Automatically-Delete-Media))
- Advanced options like setting cutoff dates and filtering by title
- Advanced options like setting cutoff dates and filtering by title ([docs](https://github.com/kieraneglin/pinchflat/wiki/Frequently-Asked-Questions#i-only-want-certain-videos-from-a-source---how-can-i-only-download-those))
- Reliable hands-off operation
- Can pass cookies to YouTube to download your private playlists ([docs](https://github.com/kieraneglin/pinchflat/wiki/YouTube-Cookies))
- Sponsorblock integration
@@ -131,9 +133,6 @@ You _must_ ensure the host directories you've mounted are writable by the user r
> [!IMPORTANT]
> It's not recommended to run the container as root. Doing so can create permission issues if other apps need to work with the downloaded media.
> [!TIP]
> If you need to run any command as root, you can run `su` from the container's shell as there is no password set for the root user.
### ADVANCED: Storing Pinchflat config directory on a network share
As pointed out in [#137](https://github.com/kieraneglin/pinchflat/issues/137), SQLite doesn't like being run in WAL mode on network shares. If you're running Pinchflat on a network share, you can disable WAL mode by setting the `JOURNAL_MODE` environment variable to `delete`. This will make Pinchflat run in rollback journal mode which is less performant but should work on network shares.
@@ -145,18 +144,23 @@ If you change this setting and it works well for you, please leave a comment on
### Environment variables
| Name | Required? | Default | Notes |
| --------------------------- | --------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `TZ` | No | `UTC` | Must follow IANA TZ format |
| `LOG_LEVEL` | No | `debug` | Can be set to `info` but `debug` is strongly recommended |
| `BASIC_AUTH_USERNAME` | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| `BASIC_AUTH_PASSWORD` | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| `EXPOSE_FEED_ENDPOINTS` | No | `false` | See [RSS feed docs](https://github.com/kieraneglin/pinchflat/wiki/Podcast-RSS-Feeds) |
| `ENABLE_IPV6` | No | `false` | Setting to _any_ non-blank value will enable IPv6 |
| `JOURNAL_MODE` | No | `wal` | Set to `delete` if your config directory is stored on a network share (not recommended) |
| `TZ_DATA_DIR` | No | `/etc/elixir_tzdata_data` | The container path where the timezone database is stored |
| `BASE_ROUTE_PATH` | No | `/` | The base path for route generation. Useful when running behind certain reverse proxies |
| `YT_DLP_WORKER_CONCURRENCY` | No | `2` | The number of concurrent workers that use `yt-dlp` _per queue_. Set to 1 if you're getting IP limited, otherwise don't touch it |
| Name | Required? | Default | Notes |
| --------------------------- | --------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `TZ` | No | `UTC` | Must follow IANA TZ format |
| `LOG_LEVEL` | No | `debug` | Can be set to `info` but `debug` is strongly recommended |
| `BASIC_AUTH_USERNAME` | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| `BASIC_AUTH_PASSWORD` | No | | See [authentication docs](https://github.com/kieraneglin/pinchflat/wiki/Username-and-Password) |
| `EXPOSE_FEED_ENDPOINTS` | No | `false` | See [RSS feed docs](https://github.com/kieraneglin/pinchflat/wiki/Podcast-RSS-Feeds) |
| `ENABLE_IPV6` | No | `false` | Setting to _any_ non-blank value will enable IPv6 |
| `JOURNAL_MODE` | No | `wal` | Set to `delete` if your config directory is stored on a network share (not recommended) |
| `TZ_DATA_DIR` | No | `/etc/elixir_tzdata_data` | The container path where the timezone database is stored |
| `BASE_ROUTE_PATH` | No | `/` | The base path for route generation. Useful when running behind certain reverse proxies - prefixes must be stripped. |
| `YT_DLP_WORKER_CONCURRENCY` | No | `2` | The number of concurrent workers that use `yt-dlp` _per queue_. Set to 1 if you're getting IP limited, otherwise don't touch it |
| `ENABLE_PROMETHEUS` | No | `false` | Setting to _any_ non-blank value will enable Prometheus. See [docs](https://github.com/kieraneglin/pinchflat/wiki/Prometheus-and-Grafana) |
### Reverse Proxies
Pinchflat makes heavy use of websockets for real-time updates. If you're running Pinchflat behind a reverse proxy then you'll need to make sure it's configured to support websockets.
## EFF donations
@@ -164,9 +168,9 @@ Prior to 2024-05-10, a portion of all donations were given to the [Electronic Fr
The EFF defends your online liberties and [backed](https://github.com/github/dmca/blob/9a85e0f021f7967af80e186b890776a50443f06c/2020/11/2020-11-16-RIAA-reversal-effletter.pdf) `youtube-dl` when Google took them down.
## Pre-release disclaimer
## Stability disclaimer
This is pre-release software and anything can break at any time. I make not guarantees about the stability of this software, forward-compatibility of updates, or integrity (both related to and independent of Pinchflat). Essentially, use at your own risk and expect there will be rough edges for now.
This software is in active development and anything can break at any time. I make no guarantees about the stability of this software, forward-compatibility of updates, or integrity (both related to and independent of Pinchflat).
## License
+11
View File
@@ -35,3 +35,14 @@ window.markVersionAsSeen = (versionString) => {
window.isVersionSeen = (versionString) => {
return localStorage.getItem('seenVersion') === versionString
}
window.dispatchFor = (elementOrId, eventName, detail = {}) => {
const element =
typeof elementOrId === 'string' ? document.getElementById(elementOrId) : elementOrId
// This is needed to ensure the DOM has updated before dispatching the event.
// Doing so ensures that the latest DOM state is what's sent to the server
setTimeout(() => {
element.dispatchEvent(new Event(eventName, { bubbles: true, detail }))
}, 0)
}
+1 -1
View File
@@ -39,7 +39,7 @@ let liveSocket = new LiveSocket(document.body.dataset.socketPath, Socket, {
}
},
hooks: {
supressEnterSubmission: {
'supress-enter-submission': {
mounted() {
this.el.addEventListener('keypress', (event) => {
if (event.key === 'Enter') {
+32
View File
@@ -347,6 +347,38 @@ module.exports = {
},
{ values }
)
}),
plugin(function ({ matchComponents, theme }) {
let iconsDir = path.join(__dirname, './vendor/simple-icons')
let values = {}
fs.readdirSync(iconsDir).forEach((file) => {
let name = path.basename(file, '.svg')
values[name] = { name, fullPath: path.join(iconsDir, file) }
})
matchComponents(
{
si: ({ name, fullPath }) => {
let content = fs
.readFileSync(fullPath)
.toString()
.replace(/\r?\n|\r/g, '')
return {
[`--si-${name}`]: `url('data:image/svg+xml;utf8,${content}')`,
'-webkit-mask': `var(--si-${name})`,
mask: `var(--si-${name})`,
'mask-repeat': 'no-repeat',
'background-color': 'currentColor',
'vertical-align': 'middle',
display: 'inline-block',
width: theme('spacing.5'),
height: theme('spacing.5')
}
}
},
{ values }
)
})
]
}
+1
View File
@@ -0,0 +1 @@
<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Discord</title><path d="M20.317 4.3698a19.7913 19.7913 0 00-4.8851-1.5152.0741.0741 0 00-.0785.0371c-.211.3753-.4447.8648-.6083 1.2495-1.8447-.2762-3.68-.2762-5.4868 0-.1636-.3933-.4058-.8742-.6177-1.2495a.077.077 0 00-.0785-.037 19.7363 19.7363 0 00-4.8852 1.515.0699.0699 0 00-.0321.0277C.5334 9.0458-.319 13.5799.0992 18.0578a.0824.0824 0 00.0312.0561c2.0528 1.5076 4.0413 2.4228 5.9929 3.0294a.0777.0777 0 00.0842-.0276c.4616-.6304.8731-1.2952 1.226-1.9942a.076.076 0 00-.0416-.1057c-.6528-.2476-1.2743-.5495-1.8722-.8923a.077.077 0 01-.0076-.1277c.1258-.0943.2517-.1923.3718-.2914a.0743.0743 0 01.0776-.0105c3.9278 1.7933 8.18 1.7933 12.0614 0a.0739.0739 0 01.0785.0095c.1202.099.246.1981.3728.2924a.077.077 0 01-.0066.1276 12.2986 12.2986 0 01-1.873.8914.0766.0766 0 00-.0407.1067c.3604.698.7719 1.3628 1.225 1.9932a.076.076 0 00.0842.0286c1.961-.6067 3.9495-1.5219 6.0023-3.0294a.077.077 0 00.0313-.0552c.5004-5.177-.8382-9.6739-3.5485-13.6604a.061.061 0 00-.0312-.0286zM8.02 15.3312c-1.1825 0-2.1569-1.0857-2.1569-2.419 0-1.3332.9555-2.4189 2.157-2.4189 1.2108 0 2.1757 1.0952 2.1568 2.419 0 1.3332-.9555 2.4189-2.1569 2.4189zm7.9748 0c-1.1825 0-2.1569-1.0857-2.1569-2.419 0-1.3332.9554-2.4189 2.1569-2.4189 1.2108 0 2.1757 1.0952 2.1568 2.419 0 1.3332-.946 2.4189-2.1568 2.4189Z"/></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+1
View File
@@ -0,0 +1 @@
<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>GitHub</title><path d="M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12"/></svg>

After

Width:  |  Height:  |  Size: 823 B

+6
View File
@@ -99,6 +99,12 @@ config :logger, :default_formatter,
# Use Jason for JSON parsing in Phoenix
config :phoenix, :json_library, Jason
config :pinchflat, Pinchflat.PromEx,
disabled: true,
manual_metrics_start_delay: :no_delay,
drop_metrics_groups: [],
metrics_server: :disabled
# Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above.
import_config "#{config_env()}.exs"
+2
View File
@@ -81,3 +81,5 @@ config :phoenix_live_view, :debug_heex_annotations, true
# Disable swoosh api client as it is only required for production adapters.
config :swoosh, :api_client, false
config :pinchflat, Pinchflat.PromEx, disabled: false
+14 -5
View File
@@ -55,11 +55,16 @@ config :pinchflat, Oban,
]
if config_env() == :prod do
config_path = "/config"
# Various paths. These ones shouldn't be tweaked if running in Docker
media_path = System.get_env("MEDIA_PATH", "/downloads")
config_path = System.get_env("CONFIG_PATH", "/config")
db_path = System.get_env("DATABASE_PATH", Path.join([config_path, "db", "pinchflat.db"]))
log_path = System.get_env("LOG_PATH", Path.join([config_path, "logs", "pinchflat.log"]))
metadata_path = System.get_env("METADATA_PATH", Path.join([config_path, "metadata"]))
extras_path = System.get_env("EXTRAS_PATH", Path.join([config_path, "extras"]))
tmpfile_path = System.get_env("TMPFILE_PATH", Path.join([System.tmp_dir!(), "pinchflat", "data"]))
# This one can be changed if you want
tz_data_path = System.get_env("TZ_DATA_PATH", Path.join([extras_path, "elixir_tz_data"]))
# For running PF as a podcast host on self-hosted environments
expose_feed_endpoints = String.length(System.get_env("EXPOSE_FEED_ENDPOINTS", "")) > 0
# For testing alternate journal modes (see issue #137)
@@ -67,28 +72,32 @@ if config_env() == :prod do
# For running PF in a subdirectory via a reverse proxy
base_route_path = System.get_env("BASE_ROUTE_PATH", "/")
enable_ipv6 = String.length(System.get_env("ENABLE_IPV6", "")) > 0
enable_prometheus = String.length(System.get_env("ENABLE_PROMETHEUS", "")) > 0
config :logger, level: String.to_existing_atom(System.get_env("LOG_LEVEL", "debug"))
config :pinchflat,
yt_dlp_executable: System.find_executable("yt-dlp"),
apprise_executable: System.find_executable("apprise"),
media_directory: "/downloads",
media_directory: media_path,
metadata_directory: metadata_path,
extras_directory: extras_path,
tmpfile_directory: Path.join([System.tmp_dir!(), "pinchflat", "data"]),
tmpfile_directory: tmpfile_path,
dns_cluster_query: System.get_env("DNS_CLUSTER_QUERY"),
expose_feed_endpoints: expose_feed_endpoints,
timezone: System.get_env("TIMEZONE") || System.get_env("TZ") || "UTC",
# This is configured in application.ex
timezone: "UTC",
log_path: log_path,
base_route_path: base_route_path
config :tzdata, :data_dir, System.get_env("TZ_DATA_DIR", "/etc/elixir_tzdata_data")
config :tzdata, :data_dir, tz_data_path
config :pinchflat, Pinchflat.Repo,
database: db_path,
journal_mode: journal_mode
config :pinchflat, Pinchflat.PromEx, disabled: !enable_prometheus
# The secret key base is used to sign/encrypt cookies and other secrets.
# A default value is used in config/dev.exs and config/test.exs but you
# want to use a different value for prod and you most likely don't want
+4 -13
View File
@@ -88,6 +88,7 @@ RUN apt-get update -y && \
ca-certificates \
python3-mutagen \
curl \
zip \
openssh-client \
nano \
python3 \
@@ -116,7 +117,8 @@ ENV LC_ALL en_US.UTF-8
WORKDIR "/app"
# Set up data volumes
RUN mkdir /config /downloads /etc/elixir_tzdata_data && chmod ugo+rw /etc/elixir_tzdata_data
RUN mkdir -p /config /downloads /etc/elixir_tzdata_data /etc/yt-dlp/plugins && \
chmod ugo+rw /etc/elixir_tzdata_data /etc/yt-dlp /etc/yt-dlp/plugins
# set runner ENV
ENV MIX_ENV="prod"
@@ -127,18 +129,7 @@ EXPOSE ${PORT}
# Only copy the final release from the build stage
COPY --from=builder /app/_build/${MIX_ENV}/rel/pinchflat ./
# NEVER do this if you're running in an environment where you don't trust the user
# (ie: most environments). This is only acceptable in a self-hosted environment.
# The user could just run the whole container as root and bypass this anyway so
# it's not a huge deal.
# This removes the root password to allow users to assume root if needed. This is
# preferrable to running the whole container as root so that the files/directories
# created by the app aren't owned by root and are therefore easier for other users
# and processes to interact with. If you want to just run the whole container as
# root, use --user 0:0 or something.
RUN passwd -d root
HEALTHCHECK --interval=120s --start-period=10s \
HEALTHCHECK --interval=30s --start-period=15s \
CMD curl --fail http://localhost:${PORT}/healthcheck || exit 1
# Start the app
+20
View File
@@ -4,10 +4,14 @@ defmodule Pinchflat.Application do
@moduledoc false
use Application
require Logger
@impl true
def start(_type, _args) do
check_and_update_timezone()
children = [
Pinchflat.PromEx,
PinchflatWeb.Telemetry,
Pinchflat.Repo,
# Must be before startup tasks
@@ -47,4 +51,20 @@ defmodule Pinchflat.Application do
:ok = Oban.Telemetry.attach_default_logger()
:telemetry.attach_many("job-telemetry-broadcast", events, &PinchflatWeb.Telemetry.job_state_change_broadcast/4, [])
end
# This has to be here (rather than runtime.exs) since the `tzdata` application
# has to be started before we can check the timezone
defp check_and_update_timezone do
attempted_timezone = System.get_env("TIMEZONE") || System.get_env("TZ") || "UTC"
valid_timezone =
if Tzdata.zone_exists?(attempted_timezone) do
attempted_timezone
else
Logger.warning("Invalid timezone #{attempted_timezone}, defaulting to UTC")
"UTC"
end
Application.put_env(:pinchflat, :timezone, valid_timezone)
end
end
@@ -16,6 +16,8 @@ defmodule Pinchflat.Boot.PreJobStartupTasks do
alias Pinchflat.Settings
alias Pinchflat.Utils.FilesystemUtils
alias Pinchflat.Lifecycle.UserScripts.CommandRunner, as: UserScriptRunner
def start_link(opts \\ []) do
GenServer.start_link(__MODULE__, %{}, opts)
end
@@ -36,6 +38,7 @@ defmodule Pinchflat.Boot.PreJobStartupTasks do
create_blank_yt_dlp_files()
create_blank_user_script_file()
apply_default_settings()
run_app_init_script()
{:ok, state}
end
@@ -95,6 +98,12 @@ defmodule Pinchflat.Boot.PreJobStartupTasks do
Settings.set(apprise_version: apprise_version)
end
defp run_app_init_script do
runner = Application.get_env(:pinchflat, :user_script_runner, UserScriptRunner)
runner.run(:app_init, %{})
end
defp yt_dlp_runner do
Application.get_env(:pinchflat, :yt_dlp_runner)
end
@@ -4,10 +4,10 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilder do
"""
alias Pinchflat.Sources
alias Pinchflat.Settings
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
alias Pinchflat.Downloading.OutputPathBuilder
alias Pinchflat.Downloading.QualityOptionBuilder
alias Pinchflat.Utils.FilesystemUtils, as: FSUtils
@@ -34,21 +34,38 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilder do
@doc """
Builds the output path for yt-dlp to download media based on the given source's
media profile. Uses the source's override output path template if it exists.
or media_item's media profile. Uses the source's override output path template if it exists.
Accepts a %MediaItem{} or %Source{} struct. If a %Source{} struct is passed, it
will use a default %MediaItem{} struct with the given source.
Returns binary()
"""
def build_output_path_for(%Source{} = source_with_preloads) do
build_output_path_for(%MediaItem{source: source_with_preloads})
end
def build_output_path_for(%MediaItem{} = media_item_with_preloads) do
output_path_template = Sources.output_path_template(media_item_with_preloads.source)
build_output_path(output_path_template, media_item_with_preloads)
end
def build_output_path_for(%Source{} = source_with_preloads) do
build_output_path_for(%MediaItem{source: source_with_preloads})
@doc """
Builds the quality options for yt-dlp to download media based on the given source's
or media_item's media profile. Useful for helping predict final filepath of downloaded
media.
returns [Keyword.t()]
"""
def build_quality_options_for(%Source{} = source_with_preloads) do
build_quality_options_for(%MediaItem{source: source_with_preloads})
end
def build_quality_options_for(%MediaItem{} = media_item_with_preloads) do
media_profile = media_item_with_preloads.source.media_profile
quality_options(media_profile)
end
defp default_options(override_opts) do
@@ -125,27 +142,7 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilder do
end
defp quality_options(media_profile) do
vcodec = Settings.get!(:video_codec_preference)
acodec = Settings.get!(:audio_codec_preference)
container = media_profile.media_container
case media_profile.preferred_resolution do
# Also be aware that :audio disabled all embedding options for subtitles
:audio ->
[:extract_audio, format_sort: "+acodec:#{acodec}", audio_format: container || "best"]
resolution_atom ->
{resolution_string, _} =
resolution_atom
|> Atom.to_string()
|> Integer.parse()
[
# Since Plex doesn't support reading metadata from MKV
remux_video: container || "mp4",
format_sort: "res:#{resolution_string},+codec:#{vcodec}:#{acodec}"
]
end
QualityOptionBuilder.build(media_profile)
end
defp sponsorblock_options(media_profile) do
@@ -155,6 +152,7 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilder do
case {behaviour, categories} do
{_, []} -> []
{:remove, _} -> [sponsorblock_remove: Enum.join(categories, ",")]
{:mark, _} -> [sponsorblock_mark: Enum.join(categories, ",")]
{:disabled, _} -> []
end
end
@@ -203,6 +201,9 @@ defmodule Pinchflat.Downloading.DownloadOptionBuilder do
source = media_item_with_preloads.source
%{
"media_item_id" => to_string(media_item_with_preloads.id),
"source_id" => to_string(source.id),
"media_profile_id" => to_string(source.media_profile_id),
"source_custom_name" => source.custom_name,
"source_collection_id" => source.collection_id,
"source_collection_name" => source.collection_name,
@@ -94,6 +94,9 @@ defmodule Pinchflat.Downloading.MediaDownloadWorker do
{:recovered, _} ->
{:error, :retry}
{:error, :unsuitable_for_download} ->
{:ok, :non_retry}
{:error, message} ->
action_on_error(message)
end
@@ -112,7 +115,7 @@ defmodule Pinchflat.Downloading.MediaDownloadWorker do
defp action_on_error(message) do
# This will attempt re-download at the next indexing, but it won't be retried
# immediately as part of job failure logic
non_retryable_errors = ["Video unavailable"]
non_retryable_errors = ["Video unavailable", "Sign in to confirm"]
if String.contains?(to_string(message), non_retryable_errors) do
Logger.error("yt-dlp download will not be retried: #{inspect(message)}")
+14 -2
View File
@@ -37,6 +37,13 @@ defmodule Pinchflat.Downloading.MediaDownloader do
{:ok, parsed_json} ->
update_media_item_from_parsed_json(media_with_preloads, parsed_json)
{:error, :unsuitable_for_download} ->
Logger.warning(
"Media item ##{media_with_preloads.id} isn't suitable for download yet. May be an active or processing live stream"
)
{:error, :unsuitable_for_download}
{:error, message, _exit_code} ->
Logger.error("yt-dlp download error for media item ##{media_with_preloads.id}: #{inspect(message)}")
@@ -106,9 +113,14 @@ defmodule Pinchflat.Downloading.MediaDownloader do
defp download_with_options(url, item_with_preloads, output_filepath, override_opts) do
{:ok, options} = DownloadOptionBuilder.build(item_with_preloads, override_opts)
runner_opts = [output_filepath: output_filepath, use_cookies: item_with_preloads.source.use_cookies]
use_cookies = item_with_preloads.source.use_cookies
runner_opts = [output_filepath: output_filepath, use_cookies: use_cookies]
YtDlpMedia.download(url, options, runner_opts)
case YtDlpMedia.get_downloadable_status(url, use_cookies: use_cookies) do
{:ok, :downloadable} -> YtDlpMedia.download(url, options, runner_opts)
{:ok, :ignorable} -> {:error, :unsuitable_for_download}
err -> err
end
end
defp recoverable_errors do
@@ -49,6 +49,8 @@ defmodule Pinchflat.Downloading.MediaRetentionWorker do
end)
end
# NOTE: Since this is a date and not a datetime, we can't add logic to have to-the-minute
# comparison like we can with retention periods. We can only compare to the day.
defp delete_media_items_from_before_cutoff do
deletable_media =
MediaQuery.new()
@@ -0,0 +1,66 @@
defmodule Pinchflat.Downloading.QualityOptionBuilder do
@moduledoc """
A standalone builder module for building quality-related options for yt-dlp to download media.
Currently exclusively used in DownloadOptionBuilder since this logic is too complex to just
place in the main module.
"""
alias Pinchflat.Settings
alias Pinchflat.Profiles.MediaProfile
@doc """
Builds the quality-related options for yt-dlp to download media based on the given media profile
Includes things like container, preferred format/codec, and audio track options.
"""
def build(%MediaProfile{preferred_resolution: :audio, media_container: container} = media_profile) do
acodec = Settings.get!(:audio_codec_preference)
[
:extract_audio,
format_sort: "+acodec:#{acodec}",
audio_format: container || "best",
format: build_format_string(media_profile)
]
end
def build(%MediaProfile{preferred_resolution: resolution_atom, media_container: container} = media_profile) do
vcodec = Settings.get!(:video_codec_preference)
acodec = Settings.get!(:audio_codec_preference)
{resolution_string, _} = resolution_atom |> Atom.to_string() |> Integer.parse()
[
# Since Plex doesn't support reading metadata from MKV
remux_video: container || "mp4",
format_sort: "res:#{resolution_string},+codec:#{vcodec}:#{acodec}",
format: build_format_string(media_profile)
]
end
defp build_format_string(%MediaProfile{preferred_resolution: :audio, audio_track: audio_track}) do
if audio_track do
"bestaudio[#{build_format_modifier(audio_track)}]/bestaudio/best"
else
"bestaudio/best"
end
end
defp build_format_string(%MediaProfile{audio_track: audio_track}) do
if audio_track do
"bestvideo+bestaudio[#{build_format_modifier(audio_track)}]/bestvideo*+bestaudio/best"
else
"bestvideo*+bestaudio/best"
end
end
# Reminder to self: this conflicts with `--extractor-args "youtube:lang=<LANG>"`
# since that will translate the format_notes as well, which means they may not match.
# At least that's what happens now - worth a re-check if I have to come back to this
defp build_format_modifier("original"), do: "format_note*=original"
defp build_format_modifier("default"), do: "format_note*='(default)'"
# This uses the carat to anchor the language to the beginning of the string
# since that's what's needed to match `en` to `en-US` and `en-GB`, etc. The user
# can always specify the full language code if they want.
defp build_format_modifier(language_code), do: "language^=#{language_code}"
end
@@ -11,13 +11,27 @@ defmodule Pinchflat.FastIndexing.FastIndexingHelpers do
alias Pinchflat.Repo
alias Pinchflat.Media
alias Pinchflat.Tasks
alias Pinchflat.Sources.Source
alias Pinchflat.FastIndexing.YoutubeRss
alias Pinchflat.FastIndexing.YoutubeApi
alias Pinchflat.Downloading.DownloadingHelpers
alias Pinchflat.FastIndexing.FastIndexingWorker
alias Pinchflat.Downloading.DownloadOptionBuilder
alias Pinchflat.YtDlp.Media, as: YtDlpMedia
@doc """
Kicks off a new fast indexing task for a source. This will delete any existing fast indexing
tasks for the source before starting a new one.
Returns {:ok, %Task{}}
"""
def kickoff_indexing_task(%Source{} = source) do
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker", include_executing: true)
FastIndexingWorker.kickoff_with_task(source)
end
@doc """
Fetches new media IDs for a source from YT's API or RSS, indexes them, and kicks off downloading
tasks for any pending media items. See comments in `FastIndexingWorker` for more info on the
@@ -27,6 +41,10 @@ defmodule Pinchflat.FastIndexing.FastIndexingHelpers do
downloaded_.
"""
def kickoff_download_tasks_from_youtube_rss_feed(%Source{} = source) do
# The media_profile is needed to determine the quality options to _then_ determine a more
# accurate predicted filepath
source = Repo.preload(source, [:media_profile])
{:ok, media_ids} = get_recent_media_ids(source)
existing_media_items = list_media_items_by_media_id_for(source, media_ids)
new_media_ids = media_ids -- Enum.map(existing_media_items, & &1.media_id)
@@ -68,7 +86,11 @@ defmodule Pinchflat.FastIndexing.FastIndexingHelpers do
defp create_media_item_from_media_id(source, media_id) do
url = "https://www.youtube.com/watch?v=#{media_id}"
case YtDlpMedia.get_media_attributes(url, use_cookies: source.use_cookies) do
command_opts =
[output: DownloadOptionBuilder.build_output_path_for(source)] ++
DownloadOptionBuilder.build_quality_options_for(source)
case YtDlpMedia.get_media_attributes(url, command_opts, use_cookies: source.use_cookies) do
{:ok, media_attrs} ->
Media.create_media_item_from_backend_attrs(source, media_attrs)
@@ -12,6 +12,7 @@ defmodule Pinchflat.Lifecycle.UserScripts.CommandRunner do
@behaviour UserScriptCommandRunner
@event_types [
:app_init,
:media_pre_download,
:media_downloaded,
:media_deleted
+7 -4
View File
@@ -15,6 +15,9 @@ defmodule Pinchflat.Media do
alias Pinchflat.Lifecycle.UserScripts.CommandRunner, as: UserScriptRunner
# Some fields should only be set on insert and not on update.
@fields_to_drop_on_update [:playlist_index]
@doc """
Returns the list of media_items.
@@ -131,8 +134,6 @@ defmodule Pinchflat.Media do
"""
def create_media_item_from_backend_attrs(source, media_attrs_struct) do
attrs = Map.merge(%{source_id: source.id}, Map.from_struct(media_attrs_struct))
# Some fields should only be set on insert and not on update.
fields_to_drop_on_update = [:playlist_index]
%MediaItem{}
|> MediaItem.changeset(attrs)
@@ -140,7 +141,7 @@ defmodule Pinchflat.Media do
on_conflict: [
set:
attrs
|> Map.drop(fields_to_drop_on_update)
|> Map.drop(@fields_to_drop_on_update)
|> Map.to_list()
],
conflict_target: [:source_id, :media_id]
@@ -153,8 +154,10 @@ defmodule Pinchflat.Media do
Returns {:ok, %MediaItem{}} | {:error, %Ecto.Changeset{}}
"""
def update_media_item(%MediaItem{} = media_item, attrs) do
update_attrs = Map.drop(attrs, @fields_to_drop_on_update)
media_item
|> MediaItem.changeset(attrs)
|> MediaItem.changeset(update_attrs)
|> Repo.update()
end
+5
View File
@@ -31,6 +31,7 @@ defmodule Pinchflat.Media.MediaItem do
:uploaded_at,
:upload_date_index,
:duration_seconds,
:predicted_media_filepath,
# these fields are captured only on download
:media_downloaded_at,
:media_filepath,
@@ -76,6 +77,7 @@ defmodule Pinchflat.Media.MediaItem do
field :duration_seconds, :integer
field :playlist_index, :integer, default: 0
field :predicted_media_filepath, :string
field :media_filepath, :string
field :media_size_bytes, :integer
field :thumbnail_filepath, :string
@@ -110,6 +112,9 @@ defmodule Pinchflat.Media.MediaItem do
|> dynamic_default(:uuid, fn _ -> Ecto.UUID.generate() end)
|> update_upload_date_index()
|> validate_required(@required_fields)
# Validate that the title does NOT start with "youtube video #" since that indicates a restriction by YouTube.
# See issue #549 for more information.
|> validate_format(:title, ~r/^(?!youtube video #)/)
|> unique_constraint([:media_id, :source_id])
end
+3 -3
View File
@@ -88,7 +88,7 @@ defmodule Pinchflat.Media.MediaQuery do
[mi, source],
fragment("""
IFNULL(retention_period_days, 0) > 0 AND
DATETIME('now', '-' || retention_period_days || ' day') > media_downloaded_at
DATETIME(media_downloaded_at, '+' || retention_period_days || ' day') < DATETIME('now')
""")
)
end
@@ -100,8 +100,8 @@ defmodule Pinchflat.Media.MediaQuery do
# downloaded_at minus the redownload_delay_days is before the upload date
fragment("""
IFNULL(redownload_delay_days, 0) > 0 AND
DATETIME('now', '-' || redownload_delay_days || ' day') > uploaded_at AND
DATETIME(media_downloaded_at, '-' || redownload_delay_days || ' day') < uploaded_at
DATE('now', '-' || redownload_delay_days || ' day') > DATE(uploaded_at) AND
DATE(media_downloaded_at, '-' || redownload_delay_days || ' day') < DATE(uploaded_at)
""")
)
end
@@ -0,0 +1,40 @@
defmodule Pinchflat.Podcasts.OpmlFeedBuilder do
@moduledoc """
Methods for building an OPML feed for a list of sources.
"""
import Pinchflat.Utils.XmlUtils, only: [safe: 1]
alias PinchflatWeb.Router.Helpers, as: Routes
@doc """
Builds an OPML feed for a given list of sources.
Returns an XML document as a string.
"""
def build(url_base, sources) do
sources_xml =
Enum.map(
sources,
&"""
<outline type="rss" text="#{safe(&1.custom_name)}" xmlUrl="#{safe(source_route(url_base, &1))}" />
"""
)
"""
<?xml version="1.0" encoding="UTF-8"?>
<opml version="2.0">
<head>
<title>All Sources</title>
</head>
<body>
#{Enum.join(sources_xml, "\n")}
</body>
</opml>
"""
end
defp source_route(url_base, source) do
Path.join(url_base, "#{Routes.podcast_path(PinchflatWeb.Endpoint, :rss_feed, source.uuid)}.xml")
end
end
+14
View File
@@ -5,11 +5,25 @@ defmodule Pinchflat.Podcasts.PodcastHelpers do
"""
use Pinchflat.Media.MediaQuery
use Pinchflat.Sources.SourcesQuery
alias Pinchflat.Repo
alias Pinchflat.Metadata.MediaMetadata
alias Pinchflat.Metadata.SourceMetadata
@doc """
Returns a list of sources that are not marked for deletion.
Returns: [%Source{}]
"""
def opml_sources() do
SourcesQuery.new()
|> select([s], %{custom_name: s.custom_name, uuid: s.uuid})
|> where([s], is_nil(s.marked_for_deletion_at))
|> order_by(asc: :custom_name)
|> Repo.all()
end
@doc """
Returns a list of media items that have been downloaded to disk
and have been proven to still exist there.
+4 -2
View File
@@ -26,6 +26,7 @@ defmodule Pinchflat.Profiles.MediaProfile do
sponsorblock_categories
shorts_behaviour
livestream_behaviour
audio_track
preferred_resolution
media_container
redownload_delay_days
@@ -54,7 +55,7 @@ defmodule Pinchflat.Profiles.MediaProfile do
field :embed_metadata, :boolean, default: false
field :download_nfo, :boolean, default: false
field :sponsorblock_behaviour, Ecto.Enum, values: [:disabled, :remove], default: :disabled
field :sponsorblock_behaviour, Ecto.Enum, values: [:disabled, :mark, :remove], default: :disabled
field :sponsorblock_categories, {:array, :string}, default: []
# NOTE: these do NOT speed up indexing - the indexer still has to go
# through the entire collection to determine if a media is a short or
@@ -65,7 +66,8 @@ defmodule Pinchflat.Profiles.MediaProfile do
# See `build_format_clauses` in the Media context for more.
field :shorts_behaviour, Ecto.Enum, values: ~w(include exclude only)a, default: :include
field :livestream_behaviour, Ecto.Enum, values: ~w(include exclude only)a, default: :include
field :preferred_resolution, Ecto.Enum, values: ~w(4320p 2160p 1080p 720p 480p 360p audio)a, default: :"1080p"
field :audio_track, :string
field :preferred_resolution, Ecto.Enum, values: ~w(4320p 2160p 1440p 1080p 720p 480p 360p audio)a, default: :"1080p"
field :media_container, :string, default: nil
field :marked_for_deletion_at, :utc_datetime
+29
View File
@@ -0,0 +1,29 @@
defmodule Pinchflat.Profiles.ProfilesQuery do
@moduledoc """
Query helpers for the Profiles context.
These methods are made to be one-ish liners used
to compose queries. Each method should strive to do
_one_ thing. These don't need to be tested as
they are just building blocks for other functionality
which, itself, will be tested.
"""
import Ecto.Query, warn: false
alias Pinchflat.Profiles.MediaProfile
# This allows the module to be aliased and query methods to be used
# all in one go
# usage: use Pinchflat.Profiles.ProfilesQuery
defmacro __using__(_opts) do
quote do
import Ecto.Query, warn: false
alias unquote(__MODULE__)
end
end
def new do
MediaProfile
end
end
+40
View File
@@ -0,0 +1,40 @@
defmodule Pinchflat.PromEx do
@moduledoc """
Configuration for the PromEx library which provides Prometheus metrics
"""
use PromEx, otp_app: :pinchflat
alias PromEx.Plugins
@impl true
def plugins do
[
Plugins.Application,
Plugins.Beam,
{Plugins.Phoenix, router: PinchflatWeb.Router, endpoint: PinchflatWeb.Endpoint},
Plugins.Ecto,
Plugins.Oban,
Plugins.PhoenixLiveView
]
end
@impl true
def dashboard_assigns do
[
default_selected_interval: "30s"
]
end
@impl true
def dashboards do
[
{:prom_ex, "application.json"},
{:prom_ex, "beam.json"},
{:prom_ex, "phoenix.json"},
{:prom_ex, "ecto.json"},
{:prom_ex, "oban.json"},
{:prom_ex, "phoenix_live_view.json"}
]
end
end
+2
View File
@@ -29,6 +29,8 @@ defmodule Pinchflat.Release do
[
"/config",
"/downloads",
"/etc/yt-dlp",
"/etc/yt-dlp/plugins",
Application.get_env(:pinchflat, :media_directory),
Application.get_env(:pinchflat, :tmpfile_directory),
Application.get_env(:pinchflat, :extras_directory),
+12 -7
View File
@@ -14,15 +14,17 @@ defmodule Pinchflat.Settings.Setting do
:apprise_server,
:video_codec_preference,
:audio_codec_preference,
:youtube_api_key
:youtube_api_key,
:extractor_sleep_interval_seconds
]
@required_fields ~w(
onboarding
pro_enabled
video_codec_preference
audio_codec_preference
)a
@required_fields [
:onboarding,
:pro_enabled,
:video_codec_preference,
:audio_codec_preference,
:extractor_sleep_interval_seconds
]
schema "settings" do
field :onboarding, :boolean, default: true
@@ -31,6 +33,8 @@ defmodule Pinchflat.Settings.Setting do
field :apprise_version, :string
field :apprise_server, :string
field :youtube_api_key, :string
field :route_token, :string
field :extractor_sleep_interval_seconds, :integer, default: 0
field :video_codec_preference, :string
field :audio_codec_preference, :string
@@ -41,5 +45,6 @@ defmodule Pinchflat.Settings.Setting do
setting
|> cast(attrs, @allowed_fields)
|> validate_required(@required_fields)
|> validate_number(:extractor_sleep_interval_seconds, greater_than: 0)
end
end
@@ -79,21 +79,21 @@ defmodule Pinchflat.SlowIndexing.MediaCollectionIndexingWorker do
case {source.index_frequency_minutes, source.last_indexed_at} do
{index_freq, _} when index_freq > 0 ->
# If the indexing is on a schedule simply run indexing and reschedule
perform_indexing_and_notification(source)
perform_indexing_and_notification(source, was_forced: args["force"])
maybe_enqueue_fast_indexing_task(source)
reschedule_indexing(source)
{_, nil} ->
# If the source has never been indexed, index it once
# even if it's not meant to reschedule
perform_indexing_and_notification(source)
perform_indexing_and_notification(source, was_forced: args["force"])
:ok
_ ->
# If the source HAS been indexed and is not meant to reschedule,
# perform a no-op (unless forced)
if args["force"] do
perform_indexing_and_notification(source)
perform_indexing_and_notification(source, was_forced: true)
end
:ok
@@ -103,11 +103,11 @@ defmodule Pinchflat.SlowIndexing.MediaCollectionIndexingWorker do
Ecto.StaleEntryError -> Logger.info("#{__MODULE__} discarded: source #{source_id} stale")
end
defp perform_indexing_and_notification(source) do
defp perform_indexing_and_notification(source, indexing_opts) do
apprise_server = Settings.get!(:apprise_server)
SourceNotifications.wrap_new_media_notification(apprise_server, source, fn ->
SlowIndexingHelpers.index_and_enqueue_download_for_media_items(source)
SlowIndexingHelpers.index_and_enqueue_download_for_media_items(source, indexing_opts)
end)
end
@@ -5,6 +5,8 @@ defmodule Pinchflat.SlowIndexing.SlowIndexingHelpers do
Many of these methods are made to be kickoff or be consumed by workers.
"""
use Pinchflat.Media.MediaQuery
require Logger
alias Pinchflat.Repo
@@ -14,30 +16,53 @@ defmodule Pinchflat.SlowIndexing.SlowIndexingHelpers do
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
alias Pinchflat.YtDlp.MediaCollection
alias Pinchflat.Utils.FilesystemUtils
alias Pinchflat.Downloading.DownloadingHelpers
alias Pinchflat.SlowIndexing.FileFollowerServer
alias Pinchflat.Downloading.DownloadOptionBuilder
alias Pinchflat.SlowIndexing.MediaCollectionIndexingWorker
alias Pinchflat.YtDlp.Media, as: YtDlpMedia
@doc """
Starts tasks for indexing a source's media regardless of the source's indexing
frequency. It's assumed the caller will check for indexing frequency.
Kills old indexing tasks and starts a new task to index the media collection.
Returns {:ok, %Task{}}.
The job is delayed based on the source's `index_frequency_minutes` setting unless
one of the following is true:
- The `force` option is set to true
- The source has never been indexed before
- The source has been indexed before, but the last indexing job was more than
`index_frequency_minutes` ago
Returns {:ok, %Task{}}
"""
def kickoff_indexing_task(%Source{} = source, job_args \\ %{}, job_opts \\ []) do
job_offset_seconds = if job_args[:force], do: 0, else: calculate_job_offset_seconds(source)
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker")
Tasks.delete_pending_tasks_for(source, "MediaCollectionIndexingWorker", include_executing: true)
MediaCollectionIndexingWorker.kickoff_with_task(source, job_args, job_opts)
MediaCollectionIndexingWorker.kickoff_with_task(source, job_args, job_opts ++ [schedule_in: job_offset_seconds])
end
@doc """
A helper method to delete all indexing-related tasks for a source.
Optionally, you can include executing tasks in the deletion process.
Returns :ok
"""
def delete_indexing_tasks(%Source{} = source, opts \\ []) do
include_executing = Keyword.get(opts, :include_executing, false)
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker", include_executing: include_executing)
Tasks.delete_pending_tasks_for(source, "MediaCollectionIndexingWorker", include_executing: include_executing)
end
@doc """
Given a media source, creates (indexes) the media by creating media_items for each
media ID in the source. Afterward, kicks off a download task for each pending media
item belonging to the source. You can't tell me the method name isn't descriptive!
Returns a list of media items or changesets (if the media item couldn't be created).
item belonging to the source. Returns a list of media items or changesets
(if the media item couldn't be created).
Indexing is slow and usually returns a list of all media data at once for record creation.
To help with this, we use a file follower to watch the file that yt-dlp writes to
@@ -45,20 +70,33 @@ defmodule Pinchflat.SlowIndexing.SlowIndexingHelpers do
clarity to the user experience. This has a few things to be aware of which are documented
below in the file watcher setup method.
Additionally, in the case of a repeat index we create a download archive file that
contains some media IDs that we've indexed in the past. Note that this archive doesn't
contain the most recent IDs but rather a subset of IDs that are offset by some amount.
Practically, this means that we'll re-index a small handful of media that we've recently
indexed, but this is a good thing since it'll let us pick up on any recent changes to the
most recent media items.
We don't create a download archive for playlists (only channels), nor do we create one if
the indexing was forced by the user.
NOTE: downloads are only enqueued if the source is set to download media. Downloads are
also enqueued for ALL pending media items, not just the ones that were indexed in this
job run. This should ensure that any stragglers are caught if, for some reason, they
weren't enqueued or somehow got de-queued.
Since indexing returns all media data EVERY TIME, we that that opportunity to update
indexing metadata for media items that have already been created.
Available options:
- `was_forced`: Whether the indexing was forced by the user
Returns [%MediaItem{} | %Ecto.Changeset{}]
"""
def index_and_enqueue_download_for_media_items(%Source{} = source) do
def index_and_enqueue_download_for_media_items(%Source{} = source, opts \\ []) do
# The media_profile is needed to determine the quality options to _then_ determine a more
# accurate predicted filepath
source = Repo.preload(source, [:media_profile])
# See the method definition below for more info on how file watchers work
# (important reading if you're not familiar with it)
{:ok, media_attributes} = setup_file_watcher_and_kickoff_indexing(source)
{:ok, media_attributes} = setup_file_watcher_and_kickoff_indexing(source, opts)
# Reload because the source may have been updated during the (long-running) indexing process
# and important settings like `download_media` may have changed.
source = Repo.reload!(source)
@@ -90,12 +128,19 @@ defmodule Pinchflat.SlowIndexing.SlowIndexingHelpers do
# It attempts a graceful shutdown of the file follower after the indexing is done,
# but the FileFollowerServer will also stop itself if it doesn't see any activity
# for a sufficiently long time.
defp setup_file_watcher_and_kickoff_indexing(source) do
defp setup_file_watcher_and_kickoff_indexing(source, opts) do
was_forced = Keyword.get(opts, :was_forced, false)
{:ok, pid} = FileFollowerServer.start_link()
handler = fn filepath -> setup_file_follower_watcher(pid, filepath, source) end
command_opts =
[output: DownloadOptionBuilder.build_output_path_for(source)] ++
DownloadOptionBuilder.build_quality_options_for(source) ++
build_download_archive_options(source, was_forced)
runner_opts = [file_listener_handler: handler, use_cookies: source.use_cookies]
result = MediaCollection.get_media_attributes_for_collection(source.original_url, runner_opts)
result = MediaCollection.get_media_attributes_for_collection(source.original_url, command_opts, runner_opts)
FileFollowerServer.stop(pid)
@@ -132,4 +177,67 @@ defmodule Pinchflat.SlowIndexing.SlowIndexingHelpers do
changeset
end
end
# Find the difference between the current time and the last time the source was indexed
defp calculate_job_offset_seconds(%Source{last_indexed_at: nil}), do: 0
defp calculate_job_offset_seconds(source) do
offset_seconds = DateTime.diff(DateTime.utc_now(), source.last_indexed_at, :second)
index_frequency_seconds = source.index_frequency_minutes * 60
max(0, index_frequency_seconds - offset_seconds)
end
# The download archive file works in tandem with --break-on-existing to stop
# yt-dlp once we've hit media items we've already indexed. But we generate
# this list with a bit of an offset so we do intentionally re-scan some media
# items to pick up any recent changes (see `get_media_items_for_download_archive`).
#
# From there, we format the media IDs in the way that yt-dlp expects (ie: "<extractor> <media_id>")
# and return the filepath to the caller.
defp create_download_archive_file(source) do
tmpfile = FilesystemUtils.generate_metadata_tmpfile(:txt)
archive_contents =
source
|> get_media_items_for_download_archive()
|> Enum.map_join("\n", fn media_item -> "youtube #{media_item.media_id}" end)
case File.write(tmpfile, archive_contents) do
:ok -> tmpfile
err -> err
end
end
# Sorting by `uploaded_at` is important because we want to re-index the most recent
# media items first but there is no guarantee of any correlation between ID and uploaded_at.
#
# The offset is important because we want to re-index some media items that we've
# recently indexed to pick up on any changes. The limit is because we want this mechanism
# to work even if, for example, the video we were using as a stopping point was deleted.
# It's not a perfect system, but it should do well enough.
#
# The chosen limit and offset are arbitary, independent, and vibes-based. Feel free to
# tweak as-needed
defp get_media_items_for_download_archive(source) do
MediaQuery.new()
|> where(^MediaQuery.for_source(source))
|> order_by(desc: :uploaded_at)
|> limit(50)
|> offset(20)
|> Repo.all()
end
# The download archive isn't useful for playlists (since those are ordered arbitrarily)
# and we don't want to use it if the indexing was forced by the user. In other words,
# only create an archive for channels that are being indexed as part of their regular
# indexing schedule
defp build_download_archive_options(%Source{collection_type: :playlist}, _was_forced), do: []
defp build_download_archive_options(_source, true), do: []
defp build_download_archive_options(source, _was_forced) do
archive_file = create_download_archive_file(source)
[:break_on_existing, download_archive: archive_file]
end
end
+2
View File
@@ -15,6 +15,7 @@ defmodule Pinchflat.Sources.Source do
alias Pinchflat.Metadata.SourceMetadata
@allowed_fields ~w(
enabled
collection_name
collection_id
collection_type
@@ -64,6 +65,7 @@ defmodule Pinchflat.Sources.Source do
)a
schema "sources" do
field :enabled, :boolean, default: true
# This is _not_ used as the primary key or internally in the database
# relations. This is only used to prevent an enumeration attack on the streaming
# and RSS feed endpoints since those _must_ be public (ie: no basic auth)
+70 -19
View File
@@ -15,8 +15,8 @@ defmodule Pinchflat.Sources do
alias Pinchflat.Metadata.SourceMetadata
alias Pinchflat.Utils.FilesystemUtils
alias Pinchflat.Downloading.DownloadingHelpers
alias Pinchflat.FastIndexing.FastIndexingWorker
alias Pinchflat.SlowIndexing.SlowIndexingHelpers
alias Pinchflat.FastIndexing.FastIndexingHelpers
alias Pinchflat.Metadata.SourceMetadataStorageWorker
@doc """
@@ -180,13 +180,22 @@ defmodule Pinchflat.Sources do
end
defp add_source_details_to_changeset(source, changeset) do
original_url = changeset.changes.original_url
use_cookies = Ecto.Changeset.get_field(changeset, :use_cookies)
# Skipping sleep interval since this is UI blocking and we want to keep this as fast as possible
addl_opts = [use_cookies: use_cookies, skip_sleep_interval: true]
case MediaCollection.get_source_details(changeset.changes.original_url, [], use_cookies: use_cookies) do
case MediaCollection.get_source_details(original_url, [], addl_opts) do
{:ok, source_details} ->
add_source_details_by_collection_type(source, changeset, source_details)
{:error, runner_error, _status_code} ->
err ->
runner_error =
case err do
{:error, error_msg, _status_code} -> error_msg
{:error, error_msg} -> error_msg
end
Ecto.Changeset.add_error(
changeset,
:original_url,
@@ -249,19 +258,40 @@ defmodule Pinchflat.Sources do
end
end
# If the source is NOT new (ie: updated) and the download_media flag has changed,
# If the source is new (ie: not persisted), do nothing
defp maybe_handle_media_tasks(%{data: %{__meta__: %{state: state}}}, _source) when state != :loaded do
:ok
end
# If the source is NOT new (ie: updated),
# enqueue or dequeue media download tasks as necessary.
defp maybe_handle_media_tasks(changeset, source) do
case {changeset.data, changeset.changes} do
{%{__meta__: %{state: :loaded}}, %{download_media: true}} ->
current_changes = changeset.changes
applied_changes = Ecto.Changeset.apply_changes(changeset)
# We need both current_changes and applied_changes to determine
# the course of action to take. For example, we only care if a source is supposed
# to be `enabled` or not - we don't care if that information comes from the
# current changes or if that's how it already was in the database.
# Rephrased, we're essentially using it in place of `get_field/2`
case {current_changes, applied_changes} do
{%{download_media: true}, %{enabled: true}} ->
DownloadingHelpers.enqueue_pending_download_tasks(source)
{%{__meta__: %{state: :loaded}}, %{download_media: false}} ->
{%{enabled: true}, %{download_media: true}} ->
DownloadingHelpers.enqueue_pending_download_tasks(source)
{%{download_media: false}, _} ->
DownloadingHelpers.dequeue_pending_download_tasks(source)
{%{enabled: false}, _} ->
DownloadingHelpers.dequeue_pending_download_tasks(source)
_ ->
:ok
nil
end
:ok
end
defp maybe_run_indexing_task(changeset, source) do
@@ -295,13 +325,22 @@ defmodule Pinchflat.Sources do
end
defp maybe_update_slow_indexing_task(changeset, source) do
case changeset.changes do
%{index_frequency_minutes: mins} when mins > 0 ->
# See comment in `maybe_handle_media_tasks` as to why we need these
current_changes = changeset.changes
applied_changes = Ecto.Changeset.apply_changes(changeset)
case {current_changes, applied_changes} do
{%{index_frequency_minutes: mins}, %{enabled: true}} when mins > 0 ->
SlowIndexingHelpers.kickoff_indexing_task(source)
%{index_frequency_minutes: _} ->
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker")
Tasks.delete_pending_tasks_for(source, "MediaCollectionIndexingWorker")
{%{enabled: true}, %{index_frequency_minutes: mins}} when mins > 0 ->
SlowIndexingHelpers.kickoff_indexing_task(source)
{%{index_frequency_minutes: _}, _} ->
SlowIndexingHelpers.delete_indexing_tasks(source, include_executing: true)
{%{enabled: false}, _} ->
SlowIndexingHelpers.delete_indexing_tasks(source, include_executing: true)
_ ->
:ok
@@ -309,13 +348,25 @@ defmodule Pinchflat.Sources do
end
defp maybe_update_fast_indexing_task(changeset, source) do
case changeset.changes do
%{fast_index: true} ->
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker")
FastIndexingWorker.kickoff_with_task(source)
# See comment in `maybe_handle_media_tasks` as to why we need these
current_changes = changeset.changes
applied_changes = Ecto.Changeset.apply_changes(changeset)
%{fast_index: false} ->
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker")
# This technically could be simplified since `maybe_update_slow_indexing_task`
# has some overlap re: deleting pending tasks, but I'm keeping it separate
# for clarity and explicitness.
case {current_changes, applied_changes} do
{%{fast_index: true}, %{enabled: true}} ->
FastIndexingHelpers.kickoff_indexing_task(source)
{%{enabled: true}, %{fast_index: true}} ->
FastIndexingHelpers.kickoff_indexing_task(source)
{%{fast_index: false}, _} ->
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker", include_executing: true)
{%{enabled: false}, _} ->
Tasks.delete_pending_tasks_for(source, "FastIndexingWorker", include_executing: true)
_ ->
:ok
+13 -1
View File
@@ -45,8 +45,20 @@ defmodule Pinchflat.Utils.FilesystemUtils do
Returns binary()
"""
def generate_metadata_tmpfile(type) do
filename = StringUtils.random_string(64)
# This "namespacing" is more to help with development since things get
# weird in my editor when there are thousands of files in a single directory
first_two = String.slice(filename, 0..1)
second_two = String.slice(filename, 2..3)
tmpfile_directory = Application.get_env(:pinchflat, :tmpfile_directory)
filepath = Path.join([tmpfile_directory, "#{StringUtils.random_string(64)}.#{type}"])
filepath =
Path.join([
tmpfile_directory,
first_two,
second_two,
"#{filename}.#{type}"
])
:ok = write_p!(filepath, "")
+14
View File
@@ -36,4 +36,18 @@ defmodule Pinchflat.Utils.NumberUtils do
end
end)
end
@doc """
Adds jitter to a number based on a percentage. Returns 0 if the number is less than or equal to 0.
Returns integer()
"""
def add_jitter(num, jitter_percentage \\ 0.5)
def add_jitter(num, _jitter_percentage) when num <= 0, do: 0
def add_jitter(num, jitter_percentage) do
jitter = :rand.uniform(round(num * jitter_percentage))
round(num + jitter)
end
end
+5 -11
View File
@@ -26,19 +26,13 @@ defmodule Pinchflat.Utils.StringUtils do
end
@doc """
Truncates a string to the given length and adds `...` if the string is longer than the given length.
Will break on a word boundary. Nothing happens if the string is shorter than the given length.
Wraps a string in double braces. Useful as a UI helper now that
LiveView 1.0.0 allows `{}` for interpolation so now we can't use braces
directly in the view.
Returns binary()
"""
def truncate(string, length) do
if String.length(string) > length do
string
|> String.slice(0..(length - 1))
|> String.replace(~r/\s+\S*$/, "")
|> Kernel.<>("...")
else
string
end
def double_brace(string) do
"{{ #{string} }}"
end
end
+31 -6
View File
@@ -3,7 +3,11 @@ defmodule Pinchflat.YtDlp.CommandRunner do
Runs yt-dlp commands using the `System.cmd/3` function
"""
require Logger
alias Pinchflat.Settings
alias Pinchflat.Utils.CliUtils
alias Pinchflat.Utils.NumberUtils
alias Pinchflat.YtDlp.YtDlpCommandRunner
alias Pinchflat.Utils.FilesystemUtils, as: FSUtils
@@ -20,23 +24,30 @@ defmodule Pinchflat.YtDlp.CommandRunner do
for a file watcher.
- :use_cookies - if true, will add a cookie file to the command options. Will not
attach a cookie file if the user hasn't set one up.
- :skip_sleep_interval - if true, will not add the sleep interval options to the command.
Usually only used for commands that would be UI-blocking
Returns {:ok, binary()} | {:error, output, status}.
"""
@impl YtDlpCommandRunner
def run(url, command_opts, output_template, addl_opts \\ []) do
# This approach lets us mock the command for testing
command = backend_executable()
def run(url, action_name, command_opts, output_template, addl_opts \\ []) do
Logger.debug("Running yt-dlp command for action: #{action_name}")
output_filepath = generate_output_filepath(addl_opts)
print_to_file_opts = [{:print_to_file, output_template}, output_filepath]
user_configured_opts = cookie_file_options(addl_opts)
user_configured_opts = cookie_file_options(addl_opts) ++ sleep_interval_opts(addl_opts)
# These must stay in exactly this order, hence why I'm giving it its own variable.
all_opts = command_opts ++ print_to_file_opts ++ user_configured_opts ++ global_options()
formatted_command_opts = [url] ++ CliUtils.parse_options(all_opts)
case CliUtils.wrap_cmd(command, formatted_command_opts, stderr_to_stdout: true) do
{_, 0} ->
case CliUtils.wrap_cmd(backend_executable(), formatted_command_opts, stderr_to_stdout: true) do
# yt-dlp exit codes:
# 0 = Everything is successful
# 100 = yt-dlp must restart for update to complete
# 101 = Download cancelled by --max-downloads etc
# 2 = Error in user-provided options
# 1 = Any other error
{_, status} when status in [0, 101] ->
# IDEA: consider deleting the file after reading it. It's in the tmp dir, so it's not
# a huge deal, but it's still a good idea to clean up after ourselves.
# (even on error? especially on error?)
@@ -87,6 +98,20 @@ defmodule Pinchflat.YtDlp.CommandRunner do
end
end
defp sleep_interval_opts(addl_opts) do
sleep_interval = Settings.get!(:extractor_sleep_interval_seconds)
if sleep_interval <= 0 || Keyword.get(addl_opts, :skip_sleep_interval) do
[]
else
[
sleep_requests: NumberUtils.add_jitter(sleep_interval),
sleep_interval: NumberUtils.add_jitter(sleep_interval),
sleep_subtitles: NumberUtils.add_jitter(sleep_interval)
]
end
end
defp add_cookie_file do
base_dir = Application.get_env(:pinchflat, :extras_directory)
filename_options_map = %{cookies: "cookies.txt"}
+50 -13
View File
@@ -11,7 +11,8 @@ defmodule Pinchflat.YtDlp.Media do
:livestream,
:short_form_content,
:uploaded_at,
:duration_seconds
:duration_seconds,
:predicted_media_filepath
]
defstruct [
@@ -23,7 +24,8 @@ defmodule Pinchflat.YtDlp.Media do
:short_form_content,
:uploaded_at,
:duration_seconds,
:playlist_index
:playlist_index,
:predicted_media_filepath
]
alias __MODULE__
@@ -39,7 +41,7 @@ defmodule Pinchflat.YtDlp.Media do
def download(url, command_opts \\ [], addl_opts \\ []) do
all_command_opts = [:no_simulate] ++ command_opts
with {:ok, output} <- backend_runner().run(url, all_command_opts, "after_move:%()j", addl_opts),
with {:ok, output} <- backend_runner().run(url, :download, all_command_opts, "after_move:%()j", addl_opts),
{:ok, parsed_json} <- Phoenix.json_library().decode(output) do
{:ok, parsed_json}
else
@@ -47,6 +49,27 @@ defmodule Pinchflat.YtDlp.Media do
end
end
@doc """
Determines if the media at the given URL is ready to be downloaded.
Common examples of non-downloadable media are upcoming or in-progress live streams.
Returns {:ok, :downloadable | :ignorable} | {:error, any}
"""
def get_downloadable_status(url, addl_opts \\ []) do
action = :get_downloadable_status
command_opts = [:simulate, :skip_download]
case backend_runner().run(url, action, command_opts, "%(.{live_status})j", addl_opts) do
{:ok, output} ->
output
|> Phoenix.json_library().decode!()
|> parse_downloadable_status()
err ->
err
end
end
@doc """
Downloads a thumbnail for a single piece of media. Usually used for
downloading thumbnails for internal use
@@ -58,20 +81,21 @@ defmodule Pinchflat.YtDlp.Media do
# NOTE: it doesn't seem like this command actually returns anything in `after_move` since
# we aren't downloading the main media file
backend_runner().run(url, all_command_opts, "after_move:%()j", addl_opts)
backend_runner().run(url, :download_thumbnail, all_command_opts, "after_move:%()j", addl_opts)
end
@doc """
Returns a map representing the media at the given URL.
Optionally takes a list of additional command options to pass to yt-dlp
or configuration-related options to pass to the runner.
Returns {:ok, %Media{}} | {:error, any, ...}.
"""
def get_media_attributes(url, addl_opts \\ []) do
runner = Application.get_env(:pinchflat, :yt_dlp_runner)
command_opts = [:simulate, :skip_download]
def get_media_attributes(url, command_opts \\ [], addl_opts \\ []) do
all_command_opts = [:simulate, :skip_download] ++ command_opts
output_template = indexing_output_template()
case runner.run(url, command_opts, output_template, addl_opts) do
case backend_runner().run(url, :get_media_attributes, all_command_opts, output_template, addl_opts) do
{:ok, output} ->
output
|> Phoenix.json_library().decode!()
@@ -87,9 +111,11 @@ defmodule Pinchflat.YtDlp.Media do
Returns the output template for yt-dlp's indexing command.
NOTE: playlist_index is really only useful for playlists that will never change their order.
NOTE: I've switched back to `original_url` (from `webpage_url`) since it's started indicating
if something is a short via the URL again
"""
def indexing_output_template do
"%(.{id,title,live_status,webpage_url,description,aspect_ratio,duration,upload_date,timestamp,playlist_index})j"
"%(.{id,title,live_status,original_url,description,aspect_ratio,duration,upload_date,timestamp,playlist_index,filename})j"
end
@doc """
@@ -103,17 +129,18 @@ defmodule Pinchflat.YtDlp.Media do
media_id: response["id"],
title: response["title"],
description: response["description"],
original_url: response["webpage_url"],
original_url: response["original_url"],
livestream: !!response["live_status"] && response["live_status"] != "not_live",
duration_seconds: response["duration"] && round(response["duration"]),
short_form_content: response["webpage_url"] && short_form_content?(response),
short_form_content: response["original_url"] && short_form_content?(response),
uploaded_at: response["upload_date"] && parse_uploaded_at(response),
playlist_index: response["playlist_index"] || 0
playlist_index: response["playlist_index"] || 0,
predicted_media_filepath: response["filename"]
}
end
defp short_form_content?(response) do
if String.contains?(response["webpage_url"], "/shorts/") do
if String.contains?(response["original_url"], "/shorts/") do
true
else
# Sometimes shorts are returned without /shorts/ in the URL,
@@ -140,6 +167,16 @@ defmodule Pinchflat.YtDlp.Media do
defp parse_uploaded_at(%{"upload_date" => nil}), do: nil
defp parse_uploaded_at(response), do: MetadataFileHelpers.parse_upload_date(response["upload_date"])
defp parse_downloadable_status(response) do
case response["live_status"] do
status when status in ["is_live", "is_upcoming", "post_live"] -> {:ok, :ignorable}
status when status in ["was_live", "not_live"] -> {:ok, :downloadable}
# This preserves my tenuous support for non-youtube sources.
nil -> {:ok, :downloadable}
_ -> {:error, "Unknown live status: #{response["live_status"]}"}
end
end
defp backend_runner do
# This approach lets us mock the command for testing
Application.get_env(:pinchflat, :yt_dlp_runner)
+13 -7
View File
@@ -11,31 +11,34 @@ defmodule Pinchflat.YtDlp.MediaCollection do
@doc """
Returns a list of maps representing the media in the collection.
Optionally takes a list of additional command options to pass to yt-dlp
or configuration-related options to pass to the runner.
Options:
Runner Options:
- :file_listener_handler - a function that will be called with the path to the
file that will be written to when yt-dlp is done. This is useful for
setting up a file watcher to know when the file is ready to be read.
- :use_cookies - whether or not to use user-provided cookies when fetching the media details
Returns {:ok, [map()]} | {:error, any, ...}.
"""
def get_media_attributes_for_collection(url, addl_opts \\ []) do
runner = Application.get_env(:pinchflat, :yt_dlp_runner)
def get_media_attributes_for_collection(url, command_opts \\ [], addl_opts \\ []) do
# `ignore_no_formats_error` is necessary because yt-dlp will error out if
# the first video has not released yet (ie: is a premier). We don't care about
# available formats since we're just getting the media details
command_opts = [:simulate, :skip_download, :ignore_no_formats_error, :no_warnings]
all_command_opts = [:simulate, :skip_download, :ignore_no_formats_error, :no_warnings] ++ command_opts
use_cookies = Keyword.get(addl_opts, :use_cookies, false)
output_template = YtDlpMedia.indexing_output_template()
output_filepath = FilesystemUtils.generate_metadata_tmpfile(:json)
file_listener_handler = Keyword.get(addl_opts, :file_listener_handler, false)
runner_opts = [output_filepath: output_filepath, use_cookies: use_cookies]
action = :get_media_attributes_for_collection
if file_listener_handler do
file_listener_handler.(output_filepath)
end
case runner.run(url, command_opts, output_template, runner_opts) do
case backend_runner().run(url, action, all_command_opts, output_template, runner_opts) do
{:ok, output} ->
parsed_lines =
output
@@ -79,11 +82,13 @@ defmodule Pinchflat.YtDlp.MediaCollection do
all_command_opts = default_opts ++ command_opts
output_template = "%(.{channel,channel_id,playlist_id,playlist_title,filename})j"
action = :get_source_details
with {:ok, output} <- backend_runner().run(source_url, all_command_opts, output_template, addl_opts),
with {:ok, output} <- backend_runner().run(source_url, action, all_command_opts, output_template, addl_opts),
{:ok, parsed_json} <- Phoenix.json_library().decode(output) do
{:ok, format_source_details(parsed_json)}
else
{:error, %Jason.DecodeError{}} -> {:error, "Error decoding JSON response"}
err -> err
end
end
@@ -117,8 +122,9 @@ defmodule Pinchflat.YtDlp.MediaCollection do
all_command_opts = [:skip_download] ++ command_opts
output_template = "playlist:%()j"
action = :get_source_metadata
with {:ok, output} <- backend_runner().run(source_url, all_command_opts, output_template, addl_opts),
with {:ok, output} <- backend_runner().run(source_url, action, all_command_opts, output_template, addl_opts),
{:ok, parsed_json} <- Phoenix.json_library().decode(output) do
{:ok, parsed_json}
else
@@ -6,7 +6,7 @@ defmodule Pinchflat.YtDlp.YtDlpCommandRunner do
yt-dlp command.
"""
@callback run(binary(), keyword(), binary()) :: {:ok, binary()} | {:error, binary(), integer()}
@callback run(binary(), keyword(), binary(), keyword()) :: {:ok, binary()} | {:error, binary(), integer()}
@callback run(binary(), atom(), keyword(), binary()) :: {:ok, binary()} | {:error, binary(), integer()}
@callback run(binary(), atom(), keyword(), binary(), keyword()) :: {:ok, binary()} | {:error, binary(), integer()}
@callback version() :: {:ok, binary()} | {:error, binary()}
end
+1
View File
@@ -100,6 +100,7 @@ defmodule PinchflatWeb do
import PinchflatWeb.CustomComponents.TextComponents
import PinchflatWeb.CustomComponents.TableComponents
import PinchflatWeb.CustomComponents.ButtonComponents
import Pinchflat.Utils.StringUtils, only: [double_brace: 1]
alias Pinchflat.Settings
alias Pinchflat.Utils.StringUtils
+52 -51
View File
@@ -82,7 +82,7 @@ defmodule PinchflatWeb.CoreComponents do
</button>
</div>
<div id={"#{@id}-content"}>
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</div>
</div>
</div>
@@ -126,9 +126,9 @@ defmodule PinchflatWeb.CoreComponents do
]}>
<main>
<h5 :if={@title} class="mb-2 text-lg font-bold">
<%= @title %>
{@title}
</h5>
<p class="mt-2 text-md leading-5 opacity-80"><%= msg %></p>
<p class="mt-2 text-md leading-5 opacity-80">{msg}</p>
</main>
<button
type="button"
@@ -208,9 +208,9 @@ defmodule PinchflatWeb.CoreComponents do
def simple_form(assigns) do
~H"""
<.form :let={f} for={@for} as={@as} {@rest}>
<%= render_slot(@inner_block, f) %>
{render_slot(@inner_block, f)}
<div :for={action <- @actions} class="mt-2 flex items-center justify-between gap-6">
<%= render_slot(action, f) %>
{render_slot(action, f)}
</div>
</.form>
"""
@@ -297,11 +297,11 @@ defmodule PinchflatWeb.CoreComponents do
class={["rounded focus:ring-0", @inputclass]}
{@rest}
/>
<%= @label %>
<span :if={@label_suffix} class="text-xs text-bodydark"><%= @label_suffix %></span>
{@label}
<span :if={@label_suffix} class="text-xs text-bodydark">{@label_suffix}</span>
</label>
<.help :if={@help}><%= if @html_help, do: Phoenix.HTML.raw(@help), else: @help %></.help>
<.error :for={msg <- @errors}><%= msg %></.error>
<.help :if={@help}>{if @html_help, do: Phoenix.HTML.raw(@help), else: @help}</.help>
<.error :for={msg <- @errors}>{msg}</.error>
</div>
"""
end
@@ -310,7 +310,7 @@ defmodule PinchflatWeb.CoreComponents do
~H"""
<div phx-feedback-for={@name}>
<.label for={@id}>
<%= @label %><span :if={@label_suffix} class="text-xs text-bodydark"><%= @label_suffix %></span>
{@label}<span :if={@label_suffix} class="text-xs text-bodydark">{@label_suffix}</span>
</.label>
<section class="grid grid-cols-1 gap-2 md:grid-cols-2 max-w-prose mb-4 ml-1">
<div :for={{option_name, option_value} <- @options} class="flex items-center">
@@ -323,12 +323,12 @@ defmodule PinchflatWeb.CoreComponents do
class={["rounded focus:ring-offset-0 ring-offset-0 focus:ring-0 h-5 w-5 ", @inputclass]}
/>
<label for={"#{@id}-#{option_value}"} class="ml-2 cursor-pointer select-none">
<%= option_name %>
{option_name}
</label>
</div>
</section>
<.help :if={@help}><%= if @html_help, do: Phoenix.HTML.raw(@help), else: @help %></.help>
<.error :for={msg <- @errors}><%= msg %></.error>
<.help :if={@help}>{if @html_help, do: Phoenix.HTML.raw(@help), else: @help}</.help>
<.error :for={msg <- @errors}>{msg}</.error>
</div>
"""
end
@@ -340,14 +340,15 @@ defmodule PinchflatWeb.CoreComponents do
end)
~H"""
<div x-data={"{ enabled: #{@checked}}"}>
<.label for={@id}>
<%= @label %>
<span :if={@label_suffix} class="text-xs text-bodydark"><%= @label_suffix %></span>
<div x-data={"{ enabled: #{@checked} }"} class="" phx-update="ignore" id={"#{@id}-wrapper"}>
<.label :if={@label} for={@id}>
{@label}
<span :if={@label_suffix} class="text-xs text-bodydark">{@label_suffix}</span>
</.label>
<div class="relative">
<div class="relative flex flex-col">
<input type="hidden" id={@id} name={@name} x-bind:value="enabled" {@rest} />
<div class="inline-block cursor-pointer" @click="enabled = !enabled">
<%!-- This triggers a `change` event on the hidden input when the toggle is clicked --%>
<div class="inline-block cursor-pointer" @click={"enabled = !enabled; dispatchFor('#{@id}', 'change')"}>
<div x-bind:class="enabled && '!bg-primary'" class="block h-8 w-14 rounded-full bg-black"></div>
<div
x-bind:class="enabled && '!right-1 !translate-x-full'"
@@ -358,8 +359,8 @@ defmodule PinchflatWeb.CoreComponents do
>
</div>
</div>
<.help :if={@help}><%= if @html_help, do: Phoenix.HTML.raw(@help), else: @help %></.help>
<.error :for={msg <- @errors}><%= msg %></.error>
<.help :if={@help}>{if @html_help, do: Phoenix.HTML.raw(@help), else: @help}</.help>
<.error :for={msg <- @errors}>{msg}</.error>
</div>
</div>
"""
@@ -369,7 +370,7 @@ defmodule PinchflatWeb.CoreComponents do
~H"""
<div phx-feedback-for={@name}>
<.label :if={@label} for={@id}>
<%= @label %><span :if={@label_suffix} class="text-xs text-bodydark"><%= @label_suffix %></span>
{@label}<span :if={@label_suffix} class="text-xs text-bodydark">{@label_suffix}</span>
</.label>
<div class="flex">
<select
@@ -384,13 +385,13 @@ defmodule PinchflatWeb.CoreComponents do
multiple={@multiple}
{@rest}
>
<option :if={@prompt} value=""><%= @prompt %></option>
<%= Phoenix.HTML.Form.options_for_select(@options, @value) %>
<option :if={@prompt} value="">{@prompt}</option>
{Phoenix.HTML.Form.options_for_select(@options, @value)}
</select>
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</div>
<.help :if={@help}><%= if @html_help, do: Phoenix.HTML.raw(@help), else: @help %></.help>
<.error :for={msg <- @errors}><%= msg %></.error>
<.help :if={@help}>{if @html_help, do: Phoenix.HTML.raw(@help), else: @help}</.help>
<.error :for={msg <- @errors}>{msg}</.error>
</div>
"""
end
@@ -399,7 +400,7 @@ defmodule PinchflatWeb.CoreComponents do
~H"""
<div phx-feedback-for={@name}>
<.label for={@id}>
<%= @label %><span :if={@label_suffix} class="text-xs text-bodydark"><%= @label_suffix %></span>
{@label}<span :if={@label_suffix} class="text-xs text-bodydark">{@label_suffix}</span>
</.label>
<textarea
id={@id}
@@ -413,8 +414,8 @@ defmodule PinchflatWeb.CoreComponents do
]}
{@rest}
><%= Phoenix.HTML.Form.normalize_value("textarea", @value) %></textarea>
<.help :if={@help}><%= if @html_help, do: Phoenix.HTML.raw(@help), else: @help %></.help>
<.error :for={msg <- @errors}><%= msg %></.error>
<.help :if={@help}>{if @html_help, do: Phoenix.HTML.raw(@help), else: @help}</.help>
<.error :for={msg <- @errors}>{msg}</.error>
</div>
"""
end
@@ -424,7 +425,7 @@ defmodule PinchflatWeb.CoreComponents do
~H"""
<div phx-feedback-for={@name}>
<.label for={@id}>
<%= @label %><span :if={@label_suffix} class="text-xs text-bodydark"><%= @label_suffix %></span>
{@label}<span :if={@label_suffix} class="text-xs text-bodydark">{@label_suffix}</span>
</.label>
<div class="flex items-center">
<input
@@ -441,10 +442,10 @@ defmodule PinchflatWeb.CoreComponents do
]}
{@rest}
/>
<%= render_slot(@input_append) %>
{render_slot(@input_append)}
</div>
<.help :if={@help}><%= if @html_help, do: Phoenix.HTML.raw(@help), else: @help %></.help>
<.error :for={msg <- @errors}><%= msg %></.error>
<.help :if={@help}>{if @html_help, do: Phoenix.HTML.raw(@help), else: @help}</.help>
<.error :for={msg <- @errors}>{msg}</.error>
</div>
"""
end
@@ -457,7 +458,7 @@ defmodule PinchflatWeb.CoreComponents do
def help(assigns) do
~H"""
<p class="mt-1 text-sm leading-5">
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</p>
"""
end
@@ -471,7 +472,7 @@ defmodule PinchflatWeb.CoreComponents do
def label(assigns) do
~H"""
<label for={@for} class="mt-5 mb-2 inline-block text-md font-medium text-black dark:text-white">
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</label>
"""
end
@@ -485,7 +486,7 @@ defmodule PinchflatWeb.CoreComponents do
~H"""
<p class="mt-1 mb-5 flex gap-3 text-md leading-6 text-rose-600 phx-no-feedback:hidden">
<.icon name="hero-exclamation-circle-mini" class="mt-0.5 h-5 w-5 flex-none" />
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</p>
"""
end
@@ -504,13 +505,13 @@ defmodule PinchflatWeb.CoreComponents do
<header class={[@actions != [] && "flex items-center justify-between gap-6", @class]}>
<div>
<h1 class="text-lg font-semibold leading-8 text-zinc-800">
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</h1>
<p :if={@subtitle != []} class="mt-2 text-sm leading-6 text-zinc-600">
<%= render_slot(@subtitle) %>
{render_slot(@subtitle)}
</p>
</div>
<div class="flex-none"><%= render_slot(@actions) %></div>
<div class="flex-none">{render_slot(@actions)}</div>
</header>
"""
end
@@ -550,9 +551,9 @@ defmodule PinchflatWeb.CoreComponents do
<table class="w-[40rem] mt-11 sm:w-full">
<thead class="text-sm text-left leading-6 text-zinc-500">
<tr>
<th :for={col <- @col} class="p-0 pb-4 pr-6 font-normal"><%= col[:label] %></th>
<th :for={col <- @col} class="p-0 pb-4 pr-6 font-normal">{col[:label]}</th>
<th :if={@action != []} class="relative p-0 pb-4">
<span class="sr-only"><%= gettext("Actions") %></span>
<span class="sr-only">{gettext("Actions")}</span>
</th>
</tr>
</thead>
@@ -570,7 +571,7 @@ defmodule PinchflatWeb.CoreComponents do
<div class="block py-4 pr-6">
<span class="absolute -inset-y-px right-0 -left-4 group-hover:bg-zinc-50 sm:rounded-l-xl" />
<span class={["relative", i == 0 && "font-semibold text-zinc-900"]}>
<%= render_slot(col, @row_item.(row)) %>
{render_slot(col, @row_item.(row))}
</span>
</div>
</td>
@@ -578,7 +579,7 @@ defmodule PinchflatWeb.CoreComponents do
<div class="relative whitespace-nowrap py-4 text-right text-sm font-medium">
<span class="absolute -inset-y-px -right-4 left-0 group-hover:bg-zinc-50 sm:rounded-r-xl" />
<span :for={action <- @action} class="relative ml-4 font-semibold leading-6 text-zinc-900 hover:text-zinc-700">
<%= render_slot(action, @row_item.(row)) %>
{render_slot(action, @row_item.(row))}
</span>
</div>
</td>
@@ -607,8 +608,8 @@ defmodule PinchflatWeb.CoreComponents do
<div class="mt-2 mb-14">
<dl class="-my-4 divide-y dark:divide-strokedark">
<div :for={item <- @item} class="flex gap-4 py-4 text-sm leading-6 sm:gap-8">
<dt class="w-1/4 flex-none dark:text-white"><%= item.title %></dt>
<dd class="dark:text-white"><%= render_slot(item) %></dd>
<dt class="w-1/4 flex-none dark:text-white">{item.title}</dt>
<dd class="dark:text-white">{render_slot(item)}</dd>
</div>
</dl>
</div>
@@ -643,12 +644,12 @@ defmodule PinchflatWeb.CoreComponents do
~H"""
<ul>
<li :for={{k, v} <- @iterable_attributes} class="mb-2 w-2/3">
<strong><%= k %>:</strong>
<strong>{k}:</strong>
<code class="inline-block text-sm font-mono text-gray p-0.5 mx-0.5">
<%= if is_binary(v) && URI.parse(v).scheme && URI.parse(v).scheme =~ "http" do %>
<TextComponents.inline_link href={v}><%= v %></TextComponents.inline_link>
<TextComponents.inline_link href={v}>{v}</TextComponents.inline_link>
<% else %>
<%= v %>
{v}
<% end %>
</code>
</li>
@@ -671,7 +672,7 @@ defmodule PinchflatWeb.CoreComponents do
<div class="mt-16">
<.link href={@href} class="text-sm font-semibold leading-6 text-zinc-900 hover:text-zinc-700">
<.icon name="hero-arrow-left-solid" class="h-3 w-3" />
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</.link>
</div>
"""
@@ -699,7 +700,7 @@ defmodule PinchflatWeb.CoreComponents do
attr :class, :string, default: nil
attr :rest, :global
def icon(%{name: "hero-" <> _} = assigns) do
def icon(assigns) do
~H"""
<span class={[@name, @class]} {@rest} />
"""
@@ -26,7 +26,7 @@ defmodule PinchflatWeb.CustomComponents.ButtonComponents do
~H"""
<button
class={[
"text-center font-medium text-white",
"text-center font-medium text-white whitespace-nowrap",
"#{@rounding} inline-flex items-center justify-center px-8 py-4",
"#{@color}",
"hover:bg-opacity-90 lg:px-8 xl:px-10",
@@ -37,7 +37,7 @@ defmodule PinchflatWeb.CustomComponents.ButtonComponents do
disabled={@disabled}
{@rest}
>
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</button>
"""
end
@@ -66,7 +66,7 @@ defmodule PinchflatWeb.CustomComponents.ButtonComponents do
"font-medium text-white hover:bg-opacity-95"
]}
>
<%= @text %>
{@text}
<CoreComponents.icon
name="hero-chevron-down"
class="fill-current duration-200 ease-linear mt-1"
@@ -81,7 +81,7 @@ defmodule PinchflatWeb.CustomComponents.ButtonComponents do
<ul class="flex flex-col">
<li :for={option <- @option}>
<span class="flex px-5 py-2 font-medium text-bodydark2 hover:text-white cursor-pointer">
<%= render_slot(option) %>
{render_slot(option)}
</span>
</li>
</ul>
@@ -126,7 +126,7 @@ defmodule PinchflatWeb.CustomComponents.ButtonComponents do
>
<span class="border-light absolute -top-1 left-1/2 -z-10 h-2 w-2 -translate-x-1/2 rotate-45 rounded-sm bg-meta-4">
</span>
<span><%= @tooltip %></span>
<span>{@tooltip}</span>
</div>
</div>
"""
@@ -34,16 +34,16 @@ defmodule PinchflatWeb.CustomComponents.TabComponents do
x-bind:class={"openTab === '#{tab.id}' ? activeClasses : inactiveClasses"}
class="border-b-2 py-4 w-full sm:w-fit text-sm font-medium hover:text-meta-5 md:text-base"
>
<span class="text-xl"><%= tab.title %></span>
<span class="text-xl">{tab.title}</span>
</a>
</div>
<div class="mx-4 my-4 lg:my-0 flex gap-5 sm:gap-10 items-center">
<%= render_slot(@tab_append) %>
{render_slot(@tab_append)}
</div>
</header>
<div class="mt-4 min-h-60">
<div :for={tab <- @tab} x-show={"openTab === '#{tab.id}'"} class="font-medium leading-relaxed">
<%= render_slot(tab) %>
{render_slot(tab)}
</div>
</div>
</div>
@@ -16,6 +16,8 @@ defmodule PinchflatWeb.CustomComponents.TableComponents do
"""
attr :rows, :list, required: true
attr :table_class, :string, default: ""
attr :sort_key, :string, default: nil
attr :sort_direction, :string, default: nil
attr :row_item, :any,
default: &Function.identity/1,
@@ -24,6 +26,7 @@ defmodule PinchflatWeb.CustomComponents.TableComponents do
slot :col, required: true do
attr :label, :string
attr :class, :string
attr :sort_key, :string
end
def table(assigns) do
@@ -31,8 +34,20 @@ defmodule PinchflatWeb.CustomComponents.TableComponents do
<table class={["w-full table-auto bg-boxdark", @table_class]}>
<thead>
<tr class="text-left bg-meta-4">
<th :for={col <- @col} class="px-4 py-4 font-medium text-white xl:pl-11">
<%= col[:label] %>
<th
:for={col <- @col}
class={["px-4 py-4 font-medium text-white", col[:sort_key] && "cursor-pointer"]}
phx-click={col[:sort_key] && "sort_update"}
phx-value-sort_key={col[:sort_key]}
>
<div class="relative">
{col[:label]}
<.icon
:if={to_string(@sort_key) == col[:sort_key]}
name={if @sort_direction == :asc, do: "hero-chevron-up", else: "hero-chevron-down"}
class="w-3 h-3 mt-2 ml-1 absolute"
/>
</div>
</th>
</tr>
</thead>
@@ -41,11 +56,11 @@ defmodule PinchflatWeb.CustomComponents.TableComponents do
<td
:for={col <- @col}
class={[
"px-4 py-5 pl-9 xl:pl-11",
"px-4 py-5",
col[:class]
]}
>
<%= render_slot(col, @row_item.(row)) %>
{render_slot(col, @row_item.(row))}
</td>
</tr>
</tbody>
@@ -70,9 +85,9 @@ defmodule PinchflatWeb.CustomComponents.TableComponents do
<li>
<span
class={[
"flex h-8 w-8 items-center justify-center rounded",
"pagination-prev h-8 w-8 items-center justify-center rounded",
@page_number != 1 && "cursor-pointer hover:bg-primary hover:text-white",
@page_number == 1 && "cursor-not-allowed"
@page_number <= 1 && "cursor-not-allowed"
]}
phx-click={@page_number != 1 && "page_change"}
phx-value-direction="dec"
@@ -88,9 +103,9 @@ defmodule PinchflatWeb.CustomComponents.TableComponents do
<li>
<span
class={[
"flex h-8 w-8 items-center justify-center rounded",
"pagination-next flex h-8 w-8 items-center justify-center rounded",
@page_number != @total_pages && "cursor-pointer hover:bg-primary hover:text-white",
@page_number == @total_pages && "cursor-not-allowed"
@page_number >= @total_pages && "cursor-not-allowed"
]}
phx-click={@page_number != @total_pages && "page_change"}
phx-value-direction="inc"
@@ -2,6 +2,7 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
@moduledoc false
use Phoenix.Component
alias Pinchflat.Utils.NumberUtils
alias PinchflatWeb.CoreComponents
@doc """
@@ -12,7 +13,7 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
def inline_code(assigns) do
~H"""
<code class="inline-block text-sm font-mono text-gray bg-boxdark rounded-md p-0.5 mx-0.5 text-nowrap">
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</code>
"""
end
@@ -26,7 +27,7 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
def inline_link(assigns) do
~H"""
<.link href={@href} target="_blank" class="text-blue-500 hover:text-blue-300">
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</.link>
"""
end
@@ -41,7 +42,7 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
def subtle_link(assigns) do
~H"""
<.link href={@href} target={@target} class="underline decoration-bodydark decoration-1 hover:decoration-white">
<%= render_slot(@inner_block) %>
{render_slot(@inner_block)}
</.link>
"""
end
@@ -62,20 +63,27 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
end
@doc """
Renders a block of text with each line broken into a separate span.
Renders a block of text with each line broken into a separate span and links highlighted.
"""
attr :text, :string, required: true
def break_on_newline(assigns) do
broken_text =
assigns.text
|> String.split("\n", trim: false)
|> Enum.intersperse(Phoenix.HTML.Tag.tag(:span, class: "inline-block mt-2"))
def render_description(assigns) do
formatted_text =
Regex.split(~r{https?://\S+}, assigns.text, include_captures: true)
|> Enum.map(fn
"http" <> _ = url ->
Phoenix.HTML.Tag.content_tag(:a, url, class: "text-blue-500 hover:text-blue-300", href: url, target: "_blank")
assigns = Map.put(assigns, :text, broken_text)
text ->
text
|> String.split("\n", trim: false)
|> Enum.intersperse(Phoenix.HTML.Tag.tag(:span, class: "inline-block mt-2"))
end)
assigns = Map.put(assigns, :text, formatted_text)
~H"""
<span><%= @text %></span>
<span>{@text}</span>
"""
end
@@ -91,7 +99,7 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
assigns = Map.put(assigns, :timezone, timezone)
~H"""
<time><%= Calendar.strftime(Timex.Timezone.convert(@datetime, @timezone), @format) %></time>
<time>{Calendar.strftime(Timex.Timezone.convert(@datetime, @timezone), @format)}</time>
"""
end
@@ -102,7 +110,7 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
def localized_number(assigns) do
~H"""
<span x-data x-text={"Intl.NumberFormat().format(#{@number})"}><%= @number %></span>
<span x-data x-text={"Intl.NumberFormat().format(#{@number})"}>{@number}</span>
"""
end
@@ -115,7 +123,27 @@ defmodule PinchflatWeb.CustomComponents.TextComponents do
def pluralize(assigns) do
~H"""
<%= @word %><%= if @count == 1, do: "", else: @suffix %>
{@word}{if @count == 1, do: "", else: @suffix}
"""
end
@doc """
Renders a human-readable byte size
"""
attr :byte_size, :integer, required: true
def readable_filesize(assigns) do
{num, suffix} = NumberUtils.human_byte_size(assigns.byte_size, precision: 2)
assigns =
Map.merge(assigns, %{
num: num,
suffix: suffix
})
~H"""
<.localized_number number={@num} /> {@suffix}
"""
end
end
+6 -4
View File
@@ -15,11 +15,12 @@ defmodule PinchflatWeb.Layouts do
attr :text, :string, required: true
attr :href, :any, required: true
attr :target, :any, default: "_self"
attr :icon_class, :string, default: ""
def sidebar_item(assigns) do
~H"""
<li class="text-bodydark1">
<.sidebar_link icon={@icon} text={@text} href={@href} target={@target} />
<.sidebar_link icon={@icon} text={@text} href={@href} target={@target} icon_class={@icon_class} />
</li>
"""
end
@@ -61,10 +62,10 @@ defmodule PinchflatWeb.Layouts do
x-on:click="selected = !selected"
>
<span class="flex items-center gap-2.5">
<.icon name={@icon} /> <%= @text %>
<.icon name={@icon} /> {@text}
</span>
<span class="text-bodydark2">
<.icon name="hero-chevron-up" x-bind:class="{ 'rotate-180': selected }" />
<.icon name="hero-chevron-down" x-bind:class="{ 'rotate-180': selected }" />
</span>
</span>
@@ -89,6 +90,7 @@ defmodule PinchflatWeb.Layouts do
attr :href, :any, required: true
attr :target, :any, default: "_self"
attr :class, :string, default: ""
attr :icon_class, :string, default: ""
def sidebar_link(assigns) do
~H"""
@@ -103,7 +105,7 @@ defmodule PinchflatWeb.Layouts do
@class
]}
>
<.icon :if={@icon} name={@icon} /> <%= @text %>
<.icon :if={@icon} name={@icon} class={@icon_class} /> {@text}
</.link>
"""
end
@@ -6,7 +6,7 @@
<main>
<div class="mx-auto max-w-screen-2xl p-4 md:p-6 2xl:p-10">
<.flash_group flash={@flash} />
<%= @inner_content %>
{@inner_content}
</div>
</main>
</div>
@@ -11,7 +11,7 @@
<main>
<div class="mx-auto max-w-screen-2xl p-4 md:p-6 2xl:p-10">
<.flash_group flash={@flash} />
<%= @inner_content %>
{@inner_content}
</div>
</main>
</div>
@@ -47,8 +47,10 @@
text="Docs"
target="_blank"
href="https://github.com/kieraneglin/pinchflat/wiki"
icon_class="scale-110"
/>
<.sidebar_item icon="hero-cog" text="Github" target="_blank" href="https://github.com/kieraneglin/pinchflat" />
<.sidebar_item icon="si-github" text="Github" target="_blank" href="https://github.com/kieraneglin/pinchflat" />
<.sidebar_item icon="si-discord" text="Discord" target="_blank" href="https://discord.gg/j7T6dCuwU4" />
<li>
<span
class={[
@@ -59,7 +61,7 @@
]}
phx-click={show_modal("donate-modal")}
>
<.icon name="hero-currency-dollar" /> Donate
<.icon name="hero-currency-dollar" class="scale-110" /> Donate
</span>
</li>
<li>
@@ -67,7 +69,7 @@
class="group relative flex items-center gap-2.5 px-4 pt-2 text-sm"
x-on:click={"markVersionAsSeen('#{Application.spec(:pinchflat)[:vsn]}')"}
>
<span>Pinchflat <%= Application.spec(:pinchflat)[:vsn] %></span>
<span>Pinchflat {Application.spec(:pinchflat)[:vsn]}</span>
<a
href="https://github.com/kieraneglin/pinchflat/releases"
target="_blank"
@@ -79,7 +81,7 @@
</a>
</span>
<span class="group relative flex items-center gap-2.5 px-4 pt-2 text-sm">
yt-dlp <%= Settings.get!(:yt_dlp_version) %>
yt-dlp {Settings.get!(:yt_dlp_version)}
</span>
</li>
</ul>
@@ -3,7 +3,7 @@ defmodule Pinchflat.UpgradeButtonLive do
def render(assigns) do
~H"""
<form id="upgradeForm" phx-change="check_matching_text" phx-hook="supressEnterSubmission">
<form id="upgradeForm" phx-change="check_matching_text" phx-hook="supress-enter-submission">
<.input type="text" name="unlock-pro-textbox" value="" />
</form>
@@ -25,6 +25,6 @@
into the text box and press the button.
</p>
<%= live_render(@conn, Pinchflat.UpgradeButtonLive) %>
{live_render(@conn, Pinchflat.UpgradeButtonLive)}
</section>
</.modal>
@@ -5,7 +5,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="csrf-token" content={get_csrf_token()} />
<.live_title>
<%= assigns[:page_title] || "Pinchflat" %>
{assigns[:page_title] || "Pinchflat"}
</.live_title>
<link phx-track-static rel="stylesheet" href={~p"/assets/app.css"} />
<link rel="icon" type="image/x-icon" href={~p"/favicon.ico?cachebust=2024-03-20"} />
@@ -21,7 +21,7 @@
class="dark text-bodydark bg-boxdark-2"
data-socket-path={Path.join(Application.get_env(:pinchflat, :base_route_path), "/live")}
>
<%= @inner_content %>
{@inner_content}
<.donate_modal conn={@conn} />
<template x-if="!proEnabled && !onboarding">
@@ -2,7 +2,7 @@
<:option>
<span x-data="{ copied: false }" x-on:click={~s"
copyWithCallbacks(
`#{Jason.Formatter.pretty_print(Phoenix.json_library().encode!(@media_item))}`,
String.raw`#{Jason.Formatter.pretty_print(Phoenix.json_library().encode!(@media_item))}`,
() => copied = true,
() => copied = false
)
@@ -1,6 +1,6 @@
<div class="mb-6 flex gap-3 flex-row items-center">
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4">
Editing "<%= StringUtils.truncate(@media_item.title, 35) %>"
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4 truncate">
Editing "{@media_item.title}"
</h2>
</div>
@@ -1,13 +1,13 @@
<%= if media_type(@media_item) == :video do %>
<video controls class="max-h-96 w-full">
<source src={~p"/media/#{@media_item.uuid}/stream"} type="video/mp4" />
<video controls class="max-h-128 w-full">
<source src={~p"/media/#{@media_item.uuid}/stream?v=#{DateTime.to_unix(@media_item.updated_at)}"} type="video/mp4" />
Your browser does not support the video element.
</video>
<% end %>
<%= if media_type(@media_item) == :audio do %>
<audio controls class="w-full">
<source src={~p"/media/#{@media_item.uuid}/stream"} type="audio/mpeg" />
<source src={~p"/media/#{@media_item.uuid}/stream?v=#{DateTime.to_unix(@media_item.updated_at)}"} type="audio/mpeg" />
Your browser does not support the audio element.
</audio>
<% end %>
@@ -1,10 +1,10 @@
<div class="mb-6 flex gap-3 flex-row items-center justify-between">
<div class="flex gap-3 items-center">
<div class="flex gap-3 items-center min-w-0">
<.link href={~p"/sources/#{@media_item.source_id}"}>
<.icon name="hero-arrow-left" class="w-10 h-10 hover:dark:text-white" />
</.link>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4">
<%= StringUtils.truncate(@media_item.title, 35) %>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4 truncate">
{@media_item.title}
</h2>
</div>
@@ -26,25 +26,29 @@
<:tab title="Media" id="media">
<div class="flex flex-col gap-10 dark:text-white">
<%= if media_file_exists?(@media_item) do %>
<section class="grid grid-cols-1 xl:grid-cols-2 xl:gap-6 mt-6">
<section class="grid grid-cols-1 xl:gap-6 mt-6">
<div>
<.media_preview media_item={@media_item} />
</div>
<aside class="mt-4 xl:mt-0">
<div>Uploaded: <%= DateTime.to_date(@media_item.uploaded_at) %></div>
<aside class="mt-4">
<h2 class="text-xl mb-2">{@media_item.title}</h2>
<div>Uploaded: {DateTime.to_date(@media_item.uploaded_at)}</div>
<div>
<span :if={URI.parse(@media_item.original_url).scheme =~ "http"}>
<.subtle_link href={@media_item.original_url} target="_blank">Open Original</.subtle_link>
<span class="mx-2">or</span>
</span>
<span>
<.subtle_link href={~p"/media/#{@media_item.uuid}/stream"} target="_blank">
<.subtle_link
href={~p"/media/#{@media_item.uuid}/stream?v=#{DateTime.to_unix(@media_item.updated_at)}"}
target="_blank"
>
Open Local Stream
</.subtle_link>
</span>
</div>
<div :if={@media_item.description} class="mt-4 text-bodydark">
<.break_on_newline text={@media_item.description} />
<.render_description text={@media_item.description} />
</div>
</aside>
</section>
@@ -54,7 +58,7 @@
<section>
<strong>Source:</strong>
<.subtle_link href={~p"/sources/#{@media_item.source_id}"}>
<%= @media_item.source.custom_name %>
{@media_item.source.custom_name}
</.subtle_link>
<.list_items_from_map map={Map.from_struct(@media_item)} />
</section>
@@ -64,13 +68,13 @@
<%= if match?([_|_], @media_item.tasks) do %>
<.table rows={@media_item.tasks} table_class="text-black dark:text-white">
<:col :let={task} label="Worker">
<%= task.job.worker %>
{task.job.worker}
</:col>
<:col :let={task} label="State">
<%= task.job.state %>
{task.job.state}
</:col>
<:col :let={task} label="Scheduled At">
<%= Calendar.strftime(task.job.scheduled_at, "%y-%m-%d %I:%M:%S %p %Z") %>
<.datetime_in_zone datetime={task.job.scheduled_at} />
</:col>
</.table>
<% else %>
@@ -1,26 +1,51 @@
defmodule PinchflatWeb.MediaProfiles.MediaProfileController do
use PinchflatWeb, :controller
use Pinchflat.Sources.SourcesQuery
use Pinchflat.Profiles.ProfilesQuery
alias Pinchflat.Repo
alias Pinchflat.Profiles
alias Pinchflat.Sources.Source
alias Pinchflat.Profiles.MediaProfile
alias Pinchflat.Profiles.MediaProfileDeletionWorker
def index(conn, _params) do
media_profiles =
MediaProfile
|> where([mp], is_nil(mp.marked_for_deletion_at))
|> order_by(asc: :name)
|> Repo.all()
media_profiles_query =
from mp in MediaProfile,
as: :media_profile,
where: is_nil(mp.marked_for_deletion_at),
order_by: [asc: mp.name],
select: map(mp, ^MediaProfile.__schema__(:fields)),
select_merge: %{
source_count:
subquery(
from s in Source,
where: s.media_profile_id == parent_as(:media_profile).id,
select: count(s.id)
)
}
render(conn, :index, media_profiles: media_profiles)
render(conn, :index, media_profiles: Repo.all(media_profiles_query))
end
def new(conn, _params) do
changeset = Profiles.change_media_profile(%MediaProfile{})
def new(conn, params) do
# Preload an existing media profile for faster creation
cs_struct =
case to_string(params["template_id"]) do
"" -> %MediaProfile{}
template_id -> Repo.get(MediaProfile, template_id) || %MediaProfile{}
end
render(conn, :new, changeset: changeset, layout: get_onboarding_layout())
render(conn, :new,
layout: get_onboarding_layout(),
changeset:
Profiles.change_media_profile(%MediaProfile{
cs_struct
| id: nil,
name: nil,
marked_for_deletion_at: nil
})
)
end
def create(conn, %{"media_profile" => media_profile_params}) do
@@ -10,6 +10,7 @@ defmodule PinchflatWeb.MediaProfiles.MediaProfileHTML do
"""
attr :changeset, Ecto.Changeset, required: true
attr :action, :string, required: true
attr :method, :string, required: true
def media_profile_form(assigns)
@@ -25,6 +26,7 @@ defmodule PinchflatWeb.MediaProfiles.MediaProfileHTML do
[
{"8k", "4320p"},
{"4k", "2160p"},
{"1440p", "1440p"},
{"1080p", "1080p"},
{"720p", "720p"},
{"480p", "480p"},
@@ -36,6 +38,7 @@ defmodule PinchflatWeb.MediaProfiles.MediaProfileHTML do
def friendly_sponsorblock_options do
[
{"Disabled (default)", "disabled"},
{"Mark Segments as Chapters", "mark"},
{"Remove Segments", "remove"}
]
end
@@ -54,7 +57,7 @@ defmodule PinchflatWeb.MediaProfiles.MediaProfileHTML do
end
def media_center_custom_output_template_options do
%{
[
season_by_year__episode_by_date: "<code>Season YYYY/sYYYYeMMDD</code>",
season_by_year__episode_by_date_and_index:
"same as the above but it handles dates better. <strong>This is the recommended option</strong>",
@@ -62,11 +65,11 @@ defmodule PinchflatWeb.MediaProfiles.MediaProfileHTML do
"<code>Season 1/s01eXX</code> where <code>XX</code> is the video's position in the playlist. Only recommended for playlists (not channels) that don't change",
static_season__episode_by_date:
"<code>Season 1/s01eYYMMDD</code>. Recommended for playlists that might change or where order isn't important"
}
]
end
def other_custom_output_template_options do
%{
[
upload_day: nil,
upload_month: nil,
upload_year: nil,
@@ -82,8 +85,11 @@ defmodule PinchflatWeb.MediaProfiles.MediaProfileHTML do
season_episode_index_from_date:
"the upload date formatted as <code>sYYYYeMMDDII</code> where <code>II</code> is an index to prevent date collisions",
media_playlist_index:
"the place of the media item in the playlist. Do not use with channels. May not work if the playlist is updated"
}
"the place of the media item in the playlist. Do not use with channels. May not work if the playlist is updated",
media_item_id: "the ID of the media item in Pinchflat's database",
source_id: "the ID of the source in Pinchflat's database",
media_profile_id: "the ID of the media profile in Pinchflat's database"
]
end
def common_output_template_options do
@@ -2,7 +2,7 @@
<:option>
<span x-data="{ copied: false }" x-on:click={~s"
copyWithCallbacks(
`#{Jason.Formatter.pretty_print(Phoenix.json_library().encode!(@media_profile))}`,
String.raw`#{Jason.Formatter.pretty_print(Phoenix.json_library().encode!(@media_profile))}`,
() => copied = true,
() => copied = false
)
@@ -11,6 +11,11 @@
<span x-show="copied" x-transition.duration.150ms><.icon name="hero-check" class="ml-2 h-4 w-4" /></span>
</span>
</:option>
<:option>
<.link href={~p"/media_profiles/new?template_id=#{@media_profile}"} method="get">
Use as Template
</.link>
</:option>
<:option>
<div class="h-px w-full bg-bodydark2"></div>
</:option>
@@ -2,15 +2,15 @@
<.link href={~p"/media_profiles"}>
<.icon name="hero-arrow-left" class="w-10 h-10 hover:dark:text-white" />
</.link>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4">
Editing "<%= @media_profile.name %>"
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4 truncate">
Editing "{@media_profile.name}"
</h2>
</div>
<div class="rounded-sm border border-stroke bg-white px-5 pb-2.5 pt-6 shadow-default dark:border-strokedark dark:bg-boxdark sm:px-7.5 xl:pb-1">
<div class="max-w-full">
<div class="flex flex-col gap-10">
<.media_profile_form changeset={@changeset} action={~p"/media_profiles/#{@media_profile}"} />
<.media_profile_form changeset={@changeset} action={~p"/media_profiles/#{@media_profile}"} method="patch" />
</div>
</div>
</div>
@@ -10,18 +10,22 @@
</.link>
</nav>
</div>
<div class="rounded-sm border border-stroke bg-white shadow-default dark:border-strokedark dark:bg-boxdark">
<div class="max-w-full overflow-x-auto">
<div class="flex flex-col gap-10 min-w-max">
<.table rows={@media_profiles} table_class="text-black dark:text-white">
<:col :let={media_profile} label="Name">
<:col :let={media_profile} label="Name" class="truncate max-w-xs">
<.subtle_link href={~p"/media_profiles/#{media_profile.id}"}>
<%= media_profile.name %>
{media_profile.name}
</.subtle_link>
</:col>
<:col :let={media_profile} label="Preferred Resolution">
<%= media_profile.preferred_resolution %>
{media_profile.preferred_resolution}
</:col>
<:col :let={media_profile} label="Sources">
<.subtle_link href={~p"/media_profiles/#{media_profile.id}/#tab-sources"}>
<.localized_number number={media_profile.source_count} />
</.subtle_link>
</:col>
<:col :let={media_profile} label="" class="flex justify-end">
<.icon_link href={~p"/media_profiles/#{media_profile.id}/edit"} icon="hero-pencil-square" class="mr-4" />
@@ -2,6 +2,7 @@
:let={f}
for={@changeset}
action={@action}
method={@method}
x-data="{ advancedMode: !!JSON.parse(localStorage.getItem('advancedMode')) }"
x-init="$watch('advancedMode', value => localStorage.setItem('advancedMode', JSON.stringify(value)))"
>
@@ -124,6 +125,16 @@
/>
</section>
<section x-show="advancedMode">
<.input
field={f[:audio_track]}
placeholder="de"
type="text"
label="Audio Track Language"
help="Only works if there are multiple audio tracks. Use either a language code, 'original' for the original audio track, or 'default' for YouTube's preference. Or just leave it blank"
/>
</section>
<h3 class="mt-10 text-2xl text-black dark:text-white">
Thumbnail Options
</h3>
@@ -8,7 +8,7 @@
<div class="rounded-sm border border-stroke bg-white px-5 pb-2.5 pt-6 shadow-default dark:border-strokedark dark:bg-boxdark sm:px-7.5 xl:pb-1">
<div class="max-w-full">
<div class="flex flex-col gap-10">
<.media_profile_form changeset={@changeset} action={~p"/media_profiles"} />
<.media_profile_form changeset={@changeset} action={~p"/media_profiles"} method="post" />
</div>
</div>
</div>
@@ -6,7 +6,9 @@
<ul class="list-disc list-inside ml-2 md:ml-5">
<li>
Liquid-style:
<.inline_code>/{{ channel }}/{{ title }} - {{ id }}.{{ ext }}</.inline_code>
<.inline_code>
{"/#{double_brace("channel")}/#{double_brace("title")} - #{double_brace("id")}.#{double_brace("ext")}"}
</.inline_code>
</li>
<li>
<code class="text-sm">yt-dlp</code>-style
@@ -17,7 +19,7 @@
</li>
<li>
Any bare words:
<.inline_code>/videos/1080p/{{ id }}.{{ ext }}</.inline_code>
<.inline_code>{"/videos/1080p/#{double_brace("id")}.#{double_brace("ext")}"}</.inline_code>
</li>
</ul>
<p class="my-2">
@@ -25,7 +27,7 @@
<em>any</em>
single-word <code class="text-sm">yt-dlp</code>
option can be used as liquid-style and it's automatically made filepath-safe. For example, the
<.inline_code>{{ duration }}</.inline_code>
<.inline_code>{double_brace("duration")}</.inline_code>
option is translated to
<.inline_code>%(duration)S</.inline_code>
</p>
@@ -33,12 +35,14 @@
<strong>Major 🔑:</strong>
these syntaxes can be mixed and matched freely! I prefer to use liquid-style and bare words
but I'll include <code class="text-sm">yt-dlp</code>-style when I need more control. For example:
<.inline_code>/1080p/{{ channel }}/{{ title }}-(%(subtitles.en.-1.ext)s).{{ ext }}</.inline_code>
<.inline_code>
{"/1080p/#{double_brace("channel")}/#{double_brace("title")}-(%(subtitles.en.-1.ext)s).#{double_brace("ext")}"}
</.inline_code>
</p>
<p class="my-2">
<strong>NOTE:</strong>
Your template <em>must</em>
end with an extension option (<.inline_code>.{{ ext }}</.inline_code>
end with an extension option (<.inline_code>.{double_brace("ext")}</.inline_code>
or
<.inline_code>.%(ext)S</.inline_code>).
Downloading won't work as expected without it.
@@ -59,21 +63,21 @@
<h3 class="text-lg font-bold my-2">Media Center Custom Aliases</h3>
<ul class="list-disc list-inside mb-4 ml-2 md:ml-5 max-w-prose">
<li :for={{k, v} <- media_center_custom_output_template_options()} class="mt-1">
<.inline_code>{{ <%= k %> }}</.inline_code>
<span :if={v}>- <%= html_escape({:safe, v}) %></span>
<.inline_code>{double_brace(k)}</.inline_code>
<span :if={v}>- {html_escape({:safe, v})}</span>
</li>
</ul>
<h3 class="text-lg font-bold mb-2">Other Custom Aliases</h3>
<ul class="list-disc list-inside mb-4 ml-2 md:ml-5 max-w-prose">
<li :for={{k, v} <- other_custom_output_template_options()} class="mt-1">
<.inline_code>{{ <%= k %> }}</.inline_code>
<span :if={v}>- <%= html_escape({:safe, v}) %></span>
<.inline_code>{double_brace(k)}</.inline_code>
<span :if={v}>- {html_escape({:safe, v})}</span>
</li>
</ul>
<h3 class="text-lg font-bold mb-2">Common Options</h3>
<ul class="list-disc list-inside ml-2 md:ml-5">
<li :for={opt <- common_output_template_options()} class="mt-1">
<.inline_code>{{ <%= opt %> }}</.inline_code>
<.inline_code>{double_brace(opt)}</.inline_code>
</li>
</ul>
</section>
@@ -1,10 +1,10 @@
<div class="mb-6 flex gap-3 flex-row items-center justify-between">
<div class="flex items-center">
<div class="flex items-center min-w-0">
<.link href={~p"/media_profiles"}>
<.icon name="hero-arrow-left" class="w-10 h-10 hover:dark:text-white" />
</.link>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-2">
<%= @media_profile.name %>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-2 truncate">
{@media_profile.name}
</h2>
</div>
@@ -33,10 +33,10 @@
<.table rows={@sources} table_class="text-black dark:text-white">
<:col :let={source} label="Name">
<.subtle_link href={~p"/sources/#{source.id}"}>
<%= source.custom_name || source.collection_name %>
{source.custom_name || source.collection_name}
</.subtle_link>
</:col>
<:col :let={source} label="Type"><%= source.collection_type %></:col>
<:col :let={source} label="Type">{source.collection_type}</:col>
<:col :let={source} label="Should Download?">
<.icon name={if source.download_media, do: "hero-check", else: "hero-x-mark"} />
</:col>
@@ -1,23 +1,5 @@
defmodule PinchflatWeb.Pages.PageHTML do
use PinchflatWeb, :html
alias Pinchflat.Utils.NumberUtils
embed_templates "page_html/*"
attr :media_filesize, :integer, required: true
def readable_media_filesize(assigns) do
{num, suffix} = NumberUtils.human_byte_size(assigns.media_filesize, precision: 2)
assigns =
Map.merge(assigns, %{
num: num,
suffix: suffix
})
~H"""
<.localized_number number={@num} /> <%= @suffix %>
"""
end
end
@@ -28,23 +28,23 @@ defmodule Pinchflat.Pages.HistoryTableLive do
</span>
<div class="max-w-full overflow-x-auto">
<.table rows={@records} table_class="text-white">
<:col :let={media_item} label="Title">
<:col :let={media_item} label="Title" class="truncate max-w-xs">
<.subtle_link href={~p"/sources/#{media_item.source_id}/media/#{media_item}"}>
<%= StringUtils.truncate(media_item.title, 35) %>
{media_item.title}
</.subtle_link>
</:col>
<:col :let={media_item} label="Upload Date">
<%= DateTime.to_date(media_item.uploaded_at) %>
{DateTime.to_date(media_item.uploaded_at)}
</:col>
<:col :let={media_item} label="Indexed At">
<%= format_datetime(media_item.inserted_at) %>
{format_datetime(media_item.inserted_at)}
</:col>
<:col :let={media_item} label="Downloaded At">
<%= format_datetime(media_item.media_downloaded_at) %>
{format_datetime(media_item.media_downloaded_at)}
</:col>
<:col :let={media_item} label="Source">
<:col :let={media_item} label="Source" class="truncate max-w-xs">
<.subtle_link href={~p"/sources/#{media_item.source_id}"}>
<%= StringUtils.truncate(media_item.source.custom_name, 35) %>
{media_item.source.custom_name}
</.subtle_link>
</:col>
</.table>
@@ -56,9 +56,9 @@ defmodule Pinchflat.Pages.HistoryTableLive do
"""
end
def mount(_params, _session, socket) do
def mount(_params, session, socket) do
page = 1
base_query = generate_base_query()
base_query = generate_base_query(session["media_state"])
pagination_attrs = fetch_pagination_attributes(base_query, page)
{:ok, assign(socket, Map.merge(pagination_attrs, %{base_query: base_query}))}
@@ -97,10 +97,17 @@ defmodule Pinchflat.Pages.HistoryTableLive do
|> Repo.preload(:source)
end
defp generate_base_query do
defp generate_base_query("pending") do
MediaQuery.new()
|> MediaQuery.require_assoc(:media_profile)
|> where(^dynamic(^MediaQuery.downloaded() or ^MediaQuery.pending()))
|> where(^dynamic(^MediaQuery.pending()))
|> order_by(desc: :id)
end
defp generate_base_query("downloaded") do
MediaQuery.new()
|> MediaQuery.require_assoc(:media_profile)
|> where(^dynamic(^MediaQuery.downloaded()))
|> order_by(desc: :id)
end
@@ -33,7 +33,7 @@
<span class="flex flex-col items-center py-2">
<span class="text-md font-medium">Library Size</span>
<h4 class="text-title-md font-bold text-white">
<.readable_media_filesize media_filesize={@media_item_size} />
<.readable_filesize byte_size={@media_item_size} />
</h4>
</span>
</div>
@@ -41,14 +41,27 @@
<div class="rounded-sm border shadow-default border-strokedark bg-boxdark mt-4 p-5">
<span class="text-2xl font-medium mb-4">Media History</span>
<section class="mt-6">
<%= live_render(@conn, Pinchflat.Pages.HistoryTableLive) %>
</section>
<.tabbed_layout>
<:tab title="Downloaded" id="downloaded">
{live_render(
@conn,
Pinchflat.Pages.HistoryTableLive,
session: %{"media_state" => "downloaded"}
)}
</:tab>
<:tab title="Pending" id="pending">
{live_render(
@conn,
Pinchflat.Pages.HistoryTableLive,
session: %{"media_state" => "pending"}
)}
</:tab>
</.tabbed_layout>
</div>
<div class="rounded-sm border shadow-default border-strokedark bg-boxdark mt-4 p-5">
<span class="text-2xl font-medium mb-4">Active Tasks</span>
<section class="mt-6 min-h-80">
<%= live_render(@conn, Pinchflat.Pages.JobTableLive) %>
{live_render(@conn, Pinchflat.Pages.JobTableLive)}
</section>
</div>
@@ -19,18 +19,18 @@ defmodule Pinchflat.Pages.JobTableLive do
<div class="max-w-full overflow-x-auto">
<.table rows={@tasks} table_class="text-white">
<:col :let={task} label="Task">
<%= worker_to_task_name(task.job.worker) %>
{worker_to_task_name(task.job.worker)}
</:col>
<:col :let={task} label="Subject">
<:col :let={task} label="Subject" class="truncate max-w-xs">
<.subtle_link href={task_to_link(task)}>
<%= StringUtils.truncate(task_to_record_name(task), 35) %>
{task_to_record_name(task)}
</.subtle_link>
</:col>
<:col :let={task} label="Attempt No.">
<%= task.job.attempt %>
{task.job.attempt}
</:col>
<:col :let={task} label="Started At">
<%= format_datetime(task.job.attempted_at) %>
{format_datetime(task.job.attempted_at)}
</:col>
</.table>
</div>
@@ -6,8 +6,19 @@ defmodule PinchflatWeb.Podcasts.PodcastController do
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
alias Pinchflat.Podcasts.RssFeedBuilder
alias Pinchflat.Podcasts.OpmlFeedBuilder
alias Pinchflat.Podcasts.PodcastHelpers
def opml_feed(conn, _params) do
url_base = url(conn, ~p"/")
xml = OpmlFeedBuilder.build(url_base, PodcastHelpers.opml_sources())
conn
|> put_resp_content_type("application/opml+xml")
|> put_resp_header("content-disposition", "inline")
|> send_resp(200, xml)
end
def rss_feed(conn, %{"uuid" => uuid}) do
source = Repo.get_by!(Source, uuid: uuid)
url_base = url(conn, ~p"/")
@@ -14,7 +14,7 @@ defmodule PinchflatWeb.Searches.SearchHTML do
~H"""
<%= for fragment <- @split_string do %>
<%= render_fragment(fragment) %>
{render_fragment(fragment)}
<% end %>
"""
end
@@ -1,6 +1,6 @@
<div class="mb-6 flex gap-3 flex-row items-center justify-between">
<h2 class="text-title-md2 font-bold text-black dark:text-white">
Results for "<%= StringUtils.truncate(@search_term, 50) %>"
<h2 class="text-title-md2 font-bold text-black dark:text-white truncate">
Results for "{@search_term}"
</h2>
</div>
@@ -9,9 +9,9 @@
<div class="flex flex-col gap-10 dark:text-white">
<%= if match?([_|_], @search_results) do %>
<.table rows={@search_results} table_class="text-black dark:text-white">
<:col :let={result} label="Title">
<:col :let={result} label="Title" class="truncate max-w-xs">
<.subtle_link href={~p"/sources/#{result.source_id}/media/#{result.id}"}>
<%= StringUtils.truncate(result.title, 35) %>
{result.title}
</.subtle_link>
</:col>
<:col :let={result} label="Excerpt">
@@ -19,17 +19,17 @@
</span>
</section>
<%= live_render(
{live_render(
@conn,
Pinchflat.Settings.AppriseServerLive,
session: %{"value" => f[:apprise_server].value}
) %>
)}
</section>
<section class="mt-8">
<section>
<h3 class="text-2xl text-black dark:text-white">
Indexing Settings
Extractor Settings
</h3>
<.input
@@ -41,6 +41,14 @@
html_help={true}
inputclass="font-mono text-sm mr-4"
/>
<.input
field={f[:extractor_sleep_interval_seconds]}
placeholder="0"
type="number"
label="Sleep Interval (seconds)"
help="Sleep interval in seconds between each extractor request. Must be a positive whole number (or set to 0 to disable)"
/>
</section>
</section>
@@ -1,12 +1,11 @@
defmodule PinchflatWeb.Sources.SourceController do
use PinchflatWeb, :controller
use Pinchflat.Media.MediaQuery
use Pinchflat.Sources.SourcesQuery
alias Pinchflat.Repo
alias Pinchflat.Tasks
alias Pinchflat.Sources
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
alias Pinchflat.Profiles.MediaProfile
alias Pinchflat.Media.FileSyncingWorker
alias Pinchflat.Sources.SourceDeletionWorker
@@ -15,33 +14,7 @@ defmodule PinchflatWeb.Sources.SourceController do
alias Pinchflat.Metadata.SourceMetadataStorageWorker
def index(conn, _params) do
source_query =
from s in Source,
as: :source,
inner_join: mp in assoc(s, :media_profile),
where: is_nil(s.marked_for_deletion_at) and is_nil(mp.marked_for_deletion_at),
preload: [media_profile: mp],
order_by: [asc: s.custom_name],
select: map(s, ^Source.__schema__(:fields)),
select_merge: %{
downloaded_count:
subquery(
from m in MediaItem,
where: m.source_id == parent_as(:source).id,
where: ^MediaQuery.downloaded(),
select: count(m.id)
),
pending_count:
subquery(
from m in MediaItem,
join: s in assoc(m, :source),
where: m.source_id == parent_as(:source).id,
where: ^MediaQuery.pending(),
select: count(m.id)
)
}
render(conn, :index, sources: Repo.all(source_query))
render(conn, :index)
end
def new(conn, params) do
@@ -67,7 +40,8 @@ defmodule PinchflatWeb.Sources.SourceController do
collection_name: nil,
collection_id: nil,
collection_type: nil,
original_url: nil
original_url: nil,
marked_for_deletion_at: nil
})
)
end
@@ -27,10 +27,28 @@ defmodule PinchflatWeb.Sources.SourceHTML do
]
end
def cutoff_date_presets do
[
{"7 days", compute_date_offset(7)},
{"14 days", compute_date_offset(14)},
{"30 days", compute_date_offset(30)},
{"60 days", compute_date_offset(60)},
{"90 days", compute_date_offset(90)},
{"180 days", compute_date_offset(180)},
{"365 days", compute_date_offset(365)}
]
end
def rss_feed_url(conn, source) do
# NOTE: The reason for this concatenation is to avoid what appears to be a bug in Phoenix
# See: https://github.com/phoenixframework/phoenix/issues/6033
url(conn, ~p"/sources/#{source.uuid}/feed") <> ".xml"
end
def opml_feed_url(conn) do
url(conn, ~p"/sources/opml.xml?#{[route_token: Settings.get!(:route_token)]}")
end
def output_path_template_override_placeholders(media_profiles) do
media_profiles
|> Enum.map(&{&1.id, &1.output_path_template})
@@ -55,4 +73,13 @@ defmodule PinchflatWeb.Sources.SourceHTML do
Must end with .{{ ext }}. Same rules as Media Profile output path templates. #{help_button} to load your media profile's output template
"""
end
defp compute_date_offset(days) do
timezone = Application.get_env(:pinchflat, :timezone)
timezone
|> Timex.now()
|> Timex.shift(days: -days)
|> Timex.format!("{YYYY}-{0M}-{0D}")
end
end
@@ -1,23 +1,21 @@
<.button_dropdown text="Actions" class="justify-center w-full sm:w-50">
<:option>
<span
x-data="{ copied: false }"
x-on:click={"
copyWithCallbacks(
'#{rss_feed_url(@conn, @source)}',
() => copied = true,
() => copied = false
)
"}
>
<.link href={rss_feed_url(@conn, @source)} x-data="{ copied: false }" x-on:click={~s"
$event.preventDefault();
copyWithCallbacks(
'#{rss_feed_url(@conn, @source)}',
() => copied = true,
() => copied = false
)
"}>
Copy RSS Feed
<span x-show="copied" x-transition.duration.150ms><.icon name="hero-check" class="ml-2 h-4 w-4" /></span>
</span>
</.link>
</:option>
<:option>
<span x-data="{ copied: false }" x-on:click={~s"
copyWithCallbacks(
`#{Jason.Formatter.pretty_print(Phoenix.json_library().encode!(@source))}`,
String.raw`#{Jason.Formatter.pretty_print(Phoenix.json_library().encode!(@source))}`,
() => copied = true,
() => copied = false
)
@@ -56,7 +54,7 @@
<.link
href={~p"/sources/#{@source}/force_index"}
method="post"
data-confirm="Are you sure you want to force an index of this source? This isn't normally needed."
data-confirm="Are you sure you want index all content from this source? This isn't normally needed."
>
Force Index
</.link>
@@ -2,8 +2,8 @@
<.link href={~p"/sources"}>
<.icon name="hero-arrow-left" class="w-10 h-10 hover:dark:text-white" />
</.link>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4">
Editing "<%= @source.custom_name %>"
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4 truncate">
Editing "{@source.custom_name}"
</h2>
</div>
@@ -1,6 +1,17 @@
<div class="mb-6 flex gap-3 flex-row items-center justify-between">
<h2 class="text-title-md2 font-bold text-black dark:text-white">Sources</h2>
<nav>
<nav class="flex items-center justify-between gap-6">
<.link href={opml_feed_url(@conn)} x-data="{ copied: false }" x-on:click={~s"
$event.preventDefault();
copyWithCallbacks(
'#{opml_feed_url(@conn)}',
() => copied = true,
() => copied = false
)
"}>
Copy OPML <span class="hidden sm:inline">Feed</span>
<span x-show="copied" x-transition.duration.150ms><.icon name="hero-check" class="ml-2 h-4 w-4" /></span>
</.link>
<.link href={~p"/sources/new"}>
<.button color="bg-primary" rounding="rounded-lg">
<span class="font-bold mx-2">+</span> New <span class="hidden sm:inline pl-1">Source</span>
@@ -11,33 +22,12 @@
<div class="rounded-sm border border-stroke bg-white shadow-default dark:border-strokedark dark:bg-boxdark">
<div class="max-w-full overflow-x-auto">
<div class="flex flex-col gap-10 min-w-max">
<.table rows={@sources} table_class="text-black dark:text-white">
<:col :let={source} label="Name">
<.subtle_link href={~p"/sources/#{source.id}"}>
<%= StringUtils.truncate(source.custom_name || source.collection_name, 35) %>
</.subtle_link>
</:col>
<:col :let={source} label="Type"><%= source.collection_type %></:col>
<:col :let={source} label="Pending"><.localized_number number={source.pending_count} /></:col>
<:col :let={source} label="Downloaded"><.localized_number number={source.downloaded_count} /></:col>
<:col :let={source} label="Retention">
<%= if source.retention_period_days && source.retention_period_days > 0 do %>
<.localized_number number={source.retention_period_days} />
<.pluralize count={source.retention_period_days} word="day" />
<% else %>
<span class="text-lg">∞</span>
<% end %>
</:col>
<:col :let={source} label="Media Profile">
<.subtle_link href={~p"/media_profiles/#{source.media_profile_id}"}>
<%= source.media_profile.name %>
</.subtle_link>
</:col>
<:col :let={source} label="" class="flex place-content-evenly">
<.icon_link href={~p"/sources/#{source.id}/edit"} icon="hero-pencil-square" class="mx-1" />
</:col>
</.table>
</div>
{live_render(@conn, PinchflatWeb.Sources.SourceLive.IndexTableLive,
session: %{
"initial_sort_key" => :custom_name,
"initial_sort_direction" => :asc,
"results_per_page" => 10
}
)}
</div>
</div>
@@ -0,0 +1,103 @@
defmodule PinchflatWeb.Sources.IndexTableLive do
use PinchflatWeb, :live_view
use Pinchflat.Media.MediaQuery
use Pinchflat.Sources.SourcesQuery
alias Pinchflat.Repo
alias Pinchflat.Sources
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
def render(assigns) do
~H"""
<.table rows={@sources} table_class="text-white">
<:col :let={source} label="Name" class="truncate max-w-xs">
<.subtle_link href={~p"/sources/#{source.id}"}>
{source.custom_name || source.collection_name}
</.subtle_link>
</:col>
<:col :let={source} label="Pending">
<.subtle_link href={~p"/sources/#{source.id}/#tab-pending"}>
<.localized_number number={source.pending_count} />
</.subtle_link>
</:col>
<:col :let={source} label="Downloaded">
<.subtle_link href={~p"/sources/#{source.id}/#tab-downloaded"}>
<.localized_number number={source.downloaded_count} />
</.subtle_link>
</:col>
<:col :let={source} label="Retention">
<%= if source.retention_period_days && source.retention_period_days > 0 do %>
<.localized_number number={source.retention_period_days} />
<.pluralize count={source.retention_period_days} word="day" />
<% else %>
<span class="text-lg">∞</span>
<% end %>
</:col>
<:col :let={source} label="Media Profile">
<.subtle_link href={~p"/media_profiles/#{source.media_profile_id}"}>
{source.media_profile.name}
</.subtle_link>
</:col>
<:col :let={source} label="Enabled?">
<.input
name={"source[#{source.id}][enabled]"}
value={source.enabled}
id={"source_#{source.id}_enabled"}
phx-hook="formless-input"
data-subscribe="change"
data-event-name="toggle_enabled"
data-identifier={source.id}
type="toggle"
/>
</:col>
<:col :let={source} label="" class="flex place-content-evenly">
<.icon_link href={~p"/sources/#{source.id}/edit"} icon="hero-pencil-square" class="mx-1" />
</:col>
</.table>
"""
end
def mount(_params, _session, socket) do
{:ok, assign(socket, %{sources: get_sources()})}
end
def handle_event("formless-input", %{"event" => "toggle_enabled"} = params, socket) do
source = Sources.get_source!(params["id"])
should_enable = params["value"] == "true"
{:ok, _} = Sources.update_source(source, %{enabled: should_enable})
{:noreply, assign(socket, %{sources: get_sources()})}
end
defp get_sources do
query =
from s in Source,
as: :source,
inner_join: mp in assoc(s, :media_profile),
where: is_nil(s.marked_for_deletion_at) and is_nil(mp.marked_for_deletion_at),
preload: [media_profile: mp],
order_by: [asc: s.custom_name],
select: map(s, ^Source.__schema__(:fields)),
select_merge: %{
downloaded_count:
subquery(
from m in MediaItem,
where: m.source_id == parent_as(:source).id,
where: ^MediaQuery.downloaded(),
select: count(m.id)
),
pending_count:
subquery(
from m in MediaItem,
join: s in assoc(m, :source),
where: m.source_id == parent_as(:source).id,
where: ^MediaQuery.pending(),
select: count(m.id)
)
}
Repo.all(query)
end
end
@@ -1,4 +1,4 @@
defmodule Pinchflat.Sources.MediaItemTableLive do
defmodule PinchflatWeb.Sources.MediaItemTableLive do
use PinchflatWeb, :live_view
use Pinchflat.Media.MediaQuery
@@ -46,16 +46,16 @@ defmodule Pinchflat.Sources.MediaItemTableLive do
</div>
</header>
<.table rows={@records} table_class="text-white">
<:col :let={media_item} label="Title">
<:col :let={media_item} label="Title" class="truncate max-w-xs">
<.subtle_link href={~p"/sources/#{@source.id}/media/#{media_item.id}"}>
<%= StringUtils.truncate(media_item.title, 50) %>
{media_item.title}
</.subtle_link>
</:col>
<:col :let={media_item} :if={@media_state == "other"} label="Manually Ignored?">
<.icon name={if media_item.prevent_download, do: "hero-check", else: "hero-x-mark"} />
</:col>
<:col :let={media_item} label="Upload Date">
<%= DateTime.to_date(media_item.uploaded_at) %>
{DateTime.to_date(media_item.uploaded_at)}
</:col>
<:col :let={media_item} label="" class="flex justify-end">
<.icon_link href={~p"/sources/#{@source.id}/media/#{media_item.id}/edit"} icon="hero-pencil-square" class="mr-4" />
@@ -1,10 +1,10 @@
<div class="mb-6 flex gap-3 flex-row items-center justify-between">
<div class="flex gap-3 items-center">
<div class="flex gap-3 items-center min-w-0">
<.link href={~p"/sources"}>
<.icon name="hero-arrow-left" class="w-10 h-10 hover:dark:text-white" />
</.link>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4">
<%= @source.custom_name %>
<h2 class="text-title-md2 font-bold text-black dark:text-white ml-4 truncate">
{@source.custom_name}
</h2>
</div>
@@ -29,7 +29,7 @@
<section>
<strong>Media Profile:</strong>
<.subtle_link href={~p"/media_profiles/#{@source.media_profile_id}"}>
<%= @source.media_profile.name %>
{@source.media_profile.name}
</.subtle_link>
</section>
@@ -37,37 +37,37 @@
</div>
</:tab>
<:tab title="Pending" id="pending">
<%= live_render(
{live_render(
@conn,
Pinchflat.Sources.MediaItemTableLive,
PinchflatWeb.Sources.MediaItemTableLive,
session: %{"source_id" => @source.id, "media_state" => "pending"}
) %>
)}
</:tab>
<:tab title="Downloaded" id="downloaded">
<%= live_render(
{live_render(
@conn,
Pinchflat.Sources.MediaItemTableLive,
PinchflatWeb.Sources.MediaItemTableLive,
session: %{"source_id" => @source.id, "media_state" => "downloaded"}
) %>
)}
</:tab>
<:tab title="Other" id="other">
<%= live_render(
{live_render(
@conn,
Pinchflat.Sources.MediaItemTableLive,
PinchflatWeb.Sources.MediaItemTableLive,
session: %{"source_id" => @source.id, "media_state" => "other"}
) %>
)}
</:tab>
<:tab title="Tasks" id="tasks">
<%= if match?([_|_], @pending_tasks) do %>
<.table rows={@pending_tasks} table_class="text-black dark:text-white">
<:col :let={task} label="Worker">
<%= task.job.worker %>
{task.job.worker}
</:col>
<:col :let={task} label="State">
<%= task.job.state %>
{task.job.state}
</:col>
<:col :let={task} label="Scheduled At">
<%= Calendar.strftime(task.job.scheduled_at, "%y-%m-%d %I:%M:%S %p %Z") %>
<.datetime_in_zone datetime={task.job.scheduled_at} />
</:col>
</.table>
<% else %>
@@ -111,16 +111,31 @@
/>
</section>
<.input
field={f[:download_cutoff_date]}
type="text"
label="Download Cutoff Date"
placeholder="YYYY-MM-DD"
maxlength="10"
pattern="((?:19|20)[0-9][0-9])-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])"
title="YYYY-MM-DD"
help="Only download media uploaded after this date. Leave blank to download all media. Must be in YYYY-MM-DD format. Old media may be deleted or downloaded if you change this date"
/>
<section x-data={"{ cutoffDate: '#{f[:download_cutoff_date].value}' }"}>
<.input
field={f[:download_cutoff_date]}
type="text"
label="Download Cutoff Date"
placeholder="YYYY-MM-DD"
maxlength="10"
pattern="((?:19|20)[0-9][0-9])-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])"
title="YYYY-MM-DD"
help="Only download media uploaded after this date. Leave blank to download all media. Must be in YYYY-MM-DD format. Old media may be deleted or downloaded if you change this date"
x-model="cutoffDate"
>
<:input_append>
<.input
prompt="Select preset"
name="download_cutoff_date_preset"
value=""
options={cutoff_date_presets()}
type="select"
inputclass="w-30 lg:w-60 ml-2 md:ml-4"
x-on:change={"cutoffDate = $event.target.value || '#{f[:download_cutoff_date].value}'"}
/>
</:input_append>
</.input>
</section>
<.input
field={f[:retention_period_days]}
@@ -0,0 +1,108 @@
defmodule PinchflatWeb.Sources.SourceLive.IndexTableLive do
use PinchflatWeb, :live_view
use Pinchflat.Media.MediaQuery
use Pinchflat.Sources.SourcesQuery
import PinchflatWeb.Helpers.SortingHelpers
import PinchflatWeb.Helpers.PaginationHelpers
alias Pinchflat.Repo
alias Pinchflat.Sources.Source
alias Pinchflat.Media.MediaItem
def mount(_params, session, socket) do
limit = session["results_per_page"]
initial_params =
Map.merge(
%{
sort_key: session["initial_sort_key"],
sort_direction: session["initial_sort_direction"]
},
get_pagination_attributes(sources_query(), 1, limit)
)
socket
|> assign(initial_params)
|> set_sources()
|> then(&{:ok, &1})
end
def handle_event("page_change", %{"direction" => direction}, %{assigns: assigns} = socket) do
new_page = update_page_number(assigns.page, direction, assigns.total_pages)
socket
|> assign(get_pagination_attributes(sources_query(), new_page, assigns.limit))
|> set_sources()
|> then(&{:noreply, &1})
end
def handle_event("sort_update", %{"sort_key" => sort_key}, %{assigns: assigns} = socket) do
new_sort_key = String.to_existing_atom(sort_key)
new_params = %{
sort_key: new_sort_key,
sort_direction: get_sort_direction(assigns.sort_key, new_sort_key, assigns.sort_direction)
}
socket
|> assign(new_params)
|> set_sources()
|> then(&{:noreply, &1})
end
defp sort_attr(:pending_count), do: dynamic([s, mp, dl, pe], pe.pending_count)
defp sort_attr(:downloaded_count), do: dynamic([s, mp, dl], dl.downloaded_count)
defp sort_attr(:media_size_bytes), do: dynamic([s, mp, dl], dl.media_size_bytes)
defp sort_attr(:media_profile_name), do: dynamic([s, mp], mp.name)
defp sort_attr(:custom_name), do: dynamic([s], s.custom_name)
defp sort_attr(:enabled), do: dynamic([s], s.enabled)
defp set_sources(%{assigns: assigns} = socket) do
sources =
sources_query()
|> order_by(^[{assigns.sort_direction, sort_attr(assigns.sort_key)}, asc: :id])
|> limit(^assigns.limit)
|> offset(^assigns.offset)
|> Repo.all()
assign(socket, %{sources: sources})
end
defp sources_query do
downloaded_subquery =
from(
m in MediaItem,
select: %{downloaded_count: count(m.id), source_id: m.source_id, media_size_bytes: sum(m.media_size_bytes)},
where: ^MediaQuery.downloaded(),
group_by: m.source_id
)
pending_subquery =
from(
m in MediaItem,
inner_join: s in assoc(m, :source),
inner_join: mp in assoc(s, :media_profile),
select: %{pending_count: count(m.id), source_id: m.source_id},
where: ^MediaQuery.pending(),
group_by: m.source_id
)
from s in Source,
as: :source,
inner_join: mp in assoc(s, :media_profile),
left_join: d in subquery(downloaded_subquery),
on: d.source_id == s.id,
left_join: p in subquery(pending_subquery),
on: p.source_id == s.id,
on: d.source_id == s.id,
where: is_nil(s.marked_for_deletion_at) and is_nil(mp.marked_for_deletion_at),
preload: [media_profile: mp],
select: map(s, ^Source.__schema__(:fields)),
select_merge: %{
downloaded_count: coalesce(d.downloaded_count, 0),
pending_count: coalesce(p.pending_count, 0),
media_size_bytes: coalesce(d.media_size_bytes, 0)
}
end
end
@@ -0,0 +1,41 @@
<div class="flex flex-col min-w-max">
<.table rows={@sources} table_class="text-white" sort_key={@sort_key} sort_direction={@sort_direction}>
<:col :let={source} label="Name" sort_key="custom_name" class="truncate max-w-xs">
<.subtle_link href={~p"/sources/#{source.id}"}>
{source.custom_name}
</.subtle_link>
</:col>
<:col :let={source} label="Pending" sort_key="pending_count">
<.subtle_link href={~p"/sources/#{source.id}/#tab-pending"}>
<.localized_number number={source.pending_count} />
</.subtle_link>
</:col>
<:col :let={source} label="Downloaded" sort_key="downloaded_count">
<.subtle_link href={~p"/sources/#{source.id}/#tab-downloaded"}>
<.localized_number number={source.downloaded_count} />
</.subtle_link>
</:col>
<:col :let={source} label="Size" sort_key="media_size_bytes">
<.readable_filesize byte_size={source.media_size_bytes} />
</:col>
<:col :let={source} label="Media Profile" sort_key="media_profile_name" class="truncate max-w-xs">
<.subtle_link href={~p"/media_profiles/#{source.media_profile_id}"}>
{source.media_profile.name}
</.subtle_link>
</:col>
<:col :let={source} label="Enabled?" sort_key="enabled">
<.live_component
module={PinchflatWeb.Sources.SourceLive.SourceEnableToggle}
source={source}
id={"source_#{source.id}_enabled"}
/>
</:col>
<:col :let={source} label="" class="flex place-content-evenly">
<.icon_link href={~p"/sources/#{source.id}/edit"} icon="hero-pencil-square" class="mx-1" />
</:col>
</.table>
<section class="flex justify-center my-5">
<.live_pagination_controls page_number={@page} total_pages={@total_pages} />
</section>
</div>
@@ -0,0 +1,35 @@
defmodule PinchflatWeb.Sources.SourceLive.SourceEnableToggle do
use PinchflatWeb, :live_component
alias Pinchflat.Sources
alias Pinchflat.Sources.Source
def render(assigns) do
~H"""
<div>
<.form :let={f} for={@form} phx-change="update" phx-target={@myself} class="enabled_toggle_form">
<.input id={"source_#{@source_id}_enabled_input"} field={f[:enabled]} type="toggle" />
</.form>
</div>
"""
end
def update(assigns, socket) do
initial_data = %{
source_id: assigns.source.id,
form: Sources.change_source(%Source{}, assigns.source)
}
socket
|> assign(initial_data)
|> then(&{:ok, &1})
end
def handle_event("update", %{"source" => source_params}, %{assigns: assigns} = socket) do
assigns.source_id
|> Sources.get_source!()
|> Sources.update_source(source_params)
{:noreply, socket}
end
end
+9
View File
@@ -32,6 +32,8 @@ defmodule PinchflatWeb.Endpoint do
plug Phoenix.Ecto.CheckRepoStatus, otp_app: :pinchflat
end
plug PromEx.Plug, prom_ex_module: Pinchflat.PromEx
plug Phoenix.LiveDashboard.RequestLogger,
param_key: "request_logger",
cookie_key: "request_logger"
@@ -73,6 +75,13 @@ defmodule PinchflatWeb.Endpoint do
Phoenix.Controller.put_router_url(conn, new_base_url)
end
# Some podcast clients require file extensions, and others still will _add_
# file extensions to XML files if they don't have them. This plug removes
# the extension from the path so that the correct route is matched, regardless
# of the provided extension.
#
# This has the downside of in-app generated verified routes not working with
# extensions so this behaviour may change in the future.
defp strip_trailing_extension(%{path_info: []} = conn, _opts), do: conn
defp strip_trailing_extension(conn, _opts) do
@@ -0,0 +1,45 @@
defmodule PinchflatWeb.Helpers.PaginationHelpers do
@moduledoc """
Methods for working with pagination, usually in the context of LiveViews or LiveComponents.
These methods are fairly simple, but they're commonly repeated across different Live entities
"""
alias Pinchflat.Repo
alias Pinchflat.Utils.NumberUtils
@doc """
Given a query, a page number, and a number of records per page, returns a map of pagination attributes.
Returns map()
"""
def get_pagination_attributes(query, page, records_per_page) do
total_record_count = Repo.aggregate(query, :count, :id)
total_pages = max(ceil(total_record_count / records_per_page), 1)
clamped_page = NumberUtils.clamp(page, 1, total_pages)
%{
page: clamped_page,
total_pages: total_pages,
total_record_count: total_record_count,
limit: records_per_page,
offset: (clamped_page - 1) * records_per_page
}
end
@doc """
Given a current page number, a direction to move in, and the total number of pages, returns the updated page number.
The updated page number is clamped to the range [1, total_pages].
Returns integer()
"""
def update_page_number(current_page, direction, total_pages) do
updated_page =
case to_string(direction) do
"inc" -> current_page + 1
"dec" -> current_page - 1
end
NumberUtils.clamp(updated_page, 1, total_pages)
end
end
@@ -0,0 +1,20 @@
defmodule PinchflatWeb.Helpers.SortingHelpers do
@moduledoc """
Methods for working with sorting, usually in the context of LiveViews or LiveComponents.
These methods are fairly simple, but they're commonly repeated across different Live entities
"""
@doc """
Given the old sort attribute, the new sort attribute, and the old sort direction, returns the new sort direction.
Returns :asc | :desc
"""
def get_sort_direction(old_sort_attr, new_sort_attr, old_sort_direction) do
case {new_sort_attr, old_sort_direction} do
{^old_sort_attr, :desc} -> :asc
{^old_sort_attr, _} -> :desc
_ -> :asc
end
end
end
+66
View File
@@ -0,0 +1,66 @@
defmodule PinchflatWeb.Plugs do
@moduledoc """
Custom plugs for PinchflatWeb.
"""
use PinchflatWeb, :router
alias Pinchflat.Settings
@doc """
If the `expose_feed_endpoints` setting is true, this plug does nothing. Otherwise, it calls `basic_auth/2`.
"""
def maybe_basic_auth(conn, opts) do
if Application.get_env(:pinchflat, :expose_feed_endpoints) do
conn
else
basic_auth(conn, opts)
end
end
@doc """
If the `basic_auth_username` and `basic_auth_password` settings are set, this plug calls `Plug.BasicAuth.basic_auth/3`.
"""
def basic_auth(conn, _opts) do
username = Application.get_env(:pinchflat, :basic_auth_username)
password = Application.get_env(:pinchflat, :basic_auth_password)
if credential_set?(username) && credential_set?(password) do
Plug.BasicAuth.basic_auth(conn, username: username, password: password, realm: "Pinchflat")
else
conn
end
end
@doc """
Removes the `x-frame-options` header from the response to allow the page to be embedded in an iframe.
"""
def allow_iframe_embed(conn, _opts) do
delete_resp_header(conn, "x-frame-options")
end
@doc """
If the `route_token` query parameter matches the `route_token` setting, this plug does nothing.
Otherwise, it sends a 401 response.
"""
def token_protected_route(%{query_params: %{"route_token" => route_token}} = conn, _opts) do
if Settings.get!(:route_token) == route_token do
conn
else
send_unauthorized(conn)
end
end
def token_protected_route(conn, _opts) do
send_unauthorized(conn)
end
defp credential_set?(credential) do
credential && credential != ""
end
defp send_unauthorized(conn) do
conn
|> send_resp(:unauthorized, "Unauthorized")
|> halt()
end
end
+18 -41
View File
@@ -1,5 +1,6 @@
defmodule PinchflatWeb.Router do
use PinchflatWeb, :router
import PinchflatWeb.Plugs
import Phoenix.LiveDashboard.Router
# IMPORTANT: `strip_trailing_extension` in endpoint.ex removes
@@ -19,8 +20,23 @@ defmodule PinchflatWeb.Router do
plug :accepts, ["json"]
end
pipeline :feeds do
plug :maybe_basic_auth
scope "/", PinchflatWeb do
pipe_through [:maybe_basic_auth, :token_protected_route]
# has to match before /sources/:id
get "/sources/opml", Podcasts.PodcastController, :opml_feed
end
# Routes in here _may not be_ protected by basic auth. This is necessary for
# media streaming to work for RSS podcast feeds.
scope "/", PinchflatWeb do
pipe_through :maybe_basic_auth
get "/sources/:uuid/feed", Podcasts.PodcastController, :rss_feed
get "/sources/:uuid/feed_image", Podcasts.PodcastController, :feed_image
get "/media/:uuid/episode_image", Podcasts.PodcastController, :episode_image
get "/media/:uuid/stream", MediaItems.MediaItemController, :stream
end
scope "/", PinchflatWeb do
@@ -48,18 +64,6 @@ defmodule PinchflatWeb.Router do
end
end
# Routes in here _may not be_ protected by basic auth. This is necessary for
# media streaming to work for RSS podcast feeds.
scope "/", PinchflatWeb do
pipe_through :feeds
get "/sources/:uuid/feed", Podcasts.PodcastController, :rss_feed
get "/sources/:uuid/feed_image", Podcasts.PodcastController, :feed_image
get "/media/:uuid/episode_image", Podcasts.PodcastController, :episode_image
get "/media/:uuid/stream", MediaItems.MediaItemController, :stream
end
# No auth or CSRF protection for the health check endpoint
scope "/", PinchflatWeb do
pipe_through :api
@@ -74,31 +78,4 @@ defmodule PinchflatWeb.Router do
metrics: PinchflatWeb.Telemetry,
ecto_repos: [Pinchflat.Repo]
end
defp maybe_basic_auth(conn, opts) do
if Application.get_env(:pinchflat, :expose_feed_endpoints) do
conn
else
basic_auth(conn, opts)
end
end
defp basic_auth(conn, _opts) do
username = Application.get_env(:pinchflat, :basic_auth_username)
password = Application.get_env(:pinchflat, :basic_auth_password)
if credential_set?(username) && credential_set?(password) do
Plug.BasicAuth.basic_auth(conn, username: username, password: password, realm: "Pinchflat")
else
conn
end
end
defp credential_set?(credential) do
credential && credential != ""
end
defp allow_iframe_embed(conn, _opts) do
delete_resp_header(conn, "x-frame-options")
end
end
+7 -6
View File
@@ -4,7 +4,7 @@ defmodule Pinchflat.MixProject do
def project do
[
app: :pinchflat,
version: "2024.10.2",
version: "2025.1.14",
elixir: "~> 1.17",
elixirc_paths: elixirc_paths(Mix.env()),
start_permanent: Mix.env() == :prod,
@@ -54,15 +54,15 @@ defmodule Pinchflat.MixProject do
{:ecto_sqlite3_extras, "~> 1.2.0"},
{:phoenix_html, "~> 3.3"},
{:phoenix_live_reload, "~> 1.2", only: :dev},
{:phoenix_live_view, "~> 0.20.1"},
{:floki, ">= 0.30.0", only: :test},
{:phoenix_live_view, "~> 1.0.0"},
{:floki, ">= 0.36.0", only: :test},
{:phoenix_live_dashboard, "~> 0.8.2"},
{:esbuild, "~> 0.8", runtime: Mix.env() == :dev},
{:tailwind, "~> 0.2.0", runtime: Mix.env() == :dev},
{:swoosh, "~> 1.3"},
{:finch, "~> 0.13"},
{:telemetry_metrics, "~> 0.6"},
{:telemetry_poller, "~> 1.0"},
{:finch, "~> 0.18"},
{:telemetry_metrics, "~> 1.0"},
{:telemetry_poller, "~> 1.1"},
{:gettext, "~> 0.20"},
{:jason, "~> 1.2"},
{:dns_cluster, "~> 0.1.1"},
@@ -70,6 +70,7 @@ defmodule Pinchflat.MixProject do
{:oban, "~> 2.17"},
{:nimble_parsec, "~> 1.4"},
{:timex, "~> 3.0"},
{:prom_ex, "~> 1.11.0"},
{:mox, "~> 1.0", only: :test},
{:credo, "~> 1.7.7", only: [:dev, :test], runtime: false},
{:credo_naming, "~> 2.1", only: [:dev, :test], runtime: false},
+17 -13
View File
@@ -1,6 +1,6 @@
%{
"bunt": {:hex, :bunt, "1.0.0", "081c2c665f086849e6d57900292b3a161727ab40431219529f13c4ddcf3e7a44", [:mix], [], "hexpm", "dc5f86aa08a5f6fa6b8096f0735c4e76d54ae5c9fa2c143e5a1fc7c1cd9bb6b5"},
"castore": {:hex, :castore, "1.0.8", "dedcf20ea746694647f883590b82d9e96014057aff1d44d03ec90f36a5c0dc6e", [:mix], [], "hexpm", "0b2b66d2ee742cb1d9cb8c8be3b43c3a70ee8651f37b75a8b982e036752983f1"},
"castore": {:hex, :castore, "1.0.11", "4bbd584741601eb658007339ea730b082cc61f3554cf2e8f39bf693a11b49073", [:mix], [], "hexpm", "e03990b4db988df56262852f20de0f659871c35154691427a5047f4967a16a62"},
"cc_precompiler": {:hex, :cc_precompiler, "0.1.10", "47c9c08d8869cf09b41da36538f62bc1abd3e19e41701c2cea2675b53c704258", [:mix], [{:elixir_make, "~> 0.7", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "f6e046254e53cd6b41c6bacd70ae728011aa82b2742a80d6e2214855c6e06b22"},
"certifi": {:hex, :certifi, "2.12.0", "2d1cca2ec95f59643862af91f001478c9863c2ac9cb6e2f89780bfd8de987329", [:rebar3], [], "hexpm", "ee68d85df22e554040cdb4be100f33873ac6051387baf6a8f6ce82272340ff1c"},
"combine": {:hex, :combine, "0.10.0", "eff8224eeb56498a2af13011d142c5e7997a80c8f5b97c499f84c841032e429f", [:mix], [], "hexpm", "1b1dbc1790073076580d0d1d64e42eae2366583e7aecd455d1215b0d16f2451b"},
@@ -10,7 +10,7 @@
"credo": {:hex, :credo, "1.7.7", "771445037228f763f9b2afd612b6aa2fd8e28432a95dbbc60d8e03ce71ba4446", [:mix], [{:bunt, "~> 0.2.1 or ~> 1.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:file_system, "~> 0.2 or ~> 1.0", [hex: :file_system, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "8bc87496c9aaacdc3f90f01b7b0582467b69b4bd2441fe8aae3109d843cc2f2e"},
"credo_naming": {:hex, :credo_naming, "2.1.0", "d44ad58890d4db552e141ce64756a74ac1573665af766d1ac64931aa90d47744", [:make, :mix], [{:credo, "~> 1.6", [hex: :credo, repo: "hexpm", optional: false]}], "hexpm", "830e23b3fba972e2fccec49c0c089fe78c1e64bc16782a2682d78082351a2909"},
"db_connection": {:hex, :db_connection, "2.7.0", "b99faa9291bb09892c7da373bb82cba59aefa9b36300f6145c5f201c7adf48ec", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "dcf08f31b2701f857dfc787fbad78223d61a32204f217f15e881dd93e4bdd3ff"},
"decimal": {:hex, :decimal, "2.1.1", "5611dca5d4b2c3dd497dec8f68751f1f1a54755e8ed2a966c2633cf885973ad6", [:mix], [], "hexpm", "53cfe5f497ed0e7771ae1a475575603d77425099ba5faef9394932b35020ffcc"},
"decimal": {:hex, :decimal, "2.2.0", "df3d06bb9517e302b1bd265c1e7f16cda51547ad9d99892049340841f3e15836", [:mix], [], "hexpm", "af8daf87384b51b7e611fb1a1f2c4d4876b65ef968fa8bd3adf44cff401c7f21"},
"dns_cluster": {:hex, :dns_cluster, "0.1.2", "3eb5be824c7888dadf9781018e1a5f1d3d1113b333c50bce90fb1b83df1015f2", [:mix], [], "hexpm", "7494272040f847637bbdb01bcdf4b871e82daf09b813e7d3cb3b84f112c6f2f8"},
"ecto": {:hex, :ecto, "3.12.3", "1a9111560731f6c3606924c81c870a68a34c819f6d4f03822f370ea31a582208", [:mix], [{:decimal, "~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "9efd91506ae722f95e48dc49e70d0cb632ede3b7a23896252a60a14ac6d59165"},
"ecto_sql": {:hex, :ecto_sql, "3.12.0", "73cea17edfa54bde76ee8561b30d29ea08f630959685006d9c6e7d1e59113b7d", [:mix], [{:db_connection, "~> 2.4.1 or ~> 2.5", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.12", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.7", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.19 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.1 or ~> 2.2", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "dc9e4d206f274f3947e96142a8fdc5f69a2a6a9abb4649ef5c882323b6d512f0"},
@@ -23,34 +23,37 @@
"exqlite": {:hex, :exqlite, "0.23.0", "6e851c937a033299d0784994c66da24845415072adbc455a337e20087bce9033", [:make, :mix], [{:cc_precompiler, "~> 0.1", [hex: :cc_precompiler, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.8", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:table, "~> 0.1.0", [hex: :table, repo: "hexpm", optional: true]}], "hexpm", "404341cceec5e6466aaed160cf0b58be2019b60af82588c215e1224ebd3ec831"},
"faker": {:hex, :faker, "0.17.0", "671019d0652f63aefd8723b72167ecdb284baf7d47ad3a82a15e9b8a6df5d1fa", [:mix], [], "hexpm", "a7d4ad84a93fd25c5f5303510753789fc2433ff241bf3b4144d3f6f291658a6a"},
"file_system": {:hex, :file_system, "0.2.10", "fb082005a9cd1711c05b5248710f8826b02d7d1784e7c3451f9c1231d4fc162d", [:mix], [], "hexpm", "41195edbfb562a593726eda3b3e8b103a309b733ad25f3d642ba49696bf715dc"},
"finch": {:hex, :finch, "0.17.0", "17d06e1d44d891d20dbd437335eebe844e2426a0cd7e3a3e220b461127c73f70", [:mix], [{:castore, "~> 0.1 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: false]}, {:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.3", [hex: :mint, repo: "hexpm", optional: false]}, {:nimble_options, "~> 0.4 or ~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:nimble_pool, "~> 0.2.6 or ~> 1.0", [hex: :nimble_pool, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "8d014a661bb6a437263d4b5abf0bcbd3cf0deb26b1e8596f2a271d22e48934c7"},
"floki": {:hex, :floki, "0.35.2", "87f8c75ed8654b9635b311774308b2760b47e9a579dabf2e4d5f1e1d42c39e0b", [:mix], [], "hexpm", "6b05289a8e9eac475f644f09c2e4ba7e19201fd002b89c28c1293e7bd16773d9"},
"finch": {:hex, :finch, "0.19.0", "c644641491ea854fc5c1bbaef36bfc764e3f08e7185e1f084e35e0672241b76d", [:mix], [{:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.6.2 or ~> 1.7", [hex: :mint, repo: "hexpm", optional: false]}, {:nimble_options, "~> 0.4 or ~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:nimble_pool, "~> 1.1", [hex: :nimble_pool, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "fc5324ce209125d1e2fa0fcd2634601c52a787aff1cd33ee833664a5af4ea2b6"},
"floki": {:hex, :floki, "0.36.3", "1102f93b16a55bc5383b85ae3ec470f82dee056eaeff9195e8afdf0ef2a43c30", [:mix], [], "hexpm", "fe0158bff509e407735f6d40b3ee0d7deb47f3f3ee7c6c182ad28599f9f6b27a"},
"gettext": {:hex, :gettext, "0.24.0", "6f4d90ac5f3111673cbefc4ebee96fe5f37a114861ab8c7b7d5b30a1108ce6d8", [:mix], [{:expo, "~> 0.5.1", [hex: :expo, repo: "hexpm", optional: false]}], "hexpm", "bdf75cdfcbe9e4622dd18e034b227d77dd17f0f133853a1c73b97b3d6c770e8b"},
"hackney": {:hex, :hackney, "1.20.1", "8d97aec62ddddd757d128bfd1df6c5861093419f8f7a4223823537bad5d064e2", [:rebar3], [{:certifi, "~> 2.12.0", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "~> 6.1.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "~> 1.0.0", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~> 1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:parse_trans, "3.4.1", [hex: :parse_trans, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~> 1.1.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}, {:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "fe9094e5f1a2a2c0a7d10918fee36bfec0ec2a979994cff8cfe8058cd9af38e3"},
"hpax": {:hex, :hpax, "0.1.2", "09a75600d9d8bbd064cdd741f21fc06fc1f4cf3d0fcc335e5aa19be1a7235c84", [:mix], [], "hexpm", "2c87843d5a23f5f16748ebe77969880e29809580efdaccd615cd3bed628a8c13"},
"hpax": {:hex, :hpax, "1.0.2", "762df951b0c399ff67cc57c3995ec3cf46d696e41f0bba17da0518d94acd4aac", [:mix], [], "hexpm", "2f09b4c1074e0abd846747329eaa26d535be0eb3d189fa69d812bfb8bfefd32f"},
"idna": {:hex, :idna, "6.1.1", "8a63070e9f7d0c62eb9d9fcb360a7de382448200fbbd1b106cc96d3d8099df8d", [:rebar3], [{:unicode_util_compat, "~> 0.7.0", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm", "92376eb7894412ed19ac475e4a86f7b413c1b9fbb5bd16dccd57934157944cea"},
"jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"},
"metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm", "69b09adddc4f74a40716ae54d140f93beb0fb8978d8636eaded0c31b6f099f16"},
"mime": {:hex, :mime, "2.0.6", "8f18486773d9b15f95f4f4f1e39b710045fa1de891fada4516559967276e4dc2", [:mix], [], "hexpm", "c9945363a6b26d747389aac3643f8e0e09d30499a138ad64fe8fd1d13d9b153e"},
"mimerl": {:hex, :mimerl, "1.3.0", "d0cd9fc04b9061f82490f6581e0128379830e78535e017f7780f37fea7545726", [:rebar3], [], "hexpm", "a1e15a50d1887217de95f0b9b0793e32853f7c258a5cd227650889b38839fe9d"},
"mint": {:hex, :mint, "1.5.2", "4805e059f96028948870d23d7783613b7e6b0e2fb4e98d720383852a760067fd", [:mix], [{:castore, "~> 0.1.0 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:hpax, "~> 0.1.1", [hex: :hpax, repo: "hexpm", optional: false]}], "hexpm", "d77d9e9ce4eb35941907f1d3df38d8f750c357865353e21d335bdcdf6d892a02"},
"mint": {:hex, :mint, "1.6.2", "af6d97a4051eee4f05b5500671d47c3a67dac7386045d87a904126fd4bbcea2e", [:mix], [{:castore, "~> 0.1.0 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:hpax, "~> 0.1.1 or ~> 0.2.0 or ~> 1.0", [hex: :hpax, repo: "hexpm", optional: false]}], "hexpm", "5ee441dffc1892f1ae59127f74afe8fd82fda6587794278d924e4d90ea3d63f9"},
"mox": {:hex, :mox, "1.1.0", "0f5e399649ce9ab7602f72e718305c0f9cdc351190f72844599545e4996af73c", [:mix], [], "hexpm", "d44474c50be02d5b72131070281a5d3895c0e7a95c780e90bc0cfe712f633a13"},
"nimble_options": {:hex, :nimble_options, "1.1.0", "3b31a57ede9cb1502071fade751ab0c7b8dbe75a9a4c2b5bbb0943a690b63172", [:mix], [], "hexpm", "8bbbb3941af3ca9acc7835f5655ea062111c9c27bcac53e004460dfd19008a99"},
"nimble_options": {:hex, :nimble_options, "1.1.1", "e3a492d54d85fc3fd7c5baf411d9d2852922f66e69476317787a7b2bb000a61b", [:mix], [], "hexpm", "821b2470ca9442c4b6984882fe9bb0389371b8ddec4d45a9504f00a66f650b44"},
"nimble_parsec": {:hex, :nimble_parsec, "1.4.0", "51f9b613ea62cfa97b25ccc2c1b4216e81df970acd8e16e8d1bdc58fef21370d", [:mix], [], "hexpm", "9c565862810fb383e9838c1dd2d7d2c437b3d13b267414ba6af33e50d2d1cf28"},
"nimble_pool": {:hex, :nimble_pool, "1.0.0", "5eb82705d138f4dd4423f69ceb19ac667b3b492ae570c9f5c900bb3d2f50a847", [:mix], [], "hexpm", "80be3b882d2d351882256087078e1b1952a28bf98d0a287be87e4a24a710b67a"},
"nimble_pool": {:hex, :nimble_pool, "1.1.0", "bf9c29fbdcba3564a8b800d1eeb5a3c58f36e1e11d7b7fb2e084a643f645f06b", [:mix], [], "hexpm", "af2e4e6b34197db81f7aad230c1118eac993acc0dae6bc83bac0126d4ae0813a"},
"oban": {:hex, :oban, "2.18.3", "1608c04f8856c108555c379f2f56bc0759149d35fa9d3b825cb8a6769f8ae926", [:mix], [{:ecto_sql, "~> 3.10", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:ecto_sqlite3, "~> 0.9", [hex: :ecto_sqlite3, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: false]}, {:postgrex, "~> 0.16", [hex: :postgrex, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "36ca6ca84ef6518f9c2c759ea88efd438a3c81d667ba23b02b062a0aa785475e"},
"octo_fetch": {:hex, :octo_fetch, "0.4.0", "074b5ecbc08be10b05b27e9db08bc20a3060142769436242702931c418695b19", [:mix], [{:castore, "~> 0.1 or ~> 1.0", [hex: :castore, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "~> 1.1", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm", "cf8be6f40cd519d7000bb4e84adcf661c32e59369ca2827c4e20042eda7a7fc6"},
"parse_trans": {:hex, :parse_trans, "3.4.1", "6e6aa8167cb44cc8f39441d05193be6e6f4e7c2946cb2759f015f8c56b76e5ff", [:rebar3], [], "hexpm", "620a406ce75dada827b82e453c19cf06776be266f5a67cff34e1ef2cbb60e49a"},
"phoenix": {:hex, :phoenix, "1.7.14", "a7d0b3f1bc95987044ddada111e77bd7f75646a08518942c72a8440278ae7825", [:mix], [{:castore, ">= 0.0.0", [hex: :castore, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.7", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:plug_crypto, "~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:websock_adapter, "~> 0.5.3", [hex: :websock_adapter, repo: "hexpm", optional: false]}], "hexpm", "c7859bc56cc5dfef19ecfc240775dae358cbaa530231118a9e014df392ace61a"},
"peep": {:hex, :peep, "3.4.1", "0e5263710fa0b42675bd0a11fdcdd3ee4f484e319105b6ad9a576c91a5d3cb55", [:mix], [{:nimble_options, "~> 1.1", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:plug, "~> 1.16", [hex: :plug, repo: "hexpm", optional: true]}, {:telemetry_metrics, "~> 1.0", [hex: :telemetry_metrics, repo: "hexpm", optional: false]}], "hexpm", "7a9b8c1f17b8b9475efb27b7048afa4d89ab84ef33a3d1df13696c85c12cd632"},
"phoenix": {:hex, :phoenix, "1.7.17", "2fcdceecc6fb90bec26fab008f96abbd0fd93bc9956ec7985e5892cf545152ca", [:mix], [{:castore, ">= 0.0.0", [hex: :castore, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.7", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:plug_crypto, "~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:websock_adapter, "~> 0.5.3", [hex: :websock_adapter, repo: "hexpm", optional: false]}], "hexpm", "50e8ad537f3f7b0efb1509b2f75b5c918f697be6a45d48e49a30d3b7c0e464c9"},
"phoenix_ecto": {:hex, :phoenix_ecto, "4.4.3", "86e9878f833829c3f66da03d75254c155d91d72a201eb56ae83482328dc7ca93", [:mix], [{:ecto, "~> 3.5", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.14.2 or ~> 3.0 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "d36c401206f3011fefd63d04e8ef626ec8791975d9d107f9a0817d426f61ac07"},
"phoenix_html": {:hex, :phoenix_html, "3.3.4", "42a09fc443bbc1da37e372a5c8e6755d046f22b9b11343bf885067357da21cb3", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: true]}], "hexpm", "0249d3abec3714aff3415e7ee3d9786cb325be3151e6c4b3021502c585bf53fb"},
"phoenix_live_dashboard": {:hex, :phoenix_live_dashboard, "0.8.3", "7ff51c9b6609470f681fbea20578dede0e548302b0c8bdf338b5a753a4f045bf", [:mix], [{:ecto, "~> 3.6.2 or ~> 3.7", [hex: :ecto, repo: "hexpm", optional: true]}, {:ecto_mysql_extras, "~> 0.5", [hex: :ecto_mysql_extras, repo: "hexpm", optional: true]}, {:ecto_psql_extras, "~> 0.7", [hex: :ecto_psql_extras, repo: "hexpm", optional: true]}, {:ecto_sqlite3_extras, "~> 1.1.7 or ~> 1.2.0", [hex: :ecto_sqlite3_extras, repo: "hexpm", optional: true]}, {:mime, "~> 1.6 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:phoenix_live_view, "~> 0.19 or ~> 1.0", [hex: :phoenix_live_view, repo: "hexpm", optional: false]}, {:telemetry_metrics, "~> 0.6 or ~> 1.0", [hex: :telemetry_metrics, repo: "hexpm", optional: false]}], "hexpm", "f9470a0a8bae4f56430a23d42f977b5a6205fdba6559d76f932b876bfaec652d"},
"phoenix_live_reload": {:hex, :phoenix_live_reload, "1.4.1", "2aff698f5e47369decde4357ba91fc9c37c6487a512b41732818f2204a8ef1d3", [:mix], [{:file_system, "~> 0.2.1 or ~> 0.3", [hex: :file_system, repo: "hexpm", optional: false]}, {:phoenix, "~> 1.4", [hex: :phoenix, repo: "hexpm", optional: false]}], "hexpm", "9bffb834e7ddf08467fe54ae58b5785507aaba6255568ae22b4d46e2bb3615ab"},
"phoenix_live_view": {:hex, :phoenix_live_view, "0.20.3", "8b6406bc0a451f295407d7acff7f234a6314be5bbe0b3f90ed82b07f50049878", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix, "~> 1.6.15 or ~> 1.7.0", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 3.3 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.15", [hex: :plug, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.2 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "a8e4385e05618b424779f894ed2df97d3c7518b7285fcd11979077ae6226466b"},
"phoenix_live_view": {:hex, :phoenix_live_view, "1.0.0", "3a10dfce8f87b2ad4dc65de0732fc2a11e670b2779a19e8d3281f4619a85bce4", [:mix], [{:floki, "~> 0.36", [hex: :floki, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix, "~> 1.6.15 or ~> 1.7.0", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 3.3 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.15", [hex: :plug, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.2 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "254caef0028765965ca6bd104cc7d68dcc7d57cc42912bef92f6b03047251d99"},
"phoenix_pubsub": {:hex, :phoenix_pubsub, "2.1.3", "3168d78ba41835aecad272d5e8cd51aa87a7ac9eb836eabc42f6e57538e3731d", [:mix], [], "hexpm", "bba06bc1dcfd8cb086759f0edc94a8ba2bc8896d5331a1e2c2902bf8e36ee502"},
"phoenix_template": {:hex, :phoenix_template, "1.0.4", "e2092c132f3b5e5b2d49c96695342eb36d0ed514c5b252a77048d5969330d639", [:mix], [{:phoenix_html, "~> 2.14.2 or ~> 3.0 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: true]}], "hexpm", "2c0c81f0e5c6753faf5cca2f229c9709919aba34fab866d3bc05060c9c444206"},
"plug": {:hex, :plug, "1.16.1", "40c74619c12f82736d2214557dedec2e9762029b2438d6d175c5074c933edc9d", [:mix], [{:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.1.1 or ~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.3 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "a13ff6b9006b03d7e33874945b2755253841b238c34071ed85b0e86057f8cddc"},
"plug_cowboy": {:hex, :plug_cowboy, "2.7.2", "fdadb973799ae691bf9ecad99125b16625b1c6039999da5fe544d99218e662e4", [:mix], [{:cowboy, "~> 2.7", [hex: :cowboy, repo: "hexpm", optional: false]}, {:cowboy_telemetry, "~> 0.3", [hex: :cowboy_telemetry, repo: "hexpm", optional: false]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "245d8a11ee2306094840c000e8816f0cbed69a23fc0ac2bcf8d7835ae019bb2f"},
"plug_crypto": {:hex, :plug_crypto, "2.1.0", "f44309c2b06d249c27c8d3f65cfe08158ade08418cf540fd4f72d4d6863abb7b", [:mix], [], "hexpm", "131216a4b030b8f8ce0f26038bc4421ae60e4bb95c5cf5395e1421437824c4fa"},
"prom_ex": {:hex, :prom_ex, "1.11.0", "1f6d67f2dead92224cb4f59beb3e4d319257c5728d9638b4a5e8ceb51a4f9c7e", [:mix], [{:absinthe, ">= 1.7.0", [hex: :absinthe, repo: "hexpm", optional: true]}, {:broadway, ">= 1.1.0", [hex: :broadway, repo: "hexpm", optional: true]}, {:ecto, ">= 3.11.0", [hex: :ecto, repo: "hexpm", optional: true]}, {:finch, "~> 0.18", [hex: :finch, repo: "hexpm", optional: false]}, {:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: false]}, {:oban, ">= 2.10.0", [hex: :oban, repo: "hexpm", optional: true]}, {:octo_fetch, "~> 0.4", [hex: :octo_fetch, repo: "hexpm", optional: false]}, {:peep, "~> 3.0", [hex: :peep, repo: "hexpm", optional: false]}, {:phoenix, ">= 1.7.0", [hex: :phoenix, repo: "hexpm", optional: true]}, {:phoenix_live_view, ">= 0.20.0", [hex: :phoenix_live_view, repo: "hexpm", optional: true]}, {:plug, ">= 1.16.0", [hex: :plug, repo: "hexpm", optional: true]}, {:plug_cowboy, ">= 2.6.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, ">= 1.0.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:telemetry_metrics, "~> 1.0", [hex: :telemetry_metrics, repo: "hexpm", optional: false]}, {:telemetry_metrics_prometheus_core, "~> 1.2", [hex: :telemetry_metrics_prometheus_core, repo: "hexpm", optional: false]}, {:telemetry_poller, "~> 1.1", [hex: :telemetry_poller, repo: "hexpm", optional: false]}], "hexpm", "76b074bc3730f0802978a7eb5c7091a65473eaaf07e99ec9e933138dcc327805"},
"ranch": {:hex, :ranch, "1.8.0", "8c7a100a139fd57f17327b6413e4167ac559fbc04ca7448e9be9057311597a1d", [:make, :rebar3], [], "hexpm", "49fbcfd3682fab1f5d109351b61257676da1a2fdbe295904176d5e521a2ddfe5"},
"sobelow": {:hex, :sobelow, "0.13.0", "218afe9075904793f5c64b8837cc356e493d88fddde126a463839351870b8d1e", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "cd6e9026b85fc35d7529da14f95e85a078d9dd1907a9097b3ba6ac7ebbe34a0d"},
"ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.7", "354c321cf377240c7b8716899e182ce4890c5938111a1296add3ec74cf1715df", [:make, :mix, :rebar3], [], "hexpm", "fe4c190e8f37401d30167c8c405eda19469f34577987c76dde613e838bbc67f8"},
@@ -58,11 +61,12 @@
"table_rex": {:hex, :table_rex, "4.0.0", "3c613a68ebdc6d4d1e731bc973c233500974ec3993c99fcdabb210407b90959b", [:mix], [], "hexpm", "c35c4d5612ca49ebb0344ea10387da4d2afe278387d4019e4d8111e815df8f55"},
"tailwind": {:hex, :tailwind, "0.2.2", "9e27288b568ede1d88517e8c61259bc214a12d7eed271e102db4c93fcca9b2cd", [:mix], [{:castore, ">= 0.0.0", [hex: :castore, repo: "hexpm", optional: false]}], "hexpm", "ccfb5025179ea307f7f899d1bb3905cd0ac9f687ed77feebc8f67bdca78565c4"},
"telemetry": {:hex, :telemetry, "1.3.0", "fedebbae410d715cf8e7062c96a1ef32ec22e764197f70cda73d82778d61e7a2", [:rebar3], [], "hexpm", "7015fc8919dbe63764f4b4b87a95b7c0996bd539e0d499be6ec9d7f3875b79e6"},
"telemetry_metrics": {:hex, :telemetry_metrics, "0.6.2", "2caabe9344ec17eafe5403304771c3539f3b6e2f7fb6a6f602558c825d0d0bfb", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "9b43db0dc33863930b9ef9d27137e78974756f5f198cae18409970ed6fa5b561"},
"telemetry_poller": {:hex, :telemetry_poller, "1.0.0", "db91bb424e07f2bb6e73926fcafbfcbcb295f0193e0a00e825e589a0a47e8453", [:rebar3], [{:telemetry, "~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "b3a24eafd66c3f42da30fc3ca7dda1e9d546c12250a2d60d7b81d264fbec4f6e"},
"telemetry_metrics": {:hex, :telemetry_metrics, "1.0.0", "29f5f84991ca98b8eb02fc208b2e6de7c95f8bb2294ef244a176675adc7775df", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "f23713b3847286a534e005126d4c959ebcca68ae9582118ce436b521d1d47d5d"},
"telemetry_metrics_prometheus_core": {:hex, :telemetry_metrics_prometheus_core, "1.2.1", "c9755987d7b959b557084e6990990cb96a50d6482c683fb9622a63837f3cd3d8", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:telemetry_metrics, "~> 0.6 or ~> 1.0", [hex: :telemetry_metrics, repo: "hexpm", optional: false]}], "hexpm", "5e2c599da4983c4f88a33e9571f1458bf98b0cf6ba930f1dc3a6e8cf45d5afb6"},
"telemetry_poller": {:hex, :telemetry_poller, "1.1.0", "58fa7c216257291caaf8d05678c8d01bd45f4bdbc1286838a28c4bb62ef32999", [:rebar3], [{:telemetry, "~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "9eb9d9cbfd81cbd7cdd24682f8711b6e2b691289a0de6826e58452f28c103c8f"},
"timex": {:hex, :timex, "3.7.11", "bb95cb4eb1d06e27346325de506bcc6c30f9c6dea40d1ebe390b262fad1862d1", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.20", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 1.1", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm", "8b9024f7efbabaf9bd7aa04f65cf8dcd7c9818ca5737677c7b76acbc6a94d1aa"},
"tzdata": {:hex, :tzdata, "1.1.2", "45e5f1fcf8729525ec27c65e163be5b3d247ab1702581a94674e008413eef50b", [:mix], [{:hackney, "~> 1.17", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "cec7b286e608371602318c414f344941d5eb0375e14cfdab605cca2fe66cba8b"},
"unicode_util_compat": {:hex, :unicode_util_compat, "0.7.0", "bc84380c9ab48177092f43ac89e4dfa2c6d62b40b8bd132b1059ecc7232f9a78", [:rebar3], [], "hexpm", "25eee6d67df61960cf6a794239566599b09e17e668d3700247bc498638152521"},
"websock": {:hex, :websock, "0.5.3", "2f69a6ebe810328555b6fe5c831a851f485e303a7c8ce6c5f675abeb20ebdadc", [:mix], [], "hexpm", "6105453d7fac22c712ad66fab1d45abdf049868f253cf719b625151460b8b453"},
"websock_adapter": {:hex, :websock_adapter, "0.5.7", "65fa74042530064ef0570b75b43f5c49bb8b235d6515671b3d250022cb8a1f9e", [:mix], [{:bandit, ">= 0.6.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.6", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "d0f478ee64deddfec64b800673fd6e0c8888b079d9f3444dd96d2a98383bdbd1"},
"websock_adapter": {:hex, :websock_adapter, "0.5.8", "3b97dc94e407e2d1fc666b2fb9acf6be81a1798a2602294aac000260a7c4a47d", [:mix], [{:bandit, ">= 0.6.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.6", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "315b9a1865552212b5f35140ad194e67ce31af45bcee443d4ecb96b5fd3f3782"},
}
+607
View File
@@ -0,0 +1,607 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
},
{
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "#73BF69",
"limit": 100,
"name": "PromEx service start",
"showIn": 0,
"tags": ["prom_ex", "pinchflat", "start"],
"type": "tags"
},
{
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "#FF9830",
"limit": 100,
"name": "PromEx service stop",
"showIn": 0,
"tags": ["prom_ex", "pinchflat", "stop"],
"type": "tags"
}
]
},
"description": "All the data that is presented here is captured by the PromEx Application plugin (https://github.com/akoutmos/prom_ex/blob/master/lib/prom_ex/plugins/application.ex)",
"editable": false,
"gnetId": null,
"graphTooltip": 1,
"id": null,
"links": [
{
"asDropdown": false,
"icon": "bolt",
"includeVars": false,
"keepTime": false,
"tags": [],
"targetBlank": true,
"title": "Sponsor PromEx",
"tooltip": "",
"type": "link",
"url": "https://github.com/sponsors/akoutmos"
},
{
"asDropdown": false,
"icon": "doc",
"includeVars": false,
"keepTime": false,
"tags": [],
"targetBlank": true,
"title": "Application Plugin Docs",
"tooltip": "",
"type": "link",
"url": "https://hexdocs.pm/prom_ex/PromEx.Plugins.Application.html"
}
],
"panels": [
{
"datasource": "prometheus",
"description": "The amount of time that the application has been running.",
"fieldConfig": {
"defaults": {
"custom": {},
"decimals": 1,
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "dtdurationms"
},
"overrides": []
},
"gridPos": {
"h": 6,
"w": 8,
"x": 0,
"y": 0
},
"id": 6,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": ["last"],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "7.1.3",
"targets": [
{
"expr": "pinchflat_prom_ex_application_uptime_milliseconds_count{job=\"$job\", instance=\"$instance\"}",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Application Uptime",
"type": "stat"
},
{
"datasource": "prometheus",
"description": "The data is populated by the PromEx Application plugin and provides information regarding your application's dependencies.",
"fieldConfig": {
"defaults": {
"custom": {
"align": "left",
"displayMode": "auto"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
}
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "Status"
},
"properties": [
{
"id": "custom.displayMode",
"value": "color-background"
},
{
"id": "mappings",
"value": [
{
"from": "",
"id": 0,
"text": "Started",
"to": "",
"type": 1,
"value": "1"
},
{
"from": "",
"id": 1,
"text": "Loaded",
"to": "",
"type": 1,
"value": "0"
}
]
},
{
"id": "custom.align",
"value": "center"
},
{
"id": "custom.width",
"value": 202
}
]
},
{
"matcher": {
"id": "byName",
"options": "Name"
},
"properties": [
{
"id": "custom.width",
"value": 349
}
]
},
{
"matcher": {
"id": "byName",
"options": "Version"
},
"properties": [
{
"id": "custom.width",
"value": 187
}
]
}
]
},
"gridPos": {
"h": 36,
"w": 16,
"x": 8,
"y": 0
},
"id": 2,
"options": {
"showHeader": true,
"sortBy": [
{
"desc": false,
"displayName": "Name"
}
]
},
"pluginVersion": "7.1.3",
"targets": [
{
"expr": "pinchflat_prom_ex_application_dependency_info{job=\"$job\", instance=\"$instance\"}",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Dependency Information",
"transformations": [
{
"id": "organize",
"options": {
"excludeByName": {
"Time": true,
"__name__": true,
"instance": true,
"job": true,
"Value": true
},
"indexByName": {
"Time": 0,
"Value": 4,
"__name__": 1,
"instance": 2,
"job": 3,
"modules": 7,
"name": 5,
"version": 6
},
"renameByName": {
"Value": "Status",
"modules": "Number of Modules Loaded",
"name": "Name",
"version": "Version"
}
}
}
],
"type": "table"
},
{
"datasource": "prometheus",
"description": "The name of the primary application that is running.",
"fieldConfig": {
"defaults": {
"custom": {},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 6,
"w": 8,
"x": 0,
"y": 6
},
"id": 11,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": ["last"],
"fields": "/^name$/",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "7.1.3",
"targets": [
{
"expr": "pinchflat_prom_ex_application_primary_info{job=\"$job\", instance=\"$instance\"}",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Application Name",
"type": "stat"
},
{
"datasource": "prometheus",
"description": "The Git SHA of the application.",
"fieldConfig": {
"defaults": {
"custom": {},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 6,
"w": 8,
"x": 0,
"y": 12
},
"id": 10,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": ["last"],
"fields": "/^sha$/",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "7.1.3",
"targets": [
{
"expr": "pinchflat_prom_ex_application_git_sha_info{job=\"$job\", instance=\"$instance\"}",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Application Git SHA",
"type": "stat"
},
{
"datasource": "prometheus",
"description": "The author of the application's last Git commit.",
"fieldConfig": {
"defaults": {
"custom": {},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 6,
"w": 8,
"x": 0,
"y": 18
},
"id": 12,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": ["last"],
"fields": "/^author$/",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "7.1.3",
"targets": [
{
"expr": "pinchflat_prom_ex_application_git_author_info{job=\"$job\", instance=\"$instance\"}",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Application Git Author",
"type": "stat"
},
{
"datasource": "prometheus",
"description": "The version of the primary application that is running.",
"fieldConfig": {
"defaults": {
"custom": {},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 6,
"w": 8,
"x": 0,
"y": 24
},
"id": 7,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": ["last"],
"fields": "/^version$/",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "7.1.3",
"targets": [
{
"expr": "pinchflat_prom_ex_application_primary_info{job=\"$job\", instance=\"$instance\"}",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Application Version",
"type": "stat"
},
{
"datasource": "prometheus",
"description": "The number of modules loaded by the primary application that is running.",
"fieldConfig": {
"defaults": {
"custom": {},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 6,
"w": 8,
"x": 0,
"y": 30
},
"id": 9,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": ["last"],
"fields": "/^modules$/",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "7.1.3",
"targets": [
{
"expr": "pinchflat_prom_ex_application_primary_info{job=\"$job\", instance=\"$instance\"}",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Application Modules Loaded",
"type": "stat"
}
],
"refresh": "5s",
"schemaVersion": 26,
"style": "dark",
"tags": ["PromEx", "Application", "pinchflat"],
"templating": {
"list": [
{
"allValue": null,
"datasource": "prometheus",
"definition": "label_values(pinchflat_prom_ex_prom_ex_status_info, job)",
"hide": 0,
"includeAll": false,
"label": "Prometheus Job",
"multi": false,
"name": "job",
"options": [],
"query": "label_values(pinchflat_prom_ex_prom_ex_status_info, job)",
"refresh": 2,
"regex": "",
"skipUrlSync": false,
"sort": 6,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
},
{
"allValue": null,
"datasource": "prometheus",
"definition": "label_values(pinchflat_prom_ex_prom_ex_status_info, instance)",
"hide": 0,
"includeAll": false,
"label": "Application Instance",
"multi": false,
"name": "instance",
"options": [],
"query": "label_values(pinchflat_prom_ex_prom_ex_status_info{job=\"$job\"}, instance)",
"refresh": 2,
"regex": "",
"skipUrlSync": false,
"sort": 0,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
}
]
},
"time": {
"from": "now-1h",
"to": "now"
},
"timepicker": {
"refresh_intervals": ["5s", "10s", "30s", "1m", "5m"]
},
"timezone": "",
"title": "Pinchflat - PromEx Application Dashboard",
"uid": "7DBBC471C5775585391E8F24D1E62319",
"version": 1
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 433 KiB

After

Width:  |  Height:  |  Size: 443 KiB

Some files were not shown because too many files have changed in this diff Show More