Fix thread sync
Reformat Metal files Rewrite Metal render loop to not use DisplayLink so we can use the amazing afterMinimumDuration API call Improve frame drop algorithm Misc fixes to HDR <-> SDR switching, deprecation warning cleanup Fix AVSB mode Fix renderingBackend selected value Add Rendering Backend setting to iPhone Let users choose which renderer to use. Refactor metrics into a singleton Fix tvOS build Port moonlight-qt's Metal renderer Bump imgui to latest Misc tweaks and refactoring Switch iOS to use Opus float32 audio, in line with other clients. It was already being sent from the server as float32, so this is simply decoding it with less loss of dynamic range. Give FrameQueue a faster data structure and full test coverage. Remove pts frame pacing, it's not a fit for iOS. Don't allow very transparent opacity for middle stats overlay box Change wording to select button instead of remote button Add tvOS gesture for double-clicking remote button to show/hide stats Remove kVTDecodeFrame_1xRealTimePlayback, this was found to seriously impact decode times Move decode time stat out of async block Enabe FQ logging Misc cleanup/fixes iPad: 26 will require UIRequiresFullScreen to be false, and for all orientations to be supported Updated iPhone and iPad settings panels Add setting for graph opacity. Enable stats and graphs to be toggled via two finger swipe Fix wait time in displaylink, fix settings spacing, stats fixes Settings for frame queue size, graphs. Misc refactoring and cleanup Bump moonlight-common-c Improve logging readability Fix host frametime and frame queue stats to be recorded during input not output. Show min/max/avg decode time Add bytes per frame graph Started on settings changes, both modes working very well Misc refactoring, move graphs around, both pacing functions working well Set frameDropTarget to 2 always Use raw RTP timestamps without converting to floating point for even more accurate playback Standard frame pacing: remove +3 logic Switch to NSRunLoopCommonModes for displayLink Add frame pacing algorithm based on moonlight-qt's Pacer class as a simpler baseline Switch to os_unfair_lock, misc tweaks Large refactor that now uses VTDecompressionSession based on PR535 from felipejfc. This now allows more control over queue size and allows dropping arbitrary frames, but introduces a lot of new problems bump moonlight-common-c to fix 32-bit rollover issue LiWait timing graph Add displaylink interval graph Graphs checkpoint ImGui graph working checkpoint ImGui working checkpoint Add pending frames to stats, add a define for verbose logging DisplayLink WIP Add imgui & implot modules Frame pacing: alternate method based on thieman's pullFrames thread Frame pacing: simplify displayLink, support passing targetTimestamp down to the sample buffer Point to my common-c with some timing changes. Add stats gestures: 2-finger swipe down to show stats, and 2-finger swipe up to hide Limit refresh rate to 60 when in low power mode Display screen refresh rate in stats overlay, remove decode time and explain why it can't be measured Adjust the refresh rate of the display to match the stream framerate and improve battery life Don't use presentationTimeMs when creating CMSampleTimingInfo Fix overlayView build for iOS Refactor CADisplayLink callback for improved frame delivery. Support a user-configurable buffered frame count. [opt] Use UILabel for stats overlay
This commit is contained in:
Travis Thieman
Acaki
@@ -0,0 +1,21 @@
|
||||
BasedOnStyle: Chromium
|
||||
AlignAfterOpenBracket: true
|
||||
AlignOperands: false
|
||||
AllowAllParametersOfDeclarationOnNextLine: true
|
||||
AlwaysBreakTemplateDeclarations: true
|
||||
BinPackArguments: false
|
||||
BinPackParameters: false
|
||||
BreakBeforeTernaryOperators: true
|
||||
ColumnLimit: 160
|
||||
ContinuationIndentWidth: 4
|
||||
DerivePointerAlignment: false
|
||||
IndentWidth: 4
|
||||
ObjCBlockIndentWidth: 4
|
||||
ObjCSpaceAfterProperty: true
|
||||
ObjCSpaceBeforeProtocolList: true
|
||||
PointerBindsToType: false
|
||||
PointerAlignment: Right
|
||||
SpacesInContainerLiterals: true
|
||||
TabWidth: 4
|
||||
UseTab: Never
|
||||
|
||||
@@ -4,3 +4,6 @@ url=https://github.com/TrueZhuangJia/voidlink-c.git
|
||||
[submodule "X1Kit"]
|
||||
path = X1Kit
|
||||
url = https://github.com/cgutman/X1Kit.git
|
||||
[submodule "third_party/imgui"]
|
||||
path = third_party/imgui
|
||||
url = https://github.com/ocornut/imgui.git
|
||||
|
||||
@@ -0,0 +1,657 @@
|
||||
// dear imgui: Renderer Backend for Metal
|
||||
// This needs to be used along with a Platform Backend (e.g. OSX)
|
||||
|
||||
// Implemented features:
|
||||
// [X] Renderer: User texture binding. Use 'MTLTexture' as texture identifier. Read the FAQ about ImTextureID/ImTextureRef!
|
||||
// [X] Renderer: Large meshes support (64k+ vertices) even with 16-bit indices (ImGuiBackendFlags_RendererHasVtxOffset).
|
||||
// [X] Renderer: Texture updates support for dynamic font atlas (ImGuiBackendFlags_RendererHasTextures).
|
||||
|
||||
// You can use unmodified imgui_impl_* files in your project. See examples/ folder for examples of using this.
|
||||
// Prefer including the entire imgui/ repository into your project (either as a copy or as a submodule), and only build the backends you need.
|
||||
// Learn about Dear ImGui:
|
||||
// - FAQ https://dearimgui.com/faq
|
||||
// - Getting Started https://dearimgui.com/getting-started
|
||||
// - Documentation https://dearimgui.com/docs (same as your local docs/ folder).
|
||||
// - Introduction, links and more at the top of imgui.cpp
|
||||
|
||||
// CHANGELOG
|
||||
// (minor and older changes stripped away, please see git history for details)
|
||||
// 2025-06-11: Added support for ImGuiBackendFlags_RendererHasTextures, for dynamic font atlas. Removed ImGui_ImplMetal_CreateFontsTexture() and ImGui_ImplMetal_DestroyFontsTexture().
|
||||
// 2025-02-03: Metal: Crash fix. (#8367)
|
||||
// 2024-01-08: Metal: Fixed memory leaks when using metal-cpp (#8276, #8166) or when using multiple contexts (#7419).
|
||||
// 2022-08-23: Metal: Update deprecated property 'sampleCount'->'rasterSampleCount'.
|
||||
// 2022-07-05: Metal: Add dispatch synchronization.
|
||||
// 2022-06-30: Metal: Use __bridge for ARC based systems.
|
||||
// 2022-06-01: Metal: Fixed null dereference on exit inside command buffer completion handler.
|
||||
// 2022-04-27: Misc: Store backend data in a per-context struct, allowing to use this backend with multiple contexts.
|
||||
// 2022-01-03: Metal: Ignore ImDrawCmd where ElemCount == 0 (very rare but can technically be manufactured by user code).
|
||||
// 2021-12-30: Metal: Added Metal C++ support. Enable with '#define IMGUI_IMPL_METAL_CPP' in your imconfig.h file.
|
||||
// 2021-08-24: Metal: Fixed a crash when clipping rect larger than framebuffer is submitted. (#4464)
|
||||
// 2021-05-19: Metal: Replaced direct access to ImDrawCmd::TextureId with a call to ImDrawCmd::GetTexID(). (will become a requirement)
|
||||
// 2021-02-18: Metal: Change blending equation to preserve alpha in output buffer.
|
||||
// 2021-01-25: Metal: Fixed texture storage mode when building on Mac Catalyst.
|
||||
// 2019-05-29: Metal: Added support for large mesh (64K+ vertices), enable ImGuiBackendFlags_RendererHasVtxOffset flag.
|
||||
// 2019-04-30: Metal: Added support for special ImDrawCallback_ResetRenderState callback to reset render state.
|
||||
// 2019-02-11: Metal: Projecting clipping rectangles correctly using draw_data->FramebufferScale to allow multi-viewports for retina display.
|
||||
// 2018-11-30: Misc: Setting up io.BackendRendererName so it can be displayed in the About Window.
|
||||
// 2018-07-05: Metal: Added new Metal backend implementation.
|
||||
|
||||
#include "imgui.h"
|
||||
#ifndef IMGUI_DISABLE
|
||||
#include "imgui_impl_metal.h"
|
||||
#import <time.h>
|
||||
#import <Metal/Metal.h>
|
||||
|
||||
#pragma mark - Support classes
|
||||
|
||||
// A wrapper around a MTLBuffer object that knows the last time it was reused
|
||||
@interface MetalBuffer : NSObject
|
||||
@property (nonatomic, strong) id<MTLBuffer> buffer;
|
||||
@property (nonatomic, assign) double lastReuseTime;
|
||||
- (instancetype)initWithBuffer:(id<MTLBuffer>)buffer;
|
||||
@end
|
||||
|
||||
// An object that encapsulates the data necessary to uniquely identify a
|
||||
// render pipeline state. These are used as cache keys.
|
||||
@interface FramebufferDescriptor : NSObject<NSCopying>
|
||||
@property (nonatomic, assign) unsigned long sampleCount;
|
||||
@property (nonatomic, assign) MTLPixelFormat colorPixelFormat;
|
||||
@property (nonatomic, assign) MTLPixelFormat depthPixelFormat;
|
||||
@property (nonatomic, assign) MTLPixelFormat stencilPixelFormat;
|
||||
- (instancetype)initWithRenderPassDescriptor:(MTLRenderPassDescriptor*)renderPassDescriptor;
|
||||
@end
|
||||
|
||||
@interface MetalTexture : NSObject
|
||||
@property (nonatomic, strong) id<MTLTexture> metalTexture;
|
||||
- (instancetype)initWithTexture:(id<MTLTexture>)metalTexture;
|
||||
@end
|
||||
|
||||
// A singleton that stores long-lived objects that are needed by the Metal
|
||||
// renderer backend. Stores the render pipeline state cache and the default
|
||||
// font texture, and manages the reusable buffer cache.
|
||||
@interface MetalContext : NSObject
|
||||
@property (nonatomic, strong) id<MTLDevice> device;
|
||||
@property (nonatomic, strong) id<MTLDepthStencilState> depthStencilState;
|
||||
@property (nonatomic, strong) FramebufferDescriptor* framebufferDescriptor; // framebuffer descriptor for current frame; transient
|
||||
@property (nonatomic, strong) NSMutableDictionary* renderPipelineStateCache; // pipeline cache; keyed on framebuffer descriptors
|
||||
@property (nonatomic, strong) NSMutableArray<MetalBuffer*>* bufferCache;
|
||||
@property (nonatomic, assign) double lastBufferCachePurge;
|
||||
- (MetalBuffer*)dequeueReusableBufferOfLength:(NSUInteger)length device:(id<MTLDevice>)device;
|
||||
- (id<MTLRenderPipelineState>)renderPipelineStateForFramebufferDescriptor:(FramebufferDescriptor*)descriptor device:(id<MTLDevice>)device;
|
||||
@end
|
||||
|
||||
struct ImGui_ImplMetal_Data
|
||||
{
|
||||
MetalContext* SharedMetalContext;
|
||||
|
||||
ImGui_ImplMetal_Data() { memset((void*)this, 0, sizeof(*this)); }
|
||||
};
|
||||
|
||||
static ImGui_ImplMetal_Data* ImGui_ImplMetal_GetBackendData() { return ImGui::GetCurrentContext() ? (ImGui_ImplMetal_Data*)ImGui::GetIO().BackendRendererUserData : nullptr; }
|
||||
static void ImGui_ImplMetal_DestroyBackendData(){ IM_DELETE(ImGui_ImplMetal_GetBackendData()); }
|
||||
|
||||
// XXX Work around missing clock symbols on iOS. The NSDate version is the method this function used previously.
|
||||
#if defined(CLOCK_UPTIME_RAW)
|
||||
static inline CFTimeInterval GetMachAbsoluteTimeInSeconds() { return (CFTimeInterval)(double)(clock_gettime_nsec_np(CLOCK_UPTIME_RAW) / 1e9); }
|
||||
#else
|
||||
static inline CFTimeInterval GetMachAbsoluteTimeInSeconds() { return (CFTimeInterval)[NSDate date].timeIntervalSince1970; }
|
||||
#endif
|
||||
|
||||
#ifdef IMGUI_IMPL_METAL_CPP
|
||||
|
||||
#pragma mark - Dear ImGui Metal C++ Backend API
|
||||
|
||||
bool ImGui_ImplMetal_Init(MTL::Device* device)
|
||||
{
|
||||
return ImGui_ImplMetal_Init((__bridge id<MTLDevice>)(device));
|
||||
}
|
||||
|
||||
void ImGui_ImplMetal_NewFrame(MTL::RenderPassDescriptor* renderPassDescriptor)
|
||||
{
|
||||
ImGui_ImplMetal_NewFrame((__bridge MTLRenderPassDescriptor*)(renderPassDescriptor));
|
||||
}
|
||||
|
||||
void ImGui_ImplMetal_RenderDrawData(ImDrawData* draw_data,
|
||||
MTL::CommandBuffer* commandBuffer,
|
||||
MTL::RenderCommandEncoder* commandEncoder)
|
||||
{
|
||||
ImGui_ImplMetal_RenderDrawData(draw_data,
|
||||
(__bridge id<MTLCommandBuffer>)(commandBuffer),
|
||||
(__bridge id<MTLRenderCommandEncoder>)(commandEncoder));
|
||||
|
||||
}
|
||||
|
||||
bool ImGui_ImplMetal_CreateDeviceObjects(MTL::Device* device)
|
||||
{
|
||||
return ImGui_ImplMetal_CreateDeviceObjects((__bridge id<MTLDevice>)(device));
|
||||
}
|
||||
|
||||
#endif // #ifdef IMGUI_IMPL_METAL_CPP
|
||||
|
||||
#pragma mark - Dear ImGui Metal Backend API
|
||||
|
||||
bool ImGui_ImplMetal_Init(id<MTLDevice> device)
|
||||
{
|
||||
ImGuiIO& io = ImGui::GetIO();
|
||||
IMGUI_CHECKVERSION();
|
||||
IM_ASSERT(io.BackendRendererUserData == nullptr && "Already initialized a renderer backend!");
|
||||
|
||||
ImGui_ImplMetal_Data* bd = IM_NEW(ImGui_ImplMetal_Data)();
|
||||
io.BackendRendererUserData = (void*)bd;
|
||||
io.BackendRendererName = "imgui_impl_metal";
|
||||
io.BackendFlags |= ImGuiBackendFlags_RendererHasVtxOffset; // We can honor the ImDrawCmd::VtxOffset field, allowing for large meshes.
|
||||
io.BackendFlags |= ImGuiBackendFlags_RendererHasTextures; // We can honor ImGuiPlatformIO::Textures[] requests during render.
|
||||
|
||||
bd->SharedMetalContext = [[MetalContext alloc] init];
|
||||
bd->SharedMetalContext.device = device;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void ImGui_ImplMetal_Shutdown()
|
||||
{
|
||||
ImGui_ImplMetal_Data* bd = ImGui_ImplMetal_GetBackendData();
|
||||
IM_UNUSED(bd);
|
||||
IM_ASSERT(bd != nullptr && "No renderer backend to shutdown, or already shutdown?");
|
||||
ImGui_ImplMetal_DestroyDeviceObjects();
|
||||
ImGui_ImplMetal_DestroyBackendData();
|
||||
|
||||
ImGuiIO& io = ImGui::GetIO();
|
||||
io.BackendRendererName = nullptr;
|
||||
io.BackendRendererUserData = nullptr;
|
||||
io.BackendFlags &= ~(ImGuiBackendFlags_RendererHasVtxOffset | ImGuiBackendFlags_RendererHasTextures);
|
||||
}
|
||||
|
||||
void ImGui_ImplMetal_NewFrame(MTLRenderPassDescriptor* renderPassDescriptor)
|
||||
{
|
||||
ImGui_ImplMetal_Data* bd = ImGui_ImplMetal_GetBackendData();
|
||||
IM_ASSERT(bd != nil && "Context or backend not initialized! Did you call ImGui_ImplMetal_Init()?");
|
||||
#ifdef IMGUI_IMPL_METAL_CPP
|
||||
bd->SharedMetalContext.framebufferDescriptor = [[[FramebufferDescriptor alloc] initWithRenderPassDescriptor:renderPassDescriptor]autorelease];
|
||||
#else
|
||||
bd->SharedMetalContext.framebufferDescriptor = [[FramebufferDescriptor alloc] initWithRenderPassDescriptor:renderPassDescriptor];
|
||||
#endif
|
||||
if (bd->SharedMetalContext.depthStencilState == nil)
|
||||
ImGui_ImplMetal_CreateDeviceObjects(bd->SharedMetalContext.device);
|
||||
}
|
||||
|
||||
static void ImGui_ImplMetal_SetupRenderState(ImDrawData* draw_data, id<MTLCommandBuffer> commandBuffer,
|
||||
id<MTLRenderCommandEncoder> commandEncoder, id<MTLRenderPipelineState> renderPipelineState,
|
||||
MetalBuffer* vertexBuffer, size_t vertexBufferOffset)
|
||||
{
|
||||
IM_UNUSED(commandBuffer);
|
||||
ImGui_ImplMetal_Data* bd = ImGui_ImplMetal_GetBackendData();
|
||||
[commandEncoder setCullMode:MTLCullModeNone];
|
||||
[commandEncoder setDepthStencilState:bd->SharedMetalContext.depthStencilState];
|
||||
|
||||
// Setup viewport, orthographic projection matrix
|
||||
// Our visible imgui space lies from draw_data->DisplayPos (top left) to
|
||||
// draw_data->DisplayPos+data_data->DisplaySize (bottom right). DisplayMin is typically (0,0) for single viewport apps.
|
||||
MTLViewport viewport =
|
||||
{
|
||||
.originX = 0.0,
|
||||
.originY = 0.0,
|
||||
.width = (double)(draw_data->DisplaySize.x * draw_data->FramebufferScale.x),
|
||||
.height = (double)(draw_data->DisplaySize.y * draw_data->FramebufferScale.y),
|
||||
.znear = 0.0,
|
||||
.zfar = 1.0
|
||||
};
|
||||
[commandEncoder setViewport:viewport];
|
||||
|
||||
float L = draw_data->DisplayPos.x;
|
||||
float R = draw_data->DisplayPos.x + draw_data->DisplaySize.x;
|
||||
float T = draw_data->DisplayPos.y;
|
||||
float B = draw_data->DisplayPos.y + draw_data->DisplaySize.y;
|
||||
float N = (float)viewport.znear;
|
||||
float F = (float)viewport.zfar;
|
||||
const float ortho_projection[4][4] =
|
||||
{
|
||||
{ 2.0f/(R-L), 0.0f, 0.0f, 0.0f },
|
||||
{ 0.0f, 2.0f/(T-B), 0.0f, 0.0f },
|
||||
{ 0.0f, 0.0f, 1/(F-N), 0.0f },
|
||||
{ (R+L)/(L-R), (T+B)/(B-T), N/(F-N), 1.0f },
|
||||
};
|
||||
[commandEncoder setVertexBytes:&ortho_projection length:sizeof(ortho_projection) atIndex:1];
|
||||
|
||||
[commandEncoder setRenderPipelineState:renderPipelineState];
|
||||
|
||||
[commandEncoder setVertexBuffer:vertexBuffer.buffer offset:0 atIndex:0];
|
||||
[commandEncoder setVertexBufferOffset:vertexBufferOffset atIndex:0];
|
||||
}
|
||||
|
||||
// Metal Render function.
|
||||
void ImGui_ImplMetal_RenderDrawData(ImDrawData* draw_data, id<MTLCommandBuffer> commandBuffer, id<MTLRenderCommandEncoder> commandEncoder)
|
||||
{
|
||||
ImGui_ImplMetal_Data* bd = ImGui_ImplMetal_GetBackendData();
|
||||
MetalContext* ctx = bd->SharedMetalContext;
|
||||
|
||||
// Avoid rendering when minimized, scale coordinates for retina displays (screen coordinates != framebuffer coordinates)
|
||||
int fb_width = (int)(draw_data->DisplaySize.x * draw_data->FramebufferScale.x);
|
||||
int fb_height = (int)(draw_data->DisplaySize.y * draw_data->FramebufferScale.y);
|
||||
if (fb_width <= 0 || fb_height <= 0 || draw_data->CmdListsCount == 0)
|
||||
return;
|
||||
|
||||
// Catch up with texture updates. Most of the times, the list will have 1 element with an OK status, aka nothing to do.
|
||||
// (This almost always points to ImGui::GetPlatformIO().Textures[] but is part of ImDrawData to allow overriding or disabling texture updates).
|
||||
if (draw_data->Textures != nullptr)
|
||||
for (ImTextureData* tex : *draw_data->Textures)
|
||||
if (tex->Status != ImTextureStatus_OK)
|
||||
ImGui_ImplMetal_UpdateTexture(tex);
|
||||
|
||||
// Try to retrieve a render pipeline state that is compatible with the framebuffer config for this frame
|
||||
// The hit rate for this cache should be very near 100%.
|
||||
id<MTLRenderPipelineState> renderPipelineState = ctx.renderPipelineStateCache[ctx.framebufferDescriptor];
|
||||
if (renderPipelineState == nil)
|
||||
{
|
||||
// No luck; make a new render pipeline state
|
||||
renderPipelineState = [ctx renderPipelineStateForFramebufferDescriptor:ctx.framebufferDescriptor device:commandBuffer.device];
|
||||
|
||||
// Cache render pipeline state for later reuse
|
||||
ctx.renderPipelineStateCache[ctx.framebufferDescriptor] = renderPipelineState;
|
||||
}
|
||||
|
||||
size_t vertexBufferLength = (size_t)draw_data->TotalVtxCount * sizeof(ImDrawVert);
|
||||
size_t indexBufferLength = (size_t)draw_data->TotalIdxCount * sizeof(ImDrawIdx);
|
||||
MetalBuffer* vertexBuffer = [ctx dequeueReusableBufferOfLength:vertexBufferLength device:commandBuffer.device];
|
||||
MetalBuffer* indexBuffer = [ctx dequeueReusableBufferOfLength:indexBufferLength device:commandBuffer.device];
|
||||
|
||||
ImGui_ImplMetal_SetupRenderState(draw_data, commandBuffer, commandEncoder, renderPipelineState, vertexBuffer, 0);
|
||||
|
||||
// Will project scissor/clipping rectangles into framebuffer space
|
||||
ImVec2 clip_off = draw_data->DisplayPos; // (0,0) unless using multi-viewports
|
||||
ImVec2 clip_scale = draw_data->FramebufferScale; // (1,1) unless using retina display which are often (2,2)
|
||||
|
||||
// Render command lists
|
||||
size_t vertexBufferOffset = 0;
|
||||
size_t indexBufferOffset = 0;
|
||||
for (int n = 0; n < draw_data->CmdListsCount; n++)
|
||||
{
|
||||
const ImDrawList* draw_list = draw_data->CmdLists[n];
|
||||
|
||||
memcpy((char*)vertexBuffer.buffer.contents + vertexBufferOffset, draw_list->VtxBuffer.Data, (size_t)draw_list->VtxBuffer.Size * sizeof(ImDrawVert));
|
||||
memcpy((char*)indexBuffer.buffer.contents + indexBufferOffset, draw_list->IdxBuffer.Data, (size_t)draw_list->IdxBuffer.Size * sizeof(ImDrawIdx));
|
||||
|
||||
for (int cmd_i = 0; cmd_i < draw_list->CmdBuffer.Size; cmd_i++)
|
||||
{
|
||||
const ImDrawCmd* pcmd = &draw_list->CmdBuffer[cmd_i];
|
||||
if (pcmd->UserCallback)
|
||||
{
|
||||
// User callback, registered via ImDrawList::AddCallback()
|
||||
// (ImDrawCallback_ResetRenderState is a special callback value used by the user to request the renderer to reset render state.)
|
||||
if (pcmd->UserCallback == ImDrawCallback_ResetRenderState)
|
||||
ImGui_ImplMetal_SetupRenderState(draw_data, commandBuffer, commandEncoder, renderPipelineState, vertexBuffer, vertexBufferOffset);
|
||||
else
|
||||
pcmd->UserCallback(draw_list, pcmd);
|
||||
}
|
||||
else
|
||||
{
|
||||
// Project scissor/clipping rectangles into framebuffer space
|
||||
ImVec2 clip_min((pcmd->ClipRect.x - clip_off.x) * clip_scale.x, (pcmd->ClipRect.y - clip_off.y) * clip_scale.y);
|
||||
ImVec2 clip_max((pcmd->ClipRect.z - clip_off.x) * clip_scale.x, (pcmd->ClipRect.w - clip_off.y) * clip_scale.y);
|
||||
|
||||
// Clamp to viewport as setScissorRect() won't accept values that are off bounds
|
||||
if (clip_min.x < 0.0f) { clip_min.x = 0.0f; }
|
||||
if (clip_min.y < 0.0f) { clip_min.y = 0.0f; }
|
||||
if (clip_max.x > fb_width) { clip_max.x = (float)fb_width; }
|
||||
if (clip_max.y > fb_height) { clip_max.y = (float)fb_height; }
|
||||
if (clip_max.x <= clip_min.x || clip_max.y <= clip_min.y)
|
||||
continue;
|
||||
if (pcmd->ElemCount == 0) // drawIndexedPrimitives() validation doesn't accept this
|
||||
continue;
|
||||
|
||||
// Apply scissor/clipping rectangle
|
||||
MTLScissorRect scissorRect =
|
||||
{
|
||||
.x = NSUInteger(clip_min.x),
|
||||
.y = NSUInteger(clip_min.y),
|
||||
.width = NSUInteger(clip_max.x - clip_min.x),
|
||||
.height = NSUInteger(clip_max.y - clip_min.y)
|
||||
};
|
||||
[commandEncoder setScissorRect:scissorRect];
|
||||
|
||||
// Bind texture, Draw
|
||||
if (ImTextureID tex_id = pcmd->GetTexID())
|
||||
[commandEncoder setFragmentTexture:(__bridge id<MTLTexture>)(void*)(intptr_t)(tex_id) atIndex:0];
|
||||
|
||||
[commandEncoder setVertexBufferOffset:(vertexBufferOffset + pcmd->VtxOffset * sizeof(ImDrawVert)) atIndex:0];
|
||||
[commandEncoder drawIndexedPrimitives:MTLPrimitiveTypeTriangle
|
||||
indexCount:pcmd->ElemCount
|
||||
indexType:sizeof(ImDrawIdx) == 2 ? MTLIndexTypeUInt16 : MTLIndexTypeUInt32
|
||||
indexBuffer:indexBuffer.buffer
|
||||
indexBufferOffset:indexBufferOffset + pcmd->IdxOffset * sizeof(ImDrawIdx)];
|
||||
}
|
||||
}
|
||||
|
||||
vertexBufferOffset += (size_t)draw_list->VtxBuffer.Size * sizeof(ImDrawVert);
|
||||
indexBufferOffset += (size_t)draw_list->IdxBuffer.Size * sizeof(ImDrawIdx);
|
||||
}
|
||||
|
||||
MetalContext* sharedMetalContext = bd->SharedMetalContext;
|
||||
[commandBuffer addCompletedHandler:^(id<MTLCommandBuffer>)
|
||||
{
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
@synchronized(sharedMetalContext.bufferCache)
|
||||
{
|
||||
[sharedMetalContext.bufferCache addObject:vertexBuffer];
|
||||
[sharedMetalContext.bufferCache addObject:indexBuffer];
|
||||
}
|
||||
});
|
||||
}];
|
||||
}
|
||||
|
||||
static void ImGui_ImplMetal_DestroyTexture(ImTextureData* tex)
|
||||
{
|
||||
MetalTexture* backend_tex = (__bridge_transfer MetalTexture*)(tex->BackendUserData);
|
||||
if (backend_tex == nullptr)
|
||||
return;
|
||||
IM_ASSERT(backend_tex.metalTexture == (__bridge id<MTLTexture>)(void*)(intptr_t)tex->TexID);
|
||||
backend_tex.metalTexture = nil;
|
||||
|
||||
// Clear identifiers and mark as destroyed (in order to allow e.g. calling InvalidateDeviceObjects while running)
|
||||
tex->SetTexID(ImTextureID_Invalid);
|
||||
tex->SetStatus(ImTextureStatus_Destroyed);
|
||||
tex->BackendUserData = nullptr;
|
||||
}
|
||||
|
||||
void ImGui_ImplMetal_UpdateTexture(ImTextureData* tex)
|
||||
{
|
||||
ImGui_ImplMetal_Data* bd = ImGui_ImplMetal_GetBackendData();
|
||||
if (tex->Status == ImTextureStatus_WantCreate)
|
||||
{
|
||||
// Create and upload new texture to graphics system
|
||||
//IMGUI_DEBUG_LOG("UpdateTexture #%03d: WantCreate %dx%d\n", tex->UniqueID, tex->Width, tex->Height);
|
||||
IM_ASSERT(tex->TexID == ImTextureID_Invalid && tex->BackendUserData == nullptr);
|
||||
IM_ASSERT(tex->Format == ImTextureFormat_RGBA32);
|
||||
|
||||
// We are retrieving and uploading the font atlas as a 4-channels RGBA texture here.
|
||||
// In theory we could call GetTexDataAsAlpha8() and upload a 1-channel texture to save on memory access bandwidth.
|
||||
// However, using a shader designed for 1-channel texture would make it less obvious to use the ImTextureID facility to render users own textures.
|
||||
// You can make that change in your implementation.
|
||||
MTLTextureDescriptor* textureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatRGBA8Unorm
|
||||
width:(NSUInteger)tex->Width
|
||||
height:(NSUInteger)tex->Height
|
||||
mipmapped:NO];
|
||||
textureDescriptor.usage = MTLTextureUsageShaderRead;
|
||||
#if TARGET_OS_OSX || TARGET_OS_MACCATALYST
|
||||
textureDescriptor.storageMode = MTLStorageModeManaged;
|
||||
#else
|
||||
textureDescriptor.storageMode = MTLStorageModeShared;
|
||||
#endif
|
||||
id <MTLTexture> texture = [bd->SharedMetalContext.device newTextureWithDescriptor:textureDescriptor];
|
||||
[texture replaceRegion:MTLRegionMake2D(0, 0, (NSUInteger)tex->Width, (NSUInteger)tex->Height) mipmapLevel:0 withBytes:tex->Pixels bytesPerRow:(NSUInteger)tex->Width * 4];
|
||||
MetalTexture* backend_tex = [[MetalTexture alloc] initWithTexture:texture];
|
||||
|
||||
// Store identifiers
|
||||
tex->SetTexID((ImTextureID)(intptr_t)texture);
|
||||
tex->SetStatus(ImTextureStatus_OK);
|
||||
tex->BackendUserData = (__bridge_retained void*)(backend_tex);
|
||||
}
|
||||
else if (tex->Status == ImTextureStatus_WantUpdates)
|
||||
{
|
||||
// Update selected blocks. We only ever write to textures regions which have never been used before!
|
||||
// This backend choose to use tex->Updates[] but you can use tex->UpdateRect to upload a single region.
|
||||
MetalTexture* backend_tex = (__bridge MetalTexture*)(tex->BackendUserData);
|
||||
for (ImTextureRect& r : tex->Updates)
|
||||
{
|
||||
[backend_tex.metalTexture replaceRegion:MTLRegionMake2D((NSUInteger)r.x, (NSUInteger)r.y, (NSUInteger)r.w, (NSUInteger)r.h)
|
||||
mipmapLevel:0
|
||||
withBytes:tex->GetPixelsAt(r.x, r.y)
|
||||
bytesPerRow:(NSUInteger)tex->Width * 4];
|
||||
}
|
||||
tex->SetStatus(ImTextureStatus_OK);
|
||||
}
|
||||
else if (tex->Status == ImTextureStatus_WantDestroy && tex->UnusedFrames > 0)
|
||||
{
|
||||
ImGui_ImplMetal_DestroyTexture(tex);
|
||||
}
|
||||
}
|
||||
|
||||
bool ImGui_ImplMetal_CreateDeviceObjects(id<MTLDevice> device)
|
||||
{
|
||||
ImGui_ImplMetal_Data* bd = ImGui_ImplMetal_GetBackendData();
|
||||
MTLDepthStencilDescriptor* depthStencilDescriptor = [[MTLDepthStencilDescriptor alloc] init];
|
||||
depthStencilDescriptor.depthWriteEnabled = NO;
|
||||
depthStencilDescriptor.depthCompareFunction = MTLCompareFunctionAlways;
|
||||
bd->SharedMetalContext.depthStencilState = [device newDepthStencilStateWithDescriptor:depthStencilDescriptor];
|
||||
#ifdef IMGUI_IMPL_METAL_CPP
|
||||
[depthStencilDescriptor release];
|
||||
#endif
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void ImGui_ImplMetal_DestroyDeviceObjects()
|
||||
{
|
||||
ImGui_ImplMetal_Data* bd = ImGui_ImplMetal_GetBackendData();
|
||||
|
||||
// Destroy all textures
|
||||
for (ImTextureData* tex : ImGui::GetPlatformIO().Textures)
|
||||
if (tex->RefCount == 1)
|
||||
ImGui_ImplMetal_DestroyTexture(tex);
|
||||
|
||||
[bd->SharedMetalContext.renderPipelineStateCache removeAllObjects];
|
||||
}
|
||||
|
||||
#pragma mark - MetalBuffer implementation
|
||||
|
||||
@implementation MetalBuffer
|
||||
- (instancetype)initWithBuffer:(id<MTLBuffer>)buffer
|
||||
{
|
||||
if ((self = [super init]))
|
||||
{
|
||||
_buffer = buffer;
|
||||
_lastReuseTime = GetMachAbsoluteTimeInSeconds();
|
||||
}
|
||||
return self;
|
||||
}
|
||||
@end
|
||||
|
||||
#pragma mark - FramebufferDescriptor implementation
|
||||
|
||||
@implementation FramebufferDescriptor
|
||||
- (instancetype)initWithRenderPassDescriptor:(MTLRenderPassDescriptor*)renderPassDescriptor
|
||||
{
|
||||
if ((self = [super init]))
|
||||
{
|
||||
_sampleCount = renderPassDescriptor.colorAttachments[0].texture.sampleCount;
|
||||
_colorPixelFormat = renderPassDescriptor.colorAttachments[0].texture.pixelFormat;
|
||||
_depthPixelFormat = renderPassDescriptor.depthAttachment.texture.pixelFormat;
|
||||
_stencilPixelFormat = renderPassDescriptor.stencilAttachment.texture.pixelFormat;
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (nonnull id)copyWithZone:(nullable NSZone*)zone
|
||||
{
|
||||
FramebufferDescriptor* copy = [[FramebufferDescriptor allocWithZone:zone] init];
|
||||
copy.sampleCount = self.sampleCount;
|
||||
copy.colorPixelFormat = self.colorPixelFormat;
|
||||
copy.depthPixelFormat = self.depthPixelFormat;
|
||||
copy.stencilPixelFormat = self.stencilPixelFormat;
|
||||
return copy;
|
||||
}
|
||||
|
||||
- (NSUInteger)hash
|
||||
{
|
||||
NSUInteger sc = _sampleCount & 0x3;
|
||||
NSUInteger cf = _colorPixelFormat & 0x3FF;
|
||||
NSUInteger df = _depthPixelFormat & 0x3FF;
|
||||
NSUInteger sf = _stencilPixelFormat & 0x3FF;
|
||||
NSUInteger hash = (sf << 22) | (df << 12) | (cf << 2) | sc;
|
||||
return hash;
|
||||
}
|
||||
|
||||
- (BOOL)isEqual:(id)object
|
||||
{
|
||||
FramebufferDescriptor* other = object;
|
||||
if (![other isKindOfClass:[FramebufferDescriptor class]])
|
||||
return NO;
|
||||
return other.sampleCount == self.sampleCount &&
|
||||
other.colorPixelFormat == self.colorPixelFormat &&
|
||||
other.depthPixelFormat == self.depthPixelFormat &&
|
||||
other.stencilPixelFormat == self.stencilPixelFormat;
|
||||
}
|
||||
|
||||
@end
|
||||
|
||||
#pragma mark - MetalTexture implementation
|
||||
|
||||
@implementation MetalTexture
|
||||
- (instancetype)initWithTexture:(id<MTLTexture>)metalTexture
|
||||
{
|
||||
if ((self = [super init]))
|
||||
self.metalTexture = metalTexture;
|
||||
return self;
|
||||
}
|
||||
|
||||
@end
|
||||
|
||||
#pragma mark - MetalContext implementation
|
||||
|
||||
@implementation MetalContext
|
||||
- (instancetype)init
|
||||
{
|
||||
if ((self = [super init]))
|
||||
{
|
||||
self.renderPipelineStateCache = [NSMutableDictionary dictionary];
|
||||
self.bufferCache = [NSMutableArray array];
|
||||
_lastBufferCachePurge = GetMachAbsoluteTimeInSeconds();
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (MetalBuffer*)dequeueReusableBufferOfLength:(NSUInteger)length device:(id<MTLDevice>)device
|
||||
{
|
||||
uint64_t now = GetMachAbsoluteTimeInSeconds();
|
||||
|
||||
@synchronized(self.bufferCache)
|
||||
{
|
||||
// Purge old buffers that haven't been useful for a while
|
||||
if (now - self.lastBufferCachePurge > 1.0)
|
||||
{
|
||||
NSMutableArray* survivors = [NSMutableArray array];
|
||||
for (MetalBuffer* candidate in self.bufferCache)
|
||||
if (candidate.lastReuseTime > self.lastBufferCachePurge)
|
||||
[survivors addObject:candidate];
|
||||
self.bufferCache = [survivors mutableCopy];
|
||||
self.lastBufferCachePurge = now;
|
||||
}
|
||||
|
||||
// See if we have a buffer we can reuse
|
||||
MetalBuffer* bestCandidate = nil;
|
||||
for (MetalBuffer* candidate in self.bufferCache)
|
||||
if (candidate.buffer.length >= length && (bestCandidate == nil || bestCandidate.lastReuseTime > candidate.lastReuseTime))
|
||||
bestCandidate = candidate;
|
||||
|
||||
if (bestCandidate != nil)
|
||||
{
|
||||
[self.bufferCache removeObject:bestCandidate];
|
||||
bestCandidate.lastReuseTime = now;
|
||||
return bestCandidate;
|
||||
}
|
||||
}
|
||||
|
||||
// No luck; make a new buffer
|
||||
id<MTLBuffer> backing = [device newBufferWithLength:length options:MTLResourceStorageModeShared];
|
||||
return [[MetalBuffer alloc] initWithBuffer:backing];
|
||||
}
|
||||
|
||||
// Bilinear sampling is required by default. Set 'io.Fonts->Flags |= ImFontAtlasFlags_NoBakedLines' or 'style.AntiAliasedLinesUseTex = false' to allow point/nearest sampling.
|
||||
- (id<MTLRenderPipelineState>)renderPipelineStateForFramebufferDescriptor:(FramebufferDescriptor*)descriptor device:(id<MTLDevice>)device
|
||||
{
|
||||
NSError* error = nil;
|
||||
|
||||
NSString* shaderSource = @""
|
||||
"#include <metal_stdlib>\n"
|
||||
"using namespace metal;\n"
|
||||
"\n"
|
||||
"struct Uniforms {\n"
|
||||
" float4x4 projectionMatrix;\n"
|
||||
"};\n"
|
||||
"\n"
|
||||
"struct VertexIn {\n"
|
||||
" float2 position [[attribute(0)]];\n"
|
||||
" float2 texCoords [[attribute(1)]];\n"
|
||||
" uchar4 color [[attribute(2)]];\n"
|
||||
"};\n"
|
||||
"\n"
|
||||
"struct VertexOut {\n"
|
||||
" float4 position [[position]];\n"
|
||||
" float2 texCoords;\n"
|
||||
" float4 color;\n"
|
||||
"};\n"
|
||||
"\n"
|
||||
"vertex VertexOut vertex_main(VertexIn in [[stage_in]],\n"
|
||||
" constant Uniforms &uniforms [[buffer(1)]]) {\n"
|
||||
" VertexOut out;\n"
|
||||
" out.position = uniforms.projectionMatrix * float4(in.position, 0, 1);\n"
|
||||
" out.texCoords = in.texCoords;\n"
|
||||
" out.color = float4(in.color) / float4(255.0);\n"
|
||||
" return out;\n"
|
||||
"}\n"
|
||||
"\n"
|
||||
"fragment half4 fragment_main(VertexOut in [[stage_in]],\n"
|
||||
" texture2d<half, access::sample> texture [[texture(0)]]) {\n"
|
||||
" constexpr sampler linearSampler(coord::normalized, min_filter::linear, mag_filter::linear, mip_filter::linear);\n"
|
||||
" half4 texColor = texture.sample(linearSampler, in.texCoords);\n"
|
||||
" return half4(in.color) * texColor;\n"
|
||||
"}\n";
|
||||
|
||||
id<MTLLibrary> library = [device newLibraryWithSource:shaderSource options:nil error:&error];
|
||||
if (library == nil)
|
||||
{
|
||||
NSLog(@"Error: failed to create Metal library: %@", error);
|
||||
return nil;
|
||||
}
|
||||
|
||||
id<MTLFunction> vertexFunction = [library newFunctionWithName:@"vertex_main"];
|
||||
id<MTLFunction> fragmentFunction = [library newFunctionWithName:@"fragment_main"];
|
||||
|
||||
if (vertexFunction == nil || fragmentFunction == nil)
|
||||
{
|
||||
NSLog(@"Error: failed to find Metal shader functions in library: %@", error);
|
||||
return nil;
|
||||
}
|
||||
|
||||
MTLVertexDescriptor* vertexDescriptor = [MTLVertexDescriptor vertexDescriptor];
|
||||
vertexDescriptor.attributes[0].offset = offsetof(ImDrawVert, pos);
|
||||
vertexDescriptor.attributes[0].format = MTLVertexFormatFloat2; // position
|
||||
vertexDescriptor.attributes[0].bufferIndex = 0;
|
||||
vertexDescriptor.attributes[1].offset = offsetof(ImDrawVert, uv);
|
||||
vertexDescriptor.attributes[1].format = MTLVertexFormatFloat2; // texCoords
|
||||
vertexDescriptor.attributes[1].bufferIndex = 0;
|
||||
vertexDescriptor.attributes[2].offset = offsetof(ImDrawVert, col);
|
||||
vertexDescriptor.attributes[2].format = MTLVertexFormatUChar4; // color
|
||||
vertexDescriptor.attributes[2].bufferIndex = 0;
|
||||
vertexDescriptor.layouts[0].stepRate = 1;
|
||||
vertexDescriptor.layouts[0].stepFunction = MTLVertexStepFunctionPerVertex;
|
||||
vertexDescriptor.layouts[0].stride = sizeof(ImDrawVert);
|
||||
|
||||
MTLRenderPipelineDescriptor* pipelineDescriptor = [[MTLRenderPipelineDescriptor alloc] init];
|
||||
pipelineDescriptor.vertexFunction = vertexFunction;
|
||||
pipelineDescriptor.fragmentFunction = fragmentFunction;
|
||||
pipelineDescriptor.vertexDescriptor = vertexDescriptor;
|
||||
pipelineDescriptor.rasterSampleCount = self.framebufferDescriptor.sampleCount;
|
||||
pipelineDescriptor.colorAttachments[0].pixelFormat = self.framebufferDescriptor.colorPixelFormat;
|
||||
pipelineDescriptor.colorAttachments[0].blendingEnabled = YES;
|
||||
pipelineDescriptor.colorAttachments[0].rgbBlendOperation = MTLBlendOperationAdd;
|
||||
pipelineDescriptor.colorAttachments[0].sourceRGBBlendFactor = MTLBlendFactorSourceAlpha;
|
||||
pipelineDescriptor.colorAttachments[0].destinationRGBBlendFactor = MTLBlendFactorOneMinusSourceAlpha;
|
||||
pipelineDescriptor.colorAttachments[0].alphaBlendOperation = MTLBlendOperationAdd;
|
||||
pipelineDescriptor.colorAttachments[0].sourceAlphaBlendFactor = MTLBlendFactorOne;
|
||||
pipelineDescriptor.colorAttachments[0].destinationAlphaBlendFactor = MTLBlendFactorOneMinusSourceAlpha;
|
||||
pipelineDescriptor.depthAttachmentPixelFormat = self.framebufferDescriptor.depthPixelFormat;
|
||||
pipelineDescriptor.stencilAttachmentPixelFormat = self.framebufferDescriptor.stencilPixelFormat;
|
||||
|
||||
id<MTLRenderPipelineState> renderPipelineState = [device newRenderPipelineStateWithDescriptor:pipelineDescriptor error:&error];
|
||||
if (error != nil)
|
||||
NSLog(@"Error: failed to create Metal pipeline state: %@", error);
|
||||
|
||||
return renderPipelineState;
|
||||
}
|
||||
|
||||
@end
|
||||
|
||||
//-----------------------------------------------------------------------------
|
||||
|
||||
#endif // #ifndef IMGUI_DISABLE
|
||||
@@ -0,0 +1,40 @@
|
||||
//
|
||||
// TemporarySettings.h
|
||||
// Moonlight
|
||||
//
|
||||
// Created by Cameron Gutman on 12/1/15.
|
||||
// Copyright © 2015 Moonlight Stream. All rights reserved.
|
||||
//
|
||||
|
||||
#import "Settings+CoreDataClass.h"
|
||||
|
||||
@interface TemporarySettings : NSObject
|
||||
|
||||
@property (nonatomic, retain) Settings * parent;
|
||||
|
||||
@property (nonatomic, retain) NSNumber * bitrate;
|
||||
@property (nonatomic, retain) NSNumber * framerate;
|
||||
@property (nonatomic, retain) NSNumber * height;
|
||||
@property (nonatomic, retain) NSNumber * width;
|
||||
@property (nonatomic, retain) NSNumber * audioConfig;
|
||||
@property (nonatomic, retain) NSNumber * onscreenControls;
|
||||
@property (nonatomic, retain) NSString * uniqueId;
|
||||
@property (nonatomic) enum {
|
||||
CODEC_PREF_AUTO,
|
||||
CODEC_PREF_H264,
|
||||
CODEC_PREF_HEVC,
|
||||
CODEC_PREF_AV1,
|
||||
} preferredCodec;
|
||||
@property (nonatomic) BOOL useFramePacing;
|
||||
@property (nonatomic) BOOL multiController;
|
||||
@property (nonatomic) BOOL swapABXYButtons;
|
||||
@property (nonatomic) BOOL playAudioOnPC;
|
||||
@property (nonatomic) BOOL optimizeGames;
|
||||
@property (nonatomic) BOOL enableHdr;
|
||||
@property (nonatomic) BOOL btMouseSupport;
|
||||
@property (nonatomic) BOOL absoluteTouchMode;
|
||||
@property (nonatomic) BOOL statsOverlay;
|
||||
|
||||
- (id) initFromSettings:(Settings*)settings;
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,36 @@
|
||||
#import <Foundation/Foundation.h>
|
||||
#import <Metal/Metal.h>
|
||||
#import <MetalKit/MetalKit.h>
|
||||
#import <UIKit/UIKit.h>
|
||||
|
||||
#import "FloatBuffer.h"
|
||||
#import "Plot.h"
|
||||
|
||||
typedef void (^MetricsHandler)(int plotId, CFTimeInterval value);
|
||||
|
||||
@interface ImGuiRenderer : UIViewController
|
||||
@end
|
||||
|
||||
@interface ImGuiRenderer () <MTKViewDelegate>
|
||||
@property (nonatomic) CGRect bounds;
|
||||
@property (nonatomic, readonly) MTKView * _Nonnull mtkView;
|
||||
@property (nonatomic, strong) id <MTLDevice> _Nonnull device;
|
||||
@property (nonatomic, strong) id <MTLCommandQueue> _Nonnull commandQueue;
|
||||
@property (nonatomic) struct PlotDef * _Nonnull plots;
|
||||
@property (nonatomic) FloatBuffer * _Nonnull frametimes;
|
||||
@property (nonatomic) BOOL enableGraphs;
|
||||
@property (nonatomic) float graphOpacity;
|
||||
@property (nonatomic) BOOL imguiRunning;
|
||||
@property (nonatomic) MetricsHandler _Nonnull metricsHandler;
|
||||
|
||||
-(nonnull instancetype) initWithFrame:(CGRect)bounds
|
||||
streamFps:(int)streamFps
|
||||
enableGraphs:(BOOL)enableGraphs
|
||||
graphOpacity:(int)graphOpacity;
|
||||
-(void) start;
|
||||
-(void) show;
|
||||
-(void) hide;
|
||||
-(void) stop;
|
||||
-(void) observeFloat:(int)plotId value:(CFTimeInterval)value;
|
||||
-(void) observeFloatReturnMetrics:(int)plotId value:(CFTimeInterval)value plotMetrics:(PlotMetrics * _Nullable)plotMetrics;
|
||||
@end
|
||||
@@ -0,0 +1,425 @@
|
||||
#import <AVFoundation/AVFoundation.h>
|
||||
#import <Metal/Metal.h>
|
||||
#import "ImGuiRenderer.h"
|
||||
#import "ImGuiPlots.h"
|
||||
|
||||
// This will fully disable ImGui by compiling it out. The in-app setting enableGraphs
|
||||
// will also remove all ImGui overhead.
|
||||
//#define IMGUI_DISABLE
|
||||
|
||||
// If this is defined, ImGui takes over all touch control while using the app.
|
||||
// This is only needed if you want to enable and interact with the demo. I'm sure it's possible
|
||||
// to pass-through touch events when touching non-ImGui portions of the screen
|
||||
// but I couldn't figure it out. I did not try to get controller support working.
|
||||
//#define IMGUI_STEALS_TOUCH
|
||||
|
||||
#import "imgui.h"
|
||||
#import "imgui_impl_metal.h"
|
||||
|
||||
// ImGui code needs to live in this file because it's an Objective-C++ class, and can call C++ code.
|
||||
|
||||
@implementation ImGuiRenderer
|
||||
|
||||
-(nonnull instancetype)initWithFrame:(CGRect)bounds
|
||||
streamFps:(int)streamFps
|
||||
enableGraphs:(BOOL)enableGraphs
|
||||
graphOpacity:(int)graphOpacity
|
||||
{
|
||||
self = [super init];
|
||||
|
||||
_enableGraphs = enableGraphs;
|
||||
_graphOpacity = (float)(graphOpacity / 100.0);
|
||||
_bounds = bounds;
|
||||
_device = MTLCreateSystemDefaultDevice();
|
||||
_commandQueue = [_device newCommandQueue];
|
||||
|
||||
_plots = [ImGuiPlots sharedInstance].plots;
|
||||
|
||||
return self;
|
||||
}
|
||||
|
||||
-(void)ImGui_Init {
|
||||
// self-contained startup for ImGui so it can be dynamically toggled easily
|
||||
|
||||
#if !defined(IMGUI_DISABLE)
|
||||
if (_enableGraphs && !_imguiRunning) {
|
||||
IMGUI_CHECKVERSION();
|
||||
ImGui::CreateContext();
|
||||
ImGuiIO& io = ImGui::GetIO(); (void)io;
|
||||
// io.ConfigFlags |= ImGuiConfigFlags_NavEnableKeyboard; // Enable Keyboard Controls
|
||||
// io.ConfigFlags |= ImGuiConfigFlags_NavEnableGamepad; // Enable Gamepad Controls
|
||||
|
||||
ImGui::StyleColorsDark();
|
||||
|
||||
ImGui_ImplMetal_Init(_device);
|
||||
_imguiRunning = YES;
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
-(void)ImGui_Deinit {
|
||||
if (_imguiRunning) {
|
||||
#if !defined(IMGUI_DISABLE)
|
||||
ImGui_ImplMetal_Shutdown();
|
||||
ImGui::DestroyContext();
|
||||
#endif
|
||||
_imguiRunning = NO;
|
||||
}
|
||||
}
|
||||
|
||||
-(MTKView *)mtkView
|
||||
{
|
||||
return (MTKView *)self.view;
|
||||
}
|
||||
|
||||
-(void)loadView
|
||||
{
|
||||
self.view = [[MTKView alloc] initWithFrame:self.bounds];
|
||||
}
|
||||
|
||||
-(void)viewDidLoad
|
||||
{
|
||||
[super viewDidLoad];
|
||||
|
||||
self.mtkView.device = self.device;
|
||||
self.mtkView.delegate = self;
|
||||
self.mtkView.preferredFramesPerSecond = 60; // ImGui overlay will always render at this rate
|
||||
self.mtkView.opaque = NO;
|
||||
self.mtkView.enableSetNeedsDisplay = NO;
|
||||
|
||||
if (_enableGraphs) {
|
||||
[self ImGui_Init];
|
||||
self.mtkView.paused = NO;
|
||||
}
|
||||
else {
|
||||
self.mtkView.paused = YES;
|
||||
}
|
||||
}
|
||||
|
||||
// start & stop are used to show/hide graphs when swiping stats in or out
|
||||
-(void)start {
|
||||
[self ImGui_Init];
|
||||
if (_enableGraphs) {
|
||||
self.mtkView.paused = NO;
|
||||
}
|
||||
}
|
||||
|
||||
-(void)show {
|
||||
[self.mtkView setHidden:NO];
|
||||
}
|
||||
|
||||
-(void)hide {
|
||||
[self.mtkView setHidden:YES];
|
||||
}
|
||||
|
||||
-(void)stop {
|
||||
self.mtkView.paused = YES;
|
||||
[self ImGui_Deinit];
|
||||
}
|
||||
|
||||
// Only called when mtkView.paused is false
|
||||
- (void)drawInMTKView:(MTKView *)view
|
||||
{
|
||||
#if !defined(IMGUI_DISABLE)
|
||||
ImGuiIO &io = ImGui::GetIO();
|
||||
io.DisplaySize.x = view.bounds.size.width;
|
||||
io.DisplaySize.y = view.bounds.size.height;
|
||||
|
||||
CGFloat framebufferScale = view.window.screen.scale ?: UIScreen.mainScreen.scale;
|
||||
io.DisplayFramebufferScale = ImVec2(framebufferScale, framebufferScale);
|
||||
|
||||
id<MTLCommandBuffer> commandBuffer = [self.commandQueue commandBuffer];
|
||||
|
||||
MTLRenderPassDescriptor* renderPassDescriptor = view.currentRenderPassDescriptor;
|
||||
if (renderPassDescriptor == nil) {
|
||||
[commandBuffer commit];
|
||||
return;
|
||||
}
|
||||
|
||||
// Start the Dear ImGui frame
|
||||
ImGui_ImplMetal_NewFrame(renderPassDescriptor);
|
||||
ImGui::NewFrame();
|
||||
|
||||
// 1. Show the big demo window (Most of the sample code is in ImGui::ShowDemoWindow()! You can browse its code to learn more about Dear ImGui!).
|
||||
static bool show_demo_window = false;
|
||||
if (show_demo_window)
|
||||
ImGui::ShowDemoWindow(&show_demo_window);
|
||||
|
||||
// Custom Moonlight stuff goes here
|
||||
[self drawStatsGraphs];
|
||||
|
||||
// Rendering
|
||||
ImGui::Render();
|
||||
ImDrawData* draw_data = ImGui::GetDrawData();
|
||||
|
||||
// This looks silly when clear_color is all zeros but the original example uses this method to tint or make transparent the rest of the viewport
|
||||
static ImVec4 clear_color = ImVec4(0, 0, 0, 0);
|
||||
renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(clear_color.x * clear_color.w, clear_color.y * clear_color.w, clear_color.z * clear_color.w, clear_color.w);
|
||||
|
||||
id <MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor];
|
||||
[renderEncoder pushDebugGroup:@"Dear ImGui rendering"];
|
||||
ImGui_ImplMetal_RenderDrawData(draw_data, commandBuffer, renderEncoder);
|
||||
[renderEncoder popDebugGroup];
|
||||
[renderEncoder endEncoding];
|
||||
|
||||
// Present
|
||||
#if TARGET_OS_SIMULATOR
|
||||
[commandBuffer presentDrawable:view.currentDrawable];
|
||||
#else
|
||||
[commandBuffer presentDrawable:view.currentDrawable afterMinimumDuration:1.0 / view.preferredFramesPerSecond];
|
||||
#endif
|
||||
[commandBuffer commit];
|
||||
#endif
|
||||
}
|
||||
|
||||
- (void)mtkView:(MTKView *)view drawableSizeWillChange:(CGSize)size
|
||||
{
|
||||
}
|
||||
|
||||
- (void)viewDidDisappear:(BOOL)animated
|
||||
{
|
||||
[super viewDidDisappear:animated];
|
||||
|
||||
[self ImGui_Deinit];
|
||||
}
|
||||
|
||||
//-----------------------------------------------------------------------------------
|
||||
// Input processing
|
||||
//-----------------------------------------------------------------------------------
|
||||
|
||||
// This touch mapping is super cheesy/hacky. We treat any touch on the screen
|
||||
// as if it were a depressed left mouse button, and we don't bother handling
|
||||
// multitouch correctly at all. This causes the "cursor" to behave very erratically
|
||||
// when there are multiple active touches. But for demo purposes, single-touch
|
||||
// interaction actually works surprisingly well.
|
||||
#if !defined(IMGUI_DISABLE)
|
||||
# if defined(IMGUI_STEALS_TOUCH)
|
||||
-(BOOL)updateIOWithTouchEvent:(UIEvent *)event
|
||||
{
|
||||
UITouch *anyTouch = event.allTouches.anyObject;
|
||||
CGPoint touchLocation = [anyTouch locationInView:self.view];
|
||||
ImGuiIO &io = ImGui::GetIO();
|
||||
io.AddMouseSourceEvent(ImGuiMouseSource_TouchScreen);
|
||||
io.AddMousePosEvent(touchLocation.x, touchLocation.y);
|
||||
|
||||
BOOL hasActiveTouch = NO;
|
||||
for (UITouch *touch in event.allTouches)
|
||||
{
|
||||
if (touch.phase != UITouchPhaseEnded && touch.phase != UITouchPhaseCancelled)
|
||||
{
|
||||
hasActiveTouch = YES;
|
||||
break;
|
||||
}
|
||||
}
|
||||
io.AddMouseButtonEvent(0, hasActiveTouch);
|
||||
return YES;
|
||||
}
|
||||
# endif
|
||||
#endif
|
||||
|
||||
#if !defined(IMGUI_DISABLE)
|
||||
# if defined(IMGUI_STEALS_TOUCH)
|
||||
-(void)touchesBegan:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self updateIOWithTouchEvent:event]; }
|
||||
-(void)touchesMoved:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self updateIOWithTouchEvent:event]; }
|
||||
-(void)touchesCancelled:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self updateIOWithTouchEvent:event]; }
|
||||
-(void)touchesEnded:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event { [self updateIOWithTouchEvent:event]; }
|
||||
# endif
|
||||
#endif
|
||||
|
||||
/// Stats Graphs, we can still track data this way even with ImGui disabled
|
||||
|
||||
- (void) observeFloat:(int)plotId value:(CFTimeInterval)value {
|
||||
[self.plots[plotId].buffer addValue:(float)value];
|
||||
}
|
||||
|
||||
- (void) observeFloatReturnMetrics:(int)plotId value:(CFTimeInterval)value plotMetrics:(PlotMetrics *)plotMetrics {
|
||||
[self.plots[plotId].buffer addValue:(float)value];
|
||||
if (plotMetrics != nil) {
|
||||
[self.plots[plotId].buffer copyMetrics:plotMetrics];
|
||||
}
|
||||
}
|
||||
|
||||
#if !defined(IMGUI_DISABLE)
|
||||
inline static float getValue(void *buffer, int idx) {
|
||||
float *fbuffer = (float *)buffer;
|
||||
float v = fbuffer[idx];
|
||||
// clip the top of frametime graphs so they're less ugly
|
||||
if (v > 50)
|
||||
v = 49.9;
|
||||
|
||||
return v;
|
||||
}
|
||||
|
||||
- (void) drawStatsGraphs {
|
||||
// we malloc a buffer for frametimes once and reuse it
|
||||
static float * buffers[8] = {
|
||||
(float *)malloc(sizeof(float) * 512),
|
||||
(float *)malloc(sizeof(float) * 512),
|
||||
(float *)malloc(sizeof(float) * 512),
|
||||
(float *)malloc(sizeof(float) * 512),
|
||||
(float *)malloc(sizeof(float) * 512),
|
||||
(float *)malloc(sizeof(float) * 512),
|
||||
(float *)malloc(sizeof(float) * 512),
|
||||
(float *)malloc(sizeof(float) * 512)
|
||||
};
|
||||
|
||||
ImGuiIO &io = ImGui::GetIO();
|
||||
|
||||
// Try to make the graphs suck less on common device sizes
|
||||
float graphW = 0.0f;
|
||||
float graphH = 0.0f;
|
||||
switch ((int)io.DisplaySize.x) {
|
||||
case 1920: // ATV 4K 1920x1080 2x
|
||||
graphW = 525.0f; graphH = 80.0f; break;
|
||||
case 1376: // iPad Pro 1376x1032 2x
|
||||
graphW = 450.0f; graphH = 45.0f; break;
|
||||
case 1366: // iPad Air 1366x1024 2x
|
||||
graphW = 446.0f; graphW = 44.0f; break;
|
||||
case 1194: // Vision Pro (iPad mode) 1194x834 2x
|
||||
graphW = 379.0f; graphH = 45.0f; break;
|
||||
case 1133: // iPad Mini 1133x744 2x
|
||||
graphW = 360.0f; graphH = 45.0f; break;
|
||||
case 874: // iPhone 16 Pro 874x402 3x
|
||||
graphW = 275.0f; graphH = 45.0f; break;
|
||||
// TODO:
|
||||
default:
|
||||
float x = io.DisplaySize.x;
|
||||
float y = io.DisplaySize.y;
|
||||
if (io.DisplayFramebufferScale.x > 2.0f) {
|
||||
x = x * io.DisplayFramebufferScale.x / 2.0f;
|
||||
y = y * io.DisplayFramebufferScale.y / 2.0f;
|
||||
}
|
||||
graphW = x * 0.327f;
|
||||
graphH = y * 0.044f;
|
||||
}
|
||||
|
||||
LogOnce(LOG_I, @"Drawing graphs of size %.1f x %.1f in viewport %.1f x %.1f scale %.0f,%.0f using opacity %.2f",
|
||||
graphW, graphH,
|
||||
io.DisplaySize.x, io.DisplaySize.y,
|
||||
io.DisplayFramebufferScale.x, io.DisplayFramebufferScale.y,
|
||||
_graphOpacity);
|
||||
|
||||
// Left side - 2 graphs
|
||||
ImVec2 windowSize(graphW, (graphH * 4));
|
||||
ImVec2 windowPos(10.0f, 10.0f);
|
||||
ImGui::SetNextWindowPos(windowPos, ImGuiCond_Always, ImVec2(0.0f, 0.0f)); // pivot (0,0) = top-left
|
||||
ImGui::SetNextWindowSize(windowSize, ImGuiCond_Always);
|
||||
ImGuiWindowFlags flags = ImGuiWindowFlags_NoDecoration | ImGuiWindowFlags_NoMove | ImGuiWindowFlags_NoNavFocus | ImGuiWindowFlags_NoBackground |
|
||||
ImGuiWindowFlags_NoSavedSettings;
|
||||
ImGui::Begin("##StatsLeft", nullptr, flags);
|
||||
|
||||
// Dimensions of each graph
|
||||
ImVec2 avail = ImGui::GetContentRegionAvail();
|
||||
float fullW = avail.x;
|
||||
|
||||
// First 2 on left
|
||||
for (int i = 0; i < PlotCount; i++) {
|
||||
if (self.plots[i].side != PLOT_LEFT) continue;
|
||||
|
||||
float minY, maxY;
|
||||
int countF = [self.plots[i].buffer copyValuesIntoBuffer:buffers[i] min:&minY max:&maxY];
|
||||
float avgF = [self.plots[i].buffer averageValue];
|
||||
if (!countF) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Ugly, but can't get ImPlot to build for iOS
|
||||
char label[64];
|
||||
switch (self.plots[i].labelType) {
|
||||
case PLOT_LABEL_MIN_MAX_AVG:
|
||||
sprintf(label, "%s %.1f/%.1f/%.1f %s", self.plots[i].title, minY, maxY, avgF, self.plots[i].unit);
|
||||
break;
|
||||
case PLOT_LABEL_MIN_MAX_AVG_INT:
|
||||
sprintf(label, "%s %d/%d/%.1f %s", self.plots[i].title, (int)minY, (int)maxY, avgF, self.plots[i].unit);
|
||||
break;
|
||||
case PLOT_LABEL_TOTAL_INT:
|
||||
sprintf(label, "%s %d %s", self.plots[i].title, (int)[self.plots[i].buffer total], self.plots[i].unit);
|
||||
break;
|
||||
}
|
||||
float scaleMin = FLT_MAX;
|
||||
float scaleMax = FLT_MAX;
|
||||
if (self.plots[i].scaleTarget) {
|
||||
// optionally center the graph on a target such as the ideal frametime
|
||||
float ideal = (float)self.plots[i].scaleTarget;
|
||||
scaleMin = ideal - (2 * ideal);
|
||||
scaleMax = ideal + (2 * ideal);
|
||||
}
|
||||
if (self.plots[i].scaleMin)
|
||||
scaleMin = self.plots[i].scaleMin;
|
||||
if (self.plots[i].scaleMax)
|
||||
scaleMax = self.plots[i].scaleMax;
|
||||
ImGui::PushID(i);
|
||||
ImGui::PushStyleColor(ImGuiCol_FrameBg, ImVec4(0.16f, 0.29f, 0.48f, _graphOpacity)); // dark
|
||||
//ImGui::PushStyleColor(ImGuiCol_FrameBg, ImVec4(0.43f, 0.43f, 0.43f, 0.39f)); // classic
|
||||
ImGui::PushStyleColor(ImGuiCol_PlotLines, ImVec4(0.0f, 1.0f, 0.0f, 1.0f)); // green
|
||||
ImGui::PlotLines("##xx", buffers[i], countF, 0, (countF > 0 ? label : "no data"), scaleMin, scaleMax, ImVec2(fullW, graphH));
|
||||
ImGui::PopStyleColor(2);
|
||||
ImGui::PopID();
|
||||
}
|
||||
ImGui::End();
|
||||
|
||||
// Right side - 2 graphs
|
||||
windowPos = ImVec2(io.DisplaySize.x - 10.0f, 10.0f); // 10px margin
|
||||
ImGui::SetNextWindowPos(windowPos, ImGuiCond_Always, ImVec2(1.0f, 0.0f)); // pivot (1,0) = top-right
|
||||
ImGui::SetNextWindowSize(windowSize, ImGuiCond_Always);
|
||||
flags = ImGuiWindowFlags_NoDecoration |
|
||||
ImGuiWindowFlags_NoMove |
|
||||
ImGuiWindowFlags_NoNavFocus |
|
||||
ImGuiWindowFlags_NoBackground |
|
||||
ImGuiWindowFlags_NoSavedSettings;
|
||||
ImGui::Begin("##StatsRight", nullptr, flags);
|
||||
|
||||
for (int i = 2; i < PlotCount; i++) {
|
||||
if (self.plots[i].side != PLOT_RIGHT) continue;
|
||||
|
||||
float minY, maxY;
|
||||
int countF = [self.plots[i].buffer copyValuesIntoBuffer:buffers[i] min:&minY max:&maxY];
|
||||
float avgF = [self.plots[i].buffer averageValue];
|
||||
if (!countF) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Ugly, but can't get ImPlot to build for iOS
|
||||
char label[64];
|
||||
switch (self.plots[i].labelType) {
|
||||
case PLOT_LABEL_MIN_MAX_AVG:
|
||||
sprintf(label, "%s %.1f/%.1f/%.1f %s", self.plots[i].title, minY, maxY, avgF, self.plots[i].unit);
|
||||
break;
|
||||
case PLOT_LABEL_MIN_MAX_AVG_INT:
|
||||
sprintf(label, "%s %d/%d/%.1f %s", self.plots[i].title, (int)minY, (int)maxY, avgF, self.plots[i].unit);
|
||||
break;
|
||||
case PLOT_LABEL_TOTAL_INT:
|
||||
sprintf(label, "%s %d %s", self.plots[i].title, (int)[self.plots[i].buffer total], self.plots[i].unit);
|
||||
break;
|
||||
}
|
||||
float scaleMin = FLT_MAX;
|
||||
float scaleMax = FLT_MAX;
|
||||
if (self.plots[i].scaleTarget) {
|
||||
// optionally center the graph on a target such as the ideal frametime
|
||||
float ideal = (float)self.plots[i].scaleTarget;
|
||||
scaleMin = ideal - (2 * ideal);
|
||||
scaleMax = ideal + (2 * ideal);
|
||||
}
|
||||
if (self.plots[i].scaleMin)
|
||||
scaleMin = self.plots[i].scaleMin;
|
||||
if (self.plots[i].scaleMax)
|
||||
scaleMax = self.plots[i].scaleMax;
|
||||
ImGui::PushID(i);
|
||||
ImGui::PushStyleColor(ImGuiCol_FrameBg, ImVec4(0.16f, 0.29f, 0.48f, _graphOpacity)); // dark
|
||||
ImGui::PushStyleColor(ImGuiCol_PlotLines, ImVec4(0.0f, 1.0f, 0.0f, 1.0f)); // green
|
||||
if (i == PLOT_FRAMETIME || i == PLOT_HOST_FRAMETIME) {
|
||||
// getValue() clips at max 50
|
||||
ImGui::PlotLines("##xx", getValue, buffers[i], countF, 0, (countF > 0 ? label : "no data"), scaleMin, scaleMax, ImVec2(fullW, graphH));
|
||||
} else {
|
||||
ImGui::PlotLines("##xx", buffers[i], countF, 0, (countF > 0 ? label : "no data"), scaleMin, scaleMax, ImVec2(fullW, graphH));
|
||||
}
|
||||
ImGui::PopStyleColor(2);
|
||||
ImGui::PopID();
|
||||
}
|
||||
ImGui::End();
|
||||
|
||||
#endif
|
||||
}
|
||||
|
||||
@end
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
//
|
||||
// StreamView.h
|
||||
// Moonlight
|
||||
//
|
||||
// Created by Cameron Gutman on 10/19/14.
|
||||
// Copyright (c) 2014 Moonlight Stream. All rights reserved.
|
||||
//
|
||||
|
||||
#import "ControllerSupport.h"
|
||||
#import "OnScreenControls.h"
|
||||
#import "Moonlight-Swift.h"
|
||||
#import "StreamConfiguration.h"
|
||||
|
||||
@protocol UserInteractionDelegate <NSObject>
|
||||
|
||||
- (void) userInteractionBegan;
|
||||
- (void) userInteractionEnded;
|
||||
|
||||
@end
|
||||
|
||||
#if TARGET_OS_TV
|
||||
@interface StreamView : UIView <X1KitMouseDelegate, UITextFieldDelegate>
|
||||
#else
|
||||
@interface StreamView : UIView <X1KitMouseDelegate, UITextFieldDelegate, UIPointerInteractionDelegate>
|
||||
#endif
|
||||
|
||||
- (void) setupStreamView:(ControllerSupport*)controllerSupport
|
||||
interactionDelegate:(id<UserInteractionDelegate>)interactionDelegate
|
||||
config:(StreamConfiguration*)streamConfig;
|
||||
- (void) showOnScreenControls;
|
||||
- (OnScreenControlsLevel) getCurrentOscState;
|
||||
|
||||
#if !TARGET_OS_TV
|
||||
- (void) updateCursorLocation:(CGPoint)location isMouse:(BOOL)isMouse;
|
||||
#endif
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,13 @@
|
||||
// When enabled, rendering occurs on the main application thread.
|
||||
// This can make responding to UI events during redraw simpler
|
||||
// to manage because UI calls usually need to occur on the main thread.
|
||||
// When disabled, rendering occurs on a background thread, allowing
|
||||
// the UI to respond more quickly in some cases because events can
|
||||
// process asynchronously from potentially CPU-intensive rendering code.
|
||||
#define RENDER_ON_MAIN_THREAD 0
|
||||
|
||||
// When enabled, the drawable's size updates automatically whenever
|
||||
// the view resizes. When disabled, you can update the drawable's
|
||||
// size explicitly outside the view class.
|
||||
#define AUTOMATICALLY_RESIZE 1
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
#import <QuartzCore/CAMetalLayer.h>
|
||||
#import <QuartzCore/QuartzCore.h>
|
||||
#import "ConnectionCallbacks.h"
|
||||
#import "Frame.h"
|
||||
#import "Plot.h"
|
||||
|
||||
@interface MetalVideoRenderer : NSObject
|
||||
|
||||
@property (atomic) CFTimeInterval averageGPUTime;
|
||||
@property (nonatomic) NSUInteger sampleCount;
|
||||
@property (nonatomic) MTLPixelFormat colorPixelFormat;
|
||||
@property (nonatomic, nonnull) CGColorSpaceRef colorspace;
|
||||
@property (nonatomic) id<CAMetalDrawable> _Nullable nextDrawable;
|
||||
|
||||
- (nonnull instancetype)initWithMetalDevice:(nonnull id<MTLDevice>)device
|
||||
drawablePixelFormat:(MTLPixelFormat)drawablePixelFormat
|
||||
framerate:(float)framerate;
|
||||
- (void)renderFrame:(nonnull Frame *)frame toLayer:(nonnull CAMetalLayer *)layer;
|
||||
- (void)waitToRenderTo:(nonnull CAMetalLayer *)layer;
|
||||
- (void)drawableResize:(CGSize)drawableSize;
|
||||
- (void)plotFrametime:(CFTimeInterval)presentedTime;
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,524 @@
|
||||
#import "MetalVideoRenderer.h"
|
||||
#import <CoreVideo/CoreVideo.h>
|
||||
#import <Metal/Metal.h>
|
||||
#import <MetalKit/MetalKit.h>
|
||||
#import "ImGuiPlots.h"
|
||||
|
||||
#include <Limelight.h>
|
||||
|
||||
#define MAX_VIDEO_PLANES 3
|
||||
|
||||
struct CscParams {
|
||||
vector_float3 matrix[3];
|
||||
vector_float3 offsets;
|
||||
};
|
||||
|
||||
struct ParamBuffer {
|
||||
struct CscParams cscParams;
|
||||
};
|
||||
|
||||
static const struct CscParams k_CscParams_Bt601Lim = {
|
||||
// CSC Matrix
|
||||
{{1.1644f, 0.0f, 1.5960f}, {1.1644f, -0.3917f, -0.8129f}, {1.1644f, 2.0172f, 0.0f}},
|
||||
|
||||
// Offsets
|
||||
{16.0f / 255.0f, 128.0f / 255.0f, 128.0f / 255.0f},
|
||||
};
|
||||
static const struct CscParams k_CscParams_Bt601Full = {
|
||||
{
|
||||
{1.0f, 0.0f, 1.4020f},
|
||||
{1.0f, -0.3441f, -0.7141f},
|
||||
{1.0f, 1.7720f, 0.0f},
|
||||
},
|
||||
{0.0f, 128.0f / 255.0f, 128.0f / 255.0f},
|
||||
};
|
||||
static const struct CscParams k_CscParams_Bt709Lim = {
|
||||
{
|
||||
{1.1644f, 0.0f, 1.7927f},
|
||||
{1.1644f, -0.2132f, -0.5329f},
|
||||
{1.1644f, 2.1124f, 0.0f},
|
||||
},
|
||||
{16.0f / 255.0f, 128.0f / 255.0f, 128.0f / 255.0f},
|
||||
};
|
||||
static const struct CscParams k_CscParams_Bt709Full = {
|
||||
{
|
||||
{1.0f, 0.0f, 1.5748f},
|
||||
{1.0f, -0.1873f, -0.4681f},
|
||||
{1.0f, 1.8556f, 0.0f},
|
||||
},
|
||||
{0.0f, 128.0f / 255.0f, 128.0f / 255.0f},
|
||||
};
|
||||
static const struct CscParams k_CscParams_Bt2020Lim = {
|
||||
{
|
||||
{1.1644f, 0.0f, 1.6781f},
|
||||
{1.1644f, -0.1874f, -0.6505f},
|
||||
{1.1644f, 2.1418f, 0.0f},
|
||||
},
|
||||
{16.0f / 255.0f, 128.0f / 255.0f, 128.0f / 255.0f},
|
||||
};
|
||||
static const struct CscParams k_CscParams_Bt2020Full = {
|
||||
{
|
||||
{1.0f, 0.0f, 1.4746f},
|
||||
{1.0f, -0.1646f, -0.5714f},
|
||||
{1.0f, 1.8814f, 0.0f},
|
||||
},
|
||||
{0.0f, 128.0f / 255.0f, 128.0f / 255.0f},
|
||||
};
|
||||
|
||||
struct Vertex {
|
||||
vector_float4 position;
|
||||
vector_float2 texCoord;
|
||||
};
|
||||
|
||||
static const NSUInteger MaxFramesInFlight = 3;
|
||||
|
||||
@implementation MetalVideoRenderer {
|
||||
dispatch_queue_t _sq;
|
||||
id<MTLDevice> _device;
|
||||
float _framerate;
|
||||
id<ConnectionCallbacks> _callbacks;
|
||||
id<MTLCommandQueue> _commandQueue;
|
||||
id<MTLLibrary> _shaderLibrary;
|
||||
id<MTLRenderPipelineState> _videoPipelineState;
|
||||
MTLRenderPassDescriptor *_renderPassDescriptor;
|
||||
id<MTLTexture> _videoTexture;
|
||||
CVMetalTextureCacheRef _textureCache;
|
||||
CVMetalTextureRef _cvMetalTextures[MAX_VIDEO_PLANES];
|
||||
|
||||
int _lastColorSpace;
|
||||
BOOL _lastFullRange;
|
||||
size_t _lastFrameWidth;
|
||||
size_t _lastFrameHeight;
|
||||
size_t _lastDrawableWidth;
|
||||
size_t _lastDrawableHeight;
|
||||
id<MTLBuffer> _CscParamsBuffer;
|
||||
id<MTLBuffer> _VideoVertexBuffer;
|
||||
CFTimeInterval _lastPresented;
|
||||
|
||||
// https://developer.apple.com/documentation/metal/synchronizing-cpu-and-gpu-work?language=objc
|
||||
dispatch_semaphore_t _inFlightSemaphore;
|
||||
}
|
||||
|
||||
- (instancetype)initWithMetalDevice:(id<MTLDevice>)device drawablePixelFormat:(MTLPixelFormat)drawablePixelFormat framerate:(float)framerate {
|
||||
self = [super init];
|
||||
if (self) {
|
||||
_sq = dispatch_queue_create("com.moonlight.MetalVideoRenderer",
|
||||
dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_SERIAL, QOS_CLASS_USER_INTERACTIVE, 0));
|
||||
_averageGPUTime = (1.0f / framerate) / 2;
|
||||
_device = device;
|
||||
_nextDrawable = nil;
|
||||
_colorPixelFormat = MTLPixelFormatBGR10A2Unorm;
|
||||
_colorspace = CGColorSpaceCreateWithName(kCGColorSpaceITUR_2100_PQ);
|
||||
_framerate = framerate;
|
||||
_commandQueue = [_device newCommandQueue];
|
||||
_lastColorSpace = -1;
|
||||
_lastFullRange = NO;
|
||||
_lastPresented = 0;
|
||||
_inFlightSemaphore = dispatch_semaphore_create(MaxFramesInFlight);
|
||||
|
||||
CFStringRef keys[1] = {kCVMetalTextureUsage};
|
||||
NSUInteger values[1] = {MTLTextureUsageShaderRead};
|
||||
CFDictionaryRef cacheAttributes = CFDictionaryCreate(kCFAllocatorDefault, (const void **)keys, (const void **)values, 1, NULL, NULL);
|
||||
CVMetalTextureCacheCreate(kCFAllocatorDefault, cacheAttributes, _device, NULL, &_textureCache);
|
||||
CFRelease(cacheAttributes);
|
||||
|
||||
_renderPassDescriptor = [MTLRenderPassDescriptor new];
|
||||
_renderPassDescriptor.colorAttachments[0].loadAction = MTLLoadActionClear;
|
||||
_renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(0, 0, 0, 0);
|
||||
_renderPassDescriptor.colorAttachments[0].storeAction = MTLStoreActionStore;
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
#if !TARGET_OS_TV
|
||||
- (void)applyEDRFromFrame:(Frame *)frame toLayer:(CAMetalLayer *)layer {
|
||||
CFDictionaryRef ext = [frame getFormatDescExtensions];
|
||||
|
||||
Log(LOG_I, @"ext: %@", ext);
|
||||
|
||||
CFDataRef masteringData = CFDictionaryGetValue(ext, kCMFormatDescriptionExtension_MasteringDisplayColorVolume);
|
||||
CFDataRef contentDataRef = CFDictionaryGetValue(ext, kCMFormatDescriptionExtension_ContentLightLevelInfo);
|
||||
|
||||
if (masteringData) {
|
||||
Log(LOG_I, @"ext MDCV %@", masteringData);
|
||||
}
|
||||
if (contentDataRef) {
|
||||
Log(LOG_I, @"ext CLLI %@", contentDataRef);
|
||||
}
|
||||
|
||||
if (masteringData && CFDataGetLength(masteringData) == 24 && contentDataRef && CFDataGetLength(contentDataRef) == 4) {
|
||||
NSData *displayData = (__bridge NSData *)masteringData;
|
||||
NSData *contentData = (__bridge NSData *)contentDataRef;
|
||||
|
||||
layer.wantsExtendedDynamicRangeContent = YES;
|
||||
layer.pixelFormat = MTLPixelFormatRGBA16Float;
|
||||
CFStringRef name = kCGColorSpaceExtendedLinearITUR_2020;
|
||||
CGColorSpaceRef colorspace = CGColorSpaceCreateWithName(name);
|
||||
layer.colorspace = colorspace;
|
||||
|
||||
layer.EDRMetadata = [CAEDRMetadata HDR10MetadataWithDisplayInfo:displayData contentInfo:contentData opticalOutputScale:100.0f];
|
||||
|
||||
Log(LOG_I, @"EDRMetadata set from MDCV %@ and CLLI %@", displayData, contentData);
|
||||
} else {
|
||||
layer.wantsExtendedDynamicRangeContent = YES;
|
||||
layer.pixelFormat = MTLPixelFormatRGBA16Float;
|
||||
CFStringRef name = kCGColorSpaceExtendedLinearITUR_2020;
|
||||
CGColorSpaceRef colorspace = CGColorSpaceCreateWithName(name);
|
||||
layer.colorspace = colorspace;
|
||||
|
||||
layer.EDRMetadata = [CAEDRMetadata HDR10MetadataWithMinLuminance:0.0005f maxLuminance:1000.0f opticalOutputScale:100.0f];
|
||||
|
||||
Log(LOG_I, @"EDRMetadata set for 1000 nits");
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
- (int)getFrameColorspaceAndRange:(Frame *)frame isFullRange:(BOOL *)isFullRange {
|
||||
CFDictionaryRef ext = [frame getFormatDescExtensions];
|
||||
|
||||
// FQLog(LOG_I, @"%@", ext);
|
||||
|
||||
// Full Range boolean
|
||||
CFBooleanRef fullRangeRef = CFDictionaryGetValue(ext, kCMFormatDescriptionExtension_FullRangeVideo);
|
||||
*isFullRange = NO;
|
||||
if (fullRangeRef && CFGetTypeID(fullRangeRef) == CFBooleanGetTypeID()) {
|
||||
*isFullRange = CFBooleanGetValue(fullRangeRef);
|
||||
}
|
||||
|
||||
// Colorspace
|
||||
CFStringRef frame_color = CFDictionaryGetValue(ext, kCVImageBufferColorPrimariesKey);
|
||||
if (CFEqual(frame_color, kCVImageBufferColorPrimaries_ITU_R_709_2)) {
|
||||
return COLORSPACE_REC_709;
|
||||
} else if (CFEqual(frame_color, kCVImageBufferColorPrimaries_ITU_R_2020)) {
|
||||
return COLORSPACE_REC_2020;
|
||||
}
|
||||
return COLORSPACE_REC_601;
|
||||
}
|
||||
|
||||
- (BOOL)updateColorSpaceForFrame:(Frame *)frame toLayer:(CAMetalLayer *)layer layerDidChange:(BOOL *)layerDidChange {
|
||||
BOOL fullRange = NO;
|
||||
int colorspace = [self getFrameColorspaceAndRange:frame isFullRange:&fullRange];
|
||||
if (colorspace != _lastColorSpace || fullRange != _lastFullRange) {
|
||||
CGColorSpaceRef newColorSpace = nil;
|
||||
MTLPixelFormat newPixelFormat = layer.pixelFormat;
|
||||
BOOL isHDR = NO;
|
||||
struct ParamBuffer paramBuffer;
|
||||
|
||||
switch (colorspace) {
|
||||
case COLORSPACE_REC_709:
|
||||
newColorSpace = CGColorSpaceCreateWithName(kCGColorSpaceITUR_709);
|
||||
newPixelFormat = MTLPixelFormatBGRA8Unorm;
|
||||
paramBuffer.cscParams = (fullRange ? k_CscParams_Bt709Full : k_CscParams_Bt709Lim);
|
||||
break;
|
||||
case COLORSPACE_REC_2020: {
|
||||
CFDictionaryRef ext = [frame getFormatDescExtensions];
|
||||
CFStringRef frame_trc = CFDictionaryGetValue(ext, kCVImageBufferTransferFunctionKey);
|
||||
if (CFEqual(frame_trc, kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ)) {
|
||||
isHDR = YES;
|
||||
newColorSpace = CGColorSpaceCreateWithName(kCGColorSpaceITUR_2100_PQ);
|
||||
newPixelFormat = MTLPixelFormatBGR10A2Unorm;
|
||||
} else {
|
||||
// SDR 2020
|
||||
newColorSpace = CGColorSpaceCreateWithName(kCGColorSpaceITUR_2020);
|
||||
newPixelFormat = MTLPixelFormatBGR10A2Unorm;
|
||||
}
|
||||
paramBuffer.cscParams = (fullRange ? k_CscParams_Bt2020Full : k_CscParams_Bt2020Lim);
|
||||
break;
|
||||
}
|
||||
case COLORSPACE_REC_601:
|
||||
newColorSpace = CGColorSpaceCreateWithName(kCGColorSpaceSRGB);
|
||||
newPixelFormat = MTLPixelFormatBGRA8Unorm;
|
||||
paramBuffer.cscParams = (fullRange ? k_CscParams_Bt601Full : k_CscParams_Bt601Lim);
|
||||
}
|
||||
|
||||
// The CAMetalLayer retains the CGColorSpace
|
||||
if (newColorSpace || newPixelFormat != layer.pixelFormat) {
|
||||
*layerDidChange = YES;
|
||||
if (newColorSpace) {
|
||||
Log(LOG_I,
|
||||
@"Frame colorspace %@ - changing MetalLayer's colorspace to %@",
|
||||
colorspace == COLORSPACE_REC_709 ? @"REC_709"
|
||||
: colorspace == COLORSPACE_REC_2020 ? @"REC_2020"
|
||||
: colorspace == COLORSPACE_REC_601 ? @"REC_601 (sRGB)"
|
||||
: [NSString stringWithFormat:@"Unknown: %d", colorspace],
|
||||
newColorSpace);
|
||||
}
|
||||
if (newPixelFormat != layer.pixelFormat) {
|
||||
Log(LOG_I,
|
||||
@"Frame pixel format %@ - changing MetalLayer's pixel format to %@",
|
||||
layer.pixelFormat == MTLPixelFormatBGRA8Unorm ? @"MTLPixelFormatBGRA8Unorm"
|
||||
: layer.pixelFormat == MTLPixelFormatBGR10A2Unorm ? @"MTLPixelFormatBGR10A2Unorm"
|
||||
: [NSString stringWithFormat:@"Unknown: %lu", layer.pixelFormat],
|
||||
newPixelFormat == MTLPixelFormatBGRA8Unorm ? @"MTLPixelFormatBGRA8Unorm"
|
||||
: newPixelFormat == MTLPixelFormatBGR10A2Unorm ? @"MTLPixelFormatBGR10A2Unorm"
|
||||
: [NSString stringWithFormat:@"Unknown: %lu", (unsigned long)layer.pixelFormat]);
|
||||
}
|
||||
#if !RENDER_ON_MAIN_THREAD
|
||||
// These can only be changed on the main thread
|
||||
dispatch_sync(dispatch_get_main_queue(), ^{
|
||||
#endif
|
||||
#if !TARGET_OS_TV
|
||||
if (isHDR) {
|
||||
layer.wantsExtendedDynamicRangeContent = YES;
|
||||
}
|
||||
#endif
|
||||
layer.colorspace = newColorSpace;
|
||||
layer.pixelFormat = newPixelFormat;
|
||||
#if !RENDER_ON_MAIN_THREAD
|
||||
});
|
||||
#endif
|
||||
CGColorSpaceRelease(newColorSpace);
|
||||
}
|
||||
|
||||
// Create the new colorspace parameter buffer for our fragment shader
|
||||
MTLResourceOptions bufferOptions = MTLResourceStorageModeShared;
|
||||
_CscParamsBuffer = [_device newBufferWithBytes:(void *)¶mBuffer length:sizeof(paramBuffer) options:bufferOptions];
|
||||
if (!_CscParamsBuffer) {
|
||||
Log(LOG_E, @"Failed to create CSC parameters buffer");
|
||||
return NO;
|
||||
}
|
||||
|
||||
_lastColorSpace = colorspace;
|
||||
_lastFullRange = fullRange;
|
||||
}
|
||||
|
||||
return YES;
|
||||
}
|
||||
|
||||
- (void)scaleSource:(CGRect *)src toDest:(CGRect *)dst {
|
||||
int dstH = ceilf((float)dst->size.width * src->size.height / src->size.width);
|
||||
int dstW = ceilf((float)dst->size.height * src->size.width / src->size.height);
|
||||
|
||||
if (dstH > dst->size.height) {
|
||||
dst->origin.x += (dst->size.width - dstW) / 2;
|
||||
dst->size.width = dstW;
|
||||
} else {
|
||||
dst->origin.y += (dst->size.height - dstH) / 2;
|
||||
dst->size.height = dstH;
|
||||
}
|
||||
}
|
||||
|
||||
- (void)screenSpace:(CGRect *)src toNormalizedDeviceCoords:(CGRect *)dst withDrawableWidth:(int)viewportWidth drawableHeight:(int)viewportHeight {
|
||||
dst->origin.x = ((float)src->origin.x / (viewportWidth / 2.0f)) - 1.0f;
|
||||
dst->origin.y = ((float)src->origin.y / (viewportHeight / 2.0f)) - 1.0f;
|
||||
dst->size.width = (float)src->size.width / (viewportWidth / 2.0f);
|
||||
dst->size.height = (float)src->size.height / (viewportHeight / 2.0f);
|
||||
}
|
||||
|
||||
- (BOOL)updateVideoRegionSizeForFrame:(Frame *)frame toLayer:(CAMetalLayer *)layer {
|
||||
int drawableWidth = layer.drawableSize.width;
|
||||
int drawableHeight = layer.drawableSize.height;
|
||||
|
||||
// Check if anything has changed since the last vertex buffer upload
|
||||
if (_VideoVertexBuffer && [frame width] == _lastFrameWidth && [frame height] == _lastFrameHeight && drawableWidth == _lastDrawableWidth &&
|
||||
drawableHeight == _lastDrawableHeight) {
|
||||
// Nothing to do
|
||||
return YES;
|
||||
}
|
||||
|
||||
// Determine the correct scaled size for the video region
|
||||
CGRect src = CGRectMake(0.0, 0.0, [frame width], [frame height]);
|
||||
CGRect dst = CGRectMake(0.0, 0.0, drawableWidth, drawableHeight);
|
||||
[self scaleSource:&src toDest:&dst];
|
||||
|
||||
// Convert screen space to normalized device coordinates
|
||||
CGRect renderRect;
|
||||
[self screenSpace:&dst toNormalizedDeviceCoords:&renderRect withDrawableWidth:drawableWidth drawableHeight:drawableHeight];
|
||||
|
||||
struct Vertex verts[] = {
|
||||
{{renderRect.origin.x, renderRect.origin.y, 0.0f, 1.0f}, {0.0f, 1.0f}},
|
||||
{{renderRect.origin.x, renderRect.origin.y + renderRect.size.height, 0.0f, 1.0f}, {0.0f, 0}},
|
||||
{{renderRect.origin.x + renderRect.size.width, renderRect.origin.y, 0.0f, 1.0f}, {1.0f, 1.0f}},
|
||||
{{renderRect.origin.x + renderRect.size.width, renderRect.origin.y + renderRect.size.height, 0.0f, 1.0f}, {1.0f, 0}},
|
||||
};
|
||||
|
||||
MTLResourceOptions bufferOptions = MTLResourceStorageModeShared;
|
||||
_VideoVertexBuffer = [_device newBufferWithBytes:verts length:sizeof(verts) options:bufferOptions];
|
||||
if (!_VideoVertexBuffer) {
|
||||
Log(LOG_E, @"Failed to create video vertex buffer");
|
||||
return NO;
|
||||
}
|
||||
|
||||
_lastFrameWidth = [frame width];
|
||||
_lastFrameHeight = [frame height];
|
||||
_lastDrawableWidth = drawableWidth;
|
||||
_lastDrawableHeight = drawableHeight;
|
||||
|
||||
return YES;
|
||||
}
|
||||
|
||||
- (void)discardNextDrawable {
|
||||
_nextDrawable = nil;
|
||||
}
|
||||
|
||||
- (void)renderFrame:(Frame *)frame toLayer:(CAMetalLayer *)layer {
|
||||
// Handle changes to the frame's colorspace from last time we rendered
|
||||
BOOL layerDidChange = NO;
|
||||
if (![self updateColorSpaceForFrame:frame toLayer:layer layerDidChange:&layerDidChange]) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (layerDidChange && frame.frameNumber > 1) {
|
||||
Log(LOG_I, @"Metal frame changed layer's colorspace and/or pixel format, returning for new drawable");
|
||||
[self discardNextDrawable];
|
||||
return;
|
||||
}
|
||||
|
||||
// Handle changes to the video size or drawable size
|
||||
if (![self updateVideoRegionSizeForFrame:frame toLayer:layer]) {
|
||||
return;
|
||||
}
|
||||
|
||||
FQLog(LOG_I, @"[%d / %.3f ms] Metal frame rendering", frame.frameNumber, frame.pts);
|
||||
|
||||
#if !TARGET_OS_TV
|
||||
// Experimental EDR handling based on frame metadata
|
||||
//[self applyEDRFromFrame:frame toLayer:layer];
|
||||
#endif
|
||||
|
||||
size_t planes = CVPixelBufferGetPlaneCount(frame.pixelBuffer);
|
||||
assert(planes == 2 || planes == 3);
|
||||
|
||||
MTLRenderPipelineDescriptor *pipelineDesc = [MTLRenderPipelineDescriptor new];
|
||||
id<MTLLibrary> defaultLibrary = [_device newDefaultLibrary];
|
||||
pipelineDesc.vertexFunction = [defaultLibrary newFunctionWithName:@"vs_draw"];
|
||||
pipelineDesc.fragmentFunction = [defaultLibrary newFunctionWithName:planes == 2 ? @"ps_draw_biplanar" : @"ps_draw_triplanar"];
|
||||
pipelineDesc.colorAttachments[0].pixelFormat = layer.pixelFormat;
|
||||
pipelineDesc.vertexBuffers[0].mutability = MTLMutabilityImmutable;
|
||||
|
||||
NSError *error = nil;
|
||||
_videoPipelineState = [_device newRenderPipelineStateWithDescriptor:pipelineDesc error:&error];
|
||||
if (!_videoPipelineState) {
|
||||
Log(LOG_E, @"Failed to create video pipeline state: %@", error);
|
||||
return;
|
||||
}
|
||||
|
||||
for (size_t i = 0; i < planes; i++) {
|
||||
MTLPixelFormat fmt;
|
||||
|
||||
switch (CVPixelBufferGetPixelFormatType(frame.pixelBuffer)) {
|
||||
case kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange:
|
||||
case kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange:
|
||||
case kCVPixelFormatType_420YpCbCr8BiPlanarFullRange:
|
||||
case kCVPixelFormatType_444YpCbCr8BiPlanarFullRange:
|
||||
fmt = (i == 0) ? MTLPixelFormatR8Unorm : MTLPixelFormatRG8Unorm;
|
||||
break;
|
||||
|
||||
case kCVPixelFormatType_420YpCbCr10BiPlanarFullRange:
|
||||
case kCVPixelFormatType_444YpCbCr10BiPlanarFullRange:
|
||||
case kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange:
|
||||
case kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange:
|
||||
fmt = (i == 0) ? MTLPixelFormatR16Unorm : MTLPixelFormatRG16Unorm;
|
||||
break;
|
||||
|
||||
default:
|
||||
Log(LOG_E, @"Unknown pixel format: %@", CVPixelBufferGetPixelFormatType(frame.pixelBuffer));
|
||||
return;
|
||||
}
|
||||
|
||||
if (_cvMetalTextures[i]) {
|
||||
CVBufferRelease(_cvMetalTextures[i]);
|
||||
}
|
||||
CVReturn err = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
|
||||
_textureCache,
|
||||
frame.pixelBuffer,
|
||||
NULL,
|
||||
fmt,
|
||||
CVPixelBufferGetWidthOfPlane(frame.pixelBuffer, i),
|
||||
CVPixelBufferGetHeightOfPlane(frame.pixelBuffer, i),
|
||||
i,
|
||||
&_cvMetalTextures[i]);
|
||||
if (err != kCVReturnSuccess) {
|
||||
Log(LOG_E, @"CVMetalTextureCacheCreateTextureFromImage() failed: %d", err);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
_renderPassDescriptor.colorAttachments[0].texture = _nextDrawable.texture;
|
||||
|
||||
id<MTLCommandBuffer> commandBuffer = [_commandQueue commandBuffer];
|
||||
id<MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:_renderPassDescriptor];
|
||||
|
||||
[renderEncoder setRenderPipelineState:_videoPipelineState];
|
||||
for (size_t i = 0; i < planes; i++) {
|
||||
[renderEncoder setFragmentTexture:CVMetalTextureGetTexture(_cvMetalTextures[i]) atIndex:i];
|
||||
}
|
||||
[commandBuffer addCompletedHandler:^(id<MTLCommandBuffer> cb) {
|
||||
const CFTimeInterval GPUTime = cb.GPUEndTime - cb.GPUStartTime;
|
||||
const double alpha = 0.25f;
|
||||
self->_averageGPUTime = (GPUTime * alpha) + (self->_averageGPUTime * (1.0 - alpha));
|
||||
|
||||
// Free textures after completion of rendering per CVMetalTextureCache requirements
|
||||
for (size_t i = 0; i < planes; i++) {
|
||||
CVBufferRelease(self->_cvMetalTextures[i]);
|
||||
self->_cvMetalTextures[i] = nil;
|
||||
}
|
||||
|
||||
CVMetalTextureCacheFlush(self->_textureCache, 0);
|
||||
}];
|
||||
|
||||
[renderEncoder setFragmentBuffer:_CscParamsBuffer offset:0 atIndex:0];
|
||||
[renderEncoder setVertexBuffer:_VideoVertexBuffer offset:0 atIndex:0];
|
||||
[renderEncoder drawPrimitives:MTLPrimitiveTypeTriangleStrip vertexStart:0 vertexCount:4];
|
||||
[renderEncoder endEncoding];
|
||||
|
||||
__block dispatch_semaphore_t block_semaphore = _inFlightSemaphore;
|
||||
[commandBuffer addCompletedHandler:^(id<MTLCommandBuffer> buffer) {
|
||||
dispatch_semaphore_signal(block_semaphore);
|
||||
}];
|
||||
|
||||
__weak typeof(self) self_ = self;
|
||||
[_nextDrawable addPresentedHandler:^(id<MTLDrawable> d) {
|
||||
if (self_) {
|
||||
[self_ plotFrametime:d.presentedTime];
|
||||
}
|
||||
}];
|
||||
|
||||
#if TARGET_OS_SIMULATOR
|
||||
[commandBuffer presentDrawable:_nextDrawable];
|
||||
#else
|
||||
// present for a minimum duration for best frame pacing
|
||||
[commandBuffer presentDrawable:_nextDrawable afterMinimumDuration:1.0f / _framerate];
|
||||
#endif
|
||||
|
||||
[commandBuffer commit];
|
||||
|
||||
// Wait for the command buffer to complete and free our CVMetalTextureCache references
|
||||
[commandBuffer waitUntilCompleted];
|
||||
|
||||
_nextDrawable = nil;
|
||||
}
|
||||
|
||||
- (void)plotFrametime:(CFTimeInterval)presentedTime {
|
||||
if (_lastPresented > 0) {
|
||||
CFTimeInterval frametime = presentedTime - _lastPresented;
|
||||
[[ImGuiPlots sharedInstance] observeFloat:PLOT_FRAMETIME value:(frametime * 1000.0)];
|
||||
}
|
||||
_lastPresented = presentedTime;
|
||||
}
|
||||
|
||||
- (void)waitToRenderTo:(nonnull CAMetalLayer *)layer {
|
||||
if (!_nextDrawable) {
|
||||
// Wait for the next available drawable
|
||||
_nextDrawable = [layer nextDrawable];
|
||||
if (!_nextDrawable) {
|
||||
Log(LOG_E, @"Error getting nextDrawable from CAMetalLayer");
|
||||
return;
|
||||
}
|
||||
|
||||
// Wait to ensure only `MaxFramesInFlight` number of frames are getting processed
|
||||
// by any stage in the Metal pipeline (CPU, GPU, Metal, Drivers, etc.).
|
||||
dispatch_semaphore_wait(_inFlightSemaphore, DISPATCH_TIME_FOREVER);
|
||||
}
|
||||
}
|
||||
|
||||
/// Responds to the drawable's size or orientation changes.
|
||||
- (void)drawableResize:(CGSize)drawableSize {
|
||||
[self resize:drawableSize];
|
||||
}
|
||||
|
||||
- (void)resize:(CGSize)size {
|
||||
// TODO?
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,36 @@
|
||||
#import <Metal/Metal.h>
|
||||
#import <QuartzCore/CAMetalDisplayLink.h>
|
||||
#import <QuartzCore/CAMetalLayer.h>
|
||||
#import "MetalConfig.h"
|
||||
|
||||
#if TARGET_OS_IOS || TARGET_OS_TV
|
||||
#import <UIKit/UIKit.h>
|
||||
#define PlatformView UIView
|
||||
#else
|
||||
#import <AppKit/AppKit.h>
|
||||
#define PlatformView NSView
|
||||
#endif
|
||||
|
||||
// The protocol to provide resize and redraw callbacks to a delegate.
|
||||
@protocol MetalViewDelegate <NSObject>
|
||||
|
||||
- (void)drawableResize:(CGSize)size;
|
||||
- (void)renderTo:(nonnull CAMetalLayer *)layer;
|
||||
- (void)waitToRenderTo:(nonnull CAMetalLayer *)layer;
|
||||
|
||||
@end
|
||||
|
||||
// The Metal game view base class.
|
||||
@interface MetalView : PlatformView <CALayerDelegate>
|
||||
|
||||
@property (nonatomic, nonnull, readonly) CAMetalLayer *metalLayer;
|
||||
@property (nonatomic, getter=isPaused) BOOL paused;
|
||||
@property (nonatomic, nullable) id<MetalViewDelegate> delegate;
|
||||
@property (nonatomic) float framerate;
|
||||
|
||||
- (void)initCommon;
|
||||
#if AUTOMATICALLY_RESIZE
|
||||
- (void)resizeDrawable:(CGFloat)scaleFactor;
|
||||
#endif
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,205 @@
|
||||
// This is based on the following Apple example:
|
||||
// https://developer.apple.com/documentation/metal/achieving-smooth-frame-rates-with-a-metal-display-link?language=objc
|
||||
// https://developer.apple.com/wwdc23/10123/
|
||||
|
||||
#import "MetalView.h"
|
||||
#import "MetalConfig.h"
|
||||
|
||||
@implementation MetalView {
|
||||
#if !RENDER_ON_MAIN_THREAD
|
||||
// The secondary thread containing the render loop.
|
||||
NSThread *_renderThread;
|
||||
|
||||
// The flag to indicate that rendering needs to cease on the main thread.
|
||||
BOOL _continueRunLoop;
|
||||
#endif
|
||||
}
|
||||
|
||||
#pragma mark - Initialization and Setup.
|
||||
|
||||
- (instancetype)initWithFrame:(CGRect)frame {
|
||||
self = [super initWithFrame:frame];
|
||||
if (self) {
|
||||
[self initCommon];
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (instancetype)initWithCoder:(NSCoder *)aDecoder {
|
||||
self = [super initWithCoder:aDecoder];
|
||||
if (self) {
|
||||
[self initCommon];
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (void)initCommon {
|
||||
#if TARGET_OS_OSX
|
||||
self.wantsLayer = YES;
|
||||
|
||||
self.layerContentsRedrawPolicy = NSViewLayerContentsRedrawDuringViewResize;
|
||||
#endif
|
||||
|
||||
_metalLayer = (CAMetalLayer *)self.layer;
|
||||
|
||||
self.layer.delegate = self;
|
||||
}
|
||||
|
||||
#if TARGET_OS_IOS || TARGET_OS_TV
|
||||
+ (Class)layerClass {
|
||||
return [CAMetalLayer class];
|
||||
}
|
||||
|
||||
- (void)didMoveToWindow {
|
||||
[self movedToWindow];
|
||||
}
|
||||
#else
|
||||
- (CALayer *)makeBackingLayer {
|
||||
return [CAMetalLayer layer];
|
||||
}
|
||||
|
||||
- (void)viewDidMoveToWindow {
|
||||
[self movedToWindow];
|
||||
}
|
||||
#endif // END TARGET_OS_IOS || TARGET_OS_TV
|
||||
|
||||
- (void)movedToWindow {
|
||||
#if !RENDER_ON_MAIN_THREAD
|
||||
// Protect _continueRunLoop with a `@synchronized` block because it's accessed by the separate
|
||||
// animation thread.
|
||||
@synchronized(self) {
|
||||
// Stop the animation loop, allowing it to complete if it's in progress.
|
||||
_continueRunLoop = NO;
|
||||
}
|
||||
|
||||
// Create and start a secondary NSThread that has another run runloop. The NSThread
|
||||
// class calls the 'runThread' method at the start of the secondary thread's execution.
|
||||
_renderThread = [[NSThread alloc] initWithTarget:self selector:@selector(runThread) object:nil];
|
||||
_continueRunLoop = YES;
|
||||
_renderThread.qualityOfService = NSQualityOfServiceUserInteractive;
|
||||
[_renderThread start];
|
||||
#endif // END !RENDER_ON_MAIN_THREAD
|
||||
|
||||
// Perform any actions that need to know the size and scale of the drawable. When UIKit calls
|
||||
// didMoveToWindow after the view initialization, this is the first opportunity to notify
|
||||
// components of the drawable's size.
|
||||
#if AUTOMATICALLY_RESIZE
|
||||
#if TARGET_OS_IOS || TARGET_OS_TV
|
||||
[self resizeDrawable:self.window.screen.nativeScale];
|
||||
#else
|
||||
[self resizeDrawable:self.window.screen.backingScaleFactor];
|
||||
#endif
|
||||
#else
|
||||
// Notify the delegate of the default drawable size when the system can calculate it.
|
||||
CGSize defaultDrawableSize = self.bounds.size;
|
||||
defaultDrawableSize.width *= self.layer.contentsScale;
|
||||
defaultDrawableSize.height *= self.layer.contentsScale;
|
||||
[self.delegate drawableResize:defaultDrawableSize];
|
||||
#endif
|
||||
}
|
||||
|
||||
#if !RENDER_ON_MAIN_THREAD
|
||||
- (void)runThread {
|
||||
// The system sets the '_continueRunLoop' ivar outside this thread, so it needs to synchronize. Create a
|
||||
// 'continueRunLoop' local var that the system can set from the _continueRunLoop ivar in a @synchronized block.
|
||||
BOOL continueRunLoop = YES;
|
||||
|
||||
// Begin the run loop.
|
||||
while (continueRunLoop) {
|
||||
@autoreleasepool {
|
||||
[_delegate waitToRenderTo:_metalLayer];
|
||||
|
||||
@synchronized(self) {
|
||||
continueRunLoop = _continueRunLoop;
|
||||
}
|
||||
if (!continueRunLoop) {
|
||||
break;
|
||||
}
|
||||
|
||||
[_delegate renderTo:_metalLayer];
|
||||
|
||||
@synchronized(self) {
|
||||
continueRunLoop = _continueRunLoop;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif // END !RENDER_ON_MAIN_THREAD
|
||||
|
||||
#pragma mark - Resizing
|
||||
|
||||
#if AUTOMATICALLY_RESIZE
|
||||
|
||||
// Override all methods that indicate the view's size has changed.
|
||||
|
||||
#if TARGET_OS_IOS || TARGET_OS_TV
|
||||
- (void)setContentScaleFactor:(CGFloat)contentScaleFactor {
|
||||
[super setContentScaleFactor:contentScaleFactor];
|
||||
[self resizeDrawable:self.window.screen.nativeScale];
|
||||
}
|
||||
|
||||
- (void)layoutSubviews {
|
||||
[super layoutSubviews];
|
||||
[self resizeDrawable:self.window.screen.nativeScale];
|
||||
}
|
||||
|
||||
- (void)setFrame:(CGRect)frame {
|
||||
[super setFrame:frame];
|
||||
[self resizeDrawable:self.window.screen.nativeScale];
|
||||
}
|
||||
|
||||
- (void)setBounds:(CGRect)bounds {
|
||||
[super setBounds:bounds];
|
||||
[self resizeDrawable:self.window.screen.nativeScale];
|
||||
}
|
||||
#else
|
||||
- (void)viewDidChangeBackingProperties {
|
||||
[super viewDidChangeBackingProperties];
|
||||
[self resizeDrawable:self.window.screen.backingScaleFactor];
|
||||
}
|
||||
|
||||
- (void)setFrameSize:(NSSize)size {
|
||||
[super setFrameSize:size];
|
||||
[self resizeDrawable:self.window.screen.backingScaleFactor];
|
||||
}
|
||||
|
||||
- (void)setBoundsSize:(NSSize)size {
|
||||
[super setBoundsSize:size];
|
||||
[self resizeDrawable:self.window.screen.backingScaleFactor];
|
||||
}
|
||||
#endif
|
||||
|
||||
- (void)resizeDrawable:(CGFloat)scaleFactor {
|
||||
CGSize newSize = self.bounds.size;
|
||||
newSize.width *= scaleFactor;
|
||||
newSize.height *= scaleFactor;
|
||||
|
||||
if (newSize.width <= 0 || newSize.width <= 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
#if RENDER_ON_MAIN_THREAD
|
||||
if (newSize.width == _metalLayer.drawableSize.width && newSize.height == _metalLayer.drawableSize.height) {
|
||||
return;
|
||||
}
|
||||
|
||||
_metalLayer.drawableSize = newSize;
|
||||
|
||||
[_delegate drawableResize:newSize];
|
||||
#else
|
||||
// The system calls all AppKit and UIKit calls that notify of a resize on the main thread. Use
|
||||
// a synchronized block to ensure that resize notifications on the delegate are atomic.
|
||||
@synchronized(_metalLayer) {
|
||||
if (newSize.width == _metalLayer.drawableSize.width && newSize.height == _metalLayer.drawableSize.height) {
|
||||
return;
|
||||
}
|
||||
|
||||
_metalLayer.drawableSize = newSize;
|
||||
|
||||
[_delegate drawableResize:newSize];
|
||||
}
|
||||
#endif
|
||||
}
|
||||
#endif // END AUTOMATICALLY_RESIZE
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,24 @@
|
||||
#import <Metal/Metal.h>
|
||||
#import "FrameQueue.h"
|
||||
#import "ImGuiRenderer.h"
|
||||
#import "MetalVideoRenderer.h"
|
||||
#import "MetalView.h"
|
||||
|
||||
#if TARGET_OS_IOS || TARGET_OS_TV
|
||||
#import <UIKit/UIKit.h>
|
||||
#define PlatformViewController UIViewController
|
||||
#else
|
||||
#import <AppKit/AppKit.h>
|
||||
#define PlatformViewController NSViewController
|
||||
#endif
|
||||
|
||||
@interface MetalViewController : PlatformViewController <MetalViewDelegate>
|
||||
|
||||
@property (nonatomic) CGRect bounds;
|
||||
|
||||
- (nonnull instancetype)initWithFrame:(CGRect)bounds
|
||||
framerate:(float)framerate
|
||||
enableHdr:(BOOL)enableHdr
|
||||
metricsHandler:(MetricsHandler _Nonnull)metricsHandler;
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,144 @@
|
||||
/*
|
||||
See the LICENSE.txt file for this sample’s licensing information.
|
||||
|
||||
Abstract:
|
||||
The implementation of the cross-platform game view controller.
|
||||
*/
|
||||
|
||||
#import "MetalViewController.h"
|
||||
#import "FrameQueue.h"
|
||||
#import "ImGuiRenderer.h"
|
||||
#import "MetalVideoRenderer.h"
|
||||
|
||||
@implementation MetalViewController {
|
||||
/// A queue to initialize the renderer asynchronously from the main thread.
|
||||
dispatch_queue_t _dispatch_queue;
|
||||
FrameQueue *_frameQueue;
|
||||
float _framerate;
|
||||
BOOL _enableHdr;
|
||||
MetalView *_metalView;
|
||||
MetalVideoRenderer *_renderer;
|
||||
MetricsHandler _metricsHandler;
|
||||
BOOL _stopping;
|
||||
}
|
||||
|
||||
- (nonnull instancetype)initWithFrame:(CGRect)bounds framerate:(float)framerate enableHdr:(BOOL)enableHdr metricsHandler:(MetricsHandler)metricsHandler {
|
||||
self = [super init];
|
||||
if (self) {
|
||||
_bounds = bounds;
|
||||
_frameQueue = [FrameQueue sharedInstance];
|
||||
_framerate = framerate;
|
||||
_enableHdr = enableHdr;
|
||||
_metricsHandler = metricsHandler;
|
||||
_stopping = NO;
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (void)loadView {
|
||||
self.view = [[MetalView alloc] initWithFrame:_bounds];
|
||||
}
|
||||
|
||||
- (void)viewDidLoad {
|
||||
[super viewDidLoad];
|
||||
|
||||
/// A queue to initialize the renderer asynchronously from the main thread.
|
||||
_dispatch_queue = dispatch_queue_create("com.moonlight.Metal", DISPATCH_QUEUE_CONCURRENT);
|
||||
|
||||
__block MetalView *view = (MetalView *)self.view;
|
||||
if (!view) {
|
||||
Log(LOG_E, @"The view attached to MetalViewController isn't a MetalView.");
|
||||
return;
|
||||
}
|
||||
_metalView = view;
|
||||
_metalView.delegate = self;
|
||||
_metalView.framerate = _framerate;
|
||||
|
||||
// Select the device to render with.
|
||||
id<MTLDevice> device = MTLCreateSystemDefaultDevice();
|
||||
if (!device) {
|
||||
Log(LOG_E, @"Metal isn't supported on this device.");
|
||||
self.view = [[PlatformView alloc] initWithFrame:self.view.frame];
|
||||
return;
|
||||
}
|
||||
view.metalLayer.device = device;
|
||||
|
||||
// Initialize the renderer.
|
||||
MetalVideoRenderer *renderer = [[MetalVideoRenderer alloc] initWithMetalDevice:device
|
||||
drawablePixelFormat:MTLPixelFormatBGR10A2Unorm
|
||||
framerate:self->_framerate];
|
||||
if (!renderer) {
|
||||
Log(LOG_E, @"The renderer couldn't be initialized.");
|
||||
return;
|
||||
}
|
||||
|
||||
// Initialize the renderer-dependent view properties.
|
||||
view.metalLayer.pixelFormat = renderer.colorPixelFormat;
|
||||
view.metalLayer.colorspace = renderer.colorspace;
|
||||
view.metalLayer.maximumDrawableCount = 3;
|
||||
|
||||
self->_renderer = renderer;
|
||||
}
|
||||
|
||||
- (void)waitToRenderTo:(nonnull CAMetalLayer *)layer {
|
||||
if (!_stopping) {
|
||||
// Renderer obtains a nextDrawable, waiting if necessary
|
||||
FQLog(LOG_I, @"[MetalViewController] caling [_renderer waitToRenderTo:layer]");
|
||||
[_renderer waitToRenderTo:layer];
|
||||
|
||||
// If we don't have a frame yet, wait on that too
|
||||
FQLog(LOG_I, @"[MetalViewController] caling [_renderer waitForEnqueue]");
|
||||
[_frameQueue waitForEnqueue];
|
||||
}
|
||||
}
|
||||
|
||||
/// Draw frame (used by manual loop)
|
||||
- (void)renderTo:(nonnull CAMetalLayer *)layer {
|
||||
if (!_renderer) {
|
||||
return;
|
||||
}
|
||||
|
||||
CFTimeInterval timeout = (1.0f / _framerate) - _renderer.averageGPUTime;
|
||||
FQLog(LOG_I, @"[MetalViewController] caling [_frameQueue dequeueWithTimeout:%f]", timeout);
|
||||
Frame *frame = [_frameQueue dequeueWithTimeout:timeout];
|
||||
if (frame) {
|
||||
FQLog(LOG_I, @"[MetalViewController] calling [_renderer renderFrame] %@", frame);
|
||||
[_renderer renderFrame:frame toLayer:layer];
|
||||
}
|
||||
}
|
||||
|
||||
- (void)drawableResize:(CGSize)size {
|
||||
[_renderer drawableResize:size];
|
||||
}
|
||||
|
||||
#if TARGET_OS_IOS
|
||||
/// Hides the Home indicator button automatically.
|
||||
- (BOOL)prefersHomeIndicatorAutoHidden {
|
||||
return YES;
|
||||
}
|
||||
#endif
|
||||
|
||||
#if TARGET_OS_OSX
|
||||
/// Makes the view controller the first responder to receive keyboard events.
|
||||
- (void)viewDidAppear {
|
||||
[_metalView.window makeFirstResponder:self];
|
||||
}
|
||||
|
||||
/// Receives the keydown events to avoid system beeps.
|
||||
///
|
||||
/// The `GameInputKeyboardMouse` class handles keyboard events.
|
||||
- (void)keyDown:(NSEvent *)event {
|
||||
// Reference the parameter to avoid an unused parameter warning.
|
||||
(void)(event);
|
||||
}
|
||||
|
||||
/// Receives the keyup events to avoid system beeps.
|
||||
///
|
||||
/// The `GameInputKeyboardMouse` class handles keyboard events.
|
||||
- (void)keyUp:(NSEvent *)event {
|
||||
// Reference the parameter to avoid an unused parameter warning.
|
||||
(void)(event);
|
||||
}
|
||||
#endif
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,63 @@
|
||||
#include <metal_stdlib>
|
||||
#include <simd/simd.h>
|
||||
|
||||
using namespace metal;
|
||||
|
||||
struct Vertex
|
||||
{
|
||||
float4 position [[ position ]];
|
||||
float2 texCoords;
|
||||
};
|
||||
|
||||
struct CscParams
|
||||
{
|
||||
float3 matrix[3];
|
||||
float3 offsets;
|
||||
};
|
||||
|
||||
constexpr sampler s(coord::normalized, address::clamp_to_edge, filter::linear);
|
||||
|
||||
vertex Vertex vs_draw(constant Vertex *vertices [[ buffer(0) ]], uint id [[ vertex_id ]])
|
||||
{
|
||||
return vertices[id];
|
||||
}
|
||||
|
||||
fragment float4 ps_draw_biplanar(Vertex v [[ stage_in ]],
|
||||
constant CscParams &cscParams [[ buffer(0) ]],
|
||||
texture2d<float> luminancePlane [[ texture(0) ]],
|
||||
texture2d<float> chrominancePlane [[ texture(1) ]])
|
||||
{
|
||||
float3 yuv = float3(luminancePlane.sample(s, v.texCoords).r,
|
||||
chrominancePlane.sample(s, v.texCoords).rg);
|
||||
yuv -= cscParams.offsets;
|
||||
|
||||
float3 rgb;
|
||||
rgb.r = dot(yuv, cscParams.matrix[0]);
|
||||
rgb.g = dot(yuv, cscParams.matrix[1]);
|
||||
rgb.b = dot(yuv, cscParams.matrix[2]);
|
||||
return float4(rgb, 1.0f);
|
||||
}
|
||||
|
||||
fragment float4 ps_draw_triplanar(Vertex v [[ stage_in ]],
|
||||
constant CscParams &cscParams [[ buffer(0) ]],
|
||||
texture2d<float> luminancePlane [[ texture(0) ]],
|
||||
texture2d<float> chrominancePlaneU [[ texture(1) ]],
|
||||
texture2d<float> chrominancePlaneV [[ texture(2) ]])
|
||||
{
|
||||
float3 yuv = float3(luminancePlane.sample(s, v.texCoords).r,
|
||||
chrominancePlaneU.sample(s, v.texCoords).r,
|
||||
chrominancePlaneV.sample(s, v.texCoords).r);
|
||||
yuv -= cscParams.offsets;
|
||||
|
||||
float3 rgb;
|
||||
rgb.r = dot(yuv, cscParams.matrix[0]);
|
||||
rgb.g = dot(yuv, cscParams.matrix[1]);
|
||||
rgb.b = dot(yuv, cscParams.matrix[2]);
|
||||
return float4(rgb, 1.0f);
|
||||
}
|
||||
|
||||
fragment float4 ps_draw_rgb(Vertex v [[ stage_in ]],
|
||||
texture2d<float> rgbTexture [[ texture(0) ]])
|
||||
{
|
||||
return rgbTexture.sample(s, v.texCoords);
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
#import <Foundation/Foundation.h>
|
||||
#import <VideoToolbox/VideoToolbox.h>
|
||||
|
||||
#import "FloatBuffer.h"
|
||||
#import "Plot.h"
|
||||
|
||||
@interface ImGuiPlots : NSObject
|
||||
|
||||
struct PlotDef {
|
||||
FloatBuffer * _Nonnull buffer;
|
||||
const char * _Nonnull title;
|
||||
PlotSide side;
|
||||
PlotLabelType labelType;
|
||||
const char * _Nonnull unit;
|
||||
double scaleMin, scaleMax, scaleTarget;
|
||||
float minY;
|
||||
float maxY;
|
||||
};
|
||||
|
||||
@property (nonatomic) struct PlotDef * _Nonnull plots;
|
||||
|
||||
+ (instancetype _Nonnull)sharedInstance;
|
||||
|
||||
- (instancetype _Nonnull)init NS_UNAVAILABLE;
|
||||
+ (instancetype _Nonnull)new NS_UNAVAILABLE;
|
||||
|
||||
- (void)observeFloat:(int)plotId value:(CFTimeInterval)value;
|
||||
- (void)observeFloatReturnMetrics:(int)plotId value:(CFTimeInterval)value plotMetrics:(PlotMetrics * _Nullable)plotMetrics;
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,95 @@
|
||||
#import "ImGuiPlots.h"
|
||||
|
||||
@implementation ImGuiPlots
|
||||
|
||||
+ (instancetype)sharedInstance {
|
||||
static ImGuiPlots *sharedInstance = nil;
|
||||
static dispatch_once_t onceToken;
|
||||
dispatch_once(&onceToken, ^{
|
||||
sharedInstance = [[super allocWithZone:NULL] _initSingleton];
|
||||
});
|
||||
return sharedInstance;
|
||||
}
|
||||
|
||||
// Prevent others from using alloc/init directly
|
||||
+ (instancetype)allocWithZone:(struct _NSZone *)zone {
|
||||
return [self sharedInstance];
|
||||
}
|
||||
|
||||
// If someone tries to copy it, just return the same instance
|
||||
- (id)copyWithZone:(NSZone *)zone {
|
||||
return self;
|
||||
}
|
||||
|
||||
- (id)mutableCopyWithZone:(NSZone *)zone {
|
||||
return self;
|
||||
}
|
||||
|
||||
-(nonnull instancetype)_initSingleton
|
||||
{
|
||||
self = [super init];
|
||||
if (self) {
|
||||
_plots = (PlotDef *)calloc(PlotCount, sizeof(PlotDef));
|
||||
|
||||
_plots[PLOT_FRAMETIME] = {
|
||||
.title = "Frametime",
|
||||
.side = PLOT_LEFT,
|
||||
.unit = "ms",
|
||||
.scaleMin = (1000.0 / 120) - 1,
|
||||
.scaleMax = 50.0f, // (1000.0 / streamFps) * 3,
|
||||
.buffer = [[FloatBuffer alloc] initWithCapacity:512]
|
||||
};
|
||||
|
||||
_plots[PLOT_HOST_FRAMETIME] = {
|
||||
.title = "Host Frametime",
|
||||
.side = PLOT_LEFT,
|
||||
.unit = "ms",
|
||||
.scaleMin = (1000.0 / 120) - 1,
|
||||
.scaleMax = 50.0f, // (1000.0 / streamFps) * 3,
|
||||
.buffer = [[FloatBuffer alloc] initWithCapacity:512]
|
||||
};
|
||||
|
||||
_plots[PLOT_QUEUED_FRAMES] = {
|
||||
.title = "Frame queue",
|
||||
.side = PLOT_RIGHT,
|
||||
.labelType = PLOT_LABEL_MIN_MAX_AVG_INT,
|
||||
.unit = "",
|
||||
.scaleMin = -0.5,
|
||||
.scaleMax = 15,
|
||||
.buffer = [[FloatBuffer alloc] initWithCapacity:512]
|
||||
};
|
||||
|
||||
_plots[PLOT_DROPPED] = {
|
||||
.title = "Frames dropped",
|
||||
.side = PLOT_RIGHT,
|
||||
.labelType = PLOT_LABEL_TOTAL_INT,
|
||||
.unit = "",
|
||||
.scaleTarget = 2,
|
||||
.buffer = [[FloatBuffer alloc] initWithCapacity:512]
|
||||
};
|
||||
|
||||
// not graphed, but used for stats
|
||||
|
||||
_plots[PLOT_DECODE] = {
|
||||
.title = "Decode time",
|
||||
.labelType = PLOT_LABEL_MIN_MAX_AVG,
|
||||
.unit = "ms",
|
||||
.buffer = [[FloatBuffer alloc] initWithCapacity:512]
|
||||
};
|
||||
}
|
||||
|
||||
return self;
|
||||
}
|
||||
|
||||
- (void) observeFloat:(int)plotId value:(CFTimeInterval)value {
|
||||
[_plots[plotId].buffer addValue:(float)value];
|
||||
}
|
||||
|
||||
- (void) observeFloatReturnMetrics:(int)plotId value:(CFTimeInterval)value plotMetrics:(PlotMetrics *)plotMetrics {
|
||||
[_plots[plotId].buffer addValue:(float)value];
|
||||
if (plotMetrics != nil) {
|
||||
[_plots[plotId].buffer copyMetrics:plotMetrics];
|
||||
}
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -31,7 +31,8 @@
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *codecSelector;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *yuv444Selector;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *hdrSelector;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *framePacingSelector;
|
||||
@property (strong, nonatomic) IBOutlet UILabel *frameQueueSizeLabel;
|
||||
@property (strong, nonatomic) IBOutlet UISlider *frameQueueSizeSlider;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *btMouseSelector;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *reverseMouseWheelDirectionSelector;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *statsOverlaySelector;
|
||||
@@ -61,6 +62,10 @@
|
||||
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *externalDisplayModeSelector;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *localMousePointerModeSelector;
|
||||
@property (strong, nonatomic) IBOutlet UILabel *enableGraphsLabel;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *enableGraphsSelector;
|
||||
@property (strong, nonatomic) IBOutlet UIStepper *graphOpacityStepper;
|
||||
@property (strong, nonatomic) IBOutlet UISegmentedControl *renderingBackendSelector;
|
||||
|
||||
#pragma clang diagnostic push
|
||||
#pragma clang diagnostic ignored "-Wunguarded-availability"
|
||||
|
||||
@@ -22,6 +22,8 @@
|
||||
bool justEnteredSettingsViewDoNotOpenOscLayoutTool;
|
||||
uint16_t oscLayoutFingers;
|
||||
CustomEdgeSlideGestureRecognizer *slideToCloseSettingsViewRecognizer;
|
||||
NSInteger _frameQueueSize;
|
||||
NSInteger _graphOpacity;
|
||||
}
|
||||
|
||||
@dynamic overrideUserInterfaceStyle;
|
||||
@@ -366,6 +368,8 @@ BOOL isCustomResolution(CGSize res) {
|
||||
|
||||
// Ensure we pick a bitrate that falls exactly onto a slider notch
|
||||
_bitrate = bitrateTable[[self getSliderValueForBitrate:[currentSettings.bitrate intValue]]];
|
||||
_frameQueueSize = [currentSettings.frameQueueSize intValue];
|
||||
_graphOpacity = [currentSettings.graphOpacity intValue];
|
||||
|
||||
// Get the size of the screen with and without safe area insets
|
||||
UIWindow *window = UIApplication.sharedApplication.windows.firstObject;
|
||||
@@ -379,9 +383,9 @@ BOOL isCustomResolution(CGSize res) {
|
||||
UITapGestureRecognizer *resolutionDisplayViewTap = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(resolutionDisplayViewTapped:)];
|
||||
[self.resolutionDisplayView addGestureRecognizer:resolutionDisplayViewTap];
|
||||
|
||||
resolutionTable[0] = CGSizeMake(640, 360);
|
||||
resolutionTable[1] = CGSizeMake(1280, 720);
|
||||
resolutionTable[2] = CGSizeMake(1920, 1080);
|
||||
resolutionTable[0] = CGSizeMake(1280, 720);
|
||||
resolutionTable[1] = CGSizeMake(1920, 1080);
|
||||
resolutionTable[2] = CGSizeMake(2560, 1440);
|
||||
resolutionTable[3] = CGSizeMake(3840, 2160);
|
||||
resolutionTable[4] = CGSizeMake(safeAreaWidth, fullScreenHeight);
|
||||
resolutionTable[5] = CGSizeMake(fullScreenWidth, fullScreenHeight);
|
||||
@@ -393,6 +397,12 @@ BOOL isCustomResolution(CGSize res) {
|
||||
resolutionTable[6] = CGSizeMake(0, 0);
|
||||
}
|
||||
|
||||
// Customize framerate list for ProMotion devices
|
||||
if ([[UIScreen mainScreen] maximumFramesPerSecond] >= 90) {
|
||||
[self.framerateSelector setEnabled:YES forSegmentAtIndex:2]; // enable 90
|
||||
[self.framerateSelector setEnabled:YES forSegmentAtIndex:3]; // enable 120
|
||||
}
|
||||
|
||||
NSInteger framerate;
|
||||
switch ([currentSettings.framerate integerValue]) {
|
||||
case 30:
|
||||
@@ -402,9 +412,12 @@ BOOL isCustomResolution(CGSize res) {
|
||||
case 60:
|
||||
framerate = 1;
|
||||
break;
|
||||
case 120:
|
||||
case 90:
|
||||
framerate = 2;
|
||||
break;
|
||||
case 120:
|
||||
framerate = 3;
|
||||
break;
|
||||
}
|
||||
|
||||
NSInteger resolution = currentSettings.resolutionSelected.integerValue;
|
||||
@@ -412,17 +425,6 @@ BOOL isCustomResolution(CGSize res) {
|
||||
resolution = 0;
|
||||
}
|
||||
|
||||
// Only show the 120 FPS option if we have a > 60-ish Hz display
|
||||
bool enable120Fps = false;
|
||||
if (@available(iOS 10.3, tvOS 10.3, *)) {
|
||||
if ([UIScreen mainScreen].maximumFramesPerSecond > 62) {
|
||||
enable120Fps = true;
|
||||
}
|
||||
}
|
||||
if (!enable120Fps) {
|
||||
[self.framerateSelector removeSegmentAtIndex:2 animated:NO];
|
||||
}
|
||||
|
||||
// Disable codec selector segments for unsupported codecs
|
||||
#if defined(__IPHONE_16_0) || defined(__TVOS_16_0)
|
||||
if (!VTIsHardwareDecodeSupported(kCMVideoCodecType_AV1))
|
||||
@@ -466,9 +468,23 @@ BOOL isCustomResolution(CGSize res) {
|
||||
|
||||
[self.yuv444Selector setSelectedSegmentIndex:currentSettings.enableYUV444 ? 1 : 0];
|
||||
[self.statsOverlaySelector setSelectedSegmentIndex:currentSettings.statsOverlayLevel.intValue];
|
||||
NSInteger renderingBackend = [currentSettings.renderingBackend integerValue];
|
||||
[self.renderingBackendSelector setSelectedSegmentIndex:renderingBackend];
|
||||
[self.enableGraphsSelector setSelectedSegmentIndex:currentSettings.enableGraphs ? 1 : 0];
|
||||
[self.enableGraphsSelector addTarget:self action:@selector(enableGraphsChanged) forControlEvents:UIControlEventValueChanged];
|
||||
[self enableGraphsChanged];
|
||||
[self.graphOpacityStepper setMinimumValue:0];
|
||||
[self.graphOpacityStepper setMaximumValue:100];
|
||||
[self.graphOpacityStepper setValue:_graphOpacity];
|
||||
[self.graphOpacityStepper addTarget:self action:@selector(graphOpacityStepperMoved) forControlEvents:UIControlEventValueChanged];
|
||||
[self updateGraphOpacityText];
|
||||
[self.btMouseSelector setSelectedSegmentIndex:currentSettings.btMouseSupport ? 1 : 0];
|
||||
[self.optimizeSettingsSelector setSelectedSegmentIndex:currentSettings.optimizeGames ? 1 : 0];
|
||||
[self.framePacingSelector setSelectedSegmentIndex:currentSettings.useFramePacing ? 1 : 0];
|
||||
[self.frameQueueSizeSlider setMinimumValue:1];
|
||||
[self.frameQueueSizeSlider setMaximumValue:5];
|
||||
[self.frameQueueSizeSlider setValue:_frameQueueSize];
|
||||
[self.frameQueueSizeSlider addTarget:self action:@selector(frameQueueSizeSliderMoved) forControlEvents:UIControlEventValueChanged];
|
||||
[self updateFrameQueueSizeText];
|
||||
[self.multiControllerSelector setSelectedSegmentIndex:currentSettings.multiController ? 1 : 0];
|
||||
[self.swapABXYButtonsSelector setSelectedSegmentIndex:currentSettings.swapABXYButtons ? 1 : 0];
|
||||
[self.audioOnPCSelector setSelectedSegmentIndex:currentSettings.playAudioOnPC ? 1 : 0];
|
||||
@@ -828,7 +844,7 @@ BOOL isCustomResolution(CGSize res) {
|
||||
}
|
||||
|
||||
defaultBitrate = round(resolutionFactor * frameRateFactor) * 1000;
|
||||
_bitrate = MIN(defaultBitrate, 100000);
|
||||
_bitrate = MIN(defaultBitrate, 150000);
|
||||
[self.bitrateSlider setValue:[self getSliderValueForBitrate:_bitrate] animated:YES];
|
||||
|
||||
[self updateBitrateText];
|
||||
@@ -1002,6 +1018,8 @@ BOOL isCustomResolution(CGSize res) {
|
||||
case 1:
|
||||
return 60;
|
||||
case 2:
|
||||
return 90;
|
||||
case 3:
|
||||
return 120;
|
||||
default:
|
||||
abort();
|
||||
@@ -1050,6 +1068,30 @@ BOOL isCustomResolution(CGSize res) {
|
||||
return resolutionTable[[self.resolutionSelector selectedSegmentIndex]].width;
|
||||
}
|
||||
|
||||
- (void) frameQueueSizeSliderMoved {
|
||||
assert(self.frameQueueSizeSlider.value >= 0 && self.frameQueueSizeSlider.value <= 5);
|
||||
_frameQueueSize = (int)self.frameQueueSizeSlider.value;
|
||||
[self updateFrameQueueSizeText];
|
||||
}
|
||||
|
||||
- (void) updateFrameQueueSizeText {
|
||||
[self.frameQueueSizeLabel setText:[NSString stringWithFormat:@"Frames to buffer: %ld", _frameQueueSize ]];
|
||||
}
|
||||
|
||||
- (void) enableGraphsChanged {
|
||||
[self.graphOpacityStepper setEnabled:[self.enableGraphsSelector selectedSegmentIndex] == 1 ? YES : NO];
|
||||
}
|
||||
|
||||
- (void) graphOpacityStepperMoved {
|
||||
assert(self.graphOpacityStepper.value >= 0 && self.graphOpacityStepper.value <= 100);
|
||||
_graphOpacity = (int)self.graphOpacityStepper.value;
|
||||
[self updateGraphOpacityText];
|
||||
}
|
||||
|
||||
- (void) updateGraphOpacityText {
|
||||
[self.enableGraphsLabel setText:[NSString stringWithFormat:@"Performance Graphs - Opacity: %ld%%", _graphOpacity ]];
|
||||
}
|
||||
|
||||
- (void) saveSettings {
|
||||
DataManager* dataMan = [[DataManager alloc] init];
|
||||
NSInteger framerate = [self getChosenFrameRate];
|
||||
@@ -1073,6 +1115,7 @@ BOOL isCustomResolution(CGSize res) {
|
||||
BOOL largerStickLR1 = [self.largerStickLR1Selector selectedSegmentIndex] == 1;
|
||||
BOOL liftStreamViewForKeyboard = [self.liftStreamViewForKeyboardSelector selectedSegmentIndex] == 1;
|
||||
BOOL showKeyboardToolbar = [self.showKeyboardToolbarSelector selectedSegmentIndex] == 1;
|
||||
NSInteger renderingBackend = [self.renderingBackendSelector selectedSegmentIndex];
|
||||
BOOL optimizeGames = [self.optimizeSettingsSelector selectedSegmentIndex] == 1;
|
||||
BOOL multiController = [self.multiControllerSelector selectedSegmentIndex] == 1;
|
||||
BOOL swapABXYButtons = [self.swapABXYButtonsSelector selectedSegmentIndex] == 1;
|
||||
@@ -1085,6 +1128,8 @@ BOOL isCustomResolution(CGSize res) {
|
||||
NSInteger touchMode = [self.touchModeSelector selectedSegmentIndex];
|
||||
NSInteger statsOverlayLevel = [self.statsOverlaySelector selectedSegmentIndex];
|
||||
BOOL statsOverlayEnabled = statsOverlayLevel != 0;
|
||||
BOOL statsOverlay = [self.statsOverlaySelector selectedSegmentIndex] == 1;
|
||||
BOOL enableGraphs = [self.enableGraphsSelector selectedSegmentIndex] == 1;
|
||||
BOOL enableHdr = [self.hdrSelector selectedSegmentIndex] == 1;
|
||||
BOOL unlockDisplayOrientation = [self.unlockDisplayOrientationSelector selectedSegmentIndex] == 1;
|
||||
NSInteger resolutionSelected = [self.resolutionSelector selectedSegmentIndex];
|
||||
@@ -1117,16 +1162,20 @@ BOOL isCustomResolution(CGSize res) {
|
||||
preferredCodec:preferredCodec
|
||||
enableYUV444:enableYUV444
|
||||
useFramePacing:useFramePacing
|
||||
frameQueueSize:_frameQueueSize
|
||||
enableHdr:enableHdr
|
||||
btMouseSupport:btMouseSupport
|
||||
// absoluteTouchMode:absoluteTouchMode
|
||||
touchMode:touchMode
|
||||
statsOverlayLevel:statsOverlayLevel
|
||||
statsOverlayEnabled:statsOverlayEnabled
|
||||
unlockDisplayOrientation:unlockDisplayOrientation
|
||||
resolutionSelected:resolutionSelected
|
||||
externalDisplayMode:externalDisplayMode
|
||||
localMousePointerMode:localMousePointerMode];
|
||||
statsOverlay:statsOverlay
|
||||
enableGraphs:enableGraphs
|
||||
graphOpacity:_graphOpacity
|
||||
renderingBackend:renderingBackend];
|
||||
}
|
||||
|
||||
- (void)didReceiveMemoryWarning {
|
||||
|
||||
@@ -0,0 +1,714 @@
|
||||
//
|
||||
// StreamFrameViewController.m
|
||||
// Moonlight
|
||||
//
|
||||
// Created by Diego Waxemberg on 1/18/14.
|
||||
// Copyright (c) 2015 Moonlight Stream. All rights reserved.
|
||||
//
|
||||
|
||||
#import "StreamFrameViewController.h"
|
||||
#import "MainFrameViewController.h"
|
||||
#import "VideoDecoderRenderer.h"
|
||||
#import "StreamManager.h"
|
||||
#import "ControllerSupport.h"
|
||||
#import "DataManager.h"
|
||||
|
||||
#include <sys/socket.h>
|
||||
#include <netinet/in.h>
|
||||
#include <arpa/inet.h>
|
||||
#include <Limelight.h>
|
||||
|
||||
#if TARGET_OS_TV
|
||||
#import <AVFoundation/AVDisplayCriteria.h>
|
||||
#import <AVKit/AVDisplayManager.h>
|
||||
#import <AVKit/UIWindow.h>
|
||||
#endif
|
||||
|
||||
@interface AVDisplayCriteria()
|
||||
@property(readonly) int videoDynamicRange;
|
||||
@property(readonly, nonatomic) float refreshRate;
|
||||
- (id)initWithRefreshRate:(float)arg1 videoDynamicRange:(int)arg2;
|
||||
@end
|
||||
|
||||
@implementation StreamFrameViewController {
|
||||
ControllerSupport *_controllerSupport;
|
||||
StreamManager *_streamMan;
|
||||
TemporarySettings *_settings;
|
||||
NSTimer *_inactivityTimer;
|
||||
NSTimer *_statsUpdateTimer;
|
||||
UITapGestureRecognizer *_menuTapGestureRecognizer;
|
||||
UITapGestureRecognizer *_menuDoubleTapGestureRecognizer;
|
||||
UITapGestureRecognizer *_playPauseTapGestureRecognizer;
|
||||
UITextView *_overlayView;
|
||||
UILabel *_stageLabel;
|
||||
UILabel *_tipLabel;
|
||||
UIActivityIndicatorView *_spinner;
|
||||
StreamView *_streamView;
|
||||
UIScrollView *_scrollView;
|
||||
BOOL _userIsInteracting;
|
||||
CGSize _keyboardSize;
|
||||
|
||||
#if !TARGET_OS_TV
|
||||
UIScreenEdgePanGestureRecognizer *_exitSwipeRecognizer;
|
||||
#endif
|
||||
}
|
||||
|
||||
- (void)viewDidAppear:(BOOL)animated
|
||||
{
|
||||
[super viewDidAppear:animated];
|
||||
|
||||
#if !TARGET_OS_TV
|
||||
[[self revealViewController] setPrimaryViewController:self];
|
||||
#endif
|
||||
}
|
||||
|
||||
#if TARGET_OS_TV
|
||||
- (void)controllerPauseButtonPressed:(id)sender { }
|
||||
- (void)controllerPauseButtonDoublePressed:(id)sender {
|
||||
Log(LOG_I, @"Menu double-pressed -- backing out of stream");
|
||||
[self returnToMainFrame];
|
||||
}
|
||||
- (void)controllerPlayPauseButtonPressed:(id)sender {
|
||||
Log(LOG_I, @"Play/Pause button pressed -- backing out of stream");
|
||||
[self returnToMainFrame];
|
||||
}
|
||||
#endif
|
||||
|
||||
|
||||
- (void)viewDidLoad
|
||||
{
|
||||
[super viewDidLoad];
|
||||
|
||||
[self.navigationController setNavigationBarHidden:YES animated:YES];
|
||||
|
||||
[UIApplication sharedApplication].idleTimerDisabled = YES;
|
||||
|
||||
_settings = [[[DataManager alloc] init] getSettings];
|
||||
|
||||
_stageLabel = [[UILabel alloc] init];
|
||||
[_stageLabel setUserInteractionEnabled:NO];
|
||||
[_stageLabel setText:[NSString stringWithFormat:@"Starting %@...", self.streamConfig.appName]];
|
||||
[_stageLabel sizeToFit];
|
||||
_stageLabel.textAlignment = NSTextAlignmentCenter;
|
||||
_stageLabel.textColor = [UIColor whiteColor];
|
||||
_stageLabel.center = CGPointMake(self.view.frame.size.width / 2, self.view.frame.size.height / 2);
|
||||
|
||||
_spinner = [[UIActivityIndicatorView alloc] init];
|
||||
[_spinner setUserInteractionEnabled:NO];
|
||||
#if TARGET_OS_TV
|
||||
[_spinner setActivityIndicatorViewStyle:UIActivityIndicatorViewStyleWhiteLarge];
|
||||
#else
|
||||
[_spinner setActivityIndicatorViewStyle:UIActivityIndicatorViewStyleWhite];
|
||||
#endif
|
||||
[_spinner sizeToFit];
|
||||
[_spinner startAnimating];
|
||||
_spinner.center = CGPointMake(self.view.frame.size.width / 2, self.view.frame.size.height / 2 - _stageLabel.frame.size.height - _spinner.frame.size.height);
|
||||
|
||||
_controllerSupport = [[ControllerSupport alloc] initWithConfig:self.streamConfig delegate:self];
|
||||
_inactivityTimer = nil;
|
||||
|
||||
_streamView = [[StreamView alloc] initWithFrame:self.view.frame];
|
||||
[_streamView setupStreamView:_controllerSupport interactionDelegate:self config:self.streamConfig];
|
||||
|
||||
#if TARGET_OS_TV
|
||||
if (!_menuTapGestureRecognizer || !_menuDoubleTapGestureRecognizer || !_playPauseTapGestureRecognizer) {
|
||||
_menuTapGestureRecognizer = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(controllerPauseButtonPressed:)];
|
||||
_menuTapGestureRecognizer.allowedPressTypes = @[@(UIPressTypeMenu)];
|
||||
|
||||
_playPauseTapGestureRecognizer = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(controllerPlayPauseButtonPressed:)];
|
||||
_playPauseTapGestureRecognizer.allowedPressTypes = @[@(UIPressTypePlayPause)];
|
||||
|
||||
_menuDoubleTapGestureRecognizer = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(controllerPauseButtonDoublePressed:)];
|
||||
_menuDoubleTapGestureRecognizer.numberOfTapsRequired = 2;
|
||||
[_menuTapGestureRecognizer requireGestureRecognizerToFail:_menuDoubleTapGestureRecognizer];
|
||||
_menuDoubleTapGestureRecognizer.allowedPressTypes = @[@(UIPressTypeMenu)];
|
||||
}
|
||||
|
||||
[self.view addGestureRecognizer:_menuTapGestureRecognizer];
|
||||
[self.view addGestureRecognizer:_menuDoubleTapGestureRecognizer];
|
||||
[self.view addGestureRecognizer:_playPauseTapGestureRecognizer];
|
||||
|
||||
#else
|
||||
_exitSwipeRecognizer = [[UIScreenEdgePanGestureRecognizer alloc] initWithTarget:self action:@selector(edgeSwiped)];
|
||||
_exitSwipeRecognizer.edges = UIRectEdgeLeft;
|
||||
_exitSwipeRecognizer.delaysTouchesBegan = NO;
|
||||
_exitSwipeRecognizer.delaysTouchesEnded = NO;
|
||||
|
||||
[self.view addGestureRecognizer:_exitSwipeRecognizer];
|
||||
#endif
|
||||
|
||||
_tipLabel = [[UILabel alloc] init];
|
||||
[_tipLabel setUserInteractionEnabled:NO];
|
||||
|
||||
#if TARGET_OS_TV
|
||||
[_tipLabel setText:@"Tip: Tap the Play/Pause button on the Apple TV Remote to disconnect from your PC"];
|
||||
#else
|
||||
[_tipLabel setText:@"Tip: Swipe from the left edge to disconnect from your PC"];
|
||||
#endif
|
||||
|
||||
[_tipLabel sizeToFit];
|
||||
_tipLabel.textColor = [UIColor whiteColor];
|
||||
_tipLabel.textAlignment = NSTextAlignmentCenter;
|
||||
_tipLabel.center = CGPointMake(self.view.frame.size.width / 2, self.view.frame.size.height * 0.9);
|
||||
|
||||
_streamMan = [[StreamManager alloc] initWithConfig:self.streamConfig
|
||||
renderView:_streamView
|
||||
connectionCallbacks:self];
|
||||
NSOperationQueue* opQueue = [[NSOperationQueue alloc] init];
|
||||
[opQueue addOperation:_streamMan];
|
||||
|
||||
[[NSNotificationCenter defaultCenter] addObserver:self
|
||||
selector:@selector(applicationWillResignActive:)
|
||||
name:UIApplicationWillResignActiveNotification
|
||||
object:nil];
|
||||
|
||||
[[NSNotificationCenter defaultCenter] addObserver: self
|
||||
selector: @selector(applicationDidBecomeActive:)
|
||||
name: UIApplicationDidBecomeActiveNotification
|
||||
object: nil];
|
||||
|
||||
[[NSNotificationCenter defaultCenter] addObserver: self
|
||||
selector: @selector(applicationDidEnterBackground:)
|
||||
name: UIApplicationDidEnterBackgroundNotification
|
||||
object: nil];
|
||||
|
||||
#if 0
|
||||
// FIXME: This doesn't work reliably on iPad for some reason. Showing and hiding the keyboard
|
||||
// several times in a row will not correctly restore the state of the UIScrollView.
|
||||
[[NSNotificationCenter defaultCenter] addObserver: self
|
||||
selector: @selector(keyboardWillShow:)
|
||||
name: UIKeyboardWillShowNotification
|
||||
object: nil];
|
||||
|
||||
[[NSNotificationCenter defaultCenter] addObserver: self
|
||||
selector: @selector(keyboardWillHide:)
|
||||
name: UIKeyboardWillHideNotification
|
||||
object: nil];
|
||||
#endif
|
||||
|
||||
// Only enable scroll and zoom in absolute touch mode
|
||||
if (_settings.absoluteTouchMode) {
|
||||
_scrollView = [[UIScrollView alloc] initWithFrame:self.view.frame];
|
||||
#if !TARGET_OS_TV
|
||||
[_scrollView.panGestureRecognizer setMinimumNumberOfTouches:2];
|
||||
#endif
|
||||
[_scrollView setShowsHorizontalScrollIndicator:NO];
|
||||
[_scrollView setShowsVerticalScrollIndicator:NO];
|
||||
[_scrollView setDelegate:self];
|
||||
[_scrollView setMaximumZoomScale:10.0f];
|
||||
|
||||
// Add StreamView inside a UIScrollView for absolute mode
|
||||
[_scrollView addSubview:_streamView];
|
||||
[self.view addSubview:_scrollView];
|
||||
}
|
||||
else {
|
||||
// Add StreamView directly in relative mode
|
||||
[self.view addSubview:_streamView];
|
||||
}
|
||||
|
||||
[self.view addSubview:_stageLabel];
|
||||
[self.view addSubview:_spinner];
|
||||
[self.view addSubview:_tipLabel];
|
||||
}
|
||||
|
||||
- (UIView *)viewForZoomingInScrollView:(UIScrollView *)scrollView {
|
||||
return _streamView;
|
||||
}
|
||||
|
||||
- (void)willMoveToParentViewController:(UIViewController *)parent {
|
||||
// Only cleanup when we're being destroyed
|
||||
if (parent == nil) {
|
||||
[_controllerSupport cleanup];
|
||||
[UIApplication sharedApplication].idleTimerDisabled = NO;
|
||||
[_streamMan stopStream];
|
||||
if (_inactivityTimer != nil) {
|
||||
[_inactivityTimer invalidate];
|
||||
_inactivityTimer = nil;
|
||||
}
|
||||
[[NSNotificationCenter defaultCenter] removeObserver:self];
|
||||
}
|
||||
}
|
||||
|
||||
#if 0
|
||||
- (void)keyboardWillShow:(NSNotification *)notification {
|
||||
_keyboardSize = [[[notification userInfo] objectForKey:UIKeyboardFrameBeginUserInfoKey] CGRectValue].size;
|
||||
|
||||
[UIView animateWithDuration:0.3 animations:^{
|
||||
CGRect frame = self->_scrollView.frame;
|
||||
frame.size.height -= self->_keyboardSize.height;
|
||||
self->_scrollView.frame = frame;
|
||||
}];
|
||||
}
|
||||
|
||||
-(void)keyboardWillHide:(NSNotification *)notification {
|
||||
// NOTE: UIKeyboardFrameEndUserInfoKey returns a different keyboard size
|
||||
// than UIKeyboardFrameBeginUserInfoKey, so it's unsuitable for use here
|
||||
// to undo the changes made by keyboardWillShow.
|
||||
|
||||
[UIView animateWithDuration:0.3 animations:^{
|
||||
CGRect frame = self->_scrollView.frame;
|
||||
frame.size.height += self->_keyboardSize.height;
|
||||
self->_scrollView.frame = frame;
|
||||
}];
|
||||
}
|
||||
#endif
|
||||
|
||||
- (void)updateStatsOverlay {
|
||||
NSString* overlayText = [self->_streamMan getStatsOverlayText];
|
||||
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
[self updateOverlayText:overlayText];
|
||||
});
|
||||
}
|
||||
|
||||
- (void)updateOverlayText:(NSString*)text {
|
||||
if (_overlayView == nil) {
|
||||
_overlayView = [[UITextView alloc] init];
|
||||
#if !TARGET_OS_TV
|
||||
[_overlayView setEditable:NO];
|
||||
#endif
|
||||
[_overlayView setUserInteractionEnabled:NO];
|
||||
[_overlayView setSelectable:NO];
|
||||
[_overlayView setScrollEnabled:NO];
|
||||
|
||||
// HACK: If not using stats overlay, center the text
|
||||
if (_statsUpdateTimer == nil) {
|
||||
[_overlayView setTextAlignment:NSTextAlignmentCenter];
|
||||
}
|
||||
|
||||
[_overlayView setTextColor:[UIColor lightGrayColor]];
|
||||
[_overlayView setBackgroundColor:[UIColor blackColor]];
|
||||
#if TARGET_OS_TV
|
||||
[_overlayView setFont:[UIFont systemFontOfSize:24]];
|
||||
#else
|
||||
[_overlayView setFont:[UIFont systemFontOfSize:12]];
|
||||
#endif
|
||||
[_overlayView setAlpha:0.5];
|
||||
[self.view addSubview:_overlayView];
|
||||
}
|
||||
|
||||
if (text != nil) {
|
||||
// We set our bounds to the maximum width in order to work around a bug where
|
||||
// sizeToFit interacts badly with the UITextView's line breaks, causing the
|
||||
// width to get smaller and smaller each time as more line breaks are inserted.
|
||||
[_overlayView setBounds:CGRectMake(self.view.frame.origin.x,
|
||||
_overlayView.frame.origin.y,
|
||||
self.view.frame.size.width,
|
||||
_overlayView.frame.size.height)];
|
||||
[_overlayView setText:text];
|
||||
[_overlayView sizeToFit];
|
||||
[_overlayView setCenter:CGPointMake(self.view.frame.size.width / 2, _overlayView.frame.size.height / 2)];
|
||||
[_overlayView setHidden:NO];
|
||||
}
|
||||
else {
|
||||
[_overlayView setHidden:YES];
|
||||
}
|
||||
}
|
||||
|
||||
- (void) returnToMainFrame {
|
||||
// Reset display mode back to default
|
||||
[self updatePreferredDisplayMode:NO];
|
||||
|
||||
[_statsUpdateTimer invalidate];
|
||||
_statsUpdateTimer = nil;
|
||||
|
||||
[self.navigationController popToRootViewControllerAnimated:YES];
|
||||
}
|
||||
|
||||
// This will fire if the user opens control center or gets a low battery message
|
||||
- (void)applicationWillResignActive:(NSNotification *)notification {
|
||||
if (_inactivityTimer != nil) {
|
||||
[_inactivityTimer invalidate];
|
||||
}
|
||||
|
||||
#if !TARGET_OS_TV
|
||||
// Terminate the stream if the app is inactive for 60 seconds
|
||||
Log(LOG_I, @"Starting inactivity termination timer");
|
||||
_inactivityTimer = [NSTimer scheduledTimerWithTimeInterval:60
|
||||
target:self
|
||||
selector:@selector(inactiveTimerExpired:)
|
||||
userInfo:nil
|
||||
repeats:NO];
|
||||
#endif
|
||||
}
|
||||
|
||||
- (void)inactiveTimerExpired:(NSTimer*)timer {
|
||||
Log(LOG_I, @"Terminating stream after inactivity");
|
||||
|
||||
[self returnToMainFrame];
|
||||
|
||||
_inactivityTimer = nil;
|
||||
}
|
||||
|
||||
- (void)applicationDidBecomeActive:(NSNotification *)notification {
|
||||
// Stop the background timer, since we're foregrounded again
|
||||
if (_inactivityTimer != nil) {
|
||||
Log(LOG_I, @"Stopping inactivity timer after becoming active again");
|
||||
[_inactivityTimer invalidate];
|
||||
_inactivityTimer = nil;
|
||||
}
|
||||
}
|
||||
|
||||
// This fires when the home button is pressed
|
||||
- (void)applicationDidEnterBackground:(UIApplication *)application {
|
||||
Log(LOG_I, @"Terminating stream immediately for backgrounding");
|
||||
|
||||
if (_inactivityTimer != nil) {
|
||||
[_inactivityTimer invalidate];
|
||||
_inactivityTimer = nil;
|
||||
}
|
||||
|
||||
[self returnToMainFrame];
|
||||
}
|
||||
|
||||
- (void)edgeSwiped {
|
||||
Log(LOG_I, @"User swiped to end stream");
|
||||
|
||||
[self returnToMainFrame];
|
||||
}
|
||||
|
||||
- (void) connectionStarted {
|
||||
Log(LOG_I, @"Connection started");
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
// Leave the spinner spinning until it's obscured by
|
||||
// the first frame of video.
|
||||
self->_stageLabel.hidden = YES;
|
||||
self->_tipLabel.hidden = YES;
|
||||
|
||||
[self->_streamView showOnScreenControls];
|
||||
|
||||
[self->_controllerSupport connectionEstablished];
|
||||
|
||||
if (self->_settings.statsOverlay) {
|
||||
self->_statsUpdateTimer = [NSTimer scheduledTimerWithTimeInterval:1.0f
|
||||
target:self
|
||||
selector:@selector(updateStatsOverlay)
|
||||
userInfo:nil
|
||||
repeats:YES];
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
- (void)connectionTerminated:(int)errorCode {
|
||||
Log(LOG_I, @"Connection terminated: %d", errorCode);
|
||||
|
||||
unsigned int portFlags = LiGetPortFlagsFromTerminationErrorCode(errorCode);
|
||||
unsigned int portTestResults = LiTestClientConnectivity(CONN_TEST_SERVER, 443, portFlags);
|
||||
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
// Allow the display to go to sleep now
|
||||
[UIApplication sharedApplication].idleTimerDisabled = NO;
|
||||
|
||||
NSString* title;
|
||||
NSString* message;
|
||||
|
||||
if (portTestResults != ML_TEST_RESULT_INCONCLUSIVE && portTestResults != 0) {
|
||||
title = @"Connection Error";
|
||||
message = @"Your device's network connection is blocking Moonlight. Streaming may not work while connected to this network.";
|
||||
}
|
||||
else {
|
||||
switch (errorCode) {
|
||||
case ML_ERROR_GRACEFUL_TERMINATION:
|
||||
[self returnToMainFrame];
|
||||
return;
|
||||
|
||||
case ML_ERROR_NO_VIDEO_TRAFFIC:
|
||||
title = @"Connection Error";
|
||||
message = @"No video received from host.";
|
||||
if (portFlags != 0) {
|
||||
char failingPorts[256];
|
||||
LiStringifyPortFlags(portFlags, "\n", failingPorts, sizeof(failingPorts));
|
||||
message = [message stringByAppendingString:[NSString stringWithFormat:@"\n\nCheck your firewall and port forwarding rules for port(s):\n%s", failingPorts]];
|
||||
}
|
||||
break;
|
||||
|
||||
case ML_ERROR_NO_VIDEO_FRAME:
|
||||
title = @"Connection Error";
|
||||
message = @"Your network connection isn't performing well. Reduce your video bitrate setting or try a faster connection.";
|
||||
break;
|
||||
|
||||
case ML_ERROR_UNEXPECTED_EARLY_TERMINATION:
|
||||
case ML_ERROR_PROTECTED_CONTENT:
|
||||
title = @"Connection Error";
|
||||
message = @"Something went wrong on your host PC when starting the stream.\n\nMake sure you don't have any DRM-protected content open on your host PC. You can also try restarting your host PC.\n\nIf the issue persists, try reinstalling your GPU drivers and GeForce Experience.";
|
||||
break;
|
||||
|
||||
case ML_ERROR_FRAME_CONVERSION:
|
||||
title = @"Connection Error";
|
||||
message = @"The host PC reported a fatal video encoding error.\n\nTry disabling HDR mode, changing the streaming resolution, or changing your host PC's display resolution.";
|
||||
break;
|
||||
|
||||
default:
|
||||
{
|
||||
NSString* errorString;
|
||||
if (abs(errorCode) > 1000) {
|
||||
// We'll assume large errors are hex values
|
||||
errorString = [NSString stringWithFormat:@"%08X", (uint32_t)errorCode];
|
||||
}
|
||||
else {
|
||||
// Smaller values will just be printed as decimal (probably errno.h values)
|
||||
errorString = [NSString stringWithFormat:@"%d", errorCode];
|
||||
}
|
||||
|
||||
title = @"Connection Terminated";
|
||||
message = [NSString stringWithFormat: @"The connection was terminated\n\nError code: %@", errorString];
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
UIAlertController* conTermAlert = [UIAlertController alertControllerWithTitle:title
|
||||
message:message
|
||||
preferredStyle:UIAlertControllerStyleAlert];
|
||||
[Utils addHelpOptionToDialog:conTermAlert];
|
||||
[conTermAlert addAction:[UIAlertAction actionWithTitle:@"OK" style:UIAlertActionStyleDefault handler:^(UIAlertAction* action){
|
||||
[self returnToMainFrame];
|
||||
}]];
|
||||
[self presentViewController:conTermAlert animated:YES completion:nil];
|
||||
});
|
||||
|
||||
[_streamMan stopStream];
|
||||
}
|
||||
|
||||
- (void) stageStarting:(const char*)stageName {
|
||||
Log(LOG_I, @"Starting %s", stageName);
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
NSString* lowerCase = [NSString stringWithFormat:@"%s in progress...", stageName];
|
||||
NSString* titleCase = [[[lowerCase substringToIndex:1] uppercaseString] stringByAppendingString:[lowerCase substringFromIndex:1]];
|
||||
[self->_stageLabel setText:titleCase];
|
||||
[self->_stageLabel sizeToFit];
|
||||
self->_stageLabel.center = CGPointMake(self.view.frame.size.width / 2, self->_stageLabel.center.y);
|
||||
});
|
||||
}
|
||||
|
||||
- (void) stageComplete:(const char*)stageName {
|
||||
}
|
||||
|
||||
- (void) stageFailed:(const char*)stageName withError:(int)errorCode portTestFlags:(int)portTestFlags {
|
||||
Log(LOG_I, @"Stage %s failed: %d", stageName, errorCode);
|
||||
|
||||
unsigned int portTestResults = LiTestClientConnectivity(CONN_TEST_SERVER, 443, portTestFlags);
|
||||
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
// Allow the display to go to sleep now
|
||||
[UIApplication sharedApplication].idleTimerDisabled = NO;
|
||||
|
||||
NSString* message = [NSString stringWithFormat:@"%s failed with error %d", stageName, errorCode];
|
||||
if (portTestFlags != 0) {
|
||||
char failingPorts[256];
|
||||
LiStringifyPortFlags(portTestFlags, "\n", failingPorts, sizeof(failingPorts));
|
||||
message = [message stringByAppendingString:[NSString stringWithFormat:@"\n\nCheck your firewall and port forwarding rules for port(s):\n%s", failingPorts]];
|
||||
}
|
||||
if (portTestResults != ML_TEST_RESULT_INCONCLUSIVE && portTestResults != 0) {
|
||||
message = [message stringByAppendingString:@"\n\nYour device's network connection is blocking Moonlight. Streaming may not work while connected to this network."];
|
||||
}
|
||||
|
||||
UIAlertController* alert = [UIAlertController alertControllerWithTitle:@"Connection Failed"
|
||||
message:message
|
||||
preferredStyle:UIAlertControllerStyleAlert];
|
||||
[Utils addHelpOptionToDialog:alert];
|
||||
[alert addAction:[UIAlertAction actionWithTitle:@"OK" style:UIAlertActionStyleDefault handler:^(UIAlertAction* action){
|
||||
[self returnToMainFrame];
|
||||
}]];
|
||||
[self presentViewController:alert animated:YES completion:nil];
|
||||
});
|
||||
|
||||
[_streamMan stopStream];
|
||||
}
|
||||
|
||||
- (void) launchFailed:(NSString*)message {
|
||||
Log(LOG_I, @"Launch failed: %@", message);
|
||||
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
// Allow the display to go to sleep now
|
||||
[UIApplication sharedApplication].idleTimerDisabled = NO;
|
||||
|
||||
UIAlertController* alert = [UIAlertController alertControllerWithTitle:@"Connection Error"
|
||||
message:message
|
||||
preferredStyle:UIAlertControllerStyleAlert];
|
||||
[Utils addHelpOptionToDialog:alert];
|
||||
[alert addAction:[UIAlertAction actionWithTitle:@"OK" style:UIAlertActionStyleDefault handler:^(UIAlertAction* action){
|
||||
[self returnToMainFrame];
|
||||
}]];
|
||||
[self presentViewController:alert animated:YES completion:nil];
|
||||
});
|
||||
}
|
||||
|
||||
- (void)rumble:(unsigned short)controllerNumber lowFreqMotor:(unsigned short)lowFreqMotor highFreqMotor:(unsigned short)highFreqMotor {
|
||||
Log(LOG_I, @"Rumble on gamepad %d: %04x %04x", controllerNumber, lowFreqMotor, highFreqMotor);
|
||||
|
||||
[_controllerSupport rumble:controllerNumber lowFreqMotor:lowFreqMotor highFreqMotor:highFreqMotor];
|
||||
}
|
||||
|
||||
- (void) rumbleTriggers:(uint16_t)controllerNumber leftTrigger:(uint16_t)leftTrigger rightTrigger:(uint16_t)rightTrigger {
|
||||
Log(LOG_I, @"Trigger rumble on gamepad %d: %04x %04x", controllerNumber, leftTrigger, rightTrigger);
|
||||
|
||||
[_controllerSupport rumbleTriggers:controllerNumber leftTrigger:leftTrigger rightTrigger:rightTrigger];
|
||||
}
|
||||
|
||||
- (void) setMotionEventState:(uint16_t)controllerNumber motionType:(uint8_t)motionType reportRateHz:(uint16_t)reportRateHz {
|
||||
Log(LOG_I, @"Set motion state on gamepad %d: %02x %u Hz", controllerNumber, motionType, reportRateHz);
|
||||
|
||||
[_controllerSupport setMotionEventState:controllerNumber motionType:motionType reportRateHz:reportRateHz];
|
||||
}
|
||||
|
||||
- (void) setControllerLed:(uint16_t)controllerNumber r:(uint8_t)r g:(uint8_t)g b:(uint8_t)b {
|
||||
Log(LOG_I, @"Set controller LED on gamepad %d: l%02x%02x%02x", controllerNumber, r, g, b);
|
||||
|
||||
[_controllerSupport setControllerLed:controllerNumber r:r g:g b:b];
|
||||
}
|
||||
|
||||
- (void)connectionStatusUpdate:(int)status {
|
||||
Log(LOG_W, @"Connection status update: %d", status);
|
||||
|
||||
// The stats overlay takes precedence over these warnings
|
||||
if (_statsUpdateTimer != nil) {
|
||||
return;
|
||||
}
|
||||
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
switch (status) {
|
||||
case CONN_STATUS_OKAY:
|
||||
[self updateOverlayText:nil];
|
||||
break;
|
||||
|
||||
case CONN_STATUS_POOR:
|
||||
if (self->_streamConfig.bitRate > 5000) {
|
||||
[self updateOverlayText:@"Slow connection to PC\nReduce your bitrate"];
|
||||
}
|
||||
else {
|
||||
[self updateOverlayText:@"Poor connection to PC"];
|
||||
}
|
||||
break;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
- (void) updatePreferredDisplayMode:(BOOL)streamActive {
|
||||
#if TARGET_OS_TV
|
||||
if (@available(tvOS 11.2, *)) {
|
||||
UIWindow* window = [[[UIApplication sharedApplication] delegate] window];
|
||||
AVDisplayManager* displayManager = [window avDisplayManager];
|
||||
|
||||
// This logic comes from Kodi and MrMC
|
||||
if (streamActive) {
|
||||
int dynamicRange;
|
||||
|
||||
if (LiGetCurrentHostDisplayHdrMode()) {
|
||||
dynamicRange = 2; // HDR10
|
||||
}
|
||||
else {
|
||||
dynamicRange = 0; // SDR
|
||||
}
|
||||
|
||||
AVDisplayCriteria* displayCriteria = [[AVDisplayCriteria alloc] initWithRefreshRate:[_settings.framerate floatValue]
|
||||
videoDynamicRange:dynamicRange];
|
||||
displayManager.preferredDisplayCriteria = displayCriteria;
|
||||
}
|
||||
else {
|
||||
// Switch back to the default display mode
|
||||
displayManager.preferredDisplayCriteria = nil;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
- (void) setHdrMode:(bool)enabled {
|
||||
Log(LOG_I, @"HDR is now: %s", enabled ? "active" : "inactive");
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
[self updatePreferredDisplayMode:YES];
|
||||
});
|
||||
}
|
||||
|
||||
- (void) videoContentShown {
|
||||
[_spinner stopAnimating];
|
||||
[self.view setBackgroundColor:[UIColor blackColor]];
|
||||
}
|
||||
|
||||
- (void)didReceiveMemoryWarning
|
||||
{
|
||||
[super didReceiveMemoryWarning];
|
||||
// Dispose of any resources that can be recreated.
|
||||
}
|
||||
|
||||
- (void)gamepadPresenceChanged {
|
||||
#if !TARGET_OS_TV
|
||||
if (@available(iOS 11.0, *)) {
|
||||
[self setNeedsUpdateOfHomeIndicatorAutoHidden];
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
- (void)mousePresenceChanged {
|
||||
#if !TARGET_OS_TV
|
||||
if (@available(iOS 14.0, *)) {
|
||||
[self setNeedsUpdateOfPrefersPointerLocked];
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
- (void) streamExitRequested {
|
||||
Log(LOG_I, @"Gamepad combo requested stream exit");
|
||||
|
||||
[self returnToMainFrame];
|
||||
}
|
||||
|
||||
- (void)userInteractionBegan {
|
||||
// Disable hiding home bar when user is interacting.
|
||||
// iOS will force it to be shown anyway, but it will
|
||||
// also discard our edges deferring system gestures unless
|
||||
// we willingly give up home bar hiding preference.
|
||||
_userIsInteracting = YES;
|
||||
#if !TARGET_OS_TV
|
||||
if (@available(iOS 11.0, *)) {
|
||||
[self setNeedsUpdateOfHomeIndicatorAutoHidden];
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
- (void)userInteractionEnded {
|
||||
// Enable home bar hiding again if conditions allow
|
||||
_userIsInteracting = NO;
|
||||
#if !TARGET_OS_TV
|
||||
if (@available(iOS 11.0, *)) {
|
||||
[self setNeedsUpdateOfHomeIndicatorAutoHidden];
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
#if !TARGET_OS_TV
|
||||
// Require a confirmation when streaming to activate a system gesture
|
||||
- (UIRectEdge)preferredScreenEdgesDeferringSystemGestures {
|
||||
return UIRectEdgeAll;
|
||||
}
|
||||
|
||||
- (BOOL)prefersHomeIndicatorAutoHidden {
|
||||
if ([_controllerSupport getConnectedGamepadCount] > 0 &&
|
||||
[_streamView getCurrentOscState] == OnScreenControlsLevelOff &&
|
||||
_userIsInteracting == NO) {
|
||||
// Autohide the home bar when a gamepad is connected
|
||||
// and the on-screen controls are disabled. We can't
|
||||
// do this all the time because any touch on the display
|
||||
// will cause the home indicator to reappear, and our
|
||||
// preferredScreenEdgesDeferringSystemGestures will also
|
||||
// be suppressed (leading to possible errant exits of the
|
||||
// stream).
|
||||
return YES;
|
||||
}
|
||||
|
||||
return NO;
|
||||
}
|
||||
|
||||
- (BOOL)shouldAutorotate {
|
||||
return YES;
|
||||
}
|
||||
|
||||
- (BOOL)prefersPointerLocked {
|
||||
// Pointer lock breaks the UIKit mouse APIs, which is a problem because
|
||||
// GCMouse is horribly broken on iOS 14.0 for certain mice. Only lock
|
||||
// the cursor if there is a GCMouse present.
|
||||
return [GCMouse mice].count > 0;
|
||||
}
|
||||
#endif
|
||||
|
||||
@end
|
||||
@@ -48,6 +48,8 @@
|
||||
</array>
|
||||
<key>NSLocalNetworkUsageDescription</key>
|
||||
<string>Moonlight uses the local network to connect to your gaming PC for streaming.</string>
|
||||
<key>UIDesignRequiresCompatibility</key>
|
||||
<true/>
|
||||
<key>UIMainStoryboardFile</key>
|
||||
<string>Main</string>
|
||||
<key>UIRequiredDeviceCapabilities</key>
|
||||
|
||||
@@ -36,6 +36,44 @@
|
||||
<string>4K</string>
|
||||
</array>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSToggleSwitchSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Request HDR Video</string>
|
||||
<key>Key</key>
|
||||
<string>enableHdr</string>
|
||||
<key>DefaultValue</key>
|
||||
<false/>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSMultiValueSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Audio Configuration</string>
|
||||
<key>Key</key>
|
||||
<string>audioConfig</string>
|
||||
<key>DefaultValue</key>
|
||||
<string>2</string>
|
||||
<key>Titles</key>
|
||||
<array>
|
||||
<string>Stereo</string>
|
||||
<string>LPCM 5.1</string>
|
||||
<string>LPCM 7.1</string>
|
||||
</array>
|
||||
<key>Values</key>
|
||||
<array>
|
||||
<string>2</string>
|
||||
<string>6</string>
|
||||
<string>8</string>
|
||||
</array>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSGroupSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Enable Match Content Rate for best results</string>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSMultiValueSpecifier</string>
|
||||
@@ -47,12 +85,16 @@
|
||||
<string>60</string>
|
||||
<key>Values</key>
|
||||
<array>
|
||||
<string>24</string>
|
||||
<string>30</string>
|
||||
<string>59.94</string>
|
||||
<string>60</string>
|
||||
</array>
|
||||
<key>Titles</key>
|
||||
<array>
|
||||
<string>24 FPS</string>
|
||||
<string>30 FPS</string>
|
||||
<string>59.94 FPS (experimental)</string>
|
||||
<string>60 FPS</string>
|
||||
</array>
|
||||
</dict>
|
||||
@@ -96,28 +138,6 @@
|
||||
<string>150000</string>
|
||||
</array>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSMultiValueSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Audio Configuration</string>
|
||||
<key>Key</key>
|
||||
<string>audioConfig</string>
|
||||
<key>DefaultValue</key>
|
||||
<string>2</string>
|
||||
<key>Titles</key>
|
||||
<array>
|
||||
<string>Stereo</string>
|
||||
<string>LPCM 5.1</string>
|
||||
<string>LPCM 7.1</string>
|
||||
</array>
|
||||
<key>Values</key>
|
||||
<array>
|
||||
<string>2</string>
|
||||
<string>6</string>
|
||||
<string>8</string>
|
||||
</array>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSGroupSpecifier</string>
|
||||
@@ -212,37 +232,53 @@
|
||||
<key>Type</key>
|
||||
<string>PSMultiValueSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Frame Pacing Preference</string>
|
||||
<string>Frames to Buffer</string>
|
||||
<key>Key</key>
|
||||
<string>useFramePacing</string>
|
||||
<string>frameQueueSize</string>
|
||||
<key>DefaultValue</key>
|
||||
<string>0</string>
|
||||
<integer>2</integer>
|
||||
<key>Values</key>
|
||||
<array>
|
||||
<string>0</string>
|
||||
<string>1</string>
|
||||
<integer>1</integer>
|
||||
<integer>2</integer>
|
||||
<integer>3</integer>
|
||||
<integer>4</integer>
|
||||
<integer>5</integer>
|
||||
</array>
|
||||
<key>Titles</key>
|
||||
<array>
|
||||
<string>Lowest Latency</string>
|
||||
<string>Smoothest Video</string>
|
||||
<string>1 - Lowest Latency</string>
|
||||
<string>2 - Recommended</string>
|
||||
<string>3 - Smoother Video</string>
|
||||
<string>4</string>
|
||||
<string>5</string>
|
||||
</array>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSToggleSwitchSpecifier</string>
|
||||
<string>PSMultiValueSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Request HDR Video</string>
|
||||
<string>Rendering Backend</string>
|
||||
<key>Key</key>
|
||||
<string>enableHdr</string>
|
||||
<string>renderingBackend</string>
|
||||
<key>DefaultValue</key>
|
||||
<false/>
|
||||
<string>0</string>
|
||||
<key>Values</key>
|
||||
<array>
|
||||
<integer>0</integer>
|
||||
<integer>1</integer>
|
||||
</array>
|
||||
<key>Titles</key>
|
||||
<array>
|
||||
<string>Metal</string>
|
||||
<string>AVSampleBuffer</string>
|
||||
</array>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSGroupSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Troubleshooting</string>
|
||||
<string>Double-click select button for stats</string>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
@@ -254,6 +290,42 @@
|
||||
<key>DefaultValue</key>
|
||||
<false/>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSToggleSwitchSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Performance Graphs</string>
|
||||
<key>Key</key>
|
||||
<string>enableGraphs</string>
|
||||
<key>DefaultValue</key>
|
||||
<false/>
|
||||
</dict>
|
||||
<dict>
|
||||
<key>Type</key>
|
||||
<string>PSMultiValueSpecifier</string>
|
||||
<key>Title</key>
|
||||
<string>Performance Graph Opacity</string>
|
||||
<key>Key</key>
|
||||
<string>graphOpacity</string>
|
||||
<key>DefaultValue</key>
|
||||
<integer>50</integer>
|
||||
<key>Values</key>
|
||||
<array>
|
||||
<integer>0</integer>
|
||||
<integer>25</integer>
|
||||
<integer>50</integer>
|
||||
<integer>75</integer>
|
||||
<integer>100</integer>
|
||||
</array>
|
||||
<key>Titles</key>
|
||||
<array>
|
||||
<string>0% - Transparent</string>
|
||||
<string>25%</string>
|
||||
<string>50%</string>
|
||||
<string>75%</string>
|
||||
<string>100% - Opaque</string>
|
||||
</array>
|
||||
</dict>
|
||||
</array>
|
||||
</dict>
|
||||
</plist>
|
||||
|
||||
@@ -0,0 +1,271 @@
|
||||
|
||||
#import <XCTest/XCTest.h>
|
||||
#import <CoreMedia/CoreMedia.h>
|
||||
|
||||
#define FRAME_QUEUE_VERBOSE
|
||||
|
||||
#import "FrameQueue.h"
|
||||
#import "Logger.h"
|
||||
#include "Limelight.h"
|
||||
|
||||
@interface FrameQueueTests : XCTestCase
|
||||
@property (nonatomic, strong) FrameQueue *queue;
|
||||
@end
|
||||
|
||||
@implementation FrameQueueTests {
|
||||
int _frameNumber;
|
||||
}
|
||||
|
||||
- (void)setUp {
|
||||
[super setUp];
|
||||
self.queue = [FrameQueue sharedInstance];
|
||||
[self.queue setHighWaterMark:3];
|
||||
_frameNumber = 0;
|
||||
}
|
||||
|
||||
- (void)tearDown {
|
||||
[self.queue clear];
|
||||
self.queue = nil;
|
||||
[super tearDown];
|
||||
}
|
||||
|
||||
- (CMSampleBufferRef)makeEmptySampleBufferFor:(uint32_t)tick {
|
||||
CMSampleTimingInfo sampleTiming = {
|
||||
.duration = kCMTimeInvalid,
|
||||
.presentationTimeStamp = CMTimeMake((int64_t)tick, 90000),
|
||||
.decodeTimeStamp = kCMTimeInvalid,
|
||||
};
|
||||
|
||||
CMSampleBufferRef sampleBuffer = NULL;
|
||||
OSStatus status = CMSampleBufferCreate(kCFAllocatorDefault,
|
||||
NULL, NO, NULL, NULL, NULL,
|
||||
1, 1, &sampleTiming, 0, NULL,
|
||||
&sampleBuffer);
|
||||
XCTAssertEqual(status, noErr, @"makeEmptySampleBufferFor:%d ok", tick);
|
||||
return sampleBuffer;
|
||||
}
|
||||
|
||||
- (Frame *)makeFrameNumber:(int)num type:(int)type pts:(uint32_t)tick {
|
||||
CMSampleBufferRef buf = [self makeEmptySampleBufferFor:tick];
|
||||
CFRetain(buf); // emulate retain that happens in CMSampleBufferCreateReadyWithImageBuffer
|
||||
Frame *frame = [[Frame alloc] initWithSampleBuffer:buf
|
||||
frameNumber:num
|
||||
frameType:type];
|
||||
_frameNumber = num; // reset the counter
|
||||
return frame;
|
||||
}
|
||||
|
||||
- (Frame *)makeFrame {
|
||||
const uint32_t ONE_FRAME = 90000 / 60;
|
||||
static uint32_t tick = 0;
|
||||
Frame *frame = [self makeFrameNumber:++_frameNumber
|
||||
type:FRAME_TYPE_PFRAME
|
||||
pts:tick];
|
||||
tick += ONE_FRAME;
|
||||
return frame;
|
||||
}
|
||||
|
||||
- (void)testEnqueueThenDequeue {
|
||||
Frame *f1 = [self makeFrameNumber:1 type:FRAME_TYPE_IDR pts:0];
|
||||
Frame *f2 = [self makeFrame];
|
||||
|
||||
[self.queue enqueue:f1];
|
||||
[self.queue enqueue:f2];
|
||||
XCTAssertEqual([self.queue count], 2);
|
||||
|
||||
Frame *out1 = [self.queue dequeue];
|
||||
XCTAssertEqual(out1.frameNumber, 1);
|
||||
XCTAssertEqual([self.queue count], 1);
|
||||
|
||||
Frame *out2 = [self.queue dequeue];
|
||||
XCTAssertEqual(out2.frameNumber, 2);
|
||||
XCTAssertEqual([self.queue count], 0);
|
||||
|
||||
// empty now
|
||||
XCTAssertNil([self.queue dequeue]);
|
||||
}
|
||||
|
||||
- (void)testOverflow {
|
||||
[self.queue setHighWaterMark:3];
|
||||
|
||||
for (int i = 0; i < self.queue.highWaterMark; i++) {
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
}
|
||||
XCTAssertEqual([self.queue count], 3);
|
||||
|
||||
// queue should be full at 3
|
||||
// 4: dropped, 5: queued, causes 1 to be dropped
|
||||
// 6: dropped, 7: queued, causes 2 to be dropped
|
||||
// 8: dropped, 9: queued, causes 3 to be dropped
|
||||
for (int i = 0; i < 6; i++) {
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
}
|
||||
XCTAssertEqual([self.queue count], 3);
|
||||
XCTAssertEqual([[self.queue frameDropMetrics] total], 6);
|
||||
Log(LOG_I, @"queue after overflow: %@", self.queue);
|
||||
for (NSNumber *expected in @[@5, @7, @9]) {
|
||||
Frame *frame = [self.queue dequeue];
|
||||
XCTAssertEqual(frame.frameNumber, expected.integerValue);
|
||||
}
|
||||
}
|
||||
|
||||
- (void)testOverflowWithIDR {
|
||||
[self.queue setHighWaterMark:5];
|
||||
|
||||
[self.queue enqueue:[self makeFrameNumber:1 type:FRAME_TYPE_IDR pts:0]];
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
[self.queue enqueue:[self makeFrameNumber:3 type:FRAME_TYPE_IDR pts:0]];
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
XCTAssertEqual([self.queue count], 5);
|
||||
|
||||
// queue should be full at 5
|
||||
// 6: dropped, 7: queued, does NOT drop 1 (IDR)
|
||||
// 8: dropped, 9: queued, does NOT drop 1 (IDR) - will be stuck here until it's dequeued
|
||||
// 10: dropped, 11: queued, does NOT drop 3 (IDR)
|
||||
for (int i = 0; i < 6; i++) {
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
}
|
||||
XCTAssertEqual([self.queue count], 8);
|
||||
XCTAssertEqual([[self.queue frameDropMetrics] total], 3);
|
||||
Log(LOG_I, @"queue after overflow: %@", self.queue);
|
||||
|
||||
for (NSNumber *expected in @[@1, @2, @3, @4, @5, @7, @9, @11]) {
|
||||
XCTAssertEqual([self.queue dequeue].frameNumber, expected.integerValue);
|
||||
}
|
||||
}
|
||||
|
||||
- (void)testClear {
|
||||
for (int i = 0; i < self.queue.highWaterMark; i++) {
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
}
|
||||
XCTAssertEqual([self.queue count], self.queue.highWaterMark);
|
||||
|
||||
[self.queue clear];
|
||||
XCTAssertEqual([self.queue count], 0);
|
||||
XCTAssertEqual([[self.queue frameDropMetrics] total], 0);
|
||||
XCTAssertNil([self.queue dequeue]);
|
||||
}
|
||||
|
||||
- (void)testDequeueWithTimeout {
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
|
||||
// should return immediately with an item in the queue
|
||||
CFTimeInterval t0 = CACurrentMediaTime();
|
||||
Frame *out = [self.queue dequeueWithTimeout:1];
|
||||
XCTAssertNotNil(out);
|
||||
XCTAssertEqual(out.frameNumber, 1);
|
||||
XCTAssertLessThan(CACurrentMediaTime() - t0, 0.9);
|
||||
|
||||
// should return nil after waiting
|
||||
t0 = CACurrentMediaTime();
|
||||
XCTAssertNil([self.queue dequeueWithTimeout:0.2]);
|
||||
XCTAssertGreaterThan(CACurrentMediaTime() - t0, 0.2);
|
||||
|
||||
// no timeout
|
||||
t0 = CACurrentMediaTime();
|
||||
XCTAssertNil([self.queue dequeueWithTimeout:0]);
|
||||
XCTAssertLessThan(CACurrentMediaTime() - t0, 0.1);
|
||||
}
|
||||
|
||||
- (void)testDuration {
|
||||
// 2nd frame will compute 1st frame's duration. makeFrame generates sequential frames at 60fps
|
||||
[self.queue clear];
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
XCTAssertEqual([self.queue count], 2);
|
||||
Frame *first = [self.queue dequeue];
|
||||
XCTAssertEqualWithAccuracy(first.duration, (float)1.0f / 60, 0.0001f);
|
||||
}
|
||||
|
||||
- (void)testEstimatedFramerate {
|
||||
// prime the start point
|
||||
[self.queue estimatedFramerate];
|
||||
for (int i = 0; i < 24 * 5; i++) {
|
||||
// these will all get dropped but they are still counted for fps
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
[NSThread sleepForTimeInterval:1.0f / 24];
|
||||
}
|
||||
XCTAssertEqualWithAccuracy([self.queue estimatedFramerate], 24.0f, 4.0f);
|
||||
}
|
||||
|
||||
- (void)testProducerConsumer {
|
||||
static const int TotalFrames = 500;
|
||||
|
||||
// We expect exactly kTotalFrames dequeues.
|
||||
XCTestExpectation *dequeueAll = [self expectationWithDescription:@"consumer dequeued all frames"];
|
||||
dequeueAll.expectedFulfillmentCount = TotalFrames;
|
||||
|
||||
// Producer
|
||||
dispatch_queue_t producerQ = dispatch_queue_create("moonlight.test.producer", DISPATCH_QUEUE_CONCURRENT);
|
||||
dispatch_async(producerQ, ^{
|
||||
for (int i = 0; i < TotalFrames; i++) {
|
||||
[self.queue enqueue:[self makeFrame]];
|
||||
[NSThread sleepForTimeInterval:0.002];
|
||||
}
|
||||
});
|
||||
|
||||
// Consumer
|
||||
dispatch_queue_t consumerQ = dispatch_queue_create("moonlight.test.consumer", DISPATCH_QUEUE_CONCURRENT);
|
||||
dispatch_async(consumerQ, ^{
|
||||
for (int expected = 0; expected < TotalFrames; expected++) {
|
||||
Frame *frame = [self.queue dequeueWithTimeout:1.0];
|
||||
XCTAssertNotNil(frame, @"Expected frame %d, but got nil", expected);
|
||||
XCTAssertEqual(frame.frameNumber, expected + 1, @"Expected frameNumber %d but got %d", expected + 1, frame.frameNumber);
|
||||
[dequeueAll fulfill];
|
||||
}
|
||||
});
|
||||
|
||||
// Wait up to 10s for all frames to be consumed
|
||||
[self waitForExpectationsWithTimeout:10.0 handler:^(NSError * _Nullable error) {
|
||||
if (error) {
|
||||
XCTFail(@"Timed out waiting for consumer: %@", error);
|
||||
}
|
||||
XCTAssertEqual([self.queue count], 0, @"Queue should be empty at end");
|
||||
}];
|
||||
}
|
||||
|
||||
- (void)testPerformance {
|
||||
XCTMeasureOptions *options = [[XCTMeasureOptions alloc] init];
|
||||
options.iterationCount = 20;
|
||||
if (@available(iOS 13.0, *)) {
|
||||
static const int TotalFrames = 1000;
|
||||
dispatch_queue_t producerQ = dispatch_queue_create("moonlight.test.producer", DISPATCH_QUEUE_CONCURRENT);
|
||||
dispatch_queue_t consumerQ = dispatch_queue_create("moonlight.test.consumer", DISPATCH_QUEUE_CONCURRENT);
|
||||
|
||||
[self measureWithOptions:options block:^{
|
||||
XCTestExpectation *dequeueAll = [self expectationWithDescription:@"consumer dequeued all frames"];
|
||||
dequeueAll.expectedFulfillmentCount = TotalFrames;
|
||||
|
||||
[self.queue setHighWaterMark:5];
|
||||
|
||||
// Producer
|
||||
dispatch_async(producerQ, ^{
|
||||
for (int i = 0; i < TotalFrames; i++) {
|
||||
//[self.queue enqueue:[self makeFrame]];
|
||||
[self.queue enqueue:[self makeFrame] withSlackSize:3];
|
||||
[NSThread sleepForTimeInterval:0.001];
|
||||
}
|
||||
});
|
||||
|
||||
// Consumer
|
||||
dispatch_async(consumerQ, ^{
|
||||
for (int expected = 0; expected < TotalFrames; expected++) {
|
||||
[self.queue dequeueWithTimeout:1.0f];
|
||||
[dequeueAll fulfill];
|
||||
}
|
||||
});
|
||||
|
||||
// Wait up to 10s for all frames to be consumed
|
||||
[self waitForExpectationsWithTimeout:100.0 handler:^(NSError * _Nullable error) {
|
||||
if (error) {
|
||||
XCTFail(@"Timed out waiting for consumer: %@", error);
|
||||
|
||||
[self.queue clear];
|
||||
}
|
||||
}];
|
||||
}];
|
||||
}
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -65,27 +65,32 @@ static NSString* DB_NAME = @"Limelight_iOS.sqlite";
|
||||
{
|
||||
// Sent when the application is about to move from active to inactive state. This can occur for certain types of temporary interruptions (such as an incoming phone call or SMS message) or when the user quits the application and it begins the transition to the background state.
|
||||
// Use this method to pause ongoing tasks, disable timers, and throttle down OpenGL ES frame rates. Games should use this method to pause the game.
|
||||
Log(LOG_I, @"TODO: applicationWillResignActive");
|
||||
}
|
||||
|
||||
- (void)applicationDidEnterBackground:(UIApplication *)application
|
||||
{
|
||||
// Use this method to release shared resources, save user data, invalidate timers, and store enough application state information to restore your application to its current state in case it is terminated later.
|
||||
// If your application supports background execution, this method is called instead of applicationWillTerminate: when the user quits.
|
||||
Log(LOG_I, @"TODO: applicationDidEnterBackground");
|
||||
}
|
||||
|
||||
- (void)applicationWillEnterForeground:(UIApplication *)application
|
||||
{
|
||||
// Called as part of the transition from the background to the inactive state; here you can undo many of the changes made on entering the background.
|
||||
Log(LOG_I, @"TODO: applicationWillEnterForeground");
|
||||
}
|
||||
|
||||
- (void)applicationDidBecomeActive:(UIApplication *)application
|
||||
{
|
||||
// Restart any tasks that were paused (or not yet started) while the application was inactive. If the application was previously in the background, optionally refresh the user interface.
|
||||
Log(LOG_I, @"TODO: applicationDidBecomeActive");
|
||||
}
|
||||
|
||||
- (void)applicationWillTerminate:(UIApplication *)application
|
||||
{
|
||||
// Saves changes in the application's managed object context before the application terminates.
|
||||
Log(LOG_I, @"applicationWillTerminate, calling saveContext");
|
||||
[self saveContext];
|
||||
}
|
||||
|
||||
|
||||
@@ -82,6 +82,7 @@
|
||||
preferredCodec:(uint32_t)preferredCodec
|
||||
enableYUV444:(BOOL)enableYUV444
|
||||
useFramePacing:(BOOL)useFramePacing
|
||||
frameQueueSize:(NSInteger)frameQueueSize
|
||||
enableHdr:(BOOL)enableHdr
|
||||
btMouseSupport:(BOOL)btMouseSupport
|
||||
// absoluteTouchMode:(BOOL)absoluteTouchMode
|
||||
@@ -93,6 +94,12 @@
|
||||
externalDisplayMode:(NSInteger)externalDisplayMode
|
||||
localMousePointerMode:(NSInteger)localMousePointerMode{
|
||||
|
||||
absoluteTouchMode:(BOOL)absoluteTouchMode
|
||||
statsOverlay:(BOOL)statsOverlay
|
||||
enableGraphs:(BOOL)enableGraphs
|
||||
graphOpacity:(NSInteger)graphOpacity
|
||||
renderingBackend:(NSInteger)renderingBackend
|
||||
{
|
||||
[_managedObjectContext performBlockAndWait:^{
|
||||
Settings* settingsToSave = [self retrieveSettings];
|
||||
settingsToSave.framerate = [NSNumber numberWithInteger:framerate];
|
||||
@@ -121,7 +128,7 @@
|
||||
settingsToSave.playAudioOnPC = audioOnPC;
|
||||
settingsToSave.preferredCodec = preferredCodec;
|
||||
settingsToSave.enableYUV444 = enableYUV444;
|
||||
settingsToSave.useFramePacing = useFramePacing;
|
||||
settingsToSave.frameQueueSize = [NSNumber numberWithInteger:frameQueueSize];
|
||||
settingsToSave.enableHdr = enableHdr;
|
||||
settingsToSave.btMouseSupport = btMouseSupport;
|
||||
// settingsToSave.absoluteTouchMode = absoluteTouchMode;
|
||||
@@ -132,6 +139,11 @@
|
||||
settingsToSave.resolutionSelected = [NSNumber numberWithInteger:resolutionSelected];
|
||||
settingsToSave.externalDisplayMode = [NSNumber numberWithInteger:externalDisplayMode];
|
||||
settingsToSave.localMousePointerMode = [NSNumber numberWithInteger:localMousePointerMode];
|
||||
settingsToSave.absoluteTouchMode = absoluteTouchMode;
|
||||
settingsToSave.statsOverlay = statsOverlay;
|
||||
settingsToSave.enableGraphs = enableGraphs;
|
||||
settingsToSave.graphOpacity = [NSNumber numberWithInteger:graphOpacity];
|
||||
settingsToSave.renderingBackend = [NSNumber numberWithInteger:renderingBackend];
|
||||
[self saveData];
|
||||
}];
|
||||
}
|
||||
|
||||
@@ -35,13 +35,13 @@
|
||||
|
||||
self.bitrate = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"bitrate"]];
|
||||
assert([self.bitrate intValue] != 0);
|
||||
self.framerate = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"framerate"]];
|
||||
assert([self.framerate intValue] != 0);
|
||||
self.framerate = [NSNumber numberWithDouble:[[NSUserDefaults standardUserDefaults] doubleForKey:@"framerate"]];
|
||||
assert([self.framerate doubleValue] != 0.0);
|
||||
self.audioConfig = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"audioConfig"]];
|
||||
assert([self.audioConfig intValue] != 0);
|
||||
self.preferredCodec = (typeof(self.preferredCodec))[[NSUserDefaults standardUserDefaults] integerForKey:@"preferredCodec"];
|
||||
self.enableYUV444 = [[NSUserDefaults standardUserDefaults] boolForKey:@"enableYUV444"];
|
||||
self.useFramePacing = [[NSUserDefaults standardUserDefaults] integerForKey:@"useFramePacing"] != 0;
|
||||
self.frameQueueSize = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"frameQueueSize"]];
|
||||
self.playAudioOnPC = [[NSUserDefaults standardUserDefaults] boolForKey:@"audioOnPC"];
|
||||
self.enableHdr = [[NSUserDefaults standardUserDefaults] boolForKey:@"enableHdr"];
|
||||
self.optimizeGames = [[NSUserDefaults standardUserDefaults] boolForKey:@"optimizeGames"];
|
||||
@@ -49,6 +49,9 @@
|
||||
self.swapABXYButtons = [[NSUserDefaults standardUserDefaults] boolForKey:@"swapABXYButtons"];
|
||||
self.btMouseSupport = [[NSUserDefaults standardUserDefaults] boolForKey:@"btMouseSupport"];
|
||||
self.statsOverlay = [[NSUserDefaults standardUserDefaults] boolForKey:@"statsOverlay"];
|
||||
self.enableGraphs = [[NSUserDefaults standardUserDefaults] boolForKey:@"enableGraphs"];
|
||||
self.graphOpacity = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"graphOpacity"]];
|
||||
self.renderingBackend = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"renderingBackend"]];
|
||||
|
||||
NSInteger _screenSize = [[NSUserDefaults standardUserDefaults] integerForKey:@"streamResolution"];
|
||||
switch (_screenSize) {
|
||||
@@ -82,7 +85,7 @@
|
||||
self.audioConfig = settings.audioConfig;
|
||||
self.preferredCodec = settings.preferredCodec;
|
||||
self.enableYUV444 = settings.enableYUV444;
|
||||
self.useFramePacing = settings.useFramePacing;
|
||||
self.frameQueueSize = settings.frameQueueSize;
|
||||
self.playAudioOnPC = settings.playAudioOnPC;
|
||||
self.enableHdr = settings.enableHdr;
|
||||
self.optimizeGames = settings.optimizeGames;
|
||||
@@ -112,6 +115,9 @@
|
||||
self.resolutionSelected = settings.resolutionSelected;
|
||||
self.externalDisplayMode = settings.externalDisplayMode;
|
||||
self.localMousePointerMode = settings.localMousePointerMode;
|
||||
self.enableGraphs = settings.enableGraphs;
|
||||
self.graphOpacity = settings.graphOpacity;
|
||||
self.renderingBackend = settings.renderingBackend;
|
||||
#endif
|
||||
self.uniqueId = settings.uniqueId;
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
#include <TargetConditionals.h>
|
||||
|
||||
#ifndef __IPHONE_3_0
|
||||
#warning "This project uses features only available in iOS SDK 3.0 and later."
|
||||
#error "This project uses features only available in iOS SDK 3.0 and later."
|
||||
#endif
|
||||
|
||||
#ifdef __OBJC__
|
||||
@@ -20,5 +20,11 @@
|
||||
#endif
|
||||
#import <Foundation/Foundation.h>
|
||||
#import <CoreData/CoreData.h>
|
||||
|
||||
#if !defined(NDEBUG)
|
||||
// # define FRAME_QUEUE_VERBOSE
|
||||
#endif
|
||||
#import "Logger.h"
|
||||
#endif
|
||||
|
||||
#import "MetalConfig.h"
|
||||
|
||||
@@ -25,12 +25,16 @@
|
||||
<attribute name="audioConfig" attributeType="Integer 32" defaultValueString="2" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="bitrate" attributeType="Integer 32" defaultValueString="10000" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="btMouseSupport" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="deviceGyroMode" optional="YES" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="enableGraphs" optional="YES" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="enableHdr" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="enableYUV444" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="externalDisplayMode" optional="YES" attributeType="Integer 16" defaultValueString="1" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="frameQueueSize" optional="YES" attributeType="Integer 32" defaultValueString="2" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="framerate" attributeType="Integer 32" defaultValueString="60" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="gyroMode" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="gyroSensitivity" optional="YES" attributeType="Float" defaultValueString="1" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="graphOpacity" optional="YES" attributeType="Integer 32" defaultValueString="50" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="height" attributeType="Integer 32" defaultValueString="720" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="keyboardToggleFingers" optional="YES" attributeType="Integer 16" defaultValueString="3" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="liftStreamViewForKeyboard" attributeType="Boolean" defaultValueString="YES" usesScalarValueType="YES" syncable="YES"/>
|
||||
@@ -50,8 +54,8 @@
|
||||
<attribute name="showKeyboardToolbar" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="slideToSettingsDistance" optional="YES" attributeType="Float" defaultValueString="0.25" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="slideToSettingsScreenEdge" optional="YES" attributeType="Integer 32" defaultValueString="2" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="statsOverlayEnabled" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="statsOverlayLevel" optional="YES" attributeType="Integer 16" defaultValueString="0" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="renderingBackend" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="statsOverlay" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="swapABXYButtons" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
|
||||
<attribute name="touchMode" optional="YES" attributeType="Integer 16" defaultValueString="1" usesScalarValueType="NO" syncable="YES"/>
|
||||
<attribute name="touchMoveEventInterval" optional="YES" attributeType="Integer 16" defaultValueString="320" usesScalarValueType="NO" syncable="YES"/>
|
||||
|
||||
@@ -6,28 +6,20 @@
|
||||
// Copyright (c) 2014 Moonlight Stream. All rights reserved.
|
||||
//
|
||||
|
||||
#import "ConnectionCallbacks.h"
|
||||
#import "VideoDecoderRenderer.h"
|
||||
#import "StreamConfiguration.h"
|
||||
#import "BandwidthTracker.h"
|
||||
#import "Plot.h"
|
||||
|
||||
#define CONN_TEST_SERVER "ios.conntest.moonlight-stream.org"
|
||||
|
||||
typedef struct {
|
||||
CFTimeInterval startTime;
|
||||
CFTimeInterval endTime;
|
||||
int totalFrames;
|
||||
int receivedFrames;
|
||||
int networkDroppedFrames;
|
||||
int totalHostProcessingLatency;
|
||||
int framesWithHostProcessingLatency;
|
||||
int maxHostProcessingLatency;
|
||||
int minHostProcessingLatency;
|
||||
} video_stats_t;
|
||||
|
||||
@interface Connection : NSOperation <NSStreamDelegate>
|
||||
|
||||
-(id) initWithConfig:(StreamConfiguration*)config renderer:(VideoDecoderRenderer*)myRenderer connectionCallbacks:(id<ConnectionCallbacks>)callbacks;
|
||||
-(void) terminate;
|
||||
-(void) main;
|
||||
-(BandwidthTracker *) getBwTracker;
|
||||
-(BOOL) getVideoStats:(video_stats_t*)stats;
|
||||
-(NSString*) getActiveCodecName;
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
//
|
||||
|
||||
#import "Connection.h"
|
||||
#import "Plot.h"
|
||||
#import "Utils.h"
|
||||
|
||||
#import <VideoToolbox/VideoToolbox.h>
|
||||
@@ -45,24 +46,28 @@ static int audioFrameSize;
|
||||
|
||||
static VideoDecoderRenderer* renderer;
|
||||
|
||||
static BandwidthTracker *bwTracker;
|
||||
|
||||
int DrDecoderSetup(int videoFormat, int width, int height, int redrawRate, void* context, int drFlags)
|
||||
{
|
||||
[renderer setupWithVideoFormat:videoFormat width:width height:height frameRate:redrawRate];
|
||||
lastFrameNumber = 0;
|
||||
activeVideoFormat = videoFormat;
|
||||
Log(LOG_I, @"Active video format: 0x%x", activeVideoFormat);
|
||||
memset(¤tVideoStats, 0, sizeof(currentVideoStats));
|
||||
memset(&lastVideoStats, 0, sizeof(lastVideoStats));
|
||||
bwTracker = [[BandwidthTracker alloc] initWithWindowSeconds:10 bucketIntervalMs:250];
|
||||
return 0;
|
||||
}
|
||||
|
||||
void DrStart(void)
|
||||
void DrCleanup(void)
|
||||
{
|
||||
[renderer start];
|
||||
[renderer cleanup];
|
||||
}
|
||||
|
||||
void DrStop(void)
|
||||
-(BandwidthTracker *) getBwTracker
|
||||
{
|
||||
[renderer stop];
|
||||
return bwTracker;
|
||||
}
|
||||
|
||||
-(BOOL) getVideoStats:(video_stats_t*)stats
|
||||
@@ -72,10 +77,12 @@ void DrStop(void)
|
||||
if (lastVideoStats.endTime != 0) {
|
||||
memcpy(stats, &lastVideoStats, sizeof(*stats));
|
||||
[videoStatsLock unlock];
|
||||
|
||||
// Pull in the separately-collected renderer stats
|
||||
[renderer getAllStats:stats];
|
||||
|
||||
return YES;
|
||||
}
|
||||
|
||||
// No stats yet
|
||||
[videoStatsLock unlock];
|
||||
return NO;
|
||||
}
|
||||
@@ -133,6 +140,8 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
|
||||
{
|
||||
int offset = 0;
|
||||
int ret;
|
||||
CFTimeInterval decodeStartTime = CACurrentMediaTime();
|
||||
|
||||
unsigned char* data = (unsigned char*) malloc(decodeUnit->fullLength);
|
||||
if (data == NULL) {
|
||||
// A frame was lost due to OOM condition
|
||||
@@ -158,8 +167,13 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
|
||||
}
|
||||
|
||||
// Any frame number greater than m_LastFrameNumber + 1 represents a dropped frame
|
||||
currentVideoStats.networkDroppedFrames += decodeUnit->frameNumber - (lastFrameNumber + 1);
|
||||
currentVideoStats.totalFrames += decodeUnit->frameNumber - (lastFrameNumber + 1);
|
||||
int droppedFrames = decodeUnit->frameNumber - (lastFrameNumber + 1);
|
||||
if (droppedFrames > 0) {
|
||||
currentVideoStats.networkDroppedFrames += droppedFrames;
|
||||
currentVideoStats.totalFrames += droppedFrames;
|
||||
|
||||
Log(LOG_W, @"Network dropped %d frame(s): %d - %d", droppedFrames, lastFrameNumber + 1, decodeUnit->frameNumber - 1);
|
||||
}
|
||||
lastFrameNumber = decodeUnit->frameNumber;
|
||||
}
|
||||
|
||||
@@ -179,6 +193,8 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
|
||||
currentVideoStats.receivedFrames++;
|
||||
currentVideoStats.totalFrames++;
|
||||
|
||||
[bwTracker addBytes:decodeUnit->fullLength];
|
||||
|
||||
PLENTRY entry = decodeUnit->bufferList;
|
||||
while (entry != NULL) {
|
||||
// Submit parameter set NALUs directly since no copy is required by the decoder
|
||||
@@ -186,7 +202,8 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
|
||||
ret = [renderer submitDecodeBuffer:(unsigned char*)entry->data
|
||||
length:entry->length
|
||||
bufferType:entry->bufferType
|
||||
decodeUnit:decodeUnit];
|
||||
decodeUnit:decodeUnit
|
||||
decodeStartTime:decodeStartTime];
|
||||
if (ret != DR_OK) {
|
||||
free(data);
|
||||
return ret;
|
||||
@@ -204,7 +221,8 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
|
||||
return [renderer submitDecodeBuffer:data
|
||||
length:offset
|
||||
bufferType:BUFFER_TYPE_PICDATA
|
||||
decodeUnit:decodeUnit];
|
||||
decodeUnit:decodeUnit
|
||||
decodeStartTime:decodeStartTime];
|
||||
}
|
||||
|
||||
int ArInit(int audioConfiguration, POPUS_MULTISTREAM_CONFIGURATION opusConfig, void* context, int flags)
|
||||
@@ -219,7 +237,7 @@ int ArInit(int audioConfiguration, POPUS_MULTISTREAM_CONFIGURATION opusConfig, v
|
||||
|
||||
SDL_zero(want);
|
||||
want.freq = opusConfig->sampleRate;
|
||||
want.format = AUDIO_S16;
|
||||
want.format = AUDIO_F32;
|
||||
want.channels = opusConfig->channelCount;
|
||||
want.samples = opusConfig->samplesPerFrame;
|
||||
|
||||
@@ -231,7 +249,7 @@ int ArInit(int audioConfiguration, POPUS_MULTISTREAM_CONFIGURATION opusConfig, v
|
||||
}
|
||||
|
||||
audioConfig = *opusConfig;
|
||||
audioFrameSize = opusConfig->samplesPerFrame * sizeof(short) * opusConfig->channelCount;
|
||||
audioFrameSize = opusConfig->samplesPerFrame * sizeof(float) * opusConfig->channelCount;
|
||||
audioBuffer = SDL_malloc(audioFrameSize);
|
||||
if (audioBuffer == NULL) {
|
||||
Log(LOG_E, @"Failed to allocate audio frame buffer");
|
||||
@@ -290,18 +308,22 @@ void ArDecodeAndPlaySample(char* sampleData, int sampleLength)
|
||||
return;
|
||||
}
|
||||
|
||||
decodeLen = opus_multistream_decode(opusDecoder, (unsigned char *)sampleData, sampleLength,
|
||||
(short*)audioBuffer, audioConfig.samplesPerFrame, 0);
|
||||
decodeLen = opus_multistream_decode_float(opusDecoder,
|
||||
(unsigned char*)sampleData,
|
||||
sampleLength,
|
||||
(float*)audioBuffer,
|
||||
audioConfig.samplesPerFrame,
|
||||
0);
|
||||
if (decodeLen > 0) {
|
||||
// Provide backpressure on the queue to ensure too many frames don't build up
|
||||
// in SDL's audio queue.
|
||||
while (SDL_GetQueuedAudioSize(audioDevice) / audioFrameSize > 10) {
|
||||
SDL_Delay(1);
|
||||
[NSThread sleepForTimeInterval:0.001f];
|
||||
}
|
||||
|
||||
if (SDL_QueueAudio(audioDevice,
|
||||
audioBuffer,
|
||||
sizeof(short) * decodeLen * audioConfig.channelCount) < 0) {
|
||||
sizeof(float) * decodeLen * audioConfig.channelCount) < 0) {
|
||||
Log(LOG_E, @"Failed to queue audio sample: %s\n", SDL_GetError());
|
||||
}
|
||||
}
|
||||
@@ -435,6 +457,19 @@ void ClSetControllerLED(uint16_t controllerNumber, uint8_t r, uint8_t g, uint8_t
|
||||
renderer = myRenderer;
|
||||
_callbacks = callbacks;
|
||||
|
||||
// Check for low power mode which limits framerate to 60
|
||||
if (config.frameRate > 60 && [[NSProcessInfo processInfo] isLowPowerModeEnabled]) {
|
||||
Log(LOG_W, @"Limiting stream to 60fps because device is in low power mode");
|
||||
config.frameRate = 60;
|
||||
}
|
||||
|
||||
// Lower to 90fps on Vision Pro
|
||||
NSInteger deviceFps = UIScreen.mainScreen.maximumFramesPerSecond;
|
||||
if (deviceFps < config.frameRate) {
|
||||
Log(LOG_W, @"Limiting stream to %dfps due to max refresh rate", deviceFps);
|
||||
config.frameRate = (int)deviceFps;
|
||||
}
|
||||
|
||||
LiInitializeStreamConfiguration(&_streamConfig);
|
||||
_streamConfig.width = config.width;
|
||||
_streamConfig.height = config.height;
|
||||
@@ -466,9 +501,9 @@ void ClSetControllerLED(uint16_t controllerNumber, uint8_t r, uint8_t g, uint8_t
|
||||
|
||||
LiInitializeVideoCallbacks(&_drCallbacks);
|
||||
_drCallbacks.setup = DrDecoderSetup;
|
||||
_drCallbacks.start = DrStart;
|
||||
_drCallbacks.stop = DrStop;
|
||||
_drCallbacks.capabilities = CAPABILITY_PULL_RENDERER |
|
||||
_drCallbacks.cleanup = DrCleanup;
|
||||
_drCallbacks.submitDecodeUnit = DrSubmitDecodeUnit;
|
||||
_drCallbacks.capabilities = CAPABILITY_DIRECT_SUBMIT |
|
||||
CAPABILITY_REFERENCE_FRAME_INVALIDATION_HEVC |
|
||||
CAPABILITY_REFERENCE_FRAME_INVALIDATION_AV1;
|
||||
|
||||
@@ -484,7 +519,9 @@ void ClSetControllerLED(uint16_t controllerNumber, uint8_t r, uint8_t g, uint8_t
|
||||
_clCallbacks.stageFailed = ClStageFailed;
|
||||
_clCallbacks.connectionStarted = ClConnectionStarted;
|
||||
_clCallbacks.connectionTerminated = ClConnectionTerminated;
|
||||
#ifdef DEBUG
|
||||
_clCallbacks.logMessage = ClLogMessage;
|
||||
#endif
|
||||
_clCallbacks.rumble = ClRumble;
|
||||
_clCallbacks.connectionStatusUpdate = ClConnectionStatusUpdate;
|
||||
_clCallbacks.setHdrMode = ClSetHdrMode;
|
||||
|
||||
@@ -6,6 +6,8 @@
|
||||
// Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
|
||||
//
|
||||
|
||||
#import "Plot.h"
|
||||
|
||||
@protocol ConnectionCallbacks <NSObject>
|
||||
|
||||
- (void) connectionStarted;
|
||||
@@ -22,4 +24,7 @@
|
||||
- (void) setControllerLed:(uint16_t)controllerNumber r:(uint8_t)r g:(uint8_t)g b:(uint8_t)b;
|
||||
- (void) videoContentShown;
|
||||
|
||||
- (void) observeFloat:(int)plotId value:(CFTimeInterval)value;
|
||||
- (void) observeFloatReturnMetrics:(int)plotId value:(CFTimeInterval)value plotMetrics:(PlotMetrics *)plotMetrics;
|
||||
|
||||
@end
|
||||
|
||||
@@ -23,6 +23,7 @@
|
||||
@property int width;
|
||||
@property int height;
|
||||
@property int frameRate;
|
||||
@property int frameRateX100;
|
||||
@property int bitRate;
|
||||
@property int riKeyId;
|
||||
@property NSData* riKey;
|
||||
@@ -35,7 +36,6 @@
|
||||
@property int audioConfiguration;
|
||||
@property int supportedVideoFormats;
|
||||
@property BOOL multiController;
|
||||
@property BOOL useFramePacing;
|
||||
@property NSData* serverCert;
|
||||
@property int localMousePointerMode;
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
#import "StreamManager.h"
|
||||
#import "CryptoManager.h"
|
||||
#import "HttpManager.h"
|
||||
#import "Plot.h"
|
||||
#import "Utils.h"
|
||||
|
||||
#import "StreamView.h"
|
||||
@@ -103,7 +104,7 @@
|
||||
|
||||
// Initializing the renderer must be done on the main thread
|
||||
dispatch_async(dispatch_get_main_queue(), ^{
|
||||
VideoDecoderRenderer* renderer = [[VideoDecoderRenderer alloc] initWithView:self->_renderView callbacks:self->_callbacks streamAspectRatio:(float)self->_config.width / (float)self->_config.height useFramePacing:self->_config.useFramePacing];
|
||||
VideoDecoderRenderer* renderer = [[VideoDecoderRenderer alloc] initWithView:self->_renderView callbacks:self->_callbacks streamAspectRatio:(float)self->_config.width / (float)self->_config.height];
|
||||
self->_connection = [[Connection alloc] initWithConfig:self->_config renderer:renderer connectionCallbacks:self->_callbacks];
|
||||
NSOperationQueue* opQueue = [[NSOperationQueue alloc] init];
|
||||
[opQueue addOperation:self->_connection];
|
||||
@@ -182,22 +183,41 @@
|
||||
(float)stats.totalHostProcessingLatency / stats.framesWithHostProcessingLatency / 10.f];
|
||||
}
|
||||
else {
|
||||
hostProcessingString = @"";
|
||||
// If all frames are duplicates this can happen, but let's avoid having the whole stats area change height
|
||||
hostProcessingString = @"Host processing latency min/max/avg: -/-/- ms\n";
|
||||
}
|
||||
|
||||
float interval = stats.endTime - stats.startTime;
|
||||
if(overlayLevel == 2) return [LocalizationHelper localizedStringForKey:@"Video stream: %dx%d %.2f FPS (Codec: %@)\nNetwork dropped frames: %.2f%%\nAverage network latency: %@%@",
|
||||
float scalePlotMetrics = stats.frameDropMetrics.nsamples > 0 ? ((float)stats.frameDropMetrics.nsamples / stats.totalFrames) : 1.0f;
|
||||
float fps = (stats.totalFrames - stats.networkDroppedFrames - (stats.frameDropMetrics.total / scalePlotMetrics)) / interval;
|
||||
|
||||
double avgVideoMbps = [_connection getBwTracker].averageMbps;
|
||||
double peakVideoMbps = [_connection getBwTracker].peakMbps;
|
||||
|
||||
return [NSString stringWithFormat:@"Video stream: %dx%d %.2f FPS (Codec: %@)\n"
|
||||
"Bitrate: %.1f Mbps, Peak: %.1f, Renderer: %@\n"
|
||||
"%@"
|
||||
"Frames buffered: %.1f\n"
|
||||
"Frames dropped by network/pacing jitter: %.1f%% / %.1f%%\n"
|
||||
"Average network latency: %@\n"
|
||||
"Decode time: %.2f/%.2f/%.2f ms",
|
||||
_config.width,
|
||||
_config.height,
|
||||
stats.totalFrames / interval,
|
||||
fps,
|
||||
[_connection getActiveCodecName],
|
||||
stats.networkDroppedFrames / interval,
|
||||
avgVideoMbps, peakVideoMbps, (stats.renderingBackend == RENDER_METAL) ? @"Metal" : @"AVSampleBuffer",
|
||||
hostProcessingString,
|
||||
stats.frameQueueMetrics.avg,
|
||||
(stats.networkDroppedFrames / stats.totalFrames) * 100.0,
|
||||
stats.frameDropMetrics.nsamples > 0 ? (stats.frameDropMetrics.total / stats.frameDropMetrics.nsamples) * 100.0 : 0.0f,
|
||||
latencyString,
|
||||
hostProcessingString];
|
||||
else return [LocalizationHelper localizedStringForKey:@"FPS: %5.2f Network dropped frames: %.2f%% Network latency: %@",
|
||||
stats.totalFrames / interval,
|
||||
stats.networkDroppedFrames / interval,
|
||||
latencyString];
|
||||
stats.decodeMetrics.min, stats.decodeMetrics.max, stats.decodeMetrics.avg];
|
||||
}
|
||||
|
||||
@end
|
||||
|
||||
@@ -9,18 +9,35 @@
|
||||
@import AVFoundation;
|
||||
|
||||
#import "ConnectionCallbacks.h"
|
||||
#import "FrameQueue.h"
|
||||
#import "Plot.h"
|
||||
|
||||
#include "Limelight.h"
|
||||
|
||||
@interface VideoDecoderRenderer : NSObject
|
||||
|
||||
- (id)initWithView:(UIView*)view callbacks:(id<ConnectionCallbacks>)callbacks streamAspectRatio:(float)aspectRatio useFramePacing:(BOOL)useFramePacing;
|
||||
@property (atomic, readonly) PlotMetrics decodeMetrics;
|
||||
@property (atomic, readonly) PlotMetrics frameQueueMetrics;
|
||||
|
||||
- (id)initWithView:(UIView*)view callbacks:(id<ConnectionCallbacks>)callbacks streamAspectRatio:(float)aspectRatio;
|
||||
|
||||
- (void)setupWithVideoFormat:(int)videoFormat width:(int)videoWidth height:(int)videoHeight frameRate:(int)frameRate;
|
||||
- (void)start;
|
||||
- (void)stop;
|
||||
- (void)renderFrame:(Frame *)frame atTime:(CMTime)targetTime;
|
||||
- (void)cleanup;
|
||||
- (void)setHdrMode:(BOOL)enabled;
|
||||
- (void)safeCopyMetricsTo:(PlotMetrics *)dst from:(PlotMetrics *)src;
|
||||
- (void)getAllStats:(video_stats_t *)stats;
|
||||
- (void)optimizeRefreshRate;
|
||||
|
||||
- (int)submitDecodeBuffer:(unsigned char *)data length:(int)length bufferType:(int)bufferType decodeUnit:(PDECODE_UNIT)du;
|
||||
- (int)submitDecodeBuffer:(unsigned char *)data
|
||||
length:(int)length
|
||||
bufferType:(int)bufferType
|
||||
decodeUnit:(PDECODE_UNIT)du
|
||||
decodeStartTime:(CFTimeInterval)decodeStartTime;
|
||||
|
||||
- (OSStatus)decodeFrameWithSampleBuffer:(CMSampleBufferRef)sampleBuffer
|
||||
frameNumber:(int)frameNumber
|
||||
frameType:(int)frameType
|
||||
decodeStartTime:(CFTimeInterval)decodeStartTime;
|
||||
|
||||
@end
|
||||
|
||||
@@ -6,43 +6,62 @@
|
||||
// Copyright (c) 2014 Moonlight Stream. All rights reserved.
|
||||
//
|
||||
|
||||
@import AVFoundation;
|
||||
@import VideoToolbox;
|
||||
|
||||
#import "DataManager.h"
|
||||
#import "TemporarySettings.h"
|
||||
#import "VideoDecoderRenderer.h"
|
||||
#import "FrameQueue.h"
|
||||
#import "StreamView.h"
|
||||
#import "Plot.h"
|
||||
#import "PlatformThreads.h"
|
||||
#import "MetalViewController.h"
|
||||
|
||||
#include <libavcodec/avcodec.h>
|
||||
#include <libavcodec/cbs.h>
|
||||
#include <libavcodec/cbs_av1.h>
|
||||
#include <libavformat/avio.h>
|
||||
#include <libavutil/mem.h>
|
||||
#include <mach/mach_time.h>
|
||||
|
||||
// Define for extra logging related to frame pacing
|
||||
#define DISPLAYLINK_VERBOSE
|
||||
|
||||
// Private libavformat API for writing the AV1 Codec Configuration Box
|
||||
extern int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size,
|
||||
int write_seq_header);
|
||||
|
||||
@implementation VideoDecoderRenderer {
|
||||
dispatch_queue_t _sq, _vtq;
|
||||
StreamView* _view;
|
||||
id<ConnectionCallbacks> _callbacks;
|
||||
float _streamAspectRatio;
|
||||
|
||||
AVSampleBufferDisplayLayer* displayLayer;
|
||||
int videoFormat;
|
||||
int frameRate;
|
||||
AVSampleBufferDisplayLayer* _displayLayer;
|
||||
int _videoFormat;
|
||||
int _frameRate;
|
||||
|
||||
NSMutableArray *parameterSetBuffers;
|
||||
NSData *masteringDisplayColorVolume;
|
||||
NSData *contentLightLevelInfo;
|
||||
CMVideoFormatDescriptionRef formatDesc;
|
||||
NSMutableArray *_parameterSetBuffers;
|
||||
NSData *_masteringDisplayColorVolume;
|
||||
NSData *_contentLightLevelInfo;
|
||||
CMVideoFormatDescriptionRef _formatDesc;
|
||||
CMVideoFormatDescriptionRef _formatDescImageBuffer;
|
||||
VTDecompressionSessionRef _decompressionSession;
|
||||
|
||||
CADisplayLink *_displayLink;
|
||||
BOOL framePacing;
|
||||
FrameQueue *_frameQueue;
|
||||
NSInteger _maxRefreshRate;
|
||||
RenderingBackend _renderingBackend;
|
||||
|
||||
}
|
||||
|
||||
- (void)reinitializeDisplayLayer
|
||||
{
|
||||
CALayer *oldLayer = displayLayer;
|
||||
CALayer *oldLayer = _displayLayer;
|
||||
|
||||
displayLayer = [[AVSampleBufferDisplayLayer alloc] init];
|
||||
displayLayer.backgroundColor = [UIColor blackColor].CGColor;
|
||||
_displayLayer = [[AVSampleBufferDisplayLayer alloc] init];
|
||||
_displayLayer.backgroundColor = [UIColor blackColor].CGColor;
|
||||
|
||||
// Ensure the AVSampleBufferDisplayLayer is sized to preserve the aspect ratio
|
||||
// of the video stream. We used to use AVLayerVideoGravityResizeAspect, but that
|
||||
@@ -55,39 +74,62 @@ extern int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size,
|
||||
} else {
|
||||
videoSize = CGSizeMake(_view.bounds.size.width, _view.bounds.size.width / _streamAspectRatio);
|
||||
}
|
||||
displayLayer.position = CGPointMake(CGRectGetMidX(_view.bounds), CGRectGetMidY(_view.bounds));
|
||||
displayLayer.bounds = CGRectMake(0, 0, videoSize.width, videoSize.height);
|
||||
displayLayer.videoGravity = AVLayerVideoGravityResize;
|
||||
_displayLayer.position = CGPointMake(CGRectGetMidX(_view.bounds), CGRectGetMidY(_view.bounds));
|
||||
_displayLayer.bounds = CGRectMake(0, 0, videoSize.width, videoSize.height);
|
||||
_displayLayer.videoGravity = AVLayerVideoGravityResize;
|
||||
|
||||
// Hide the layer until we get an IDR frame. This ensures we
|
||||
// can see the loading progress label as the stream is starting.
|
||||
displayLayer.hidden = YES;
|
||||
_displayLayer.hidden = YES;
|
||||
|
||||
if (oldLayer != nil) {
|
||||
// Switch out the old display layer with the new one
|
||||
[_view.layer replaceSublayer:oldLayer with:displayLayer];
|
||||
[_view.layer replaceSublayer:oldLayer with:_displayLayer];
|
||||
}
|
||||
else {
|
||||
[_view.layer addSublayer:displayLayer];
|
||||
[_view.layer addSublayer:_displayLayer];
|
||||
}
|
||||
|
||||
if (formatDesc != nil) {
|
||||
CFRelease(formatDesc);
|
||||
formatDesc = nil;
|
||||
if (_formatDesc != nil) {
|
||||
CFRelease(_formatDesc);
|
||||
_formatDesc = nil;
|
||||
}
|
||||
|
||||
if (_formatDescImageBuffer != nil) {
|
||||
CFRelease(_formatDescImageBuffer);
|
||||
_formatDescImageBuffer = nil;
|
||||
}
|
||||
|
||||
if (_decompressionSession != nil){
|
||||
VTDecompressionSessionInvalidate(_decompressionSession);
|
||||
CFRelease(_decompressionSession);
|
||||
_decompressionSession = nil;
|
||||
}
|
||||
}
|
||||
|
||||
- (id)initWithView:(StreamView*)view callbacks:(id<ConnectionCallbacks>)callbacks streamAspectRatio:(float)aspectRatio useFramePacing:(BOOL)useFramePacing
|
||||
- (id)initWithView:(StreamView*)view callbacks:(id<ConnectionCallbacks>)callbacks streamAspectRatio:(float)aspectRatio
|
||||
{
|
||||
NSLog(@"initializing video decoder %f", CACurrentMediaTime());
|
||||
self = [super init];
|
||||
|
||||
_sq = dispatch_queue_create("com.moonlight.VideoDecoderRenderer",
|
||||
dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_SERIAL, QOS_CLASS_USER_INTERACTIVE, 0));
|
||||
|
||||
// Video decoder needs to run at the highest priority since DisplayLink waits on it
|
||||
_vtq = dispatch_queue_create("com.moonlight.VideoDecoderRenderer.VTDecoder",
|
||||
dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_SERIAL, QOS_CLASS_USER_INTERACTIVE, 0));
|
||||
|
||||
_view = view;
|
||||
_callbacks = callbacks;
|
||||
_streamAspectRatio = aspectRatio;
|
||||
framePacing = useFramePacing;
|
||||
|
||||
parameterSetBuffers = [[NSMutableArray alloc] init];
|
||||
_parameterSetBuffers = [[NSMutableArray alloc] init];
|
||||
_frameQueue = [FrameQueue sharedInstance];
|
||||
_maxRefreshRate = [[UIScreen mainScreen] maximumFramesPerSecond];
|
||||
|
||||
DataManager* dataMan = [[DataManager alloc] init];
|
||||
|
||||
[_frameQueue setHighWaterMark:(int)[[dataMan getSettings].frameQueueSize integerValue]];
|
||||
|
||||
[self reinitializeDisplayLayer];
|
||||
|
||||
@@ -99,50 +141,196 @@ extern int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size,
|
||||
return self;
|
||||
}
|
||||
|
||||
# pragma mark DisplayLink vsync callback
|
||||
|
||||
- (void)setupWithVideoFormat:(int)videoFormat width:(int)videoWidth height:(int)videoHeight frameRate:(int)frameRate
|
||||
{
|
||||
self->videoFormat = videoFormat;
|
||||
self->frameRate = frameRate;
|
||||
}
|
||||
self->_videoFormat = videoFormat;
|
||||
self->_frameRate = frameRate;
|
||||
|
||||
DataManager* dataMan = [[DataManager alloc] init];
|
||||
if ([[dataMan getSettings].renderingBackend integerValue] == RENDER_AVSB) {
|
||||
// PACING_MODE_VSYNC:
|
||||
// Deliver 1 frame at each vsync interval. Ignores server pts timestamps.
|
||||
// Drop frames intelligently to maintain chosen queue size.
|
||||
_renderingBackend = RENDER_AVSB;
|
||||
_displayLink = [CADisplayLink displayLinkWithTarget:self selector:@selector(renderModeAVSB:)];
|
||||
|
||||
- (void)start
|
||||
{
|
||||
_displayLink = [CADisplayLink displayLinkWithTarget:self selector:@selector(displayLinkCallback:)];
|
||||
if (@available(iOS 15.0, tvOS 15.0, *)) {
|
||||
_displayLink.preferredFrameRateRange = CAFrameRateRangeMake(self->frameRate, self->frameRate, self->frameRate);
|
||||
_displayLink.preferredFrameRateRange = CAFrameRateRangeMake(self->_frameRate, self->_frameRate, self->_frameRate);
|
||||
}
|
||||
else {
|
||||
_displayLink.preferredFramesPerSecond = self->frameRate;
|
||||
_displayLink.preferredFramesPerSecond = self->_frameRate;
|
||||
}
|
||||
[_displayLink addToRunLoop:[NSRunLoop mainRunLoop] forMode:NSRunLoopCommonModes];
|
||||
} else {
|
||||
_renderingBackend = RENDER_METAL;
|
||||
// RENDER_METAL begins in StreamFrameViewController.
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
- (void)setupDecompressionSessionWithAttributes:(NSDictionary *)destinationPixelBufferAttributes {
|
||||
if (_decompressionSession != NULL) {
|
||||
VTDecompressionSessionInvalidate(_decompressionSession);
|
||||
CFRelease(_decompressionSession);
|
||||
_decompressionSession = nil;
|
||||
}
|
||||
|
||||
int status = VTDecompressionSessionCreate(kCFAllocatorDefault,
|
||||
_formatDesc,
|
||||
nil,
|
||||
(__bridge CFDictionaryRef)destinationPixelBufferAttributes,
|
||||
nil,
|
||||
&_decompressionSession);
|
||||
if (status != noErr) {
|
||||
Log(LOG_E, @"Failed to create VTDecompressionSession, status %d", status);
|
||||
}
|
||||
}
|
||||
|
||||
- (void)setupDecompressionSession {
|
||||
#if TARGET_OS_SIMULATOR
|
||||
NSNumber *pixelFormat = @(kCVPixelFormatType_32BGRA);
|
||||
#else
|
||||
NSNumber *pixelFormat = nil;
|
||||
if (self->_videoFormat & VIDEO_FORMAT_MASK_YUV444) {
|
||||
pixelFormat = @(kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange);
|
||||
}
|
||||
else {
|
||||
pixelFormat = @(kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange);
|
||||
}
|
||||
#endif
|
||||
|
||||
NSDictionary *destinationPixelBufferAttributes = @{(id)kCVPixelBufferPixelFormatTypeKey : pixelFormat};
|
||||
|
||||
return [self setupDecompressionSessionWithAttributes:destinationPixelBufferAttributes];
|
||||
}
|
||||
|
||||
- (void) checkDisplayLayer {
|
||||
// Check for issues with the SampleBuffer, this should be much less likely since
|
||||
// AVSB is not actually decoding the frames anymore
|
||||
if (self->_displayLayer.status == AVQueuedSampleBufferRenderingStatusFailed) {
|
||||
Log(LOG_E, @"Display layer rendering failed: %@", _displayLayer.error);
|
||||
|
||||
// Recreate the display layer. We are already on the main thread,
|
||||
// so this is safe to do right here.
|
||||
[self reinitializeDisplayLayer];
|
||||
|
||||
// Request an IDR frame to initialize the new decoder
|
||||
LiRequestIdrFrame();
|
||||
}
|
||||
[_displayLink addToRunLoop:[NSRunLoop mainRunLoop] forMode:NSDefaultRunLoopMode];
|
||||
}
|
||||
|
||||
// TODO: Refactor this
|
||||
int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
|
||||
- (void)displayLinkCallback:(CADisplayLink *)sender
|
||||
{
|
||||
VIDEO_FRAME_HANDLE handle;
|
||||
PDECODE_UNIT du;
|
||||
#pragma mark DisplayLink - Frame Pacing - Vsync with FrameQueue
|
||||
|
||||
while (LiPollNextVideoFrame(&handle, &du)) {
|
||||
LiCompleteVideoFrame(handle, DrSubmitDecodeUnit(du));
|
||||
// This frame pacing method was inspired by the behavior of moonlight-qt's Pacer class, although it has evolved
|
||||
// a few additional features. Incoming frames from Sunshine are asynchronously processed into a queue by the VideoRecv thread.
|
||||
// DisplayLink calls us every vsync we we try to present the most recent frame. We try to maintain a user-configurable buffer
|
||||
// of 1-5 frames. If the buffer is full, every other frame is dropped which just appears to the user as a lower framerate stream.
|
||||
- (void)renderModeAVSB:(CADisplayLink *)link {
|
||||
CFTimeInterval start = link.timestamp;
|
||||
CFTimeInterval deadline = link.targetTimestamp;
|
||||
static CFTimeInterval lastTargetLocal = 0.0f;
|
||||
CFTimeInterval dl0 = CACurrentMediaTime();
|
||||
|
||||
if (framePacing) {
|
||||
// Calculate the actual display refresh rate
|
||||
double displayRefreshRate = 1 / (_displayLink.targetTimestamp - _displayLink.timestamp);
|
||||
static int lateCallbacks = 0;
|
||||
if (dl0 > deadline) {
|
||||
// we already missed it, count how often this happens
|
||||
lateCallbacks++;
|
||||
return;
|
||||
}
|
||||
|
||||
// Only pace frames if the display refresh rate is >= 90% of our stream frame rate.
|
||||
// Battery saver, accessibility settings, or device thermals can cause the actual
|
||||
// refresh rate of the display to drop below the physical maximum.
|
||||
if (displayRefreshRate >= frameRate * 0.9f) {
|
||||
// Keep one pending frame to smooth out gaps due to
|
||||
// network jitter at the cost of 1 frame of latency
|
||||
if (LiGetPendingVideoFrames() == 1) {
|
||||
break;
|
||||
[self checkDisplayLayer];
|
||||
|
||||
static CFTimeInterval avgOverhead = 0.004f; // averaged each callback
|
||||
CFTimeInterval waitFor = deadline - dl0 - avgOverhead;
|
||||
if (waitFor < 0.001f) {
|
||||
waitFor = 0.0f;
|
||||
}
|
||||
|
||||
// Get the next frame or wait if necessary. If no frame arrives the previous one will be redisplayed automatically.
|
||||
Frame *frame = [_frameQueue dequeueWithTimeout:waitFor];
|
||||
if (frame) {
|
||||
CFTimeInterval dl1 = CACurrentMediaTime();
|
||||
|
||||
LogOnce(LOG_I, @"Frame pacing: using AVSampleBufferDisplayLayer target %f Hz with %d FPS stream", 1.0f / (deadline - start), self->_frameRate);
|
||||
|
||||
// The system works best with properly timed video frames, which we time to the end of the next vsync period,
|
||||
// the earliest they can be displayed due to double-buffering.
|
||||
CFTimeInterval targetLocal = deadline + link.duration;
|
||||
|
||||
[self renderFrame:frame atTime:CMTimeMakeWithSeconds(targetLocal, NSEC_PER_SEC)];
|
||||
|
||||
#ifdef DISPLAYLINK_VERBOSE
|
||||
Log(LOG_I, @"[%.3f] rendering frame %d, waitFor %.3f ms, overhead %.3f ms, lateCallbacks %d, queue size %d",
|
||||
deadline, frame.frameNumber, waitFor * 1000.0, avgOverhead * 1000.0, lateCallbacks, [_frameQueue count]);
|
||||
#endif
|
||||
|
||||
// Update metrics
|
||||
if (lastTargetLocal != 0) {
|
||||
CFTimeInterval frametime = targetLocal - lastTargetLocal;
|
||||
if (frametime > deadline - start + 0.0005f) {
|
||||
// we missed a callback
|
||||
// Log(LOG_W, @"*** slow frametime %.3f ms", frametime * 1000.0);
|
||||
}
|
||||
[self->_callbacks observeFloat:PLOT_FRAMETIME value:frametime * 1000.0];
|
||||
}
|
||||
lastTargetLocal = targetLocal;
|
||||
|
||||
// weighted moving average of how much time displayLink needs after dequeuing a frame.
|
||||
// This is used to avoid overshooting a vsync by waiting too long.
|
||||
const double alpha = 0.1f;
|
||||
avgOverhead = ((CACurrentMediaTime() - dl1) * alpha) + (avgOverhead * (1.0 - alpha));
|
||||
}
|
||||
}
|
||||
|
||||
// Render frame at a specific targetTime
|
||||
- (void)renderFrame:(Frame *)frame atTime:(CMTime)targetTime {
|
||||
CMSampleBufferSetOutputPresentationTimeStamp(frame.sampleBuffer, targetTime);
|
||||
|
||||
if (frame.frameNumber == 1) {
|
||||
// On first frame, set timebase to the initial presentation time.
|
||||
// This will let us present frames using the local clock (vsync pacing) or
|
||||
// the pts timestamps from the host.
|
||||
CMTimebaseRef timebase = NULL;
|
||||
CMTimebaseCreateWithSourceClock(CFAllocatorGetDefault(), CMClockGetHostTimeClock(), &timebase);
|
||||
|
||||
// Set the timebase to the initial pts here
|
||||
CMTime pts = CMSampleBufferGetOutputPresentationTimeStamp(frame.sampleBuffer);
|
||||
CMTimebaseSetTime(timebase, pts);
|
||||
CMTimebaseSetRate(timebase, 1.0);
|
||||
|
||||
[self->_displayLayer setControlTimebase:timebase];
|
||||
Log(LOG_I, @"Setting timebase for stream to %d / %d", pts.value, pts.timescale);
|
||||
}
|
||||
|
||||
[self->_displayLayer enqueueSampleBuffer:frame.sampleBuffer];
|
||||
|
||||
#ifdef DISPLAYLINK_VERBOSE
|
||||
// Some OS-level metrics I'm not sure what to do with
|
||||
if (@available(iOS 17.4, tvOS 17.4, *)) {
|
||||
if (frame.frameNumber % 600 == 0) {
|
||||
[self->_displayLayer.sampleBufferRenderer loadVideoPerformanceMetricsWithCompletionHandler:^(AVVideoPerformanceMetrics * videoMetrics) {
|
||||
Log(LOG_I, @"AVVideoPerformanceMetrics: frames %d, dropped %d (%.1f%%), optimized %d (%.1f%%), accumulatedDelay %f",
|
||||
videoMetrics.totalNumberOfFrames, // The total number of frames that display if no frames drop.
|
||||
videoMetrics.numberOfDroppedFrames, // The total number of frames the system drops prior to decoding or from missing the display deadline
|
||||
((double)videoMetrics.numberOfDroppedFrames / videoMetrics.totalNumberOfFrames) * 100.0,
|
||||
videoMetrics.numberOfFramesDisplayedUsingOptimizedCompositing, // The total number of full screen frames rendered in a special power-efficient mode that didn’t require compositing with other UI elements.
|
||||
((double)videoMetrics.numberOfFramesDisplayedUsingOptimizedCompositing / videoMetrics.totalNumberOfFrames) * 100.0,
|
||||
videoMetrics.totalAccumulatedFrameDelay); // The accumulated amount of time between the prescribed presentation times of displayed video frames and their actual time of display.
|
||||
}];
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
if (frame.frameType == FRAME_TYPE_IDR) {
|
||||
// Ensure the layer is visible now
|
||||
self->_displayLayer.hidden = NO;
|
||||
|
||||
// Tell our parent VC to hide the progress indicator
|
||||
[self->_callbacks videoContentShown];
|
||||
}
|
||||
}
|
||||
|
||||
@@ -150,6 +338,18 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
[[NSNotificationCenter defaultCenter] removeObserver:self];
|
||||
[_displayLink invalidate];
|
||||
}
|
||||
- (void)cleanup
|
||||
{
|
||||
if (_renderingBackend == RENDER_AVSB) {
|
||||
[_displayLink invalidate];
|
||||
}
|
||||
|
||||
if (_decompressionSession != NULL) {
|
||||
VTDecompressionSessionInvalidate(_decompressionSession);
|
||||
CFRelease(_decompressionSession);
|
||||
_decompressionSession = nil;
|
||||
}
|
||||
}
|
||||
|
||||
#define NALU_START_PREFIX_SIZE 3
|
||||
#define NAL_LENGTH_PREFIX_SIZE 4
|
||||
@@ -351,6 +551,7 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
break;
|
||||
}
|
||||
|
||||
Log(LOG_I, @"AV1 video range: %@", seqHeader->color_config.color_range == 1 ? @"full" : @"limited");
|
||||
SET_EXTENSION(kCMFormatDescriptionExtension_FullRangeVideo, @(seqHeader->color_config.color_range == 1));
|
||||
|
||||
// Progressive content
|
||||
@@ -372,12 +573,12 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
break;
|
||||
}
|
||||
|
||||
if (contentLightLevelInfo) {
|
||||
SET_EXTENSION(kCMFormatDescriptionExtension_ContentLightLevelInfo, contentLightLevelInfo);
|
||||
if (_contentLightLevelInfo) {
|
||||
SET_EXTENSION(kCMFormatDescriptionExtension_ContentLightLevelInfo, _contentLightLevelInfo);
|
||||
}
|
||||
|
||||
if (masteringDisplayColorVolume) {
|
||||
SET_EXTENSION(kCMFormatDescriptionExtension_MasteringDisplayColorVolume, masteringDisplayColorVolume);
|
||||
if (_masteringDisplayColorVolume) {
|
||||
SET_EXTENSION(kCMFormatDescriptionExtension_MasteringDisplayColorVolume, _masteringDisplayColorVolume);
|
||||
}
|
||||
|
||||
// Referenced the VP9 code in Chrome that performs a similar function
|
||||
@@ -402,13 +603,21 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
formatDesc = NULL;
|
||||
}
|
||||
|
||||
Log(LOG_I, @"AV1 extensions: %@, format description: %@", extensions, formatDesc);
|
||||
|
||||
ff_cbs_fragment_free(&cbsFrag);
|
||||
ff_cbs_close(&cbsCtx);
|
||||
return formatDesc;
|
||||
}
|
||||
|
||||
#pragma mark VideoRecv thread - Decoder
|
||||
|
||||
// This function must free data for bufferType == BUFFER_TYPE_PICDATA
|
||||
- (int)submitDecodeBuffer:(unsigned char *)data length:(int)length bufferType:(int)bufferType decodeUnit:(PDECODE_UNIT)du
|
||||
- (int)submitDecodeBuffer:(unsigned char *)data
|
||||
length:(int)length
|
||||
bufferType:(int)bufferType
|
||||
decodeUnit:(PDECODE_UNIT)du
|
||||
decodeStartTime:(CFTimeInterval)decodeStartTime
|
||||
{
|
||||
OSStatus status;
|
||||
|
||||
@@ -418,7 +627,7 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
if (bufferType == BUFFER_TYPE_VPS || bufferType == BUFFER_TYPE_SPS || bufferType == BUFFER_TYPE_PPS) {
|
||||
// Add new parameter set into the parameter set array
|
||||
int startLen = data[2] == 0x01 ? 3 : 4;
|
||||
[parameterSetBuffers addObject:[NSData dataWithBytes:&data[startLen] length:length - startLen]];
|
||||
[_parameterSetBuffers addObject:[NSData dataWithBytes:&data[startLen] length:length - startLen]];
|
||||
}
|
||||
|
||||
// Data is NOT to be freed here. It's a direct usage of the caller's buffer.
|
||||
@@ -433,18 +642,18 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
// NB: This logic depends on the fact that we submit all picture data in one buffer!
|
||||
|
||||
// Free the old format description
|
||||
if (formatDesc != NULL) {
|
||||
CFRelease(formatDesc);
|
||||
formatDesc = NULL;
|
||||
if (_formatDesc != NULL) {
|
||||
CFRelease(_formatDesc);
|
||||
_formatDesc = NULL;
|
||||
}
|
||||
|
||||
if (videoFormat & VIDEO_FORMAT_MASK_H264) {
|
||||
if (_videoFormat & VIDEO_FORMAT_MASK_H264) {
|
||||
// Construct parameter set arrays for the format description
|
||||
size_t parameterSetCount = [parameterSetBuffers count];
|
||||
size_t parameterSetCount = [_parameterSetBuffers count];
|
||||
const uint8_t* parameterSetPointers[parameterSetCount];
|
||||
size_t parameterSetSizes[parameterSetCount];
|
||||
for (int i = 0; i < parameterSetCount; i++) {
|
||||
NSData* parameterSet = parameterSetBuffers[i];
|
||||
NSData* parameterSet = _parameterSetBuffers[i];
|
||||
parameterSetPointers[i] = parameterSet.bytes;
|
||||
parameterSetSizes[i] = parameterSet.length;
|
||||
}
|
||||
@@ -455,22 +664,22 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
parameterSetPointers,
|
||||
parameterSetSizes,
|
||||
NAL_LENGTH_PREFIX_SIZE,
|
||||
&formatDesc);
|
||||
&_formatDesc);
|
||||
if (status != noErr) {
|
||||
Log(LOG_E, @"Failed to create H264 format description: %d", (int)status);
|
||||
formatDesc = NULL;
|
||||
_formatDesc = NULL;
|
||||
}
|
||||
|
||||
// Free parameter set buffers after submission
|
||||
[parameterSetBuffers removeAllObjects];
|
||||
[_parameterSetBuffers removeAllObjects];
|
||||
}
|
||||
else if (videoFormat & VIDEO_FORMAT_MASK_H265) {
|
||||
else if (_videoFormat & VIDEO_FORMAT_MASK_H265) {
|
||||
// Construct parameter set arrays for the format description
|
||||
size_t parameterSetCount = [parameterSetBuffers count];
|
||||
size_t parameterSetCount = [_parameterSetBuffers count];
|
||||
const uint8_t* parameterSetPointers[parameterSetCount];
|
||||
size_t parameterSetSizes[parameterSetCount];
|
||||
for (int i = 0; i < parameterSetCount; i++) {
|
||||
NSData* parameterSet = parameterSetBuffers[i];
|
||||
NSData* parameterSet = _parameterSetBuffers[i];
|
||||
parameterSetPointers[i] = parameterSet.bytes;
|
||||
parameterSetSizes[i] = parameterSet.length;
|
||||
}
|
||||
@@ -479,12 +688,12 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
|
||||
NSMutableDictionary* videoFormatParams = [[NSMutableDictionary alloc] init];
|
||||
|
||||
if (contentLightLevelInfo) {
|
||||
[videoFormatParams setObject:contentLightLevelInfo forKey:(__bridge NSString*)kCMFormatDescriptionExtension_ContentLightLevelInfo];
|
||||
if (_contentLightLevelInfo) {
|
||||
[videoFormatParams setObject:_contentLightLevelInfo forKey:(__bridge NSString*)kCMFormatDescriptionExtension_ContentLightLevelInfo];
|
||||
}
|
||||
|
||||
if (masteringDisplayColorVolume) {
|
||||
[videoFormatParams setObject:masteringDisplayColorVolume forKey:(__bridge NSString*)kCMFormatDescriptionExtension_MasteringDisplayColorVolume];
|
||||
if (_masteringDisplayColorVolume) {
|
||||
[videoFormatParams setObject:_masteringDisplayColorVolume forKey:(__bridge NSString*)kCMFormatDescriptionExtension_MasteringDisplayColorVolume];
|
||||
}
|
||||
|
||||
status = CMVideoFormatDescriptionCreateFromHEVCParameterSets(kCFAllocatorDefault,
|
||||
@@ -493,21 +702,21 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
parameterSetSizes,
|
||||
NAL_LENGTH_PREFIX_SIZE,
|
||||
(__bridge CFDictionaryRef)videoFormatParams,
|
||||
&formatDesc);
|
||||
&_formatDesc);
|
||||
|
||||
if (status != noErr) {
|
||||
Log(LOG_E, @"Failed to create HEVC format description: %d", (int)status);
|
||||
formatDesc = NULL;
|
||||
_formatDesc = NULL;
|
||||
}
|
||||
|
||||
// Free parameter set buffers after submission
|
||||
[parameterSetBuffers removeAllObjects];
|
||||
[_parameterSetBuffers removeAllObjects];
|
||||
}
|
||||
else if (videoFormat & VIDEO_FORMAT_MASK_AV1) {
|
||||
else if (_videoFormat & VIDEO_FORMAT_MASK_AV1) {
|
||||
NSData* fullFrameData = [NSData dataWithBytesNoCopy:data length:length freeWhenDone:NO];
|
||||
|
||||
Log(LOG_I, @"Constructing new AV1 format description");
|
||||
formatDesc = [self createAV1FormatDescriptionForIDRFrame:fullFrameData];
|
||||
_formatDesc = [self createAV1FormatDescriptionForIDRFrame:fullFrameData];
|
||||
}
|
||||
else {
|
||||
// Unsupported codec!
|
||||
@@ -515,25 +724,12 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
}
|
||||
}
|
||||
|
||||
if (formatDesc == NULL) {
|
||||
if (_formatDesc == NULL) {
|
||||
// Can't decode if we haven't gotten our parameter sets yet
|
||||
free(data);
|
||||
return DR_NEED_IDR;
|
||||
}
|
||||
|
||||
// Check for previous decoder errors before doing anything
|
||||
if (displayLayer.status == AVQueuedSampleBufferRenderingStatusFailed) {
|
||||
Log(LOG_E, @"Display layer rendering failed: %@", displayLayer.error);
|
||||
|
||||
// Recreate the display layer. We are already on the main thread,
|
||||
// so this is safe to do right here.
|
||||
[self reinitializeDisplayLayer];
|
||||
|
||||
// Request an IDR frame to initialize the new decoder
|
||||
free(data);
|
||||
return DR_NEED_IDR;
|
||||
}
|
||||
|
||||
// Now we're decoding actual frame data here
|
||||
CMBlockBufferRef frameBlockBuffer;
|
||||
CMBlockBufferRef dataBlockBuffer;
|
||||
@@ -555,7 +751,7 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
}
|
||||
|
||||
// H.264 and HEVC formats require NAL prefix fixups from Annex B to length-delimited
|
||||
if (videoFormat & (VIDEO_FORMAT_MASK_H264 | VIDEO_FORMAT_MASK_H265)) {
|
||||
if (_videoFormat & (VIDEO_FORMAT_MASK_H264 | VIDEO_FORMAT_MASK_H265)) {
|
||||
int lastOffset = -1;
|
||||
for (int i = 0; i < length - NALU_START_PREFIX_SIZE; i++) {
|
||||
// Search for a NALU
|
||||
@@ -584,13 +780,18 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
}
|
||||
}
|
||||
|
||||
// Set the current frame's pts, in RTP 90khz units. We will set the duration
|
||||
// later in FrameQueue because it requires the next frame's timestamp.
|
||||
CMSampleTimingInfo sampleTiming = {
|
||||
.duration = kCMTimeInvalid,
|
||||
.presentationTimeStamp = CMTimeMake((int64_t)du->rtpTimestamp, 90000),
|
||||
.decodeTimeStamp = kCMTimeInvalid,
|
||||
};
|
||||
|
||||
CMSampleBufferRef sampleBuffer;
|
||||
|
||||
CMSampleTimingInfo sampleTiming = {kCMTimeInvalid, CMTimeMake(du->presentationTimeMs, 1000), kCMTimeInvalid};
|
||||
|
||||
status = CMSampleBufferCreateReady(kCFAllocatorDefault,
|
||||
frameBlockBuffer,
|
||||
formatDesc, 1, 1,
|
||||
_formatDesc, 1, 1,
|
||||
&sampleTiming, 0, NULL,
|
||||
&sampleBuffer);
|
||||
if (status != noErr) {
|
||||
@@ -600,17 +801,18 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
return DR_NEED_IDR;
|
||||
}
|
||||
|
||||
// Enqueue the next frame
|
||||
[self->displayLayer enqueueSampleBuffer:sampleBuffer];
|
||||
|
||||
if (du->frameType == FRAME_TYPE_IDR) {
|
||||
// Ensure the layer is visible now
|
||||
self->displayLayer.hidden = NO;
|
||||
|
||||
// Tell our parent VC to hide the progress indicator
|
||||
[self->_callbacks videoContentShown];
|
||||
OSStatus decodeStatus;
|
||||
if (0 && _renderingBackend == RENDER_METAL) {
|
||||
decodeStatus = [self decodeFrameToLinearColorspaceWithSampleBuffer:sampleBuffer
|
||||
frameNumber:du->frameNumber
|
||||
frameType:du->frameType
|
||||
decodeStartTime:decodeStartTime];
|
||||
} else {
|
||||
decodeStatus = [self decodeFrameWithSampleBuffer:sampleBuffer
|
||||
frameNumber:du->frameNumber
|
||||
frameType:du->frameType
|
||||
decodeStartTime:decodeStartTime];
|
||||
}
|
||||
|
||||
// Dereference the buffers
|
||||
CFRelease(dataBlockBuffer);
|
||||
CFRelease(frameBlockBuffer);
|
||||
@@ -619,6 +821,153 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
return DR_OK;
|
||||
}
|
||||
|
||||
// For experimenting with improved HDR tone-mapping by directly decoding to linear so we can tonemap using
|
||||
// the host's metadata.
|
||||
- (OSStatus)decodeFrameToLinearColorspaceWithSampleBuffer:(CMSampleBufferRef)sampleBuffer
|
||||
frameNumber:(int)frameNumber
|
||||
frameType:(int)frameType
|
||||
decodeStartTime:(CFTimeInterval)decodeStartTime
|
||||
{
|
||||
NSDictionary *linearAttributes = @{
|
||||
(id)kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_64RGBAHalf), // half-float RGBA
|
||||
(id)kCVImageBufferColorPrimariesKey : (id)kCVImageBufferColorPrimaries_ITU_R_2020, // wide-gamut primaries (Rec.2020)
|
||||
(id)kCVImageBufferTransferFunctionKey : (id)kCVImageBufferTransferFunction_Linear, // linear transfer function
|
||||
(id)kCVImageBufferYCbCrMatrixKey : (id)kCVImageBufferYCbCrMatrix_ITU_R_2020, // Y′CbCr matrix → RGB matrix (Rec.2020)
|
||||
(id)kCVPixelBufferMetalCompatibilityKey : @YES // make it GPU-compatible
|
||||
};
|
||||
|
||||
if (frameType == FRAME_TYPE_IDR || _decompressionSession == nil) {
|
||||
[self setupDecompressionSessionWithAttributes:linearAttributes];
|
||||
}
|
||||
|
||||
OSStatus status = VTDecompressionSessionDecodeFrameWithOutputHandler(
|
||||
_decompressionSession, sampleBuffer, 0, NULL,
|
||||
^(OSStatus status, VTDecodeInfoFlags infoFlags, CVImageBufferRef _Nullable imageBuffer, CMTime pts, CMTime duration) {
|
||||
CVPixelBufferRef pixelBuffer = CVPixelBufferRetain((CVPixelBufferRef)imageBuffer);
|
||||
Log(LOG_D, @"Decoded to PixelBuffer %@", pixelBuffer); // dumps full frame details
|
||||
|
||||
// Dispatch onto our higher priority queue
|
||||
dispatch_async(self->_vtq, ^{
|
||||
Frame *frame = [[Frame alloc] initWithPixelBufffer:pixelBuffer
|
||||
frameNumber:frameNumber
|
||||
frameType:frameType
|
||||
pts:pts];
|
||||
[frame setFormatDesc:self->_formatDesc];
|
||||
int framesDropped = [self->_frameQueue enqueue:frame withSlackSize:3];
|
||||
|
||||
static PlotMetrics frameQueueMetrics = {};
|
||||
[self->_callbacks observeFloatReturnMetrics:PLOT_QUEUED_FRAMES value:[self->_frameQueue count] plotMetrics:&frameQueueMetrics];
|
||||
[self safeCopyMetricsTo:&self->_frameQueueMetrics from:&frameQueueMetrics];
|
||||
|
||||
[self->_callbacks observeFloat:PLOT_DROPPED value:framesDropped];
|
||||
|
||||
// It's important we capture host metrics on the incoming thread, as this frame object
|
||||
// may have been dropped by the above enqueue
|
||||
static CFTimeInterval lastHostFrame = 0.0f;
|
||||
if (lastHostFrame != 0) {
|
||||
[self->_callbacks observeFloat:PLOT_HOST_FRAMETIME value:(frame.pts - lastHostFrame) * 1000.0];
|
||||
}
|
||||
lastHostFrame = frame.pts;
|
||||
|
||||
// Decode time is not graphed because it is marked as hidden, but we can use the same mechanism for the value used by stats
|
||||
static PlotMetrics decodeMetrics = {};
|
||||
[self->_callbacks observeFloatReturnMetrics:PLOT_DECODE value:(CACurrentMediaTime() - decodeStartTime) * 1000.0 plotMetrics:&decodeMetrics];
|
||||
[self safeCopyMetricsTo:&self->_decodeMetrics from:&decodeMetrics];
|
||||
});
|
||||
});
|
||||
return status;
|
||||
}
|
||||
|
||||
- (OSStatus)decodeFrameWithSampleBuffer:(CMSampleBufferRef)sampleBuffer
|
||||
frameNumber:(int)frameNumber
|
||||
frameType:(int)frameType
|
||||
decodeStartTime:(CFTimeInterval)decodeStartTime {
|
||||
if (frameType == FRAME_TYPE_IDR || _decompressionSession == nil) {
|
||||
[self setupDecompressionSession];
|
||||
}
|
||||
|
||||
OSStatus status = VTDecompressionSessionDecodeFrameWithOutputHandler(
|
||||
_decompressionSession,
|
||||
sampleBuffer,
|
||||
0,
|
||||
NULL,
|
||||
^(OSStatus status, VTDecodeInfoFlags infoFlags, CVImageBufferRef _Nullable imageBuffer, CMTime presentationTimestamp, CMTime presentationDuration) {
|
||||
if (status != noErr || !imageBuffer) {
|
||||
NSError *error = [NSError errorWithDomain:NSOSStatusErrorDomain code:status userInfo:nil];
|
||||
Log(LOG_E, @"Decompression session error: %@", error);
|
||||
LiRequestIdrFrame();
|
||||
return;
|
||||
}
|
||||
|
||||
CMSampleBufferRef sampleBufferOut = nil;
|
||||
CVPixelBufferRef pixelBuffer = nil;
|
||||
|
||||
// AVSampleBuffer path: package into a SampleBuffer
|
||||
if (self->_renderingBackend == RENDER_AVSB) {
|
||||
if (self->_formatDescImageBuffer == NULL || !CMVideoFormatDescriptionMatchesImageBuffer(self->_formatDescImageBuffer, imageBuffer)) {
|
||||
OSStatus res = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, imageBuffer, &(self->_formatDescImageBuffer));
|
||||
if (res != noErr) {
|
||||
Log(LOG_E, @"Failed to create video format description from imageBuffer");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
if (self->_formatDescImageBuffer == NULL || !CMVideoFormatDescriptionMatchesImageBuffer(self->_formatDescImageBuffer, imageBuffer)) {
|
||||
OSStatus res = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, imageBuffer, &(self->_formatDescImageBuffer));
|
||||
if (res != noErr) {
|
||||
Log(LOG_E, @"Failed to create video format description from imageBuffer");
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
CMSampleTimingInfo sampleTiming = {kCMTimeInvalid, presentationTimestamp, presentationDuration};
|
||||
|
||||
OSStatus err =
|
||||
CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, imageBuffer, self->_formatDescImageBuffer, &sampleTiming, &sampleBufferOut);
|
||||
if (err != noErr) {
|
||||
Log(LOG_E, @"Error creating sample buffer for decompressed image buffer %d", (int)err);
|
||||
return;
|
||||
}
|
||||
} else if (self->_renderingBackend == RENDER_METAL) {
|
||||
// Metal path: retain the pixelBuffer here so it survives the dispatch
|
||||
pixelBuffer = CVPixelBufferRetain((CVPixelBufferRef)imageBuffer);
|
||||
}
|
||||
|
||||
// Dispatch onto our higher priority queue
|
||||
dispatch_async(self->_vtq, ^{
|
||||
Frame *frame = nil;
|
||||
if (self->_renderingBackend == RENDER_AVSB) {
|
||||
frame = [[Frame alloc] initWithSampleBuffer:sampleBufferOut frameNumber:frameNumber frameType:frameType];
|
||||
} else {
|
||||
frame = [[Frame alloc] initWithPixelBufffer:pixelBuffer frameNumber:frameNumber frameType:frameType pts:presentationTimestamp];
|
||||
[frame setFormatDesc:self->_formatDesc];
|
||||
}
|
||||
int framesDropped = [self->_frameQueue enqueue:frame withSlackSize:3];
|
||||
|
||||
static PlotMetrics frameQueueMetrics = {};
|
||||
[self->_callbacks observeFloatReturnMetrics:PLOT_QUEUED_FRAMES value:[self->_frameQueue count] plotMetrics:&frameQueueMetrics];
|
||||
[self safeCopyMetricsTo:&self->_frameQueueMetrics from:&frameQueueMetrics];
|
||||
|
||||
[self->_callbacks observeFloat:PLOT_DROPPED value:framesDropped];
|
||||
|
||||
// It's important we capture host metrics on the incoming thread, as this frame object
|
||||
// may have been dropped by the above enqueue
|
||||
static CFTimeInterval lastHostFrame = 0.0f;
|
||||
if (lastHostFrame != 0) {
|
||||
[self->_callbacks observeFloat:PLOT_HOST_FRAMETIME value:(frame.pts - lastHostFrame) * 1000.0];
|
||||
}
|
||||
lastHostFrame = frame.pts;
|
||||
|
||||
// Decode time is not graphed because it is marked as hidden, but we can use the same mechanism for the value used by stats
|
||||
static PlotMetrics decodeMetrics = {};
|
||||
[self->_callbacks observeFloatReturnMetrics:PLOT_DECODE value:(CACurrentMediaTime() - decodeStartTime) * 1000.0 plotMetrics:&decodeMetrics];
|
||||
[self safeCopyMetricsTo:&self->_decodeMetrics from:&decodeMetrics];
|
||||
});
|
||||
});
|
||||
|
||||
return status;
|
||||
}
|
||||
|
||||
- (void)setHdrMode:(BOOL)enabled {
|
||||
SS_HDR_METADATA hdrMetadata;
|
||||
|
||||
@@ -650,13 +999,20 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
mdcv.luminance_min = __builtin_bswap32(hdrMetadata.minDisplayLuminance);
|
||||
|
||||
NSData* newMdcv = [NSData dataWithBytes:&mdcv length:sizeof(mdcv)];
|
||||
if (masteringDisplayColorVolume == nil || ![newMdcv isEqualToData:masteringDisplayColorVolume]) {
|
||||
masteringDisplayColorVolume = newMdcv;
|
||||
if (_masteringDisplayColorVolume == nil || ![newMdcv isEqualToData:_masteringDisplayColorVolume]) {
|
||||
_masteringDisplayColorVolume = newMdcv;
|
||||
metadataChanged = YES;
|
||||
|
||||
Log(LOG_I, @"HDR Mastering Display Color Volume: G(%d,%d) B(%d,%d) R(%d,%d) white point(%d,%d) luminance (%d,%d)",
|
||||
mdcv.primaries[0].x, mdcv.primaries[0].y,
|
||||
mdcv.primaries[1].x, mdcv.primaries[1].y,
|
||||
mdcv.primaries[2].x, mdcv.primaries[2].y,
|
||||
mdcv.white_point.x, mdcv.white_point.y,
|
||||
mdcv.luminance_max, mdcv.luminance_min);
|
||||
}
|
||||
}
|
||||
else if (masteringDisplayColorVolume != nil) {
|
||||
masteringDisplayColorVolume = nil;
|
||||
else if (_masteringDisplayColorVolume != nil) {
|
||||
_masteringDisplayColorVolume = nil;
|
||||
metadataChanged = YES;
|
||||
}
|
||||
|
||||
@@ -671,13 +1027,16 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
cll.max_frame_average_light_level = __builtin_bswap16(hdrMetadata.maxFrameAverageLightLevel);
|
||||
|
||||
NSData* newCll = [NSData dataWithBytes:&cll length:sizeof(cll)];
|
||||
if (contentLightLevelInfo == nil || ![newCll isEqualToData:contentLightLevelInfo]) {
|
||||
contentLightLevelInfo = newCll;
|
||||
if (_contentLightLevelInfo == nil || ![newCll isEqualToData:_contentLightLevelInfo]) {
|
||||
_contentLightLevelInfo = newCll;
|
||||
metadataChanged = YES;
|
||||
|
||||
Log(LOG_I, @"HDR maxCLL: %d maxFALL: %d",
|
||||
cll.max_content_light_level, cll.max_frame_average_light_level);
|
||||
}
|
||||
}
|
||||
else if (contentLightLevelInfo != nil) {
|
||||
contentLightLevelInfo = nil;
|
||||
else if (_contentLightLevelInfo != nil) {
|
||||
_contentLightLevelInfo = nil;
|
||||
metadataChanged = YES;
|
||||
}
|
||||
|
||||
@@ -687,6 +1046,75 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
|
||||
}
|
||||
}
|
||||
|
||||
- (void)safeCopyMetricsTo:(PlotMetrics *)dst from:(PlotMetrics *)src {
|
||||
if (dst != nil && src != nil) {
|
||||
dispatch_sync(_sq, ^{
|
||||
memcpy(dst, src, sizeof(PlotMetrics));
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
- (void)getAllStats:(video_stats_t *)stats {
|
||||
dispatch_sync(_sq, ^{
|
||||
stats->renderingBackend = _renderingBackend;
|
||||
memcpy(&stats->decodeMetrics, &_decodeMetrics, sizeof(PlotMetrics));
|
||||
memcpy(&stats->frameQueueMetrics, &_frameQueueMetrics, sizeof(PlotMetrics));
|
||||
[_frameQueue.frameDropMetrics copyMetrics:&stats->frameDropMetrics];
|
||||
});
|
||||
}
|
||||
|
||||
// When streaming lower framerate content on a ProMotion display, the screen refresh rate can be
|
||||
// reduced, optimizing battery life. Not currently used, it doesn't seem as reliable as I'd like.
|
||||
- (void)optimizeRefreshRate {
|
||||
static NSArray<NSNumber *> *supportedRates;
|
||||
static dispatch_once_t onceToken;
|
||||
static int lastTargetRate = 0;
|
||||
int targetRate = (int)_maxRefreshRate;
|
||||
|
||||
if (_maxRefreshRate <= 60 || _maxRefreshRate == 90) {
|
||||
return;
|
||||
}
|
||||
|
||||
dispatch_once(&onceToken, ^{
|
||||
// https://developer.apple.com/documentation/quartzcore/optimizing-promotion-refresh-rates-for-iphone-13-pro-and-ipad-pro?language=objc
|
||||
UIDevice *device = [UIDevice currentDevice];
|
||||
if (device.userInterfaceIdiom == UIUserInterfaceIdiomPad) {
|
||||
supportedRates = @[@24, @30, @40, @60, @120];
|
||||
}
|
||||
else if (device.userInterfaceIdiom == UIUserInterfaceIdiomPhone) {
|
||||
supportedRates = @[@10, @12, @15, @16, @20, @24, @30, @40, @48, @60, @80, @120];
|
||||
}
|
||||
else {
|
||||
supportedRates = @[@30, @60];
|
||||
}
|
||||
});
|
||||
|
||||
CFTimeInterval streamFps = [_frameQueue estimatedFramerate];
|
||||
if (streamFps > _maxRefreshRate) {
|
||||
streamFps = _maxRefreshRate;
|
||||
}
|
||||
|
||||
for (NSNumber *r in supportedRates) {
|
||||
NSInteger rate = r.integerValue;
|
||||
if (rate >= (int)streamFps) {
|
||||
targetRate = (int)rate;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (targetRate == lastTargetRate) {
|
||||
return;
|
||||
}
|
||||
lastTargetRate = targetRate;
|
||||
|
||||
Log(LOG_I, @"optimizeRefreshRate: new rate %d Hz based on streamFps of %.2f fps", targetRate, streamFps);
|
||||
|
||||
if (@available(iOS 15.0, tvOS 15.0, *)) {
|
||||
_displayLink.preferredFrameRateRange = CAFrameRateRangeMake(targetRate, _maxRefreshRate, targetRate);
|
||||
}
|
||||
else {
|
||||
_displayLink.preferredFramesPerSecond = targetRate;
|
||||
}
|
||||
}
|
||||
|
||||
@end
|
||||
|
||||
@@ -38,7 +38,7 @@ static const int LABEL_DY = 20;
|
||||
#if TARGET_OS_TV
|
||||
self.frame = CGRectMake(0, 0, 400, 400);
|
||||
#else
|
||||
if (UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPad) {
|
||||
if ([UIDevice currentDevice].userInterfaceIdiom == UIUserInterfaceIdiomPad) {
|
||||
self.frame = CGRectMake(0, 0, 200, 200);
|
||||
} else {
|
||||
self.frame = CGRectMake(0, 0, 100, 100);
|
||||
@@ -59,7 +59,7 @@ static const int LABEL_DY = 20;
|
||||
_hostLabel.textColor = [UIColor whiteColor];
|
||||
|
||||
_hostOverlay = [[UIImageView alloc] initWithFrame:CGRectMake(self.frame.size.width / 3, _hostIcon.frame.size.height / 4, _hostIcon.frame.size.width / 3, self.frame.size.height / 3)];
|
||||
_hostSpinner = [[UIActivityIndicatorView alloc] initWithActivityIndicatorStyle:UIActivityIndicatorViewStyleWhiteLarge];
|
||||
_hostSpinner = [[UIActivityIndicatorView alloc] initWithActivityIndicatorStyle:UIActivityIndicatorViewStyleLarge];
|
||||
[_hostSpinner setFrame:_hostOverlay.frame];
|
||||
_hostSpinner.userInteractionEnabled = NO;
|
||||
_hostSpinner.hidesWhenStopped = YES;
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
#import <Foundation/Foundation.h>
|
||||
|
||||
NS_ASSUME_NONNULL_BEGIN
|
||||
|
||||
/**
|
||||
* @brief The BandwidthTracker class tracks network bandwidth usage over a sliding time window (default 10s).
|
||||
*
|
||||
* Byte totals are grouped into fixed time interval buckets (default 250ms). This provides an element of smoothing
|
||||
* and deals well with spikes.
|
||||
*
|
||||
* GetAverageMbps() is calculated using the 25% most recent fully completed buckets. The default settings will
|
||||
* return an average of the past 2.5s of data, ignoring the in-progress bucket. Using only 2.5s of data for the
|
||||
* average provides a good balance of reactivity and smoothness.
|
||||
*
|
||||
* GetPeakMbps() returns the peak bandwidth seen during any one bucket interval across the full time window.
|
||||
*
|
||||
* All public methods are thread safe. A typical use case is calling AddBytes() in a data processing thread while
|
||||
* calling GetAverageMbps() from a UI thread.
|
||||
*
|
||||
* This class was mostly auto-converted from the C++ version to use Obj-C APIs and naming conventions.
|
||||
*/
|
||||
@interface BandwidthTracker : NSObject
|
||||
|
||||
/**
|
||||
* @brief Constructs a new BandwidthTracker object.
|
||||
*
|
||||
* Initializes the tracker to maintain statistics over a sliding window of time.
|
||||
* The window is divided into buckets of fixed duration (bucketIntervalMs).
|
||||
*
|
||||
* @param windowSeconds The duration of the tracking window in seconds. Default is 10 seconds.
|
||||
* @param bucketIntervalMs The interval for each bucket in milliseconds. Default is 250 ms.
|
||||
*/
|
||||
- (instancetype)initWithWindowSeconds:(NSUInteger)windowSeconds
|
||||
bucketIntervalMs:(NSUInteger)bucketIntervalMs NS_DESIGNATED_INITIALIZER;
|
||||
|
||||
/// Convenience init → defaults to a 10 s window with 250 ms buckets.
|
||||
- (instancetype)init;
|
||||
|
||||
/**
|
||||
* @brief Record bytes that were received or sent.
|
||||
*
|
||||
* This method updates the corresponding bucket for the current time interval with the new data.
|
||||
* It is thread-safe. Bytes are associated with the bucket for "now" and it is not possible to
|
||||
* submit data for old buckets. This function should be called as needed at the time the bytes
|
||||
* were received. Callers should not maintain their own byte totals.
|
||||
*
|
||||
* @param bytes The number of bytes to add.
|
||||
*/
|
||||
- (void)addBytes:(NSUInteger)bytes;
|
||||
|
||||
/**
|
||||
* @brief Computes and returns the average bandwidth in Mbps for the most recent 25% of buckets.
|
||||
*
|
||||
* @return The average bandwidth in megabits per second.
|
||||
*/
|
||||
- (double)averageMbps;
|
||||
|
||||
/**
|
||||
* @brief Returns the peak bandwidth in Mbps observed in any single bucket within the current window.
|
||||
*
|
||||
* This value represents the highest instantaneous throughput measured over one bucket interval.
|
||||
*
|
||||
* @return The peak bandwidth in megabits per second.
|
||||
*/
|
||||
- (double)peakMbps;
|
||||
|
||||
/**
|
||||
* @brief Retrieves the duration of the tracking window.
|
||||
*
|
||||
* This is useful when displaying the length of the peak, e.g.
|
||||
* @code
|
||||
* printf("Bitrate: %.1f Mbps Peak (%us): %.1f\n",
|
||||
* [bw averageMbps], [bw windowSeconds], [bw peakMbps]);
|
||||
* @endcode
|
||||
*
|
||||
* @return The window duration in seconds.
|
||||
*/
|
||||
- (NSUInteger)windowSeconds;
|
||||
|
||||
@end
|
||||
|
||||
NS_ASSUME_NONNULL_END
|
||||
@@ -0,0 +1,136 @@
|
||||
#import "BandwidthTracker.h"
|
||||
#import <QuartzCore/QuartzCore.h> // for CACurrentMediaTime()
|
||||
|
||||
// A single time‐bucket: start time (ms) + bytes counted.
|
||||
typedef struct {
|
||||
CFTimeInterval startMs;
|
||||
NSUInteger bytes;
|
||||
} Bucket;
|
||||
|
||||
@interface BandwidthTracker () {
|
||||
// Configuration
|
||||
NSUInteger _windowSeconds; // e.g. 10
|
||||
NSUInteger _bucketIntervalMs; // e.g. 250
|
||||
NSUInteger _bucketCount; // = (_windowSeconds * 1000) / _bucketIntervalMs
|
||||
|
||||
// State
|
||||
Bucket *_buckets; // C-array of _bucketCount entries
|
||||
NSLock *_lock; // protects all bucket ops
|
||||
}
|
||||
@end
|
||||
|
||||
@implementation BandwidthTracker
|
||||
|
||||
- (instancetype)init {
|
||||
return [self initWithWindowSeconds:10 bucketIntervalMs:250];
|
||||
}
|
||||
|
||||
- (instancetype)initWithWindowSeconds:(NSUInteger)windowSeconds bucketIntervalMs:(NSUInteger)bucketIntervalMs {
|
||||
self = [super init];
|
||||
if (self) {
|
||||
_windowSeconds = (windowSeconds > 0 ? windowSeconds : 10);
|
||||
_bucketIntervalMs = (bucketIntervalMs > 0 ? bucketIntervalMs : 250);
|
||||
|
||||
_bucketCount = (_windowSeconds * 1000) / _bucketIntervalMs;
|
||||
_buckets = calloc(_bucketCount, sizeof(Bucket));
|
||||
_lock = [[NSLock alloc] init];
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (void)dealloc {
|
||||
free(_buckets);
|
||||
}
|
||||
|
||||
- (NSUInteger)windowSeconds {
|
||||
return _windowSeconds;
|
||||
}
|
||||
|
||||
#pragma mark – Public API
|
||||
|
||||
- (void)addBytes:(NSUInteger)bytes {
|
||||
[_lock lock];
|
||||
CFTimeInterval nowMs = CACurrentMediaTime() * 1000.0;
|
||||
[self _updateBucketWithBytes:bytes nowMs:nowMs];
|
||||
[_lock unlock];
|
||||
}
|
||||
|
||||
- (double)averageMbps {
|
||||
[_lock lock];
|
||||
CFTimeInterval nowMs = CACurrentMediaTime() * 1000.0;
|
||||
NSUInteger currentIndex = ((NSUInteger)nowMs / _bucketIntervalMs) % _bucketCount;
|
||||
NSUInteger maxBuckets = _bucketCount / 4;
|
||||
NSUInteger totalBytes = 0;
|
||||
CFTimeInterval oldestStart = nowMs;
|
||||
|
||||
for (NSUInteger i = 0; i < maxBuckets; i++) {
|
||||
NSUInteger idx = (currentIndex + _bucketCount - i) % _bucketCount;
|
||||
Bucket b = _buckets[idx];
|
||||
if ([self _isBucketValid:&b nowMs:nowMs] && (nowMs - b.startMs >= _bucketIntervalMs)) {
|
||||
totalBytes += b.bytes;
|
||||
if (b.startMs < oldestStart)
|
||||
oldestStart = b.startMs;
|
||||
}
|
||||
}
|
||||
|
||||
// elapsed time in seconds
|
||||
double elapsedSec = (nowMs - oldestStart) / 1000.0;
|
||||
double avg = (elapsedSec > 0.0 ? totalBytes * 8.0 / 1000000.0 / elapsedSec : 0.0);
|
||||
[_lock unlock];
|
||||
return avg;
|
||||
}
|
||||
|
||||
- (double)peakMbps {
|
||||
[_lock lock];
|
||||
CFTimeInterval nowMs = CACurrentMediaTime() * 1000.0;
|
||||
double peak = 0.0;
|
||||
|
||||
for (NSUInteger i = 0; i < _bucketCount; i++) {
|
||||
Bucket b = _buckets[i];
|
||||
if ([self _isBucketValid:&b nowMs:nowMs]) {
|
||||
double mbps = [self _bucketMbps:&b];
|
||||
if (mbps > peak)
|
||||
peak = mbps;
|
||||
}
|
||||
}
|
||||
|
||||
[_lock unlock];
|
||||
return peak;
|
||||
}
|
||||
|
||||
#pragma mark – Private Helpers
|
||||
|
||||
// Check if a bucket's data is still valid (within the window)
|
||||
- (BOOL)_isBucketValid:(Bucket *)bucket nowMs:(CFTimeInterval)nowMs {
|
||||
return (nowMs - bucket->startMs) <= (_windowSeconds * 1000.0);
|
||||
}
|
||||
|
||||
- (double)_bucketMbps:(Bucket *)bucket {
|
||||
// bucketIntervalMs is ms → divide by 1000 to get seconds
|
||||
return bucket->bytes * 8.0 / 1000000.0 / (_bucketIntervalMs / 1000.0);
|
||||
}
|
||||
|
||||
- (void)_updateBucketWithBytes:(NSUInteger)bytes nowMs:(CFTimeInterval)nowMs {
|
||||
NSUInteger ms = (NSUInteger)nowMs;
|
||||
NSUInteger idx = (ms / _bucketIntervalMs) % _bucketCount;
|
||||
NSUInteger alignedMs = ms - (ms % _bucketIntervalMs);
|
||||
CFTimeInterval startMs = alignedMs;
|
||||
|
||||
Bucket *b = &_buckets[idx];
|
||||
|
||||
// If this bucket is stale (older than the window), reset it
|
||||
if (nowMs - b->startMs > (_windowSeconds * 1000.0)) {
|
||||
b->bytes = 0;
|
||||
b->startMs = startMs;
|
||||
}
|
||||
|
||||
// If we’ve moved into a new sub-interval, overwrite; otherwise accumulate
|
||||
if (b->startMs != startMs) {
|
||||
b->bytes = bytes;
|
||||
b->startMs = startMs;
|
||||
} else {
|
||||
b->bytes += bytes;
|
||||
}
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,60 @@
|
||||
#import <Foundation/Foundation.h>
|
||||
#import "Plot.h"
|
||||
|
||||
NS_ASSUME_NONNULL_BEGIN
|
||||
|
||||
/// A thread-safe circular buffer of `float` values (capacity must be a power of two).
|
||||
/// This is a mostly automated conversion of my C++ FloatBuffer.h class to Objective-C.
|
||||
@interface FloatBuffer : NSObject
|
||||
|
||||
/// The total capacity (always a power of two).
|
||||
@property(nonatomic, readonly) int capacity;
|
||||
|
||||
/// The current number of valid entries (≤ capacity).
|
||||
@property(nonatomic, readonly) int count;
|
||||
|
||||
/// The minimum value among all entries currently in the buffer.
|
||||
@property(nonatomic, readonly) float minValue;
|
||||
|
||||
/// The maximum value among all entries currently in the buffer.
|
||||
@property(nonatomic, readonly) float maxValue;
|
||||
|
||||
/// The arithmetic average (sum / count) of all entries. Zero if count=0.
|
||||
@property (nonatomic, readonly) float averageValue;
|
||||
|
||||
@property (nonatomic, readonly) float total;
|
||||
|
||||
/// Designated initializer. `capacity` must be >0 and a power of two, otherwise throws an exception.
|
||||
- (instancetype)initWithCapacity:(int)capacity NS_DESIGNATED_INITIALIZER;
|
||||
|
||||
/// Convenience initializer: same as `initWithCapacity:`.
|
||||
- (instancetype)init;
|
||||
|
||||
/// Adds a new float into the buffer (overwriting the oldest if full). Thread-safe.
|
||||
- (void)addValue:(float)value;
|
||||
|
||||
/// Most recent value
|
||||
- (float)newestValue;
|
||||
|
||||
/// Oldest timestamp
|
||||
- (CFTimeInterval)oldestTimestamp;
|
||||
|
||||
/// Copies the buffer’s contents (oldest→newest) into `outBuffer`, which must be able to hold at least `self.count` floats.
|
||||
/// Returns the number of floats written (i.e. the current `count`).
|
||||
/// If non-NULL, `outMin` and `outMax` are set to the buffer’s current minimum/maximum.
|
||||
- (int)copyValuesIntoBuffer:(float *)outBuffer
|
||||
min:(float * _Nullable)outMin
|
||||
max:(float * _Nullable)outMax;
|
||||
|
||||
/// Export metrics for use by the classic stats overlay
|
||||
- (void)copyMetrics:(PlotMetrics *)plotMetrics;
|
||||
|
||||
/// Clear and reset the buffer to 0.
|
||||
- (void)clear;
|
||||
|
||||
/// Calls `block(value, timestamp)` for each value in the buffer (from oldest to newest)
|
||||
- (void)enumerateValuesWithBlock:(void (^)(float value, BOOL *stop))block;
|
||||
|
||||
@end
|
||||
|
||||
NS_ASSUME_NONNULL_END
|
||||
@@ -0,0 +1,273 @@
|
||||
#import <UIKit/UIKit.h>
|
||||
#import "FloatBuffer.h"
|
||||
|
||||
@implementation FloatBuffer {
|
||||
float *_buffer; // raw C array holding up to capacity floats
|
||||
CFTimeInterval *_timestamps; // timestamp the entry was observed
|
||||
int _head; // index of next write (0…capacity−1)
|
||||
int _count; // how many valid entries are in the buffer (≤ capacity)
|
||||
float _minValue; // current minimum across all valid entries
|
||||
float _maxValue; // current maximum across all valid entries
|
||||
double _sum; // running sum of all valid entries (for average)
|
||||
dispatch_queue_t _sq; // serial queue for thread safety
|
||||
}
|
||||
|
||||
@synthesize capacity = _capacity;
|
||||
@synthesize count = _count;
|
||||
@synthesize minValue = _minValue;
|
||||
@synthesize maxValue = _maxValue;
|
||||
@synthesize averageValue = _averageValue; // custom getter below
|
||||
|
||||
- (instancetype)init {
|
||||
return [self initWithCapacity:256];
|
||||
}
|
||||
|
||||
- (instancetype)initWithCapacity:(int)capacity {
|
||||
self = [super init];
|
||||
if (self) {
|
||||
// Enforce capacity > 0 and a power of two
|
||||
if (capacity <= 0 || (capacity & (capacity - 1)) != 0) {
|
||||
@throw [NSException exceptionWithName:NSInvalidArgumentException reason:@"FloatBuffer capacity must be a nonzero power of two" userInfo:nil];
|
||||
}
|
||||
_capacity = capacity;
|
||||
_buffer = calloc(capacity, sizeof(float));
|
||||
_timestamps = calloc(capacity, sizeof(CFTimeInterval));
|
||||
_head = 0;
|
||||
_count = 0;
|
||||
_minValue = 0.0f;
|
||||
_maxValue = 0.0f;
|
||||
_sum = 0.0;
|
||||
|
||||
_sq = dispatch_queue_create("com.floatbuffer.serial", DISPATCH_QUEUE_SERIAL);
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (void)dealloc {
|
||||
if (_buffer) {
|
||||
free(_buffer);
|
||||
_buffer = NULL;
|
||||
}
|
||||
if (_timestamps) {
|
||||
free(_timestamps);
|
||||
_timestamps = NULL;
|
||||
}
|
||||
}
|
||||
|
||||
- (void)addValue:(float)value {
|
||||
dispatch_sync(_sq, ^{
|
||||
[self _unsafeAddValue:value];
|
||||
});
|
||||
}
|
||||
|
||||
- (void)_unsafeAddValue:(float)value {
|
||||
BOOL wasFull = (_count == _capacity);
|
||||
float overwrittenValue = 0.0f;
|
||||
if (wasFull) {
|
||||
// The slot at _head is about to be overwritten
|
||||
overwrittenValue = _buffer[_head];
|
||||
}
|
||||
|
||||
// 1) Write the new value into the “head” slot:
|
||||
_buffer[_head] = value;
|
||||
_timestamps[_head] = CACurrentMediaTime();
|
||||
_head = (_head + 1) & (_capacity - 1); // wrap via bitmask
|
||||
|
||||
// 2) Update count / sum / min / max
|
||||
if (!wasFull) {
|
||||
// We had room to grow
|
||||
_count += 1;
|
||||
_sum += value;
|
||||
|
||||
if (_count == 1) {
|
||||
// Very first element:
|
||||
_minValue = value;
|
||||
_maxValue = value;
|
||||
} else {
|
||||
// Compare to existing min/max
|
||||
if (value < _minValue)
|
||||
_minValue = value;
|
||||
if (value > _maxValue)
|
||||
_maxValue = value;
|
||||
}
|
||||
} else {
|
||||
// Buffer was full: dropped overwrittenValue, added new value
|
||||
_sum += (double)value - (double)overwrittenValue;
|
||||
|
||||
// If overwrittenValue was equal to old min or max, we must rescan
|
||||
if (overwrittenValue == _minValue || overwrittenValue == _maxValue) {
|
||||
float newMin = _buffer[0];
|
||||
float newMax = _buffer[0];
|
||||
for (int i = 1; i < _capacity; i++) {
|
||||
float v = _buffer[i];
|
||||
if (v < newMin)
|
||||
newMin = v;
|
||||
if (v > newMax)
|
||||
newMax = v;
|
||||
}
|
||||
_minValue = newMin;
|
||||
_maxValue = newMax;
|
||||
}
|
||||
// Finally, make sure the newly written `value` updates min/max if needed
|
||||
if (value < _minValue)
|
||||
_minValue = value;
|
||||
if (value > _maxValue)
|
||||
_maxValue = value;
|
||||
}
|
||||
}
|
||||
|
||||
- (float)averageValue {
|
||||
__block float result;
|
||||
dispatch_sync(_sq, ^{
|
||||
result = (self->_count > 0) ? (float)(self->_sum / (double)self->_count) : 0.0f;
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
- (float)total {
|
||||
__block float result;
|
||||
dispatch_sync(_sq, ^{
|
||||
result = (self->_count > 0) ? (float)(self->_sum) : 0.0f;
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
- (float)newestValue {
|
||||
__block float result;
|
||||
dispatch_sync(_sq, ^{
|
||||
result = (self->_count > 0) ? _buffer[_head] : 0.0f;
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
- (CFTimeInterval)oldestTimestamp {
|
||||
__block CFTimeInterval result;
|
||||
dispatch_sync(_sq, ^{
|
||||
NSUInteger tail = (_head + _capacity - _count) & (_capacity - 1);
|
||||
result = (self->_count > 0) ? _timestamps[tail] : 0.0f;
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
- (int)copyValuesIntoBuffer:(float *)outBuffer min:(nullable float *)outMin max:(nullable float *)outMax {
|
||||
__block int result;
|
||||
dispatch_sync(_sq, ^{
|
||||
result = [self _unsafeCopyValuesIntoBuffer:outBuffer min:outMin max:outMax];
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
- (int)_unsafeCopyValuesIntoBuffer:(float *)outBuffer min:(nullable float *)outMin max:(nullable float *)outMax {
|
||||
if (_count == 0) {
|
||||
// Empty buffer → zero, and set min/max to zero if requested
|
||||
if (outMin)
|
||||
*outMin = 0.0f;
|
||||
if (outMax)
|
||||
*outMax = 0.0f;
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Compute "tail" index (oldest element)
|
||||
NSUInteger tail = (_head + _capacity - _count) & (_capacity - 1);
|
||||
|
||||
// 1) Copy first chunk from buffer[tail] up to either end or _count elements
|
||||
NSUInteger firstChunkSize = MIN(_capacity - tail, _count);
|
||||
memcpy(outBuffer, &_buffer[tail], firstChunkSize * sizeof(float));
|
||||
|
||||
NSUInteger copiedSoFar = firstChunkSize;
|
||||
|
||||
// 2) If wrapped, copy remainder from index 0
|
||||
if (firstChunkSize < _count) {
|
||||
NSUInteger remainder = _count - firstChunkSize;
|
||||
memcpy(&outBuffer[copiedSoFar], _buffer, remainder * sizeof(float));
|
||||
}
|
||||
|
||||
if (outMin)
|
||||
*outMin = _minValue;
|
||||
if (outMax)
|
||||
*outMax = _maxValue;
|
||||
|
||||
return _count;
|
||||
}
|
||||
|
||||
- (void)copyMetrics:(PlotMetrics *)plotMetrics {
|
||||
dispatch_sync(_sq, ^{
|
||||
[self _unsafeCopyMetrics:plotMetrics];
|
||||
});
|
||||
}
|
||||
|
||||
- (void)_unsafeCopyMetrics:(PlotMetrics *)plotMetrics {
|
||||
plotMetrics->min = _minValue;
|
||||
plotMetrics->max = _maxValue;
|
||||
plotMetrics->avg = (self->_count > 0) ? (float)(self->_sum / (double)self->_count) : 0.0f;
|
||||
plotMetrics->total = (self->_count > 0) ? (float)(self->_sum) : 0.0f;
|
||||
plotMetrics->nsamples = self->_count;
|
||||
plotMetrics->samplerate = 0.0f;
|
||||
|
||||
if (plotMetrics->nsamples > 1) {
|
||||
NSUInteger tail = (_head + _capacity - _count) & (_capacity - 1);
|
||||
NSUInteger newest = (_head + _capacity - 1) & (_capacity - 1);
|
||||
CFTimeInterval elapsed = _timestamps[newest] - _timestamps[tail];
|
||||
if (elapsed > 0.0) {
|
||||
// nsamples-1 intervals over elapsed seconds
|
||||
plotMetrics->samplerate = (float)((plotMetrics->nsamples - 1) / elapsed);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
- (void)clear {
|
||||
dispatch_sync(_sq, ^{
|
||||
memset(_buffer, 0, sizeof(float) * _capacity);
|
||||
memset(_timestamps, 0, sizeof(CFTimeInterval) * _capacity);
|
||||
_head = 0;
|
||||
_count = 0;
|
||||
_minValue = 0.0f;
|
||||
_maxValue = 0.0f;
|
||||
_sum = 0.0;
|
||||
});
|
||||
}
|
||||
|
||||
- (void)enumerateValuesWithBlock:(void (^)(float value, BOOL *stop))block {
|
||||
if (!block) return;
|
||||
dispatch_sync(_sq, ^{
|
||||
if (_count == 0) return;
|
||||
int tail = (_head + _capacity - _count) & (_capacity - 1);
|
||||
BOOL stop = NO;
|
||||
for (int i = 0; i < _count && !stop; i++) {
|
||||
int idx = (tail + i) & (_capacity - 1);
|
||||
block(_buffer[idx], &stop);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Debug output for use with %@
|
||||
- (NSString *)description {
|
||||
__block NSString *desc;
|
||||
dispatch_sync(_sq, ^{
|
||||
int tail = (_head + _capacity - _count) & (_capacity - 1);
|
||||
|
||||
NSMutableString *values = [NSMutableString stringWithString:@"["];
|
||||
for (int i = 0; i < _count; i++) {
|
||||
float v = _buffer[(tail + i) & (_capacity - 1)];
|
||||
[values appendFormat:@"%0.3f", v];
|
||||
if (i < _count - 1) [values appendString:@", "];
|
||||
}
|
||||
[values appendString:@"]"];
|
||||
|
||||
float avg = (_count > 0) ? (float)(_sum / _count) : 0.0f;
|
||||
|
||||
desc = [NSString stringWithFormat:@"<%@: %p; capacity=%d; count=%d; min=%0.3f; max=%0.3f; avg=%0.3f; sum=%0.3f, values=%@>",
|
||||
NSStringFromClass([self class]),
|
||||
self,
|
||||
_capacity,
|
||||
_count,
|
||||
_minValue,
|
||||
_maxValue,
|
||||
avg,
|
||||
_sum,
|
||||
values];
|
||||
});
|
||||
return desc;
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,25 @@
|
||||
#import <Foundation/Foundation.h>
|
||||
#import <VideoToolbox/VideoToolbox.h>
|
||||
|
||||
@interface Frame : NSObject
|
||||
@property (nonatomic) int frameNumber;
|
||||
@property (nonatomic) int frameType;
|
||||
@property (nonatomic) CFTimeInterval decodedAt;
|
||||
@property (nonatomic) CMTime pts90;
|
||||
@property (nonatomic) CMTime duration90;
|
||||
@property (nonatomic, assign) CMVideoFormatDescriptionRef formatDesc;
|
||||
@property (nonatomic) CMSampleBufferRef sampleBuffer;
|
||||
@property (nonatomic) CVPixelBufferRef pixelBuffer;
|
||||
|
||||
- (instancetype)initWithPixelBufffer:(CVPixelBufferRef)pixelBuffer frameNumber:(int)frameNumber frameType:(int)frameType pts:(CMTime)pts;
|
||||
- (instancetype)initWithSampleBuffer:(CMSampleBufferRef)sampleBuffer frameNumber:(int)frameNumber frameType:(int)frameType;
|
||||
- (void)dealloc;
|
||||
- (CFTimeInterval)pts;
|
||||
- (CFTimeInterval)duration;
|
||||
- (void)setDurationFromNext:(Frame *)nextFrame;
|
||||
- (BOOL)durationIsValid;
|
||||
- (CFDictionaryRef)getFormatDescExtensions;
|
||||
- (size_t)width;
|
||||
- (size_t)height;
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,142 @@
|
||||
#import <CoreVideo/CVImageBuffer.h>
|
||||
#import "Frame.h"
|
||||
#import "Logger.h"
|
||||
#include <Limelight.h>
|
||||
|
||||
@implementation Frame {
|
||||
CFDictionaryRef _formatDescExt;
|
||||
}
|
||||
|
||||
- (instancetype)initWithPixelBufffer:(CVPixelBufferRef)pixelBuffer
|
||||
frameNumber:(int)frameNumber
|
||||
frameType:(int)frameType
|
||||
pts:(CMTime)pts {
|
||||
self = [super init];
|
||||
if (self) {
|
||||
_decodedAt = CACurrentMediaTime();
|
||||
_formatDescExt = nil;
|
||||
_frameNumber = frameNumber;
|
||||
_frameType = frameType;
|
||||
_pixelBuffer = pixelBuffer; // already retained
|
||||
_sampleBuffer = nil;
|
||||
|
||||
// 90 kHz pts from RTP
|
||||
_pts90 = pts;
|
||||
_duration90 = kCMTimeInvalid;
|
||||
|
||||
FQLog(LOG_I, @"init Frame %d - type %@ [host pts %.3f]",
|
||||
_frameNumber, _frameType == FRAME_TYPE_IDR ? @"IDR" : @"P",
|
||||
CMTimeGetSeconds(_pts90));
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (instancetype)initWithSampleBuffer:(CMSampleBufferRef)sampleBuffer frameNumber:(int)frameNumber frameType:(int)frameType {
|
||||
self = [super init];
|
||||
if (self) {
|
||||
_frameNumber = frameNumber;
|
||||
_frameType = frameType;
|
||||
_pixelBuffer = nil;
|
||||
_sampleBuffer = sampleBuffer;
|
||||
|
||||
// 90 kHz pts from RTP
|
||||
_pts90 = CMSampleBufferGetOutputPresentationTimeStamp(sampleBuffer);
|
||||
_duration90 = kCMTimeInvalid;
|
||||
|
||||
FQLog(LOG_I, @"init Frame %d - type %@ [host pts %.3f]",
|
||||
_frameNumber, _frameType == FRAME_TYPE_IDR ? @"IDR" : @"P",
|
||||
CMTimeGetSeconds(_pts90));
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (void)dealloc {
|
||||
FQLog(LOG_I, @"[%d / %f] Frame dealloc", _frameNumber, CMTimeGetSeconds(_pts90));
|
||||
|
||||
if (_formatDesc) {
|
||||
CFRelease(_formatDesc);
|
||||
_formatDesc = nil;
|
||||
}
|
||||
if (_formatDescExt) {
|
||||
CFRelease(_formatDescExt);
|
||||
_formatDescExt = nil;
|
||||
}
|
||||
if (_pixelBuffer) {
|
||||
CVPixelBufferRelease(_pixelBuffer);
|
||||
_pixelBuffer = nil;
|
||||
}
|
||||
|
||||
// sampleBuffer comes from CMSampleBufferCreateReadyWithImageBuffer
|
||||
// so we don't need to CFRetain in init, but do need to release it
|
||||
if (_sampleBuffer) {
|
||||
CFRelease(_sampleBuffer);
|
||||
_sampleBuffer = nil;
|
||||
}
|
||||
}
|
||||
|
||||
- (void)setFormatDesc:(CMVideoFormatDescriptionRef)formatDesc {
|
||||
if (_formatDesc) {
|
||||
CFRelease(_formatDesc);
|
||||
}
|
||||
_formatDesc = CFRetain(formatDesc);
|
||||
}
|
||||
|
||||
- (CFDictionaryRef)getFormatDescExtensions {
|
||||
if (!_formatDescExt && _formatDesc) {
|
||||
_formatDescExt = CFRetain(CMFormatDescriptionGetExtensions(_formatDesc));
|
||||
}
|
||||
return _formatDescExt;
|
||||
}
|
||||
|
||||
- (CFTimeInterval)pts {
|
||||
return CMTimeGetSeconds(_pts90);
|
||||
}
|
||||
|
||||
- (CFTimeInterval)duration {
|
||||
if (CMTIME_IS_VALID(_duration90)) {
|
||||
return CMTimeGetSeconds(_duration90);
|
||||
}
|
||||
return NAN;
|
||||
}
|
||||
|
||||
- (void)setDurationFromNext:(Frame *)nextFrame {
|
||||
if (nextFrame.frameNumber == _frameNumber + 1) {
|
||||
_duration90 = CMTimeSubtract(nextFrame.pts90, _pts90);
|
||||
_duration90.flags = kCMTimeFlags_Valid;
|
||||
|
||||
FQLog(LOG_I, @"frame [%d / %.3f] set duration %.3f ms",
|
||||
_frameNumber, CMTimeGetSeconds(_pts90),
|
||||
CMTimeGetSeconds(_duration90) * 1000.0);
|
||||
}
|
||||
}
|
||||
|
||||
- (BOOL)durationIsValid {
|
||||
return CMTIME_IS_VALID(_duration90);
|
||||
}
|
||||
|
||||
- (size_t)width {
|
||||
if (_pixelBuffer) {
|
||||
return CVPixelBufferGetWidth(_pixelBuffer);
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
- (size_t)height {
|
||||
if (_pixelBuffer) {
|
||||
return CVPixelBufferGetHeight(_pixelBuffer);
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Debug output when using %@
|
||||
- (NSString *)description {
|
||||
return [NSString stringWithFormat:@"{Frame: %d, type %@, pts90 %lld, pts %.3f ms, duration %@}",
|
||||
self.frameNumber,
|
||||
self.frameType == 1 ? @"IDR" : @"PFRAME",
|
||||
self.pts90.value,
|
||||
self.pts * 1000.0,
|
||||
[self durationIsValid] ? [ NSString stringWithFormat:@"%.3f ms", self.duration * 1000.0] : @"--"
|
||||
];
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,32 @@
|
||||
#import <Foundation/Foundation.h>
|
||||
#import <VideoToolbox/VideoToolbox.h>
|
||||
|
||||
#import "Frame.h"
|
||||
#import "FloatBuffer.h"
|
||||
|
||||
NS_ASSUME_NONNULL_BEGIN
|
||||
|
||||
@interface FrameQueue : NSObject
|
||||
|
||||
@property (nonatomic, readonly) NSUInteger count;
|
||||
@property (nonatomic) FloatBuffer *frameDropMetrics;
|
||||
@property (nonatomic) int highWaterMark;
|
||||
@property (nonatomic, readonly) int maxCapacity;
|
||||
|
||||
+ (instancetype)sharedInstance;
|
||||
|
||||
- (instancetype)init NS_UNAVAILABLE;
|
||||
+ (instancetype)new NS_UNAVAILABLE;
|
||||
- (BOOL)isEmpty;
|
||||
- (void)clear;
|
||||
- (int)enqueue:(Frame *)frame;
|
||||
- (int)enqueue:(Frame *)frame withSlackSize:(int)slack;
|
||||
- (nullable Frame *)dequeue;
|
||||
- (nullable Frame *)dequeueWithTimeout:(CFTimeInterval)timeout;
|
||||
- (CFTimeInterval)estimatedFramerate;
|
||||
- (int)currentSoftCap;
|
||||
- (void)waitForEnqueue;
|
||||
|
||||
@end
|
||||
|
||||
NS_ASSUME_NONNULL_END
|
||||
@@ -0,0 +1,331 @@
|
||||
@import AVFoundation;
|
||||
@import VideoToolbox;
|
||||
|
||||
#import <os/lock.h>
|
||||
#import <Limelight.h>
|
||||
#import "Logger.h"
|
||||
#import "FloatBuffer.h"
|
||||
#import "FrameQueue.h"
|
||||
#import "ImGuiPlots.h"
|
||||
|
||||
@implementation FrameQueue {
|
||||
NSMutableArray<id> *_buffer;
|
||||
int _capacity;
|
||||
int _head;
|
||||
int _tail;
|
||||
int _count;
|
||||
|
||||
BOOL _droppedLast;
|
||||
int _framesIn;
|
||||
CMTime _ptsCorrection;
|
||||
os_unfair_lock _lock;
|
||||
|
||||
FloatBuffer *_queueSizeHistory;
|
||||
int _currentSoftCap;
|
||||
dispatch_queue_t _sq;
|
||||
dispatch_semaphore_t _frameSemaphore;
|
||||
}
|
||||
|
||||
+ (instancetype)sharedInstance {
|
||||
static FrameQueue *sharedInstance = nil;
|
||||
static dispatch_once_t onceToken;
|
||||
dispatch_once(&onceToken, ^{
|
||||
sharedInstance = [[super allocWithZone:NULL] _initSingleton];
|
||||
});
|
||||
return sharedInstance;
|
||||
}
|
||||
|
||||
// Prevent others from using alloc/init directly
|
||||
+ (instancetype)allocWithZone:(struct _NSZone *)zone {
|
||||
return [self sharedInstance];
|
||||
}
|
||||
|
||||
// If someone tries to copy it, just return the same instance
|
||||
- (id)copyWithZone:(NSZone *)zone {
|
||||
return self;
|
||||
}
|
||||
|
||||
- (id)mutableCopyWithZone:(NSZone *)zone {
|
||||
return self;
|
||||
}
|
||||
|
||||
- (instancetype)_initSingleton {
|
||||
self = [super init];
|
||||
if (self) {
|
||||
_droppedLast = NO;
|
||||
_frameDropMetrics = [[FloatBuffer alloc] initWithCapacity:512];
|
||||
_framesIn = 0;
|
||||
_highWaterMark = 2;
|
||||
_maxCapacity = 15;
|
||||
_ptsCorrection = CMTimeMake(0, 90000);
|
||||
_queueSizeHistory = [[FloatBuffer alloc] initWithCapacity:64];
|
||||
_lock = OS_UNFAIR_LOCK_INIT;
|
||||
|
||||
// ring buffer
|
||||
_capacity = _maxCapacity;
|
||||
_buffer = [NSMutableArray arrayWithCapacity:_capacity];
|
||||
for (int i = 0; i < _capacity; i++) {
|
||||
[_buffer addObject:[NSNull null]];
|
||||
}
|
||||
_head = _tail = _count = 0;
|
||||
|
||||
_sq = dispatch_queue_create("com.moonlight.FrameQueue",
|
||||
dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_SERIAL, QOS_CLASS_USER_INTERACTIVE, 0));
|
||||
_frameSemaphore = dispatch_semaphore_create(0);
|
||||
|
||||
// ping estimatedFramerate to set initial last value
|
||||
[self estimatedFramerate];
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
// Push into buffer at _tail
|
||||
- (void)_pushFrame:(Frame *)frame {
|
||||
[_buffer replaceObjectAtIndex:_tail withObject:frame];
|
||||
_tail = (_tail + 1) % _capacity;
|
||||
_count++;
|
||||
|
||||
// I think it's ok to signal the cond from within the unfair lock, the render
|
||||
// loop will just end up waiting on it in a call to dequeue.
|
||||
dispatch_semaphore_signal(_frameSemaphore);
|
||||
|
||||
FQLog(LOG_I, @"[-> %@ %d / %f] enqueue frame, queue size %d / %d",
|
||||
frame.frameType == FRAME_TYPE_IDR ? @"IDR" : @"P",
|
||||
frame.frameNumber, frame.pts, _count, _highWaterMark);
|
||||
}
|
||||
|
||||
// Pop oldest frame from _head
|
||||
- (Frame *)_popFrame {
|
||||
id obj = _buffer[_head];
|
||||
Frame *frame = (obj == [NSNull null] ? nil : obj);
|
||||
[_buffer replaceObjectAtIndex:_head withObject:[NSNull null]];
|
||||
_head = (_head + 1) % _capacity;
|
||||
_count--;
|
||||
return frame;
|
||||
}
|
||||
|
||||
// Peek next frame (without removing)
|
||||
- (Frame *)_peekFrame {
|
||||
id frame = (_count > 0) ? _buffer[_head] : nil;
|
||||
return (frame == [NSNull null]) ? nil : frame;
|
||||
}
|
||||
|
||||
// enumerate in‐buffer frames
|
||||
- (void)_enumerateFrames:(void(^)(Frame *frame, NSUInteger idx, BOOL *stop))block {
|
||||
BOOL stop = NO;
|
||||
for (int i = 0; i < _count; i++) {
|
||||
int idx = (_head + i) % _capacity;
|
||||
block([_buffer objectAtIndex:idx], i, &stop);
|
||||
if (stop) break;
|
||||
}
|
||||
}
|
||||
|
||||
- (void)_noteDroppedFrame:(Frame *)frame {
|
||||
if ([frame durationIsValid]) {
|
||||
_ptsCorrection = CMTimeAdd(_ptsCorrection, frame.duration90);
|
||||
FQLog(LOG_W, @"dropped frame %d with duration %.3f ms", frame.frameNumber, frame.duration * 1000.0);
|
||||
} else {
|
||||
// count unknowns as 1 avg frametime
|
||||
CFTimeInterval fps = (_framesIn > 1000) ? [self _unsafeEstimatedFramerate] : 0.0f;
|
||||
CMTime oneFrame = (_framesIn > 1000) ? CMTimeMake((int)(90000.0f / fps), 90000) : kCMTimeZero;
|
||||
_ptsCorrection = CMTimeAdd(_ptsCorrection, oneFrame);
|
||||
FQLog(LOG_W, @"dropped frame %d with unknown duration, using %.3f ms (%.1f fps) instead",
|
||||
frame.frameNumber, CMTimeGetSeconds(oneFrame) * 1000.0, fps);
|
||||
}
|
||||
}
|
||||
|
||||
- (int)_unsafeEnqueue:(Frame *)frame withDropTarget:(int)frameDropTarget {
|
||||
int dropCount = 0;
|
||||
// Always accept IDR frames, allow exceeding HWM
|
||||
if (frame.frameType == FRAME_TYPE_IDR || _count < frameDropTarget) {
|
||||
[self _pushFrame:frame];
|
||||
_droppedLast = NO;
|
||||
} else {
|
||||
if (!_droppedLast) {
|
||||
// alternate between: drop newest...
|
||||
[self _noteDroppedFrame:frame];
|
||||
dropCount = 1;
|
||||
_droppedLast = YES;
|
||||
} else {
|
||||
// and: drop oldest & enqueue new
|
||||
if ([self _peekFrame].frameType != FRAME_TYPE_IDR) {
|
||||
Frame *oldest = [self _popFrame];
|
||||
[oldest setDurationFromNext:[self _peekFrame]];
|
||||
[self _noteDroppedFrame:oldest];
|
||||
dropCount = 1;
|
||||
}
|
||||
[self _pushFrame:frame];
|
||||
_droppedLast = NO;
|
||||
}
|
||||
}
|
||||
// regardless of drop status, every enqueue is a frame
|
||||
// for estimatedFramerate purposes
|
||||
_framesIn++;
|
||||
[_frameDropMetrics addValue:(float)dropCount];
|
||||
|
||||
// Stats displays the soft cap
|
||||
_currentSoftCap = frameDropTarget;
|
||||
return dropCount;
|
||||
}
|
||||
|
||||
// enqueue with simple alternate-drop logic
|
||||
- (int)enqueue:(Frame *)frame {
|
||||
os_unfair_lock_lock(&_lock);
|
||||
int dropCount = [self _unsafeEnqueue:frame withDropTarget:_highWaterMark];
|
||||
os_unfair_lock_unlock(&_lock);
|
||||
return dropCount;
|
||||
}
|
||||
|
||||
// enqueue that is a bit more flexixble, using the same 500ms queue size history method as moonlight-qt.
|
||||
- (int)enqueue:(Frame *)frame withSlackSize:(int)slack {
|
||||
os_unfair_lock_lock(&_lock);
|
||||
CFTimeInterval now = CACurrentMediaTime();
|
||||
|
||||
// new data point for queue health
|
||||
[_queueSizeHistory addValue:(float)_count];
|
||||
|
||||
// The "target" initially starts as the size of the buffer chosen by the user. 1-5 default 2. We drop a frame
|
||||
// when the queue size exceeds this amount.
|
||||
int frameDropTarget = _highWaterMark;
|
||||
|
||||
// CFTimeInterval t0 = [_queueSizeHistory oldestTimestamp];
|
||||
// if (now - t0 > 0.5f) {
|
||||
// // Get the current drop percentage over the past 512 frames
|
||||
// // TODO: design a better API e.g. [ImGuiPlots PLOT_DROPPED].
|
||||
// float dropRate = [[ImGuiPlots sharedInstance].plots[PLOT_DROPPED].buffer averageValue];
|
||||
// if (dropRate > 0.25f) {
|
||||
// FQLog(LOG_I, @"queue is dropping at %.2f%%, forcing a drop", dropRate * 100.0);
|
||||
// frameDropTarget--;
|
||||
// } else if (_queueSizeHistory.minValue > 0 && _queueSizeHistory.minValue == _queueSizeHistory.maxValue) {
|
||||
// // If the queue has been in the same state for the entire period, it's possible to get stuck there for a while.
|
||||
// // Lower the FDT by 1 to force a frame to be dropped and hopefully unstick things.
|
||||
// FQLog(LOG_I, @"queue stuck at %.0f, forcing a drop", _queueSizeHistory.minValue);
|
||||
// frameDropTarget--;
|
||||
// } else if (_queueSizeHistory.minValue < 1.0f) {
|
||||
// // If the queue has cleared out at least once in the past history period, be more lenient about dropping frames.
|
||||
// frameDropTarget += slack;
|
||||
// // FQLog(LOG_I, @"allowing frameDropTarget of %d (history %.0f/%.0f/%.2f)", frameDropTarget,
|
||||
// // _queueSizeHistory.minValue, _queueSizeHistory.maxValue, _queueSizeHistory.averageValue);
|
||||
// }
|
||||
//
|
||||
// // TODO: New logic
|
||||
// // Use timestamps, check initial interval is long enough
|
||||
// // Smarter when checking min==max, a steady state can be fine if no frames are being dropped
|
||||
// // If min==max && frame drops are >= 25%, force a drop
|
||||
// // Maybe only look at >= 25%.
|
||||
// // Test condition variables, manual present, no displaylink
|
||||
// }
|
||||
|
||||
// original enqueue logic still applies if we're full
|
||||
int dropCount = [self _unsafeEnqueue:frame withDropTarget:frameDropTarget];
|
||||
|
||||
os_unfair_lock_unlock(&_lock);
|
||||
return dropCount;
|
||||
}
|
||||
|
||||
// Allows the render loop to wait if the queue is empty
|
||||
- (void)waitForEnqueue {
|
||||
while ([self isEmpty]) {
|
||||
FQLog(LOG_I, @"waitForEnqueue...");
|
||||
long result = dispatch_semaphore_wait(_frameSemaphore, DISPATCH_TIME_FOREVER);
|
||||
}
|
||||
}
|
||||
|
||||
- (Frame *)dequeue {
|
||||
os_unfair_lock_lock(&_lock);
|
||||
Frame *frame = nil;
|
||||
if (_count > 0) {
|
||||
frame = [self _popFrame];
|
||||
// compute duration from next
|
||||
Frame *next = [self _peekFrame];
|
||||
if (next) {
|
||||
[frame setDurationFromNext:next];
|
||||
}
|
||||
FQLog(LOG_I, @"[<- %d / %f%@] dequeue frame, queue size %d",
|
||||
frame.frameNumber, frame.pts,
|
||||
[frame durationIsValid] ? [NSString stringWithFormat:@" dur %.3f ms", frame.duration * 1000.0] : @"",
|
||||
_count);
|
||||
}
|
||||
os_unfair_lock_unlock(&_lock);
|
||||
return frame;
|
||||
}
|
||||
|
||||
- (Frame *)dequeueWithTimeout:(CFTimeInterval)timeout {
|
||||
CFTimeInterval start = CACurrentMediaTime();
|
||||
CFTimeInterval deadline = start + timeout;
|
||||
int round = 0;
|
||||
|
||||
// Always attempt to dequeue at least once
|
||||
do {
|
||||
if (round > 0) {
|
||||
usleep(100); // 0.1ms
|
||||
}
|
||||
Frame *frame = [self dequeue];
|
||||
if (frame) {
|
||||
return frame;
|
||||
}
|
||||
round++;
|
||||
} while (CACurrentMediaTime() < deadline);
|
||||
|
||||
FQLog(LOG_I, @"dequeueWithTimeout timed out after %.3f ms", (CACurrentMediaTime() - start) * 1000.0);
|
||||
return nil;
|
||||
}
|
||||
|
||||
- (NSUInteger)count {
|
||||
os_unfair_lock_lock(&_lock);
|
||||
NSUInteger c = _count;
|
||||
os_unfair_lock_unlock(&_lock);
|
||||
return c;
|
||||
}
|
||||
|
||||
- (BOOL)isEmpty {
|
||||
return [self count] == 0;
|
||||
}
|
||||
|
||||
- (void)clear {
|
||||
os_unfair_lock_lock(&_lock);
|
||||
_head = _tail = _count = 0;
|
||||
_frameDropMetrics = [[FloatBuffer alloc] initWithCapacity:512];
|
||||
os_unfair_lock_unlock(&_lock);
|
||||
}
|
||||
|
||||
- (CFTimeInterval)_unsafeEstimatedFramerate {
|
||||
static CFTimeInterval lastTime = 0;
|
||||
static int lastFrames = 0;
|
||||
static CFTimeInterval estimate = 0;
|
||||
|
||||
CFTimeInterval now = CACurrentMediaTime();
|
||||
if (now - lastTime > 1.0) {
|
||||
estimate = (_framesIn - lastFrames) / (now - lastTime);
|
||||
// FQLog(LOG_I, @"fps calc using framesIn %d - lastFrames %d / now %f - lastTime %f = %.1f fps",
|
||||
// _framesIn, lastFrames, now, lastTime, estimate);
|
||||
lastTime = now;
|
||||
lastFrames = _framesIn;
|
||||
}
|
||||
return estimate;
|
||||
}
|
||||
|
||||
- (CFTimeInterval)estimatedFramerate {
|
||||
os_unfair_lock_lock(&_lock);
|
||||
CFTimeInterval fps = [self _unsafeEstimatedFramerate];
|
||||
os_unfair_lock_unlock(&_lock);
|
||||
return fps;
|
||||
}
|
||||
|
||||
- (int)currentSoftCap {
|
||||
os_unfair_lock_lock(&_lock);
|
||||
int cap = _currentSoftCap;
|
||||
os_unfair_lock_unlock(&_lock);
|
||||
return cap;
|
||||
}
|
||||
|
||||
// For use with NSLog("%@", franeQueue);
|
||||
- (NSString *)description {
|
||||
__block NSMutableArray *parts = [NSMutableArray arrayWithCapacity:_count];
|
||||
[self _enumerateFrames:^(Frame *frame, NSUInteger idx, BOOL *stop) {
|
||||
[parts addObject:[frame description]];
|
||||
}];
|
||||
return [NSString stringWithFormat:@"[%@]", [parts componentsJoinedByString:@",\n"]];
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -9,8 +9,13 @@
|
||||
#ifndef Limelight_Logger_h
|
||||
#define Limelight_Logger_h
|
||||
|
||||
#import <dispatch/dispatch.h>
|
||||
#import <stdarg.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
typedef enum {
|
||||
LOG_D,
|
||||
LOG_I,
|
||||
@@ -26,4 +31,54 @@ typedef enum {
|
||||
void Log(LogLevel level, NSString* fmt, ...);
|
||||
void LogTag(LogLevel level, NSString* tag, NSString* fmt, ...);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
||||
// LogOnce() is a one-time log message for use in hot areas of the code
|
||||
#define CONCAT(a,b) CONCAT2(a,b)
|
||||
#define CONCAT2(a,b) a##b
|
||||
|
||||
#define LogOnce(level, fmt, ...) \
|
||||
do { \
|
||||
static dispatch_once_t CONCAT(_onceToken_, __LINE__); \
|
||||
dispatch_once(&CONCAT(_onceToken_, __LINE__), ^{ \
|
||||
Log(level, fmt, ##__VA_ARGS__); \
|
||||
}); \
|
||||
} while (0)
|
||||
|
||||
#endif
|
||||
|
||||
// Disable all logging in release mode for performance
|
||||
#ifdef DEBUG
|
||||
#define Log(level, fmt, ...) \
|
||||
LogTag(level, NULL, fmt, ##__VA_ARGS__)
|
||||
#else
|
||||
#define Log(level, fmt, ...) do {} while(0)
|
||||
#endif
|
||||
|
||||
// FrameQueue log helpers FQLog() is a no-op unless FRAME_QUEUE_VERBOSE is defined
|
||||
|
||||
static inline NSString *FQQoSString(qos_class_t qos) {
|
||||
switch (qos) {
|
||||
case QOS_CLASS_USER_INTERACTIVE: return @"UI-25";
|
||||
case QOS_CLASS_USER_INITIATED: return @"IN-19";
|
||||
case QOS_CLASS_DEFAULT: return @"DF-15";
|
||||
case QOS_CLASS_UTILITY: return @"UT-11";
|
||||
case QOS_CLASS_BACKGROUND: return @"BG-09";
|
||||
default: return [NSString stringWithFormat:@"??-%d", qos];
|
||||
}
|
||||
}
|
||||
|
||||
static inline NSString *FQLogPrefix(void) {
|
||||
CFTimeInterval now = CACurrentMediaTime();
|
||||
NSString *qos = FQQoSString(qos_class_self());
|
||||
return [NSString stringWithFormat:@"[%.3f] [%@]", now, qos];
|
||||
}
|
||||
|
||||
#if defined(FRAME_QUEUE_VERBOSE)
|
||||
#define FQLog(level, fmt, ...) \
|
||||
Log(level, @"%@ " fmt, FQLogPrefix(), ##__VA_ARGS__)
|
||||
#else
|
||||
#define FQLog(level, fmt, ...) do {} while(0)
|
||||
#endif
|
||||
|
||||
+11
-16
@@ -8,23 +8,11 @@
|
||||
|
||||
#import "Logger.h"
|
||||
|
||||
#if defined(NDEBUG)
|
||||
static LogLevel LoggerLogLevel = LOG_I;
|
||||
|
||||
void LogTagv(LogLevel level, NSString* tag, NSString* fmt, va_list args);
|
||||
|
||||
void Log(LogLevel level, NSString* fmt, ...) {
|
||||
va_list args;
|
||||
va_start(args, fmt);
|
||||
LogTagv(level, NULL, fmt, args);
|
||||
va_end(args);
|
||||
}
|
||||
|
||||
void LogTag(LogLevel level, NSString* tag, NSString* fmt, ...) {
|
||||
va_list args;
|
||||
va_start(args, fmt);
|
||||
LogTagv(level, tag, fmt, args);
|
||||
va_end(args);
|
||||
}
|
||||
#else
|
||||
static LogLevel LoggerLogLevel = LOG_D;
|
||||
#endif
|
||||
|
||||
void LogTagv(LogLevel level, NSString* tag, NSString* fmt, va_list args) {
|
||||
NSString* levelPrefix = @"";
|
||||
@@ -59,3 +47,10 @@ void LogTagv(LogLevel level, NSString* tag, NSString* fmt, va_list args) {
|
||||
}
|
||||
NSLogv(prefixedString, args);
|
||||
}
|
||||
|
||||
void LogTag(LogLevel level, NSString* tag, NSString* fmt, ...) {
|
||||
va_list args;
|
||||
va_start(args, fmt);
|
||||
LogTagv(level, tag, fmt, args);
|
||||
va_end(args);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
//
|
||||
// PaddedLabel.h
|
||||
// Moonlight
|
||||
//
|
||||
// Created by Travis Thieman on 5/6/25.
|
||||
// Copyright © 2025 Moonlight Game Streaming Project. All rights reserved.
|
||||
//
|
||||
|
||||
#import <UIKit/UIKit.h>
|
||||
|
||||
@interface PaddedLabel : UILabel
|
||||
|
||||
@property (nonatomic, assign) UIEdgeInsets textInsets;
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,37 @@
|
||||
//
|
||||
// PaddedLabel.m
|
||||
// Moonlight
|
||||
//
|
||||
// Created by Travis Thieman on 5/6/25.
|
||||
// Copyright © 2025 Moonlight Game Streaming Project. All rights reserved.
|
||||
//
|
||||
|
||||
#import "PaddedLabel.h"
|
||||
|
||||
@implementation PaddedLabel
|
||||
|
||||
- (instancetype)initWithFrame:(CGRect)frame {
|
||||
if (self = [super initWithFrame:frame]) {
|
||||
self.textInsets = UIEdgeInsetsMake(8, 8, 8, 8); // default insets
|
||||
}
|
||||
return self;
|
||||
}
|
||||
|
||||
- (void)drawTextInRect:(CGRect)rect {
|
||||
UIEdgeInsets insets = self.textInsets;
|
||||
[super drawTextInRect:UIEdgeInsetsInsetRect(rect, insets)];
|
||||
}
|
||||
|
||||
- (CGSize)intrinsicContentSize {
|
||||
CGSize size = [super intrinsicContentSize];
|
||||
return CGSizeMake(size.width + self.textInsets.left + self.textInsets.right,
|
||||
size.height + self.textInsets.top + self.textInsets.bottom);
|
||||
}
|
||||
|
||||
- (CGSize)sizeThatFits:(CGSize)size {
|
||||
CGSize fittingSize = [super sizeThatFits:size];
|
||||
return CGSizeMake(fittingSize.width + self.textInsets.left + self.textInsets.right,
|
||||
fittingSize.height + self.textInsets.top + self.textInsets.bottom);
|
||||
}
|
||||
|
||||
@end
|
||||
@@ -0,0 +1,56 @@
|
||||
typedef enum {
|
||||
PACING_MODE_VSYNC,
|
||||
PACING_MODE_PTS
|
||||
} FramePacingMode;
|
||||
|
||||
typedef enum {
|
||||
PLOT_FRAMETIME = 0,
|
||||
PLOT_HOST_FRAMETIME,
|
||||
PLOT_QUEUED_FRAMES,
|
||||
PLOT_DROPPED,
|
||||
|
||||
PLOT_DECODE,
|
||||
PlotCount
|
||||
} PlotType;
|
||||
|
||||
typedef enum {
|
||||
PLOT_LABEL_MIN_MAX_AVG = 0,
|
||||
PLOT_LABEL_MIN_MAX_AVG_INT,
|
||||
PLOT_LABEL_TOTAL_INT
|
||||
} PlotLabelType;
|
||||
|
||||
typedef enum {
|
||||
PLOT_HIDDEN,
|
||||
PLOT_LEFT,
|
||||
PLOT_RIGHT
|
||||
} PlotSide;
|
||||
|
||||
typedef struct {
|
||||
float min;
|
||||
float max;
|
||||
float avg;
|
||||
float total;
|
||||
int nsamples;
|
||||
float samplerate;
|
||||
} PlotMetrics;
|
||||
|
||||
typedef enum {
|
||||
RENDER_METAL = 0,
|
||||
RENDER_AVSB
|
||||
} RenderingBackend;
|
||||
|
||||
typedef struct {
|
||||
CFTimeInterval startTime;
|
||||
CFTimeInterval endTime;
|
||||
int totalFrames;
|
||||
int receivedFrames;
|
||||
int networkDroppedFrames;
|
||||
int totalHostProcessingLatency;
|
||||
int framesWithHostProcessingLatency;
|
||||
int maxHostProcessingLatency;
|
||||
int minHostProcessingLatency;
|
||||
PlotMetrics decodeMetrics;
|
||||
PlotMetrics frameQueueMetrics;
|
||||
PlotMetrics frameDropMetrics;
|
||||
RenderingBackend renderingBackend;
|
||||
} video_stats_t;
|
||||
@@ -810,7 +810,6 @@ static NSMutableSet* hostList;
|
||||
_streamConfig.bitRate = [streamSettings.bitrate intValue];
|
||||
_streamConfig.optimizeGameSettings = streamSettings.optimizeGames;
|
||||
_streamConfig.playAudioOnPC = streamSettings.playAudioOnPC;
|
||||
_streamConfig.useFramePacing = streamSettings.useFramePacing;
|
||||
_streamConfig.swapABXYButtons = streamSettings.swapABXYButtons;
|
||||
_streamConfig.asyncNativeTouchPriority = streamSettings.asyncNativeTouchPriority; // new streamConfig segment
|
||||
_streamConfig.gyroMode = [streamSettings.gyroMode intValue];
|
||||
|
||||
@@ -10,6 +10,9 @@
|
||||
//
|
||||
|
||||
#import "Connection.h"
|
||||
#import "FloatBuffer.h"
|
||||
#import "ImGuiRenderer.h"
|
||||
#import "MetalViewController.h"
|
||||
#import "StreamConfiguration.h"
|
||||
#import "StreamView.h"
|
||||
#import "LayoutOnScreenControlsViewController.h"
|
||||
@@ -28,6 +31,8 @@
|
||||
@property (nonatomic) StreamConfiguration* streamConfig;
|
||||
@property (nonatomic, assign) MainFrameViewController *mainFrameViewcontroller;
|
||||
|
||||
@property (nonatomic, strong) MetalViewController *metalViewController;
|
||||
@property (nonatomic, strong) ImGuiRenderer *imguiView;
|
||||
|
||||
-(void)updatePreferredDisplayMode:(BOOL)streamActive;
|
||||
-(void)reConfigStreamViewRealtime;
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
#pragma mark DisplayLink - RTP PTS timestamp-based frame pacing (experimental)
|
||||
|
||||
// TODO: handle presentationTimeUs rollover every 13 hours
|
||||
|
||||
// This frame pacing method attempts to use timestamps from Sunshine to determine when
|
||||
// frames should be displayed. Sunshine v2025.600+ required.
|
||||
- (void)framePacingUsingTimestamps:(CADisplayLink *)link {
|
||||
// All times are in seconds
|
||||
static CFTimeInterval anchorLocal = 0.0f;
|
||||
static CMTime anchorHost;
|
||||
static CFTimeInterval lastTargetLocal = 0.0f;
|
||||
|
||||
CFTimeInterval start = link.timestamp;
|
||||
CFTimeInterval deadline = link.targetTimestamp;
|
||||
_displayRefreshRate = 1.0f / (deadline - start);
|
||||
|
||||
[self checkDisplayLayer];
|
||||
|
||||
Frame *frame = nil;
|
||||
|
||||
CFTimeInterval dl0 = CACurrentMediaTime();
|
||||
static CFTimeInterval avgOverhead = 0.004f; // averaged each callback
|
||||
CFTimeInterval waitFor = deadline - dl0 - avgOverhead;
|
||||
if (waitFor < 0.001f) {
|
||||
waitFor = 0.001f;
|
||||
}
|
||||
|
||||
// Process the next frame for display
|
||||
frame = [_frameQueue dequeueWithTimeout:waitFor];
|
||||
if (frame) {
|
||||
CFTimeInterval now = CACurrentMediaTime();
|
||||
|
||||
// during stream startup, wait and let the first frames through untouched
|
||||
if (!_seenFrameOne) {
|
||||
Log(LOG_I, @"Frame pacing (pts mode): target %f Hz with %d FPS stream", _displayRefreshRate, self->frameRate);
|
||||
_seenFrameOne = YES;
|
||||
|
||||
// Just let frame 1 through, we don't want to anchor to it
|
||||
Log(LOG_I, @"[%.3f] rendering frame %d / %.3f, queue size %d",
|
||||
deadline, frame.frameNumber, frame.pts, [_frameQueue count]);
|
||||
[self renderFrame:frame atTime:frame.pts90];
|
||||
return;
|
||||
}
|
||||
|
||||
if (!CMTIME_IS_VALID(anchorHost)) {
|
||||
// Use this frame as the anchor point. This is normally frame 2.
|
||||
anchorHost = frame.pts90;
|
||||
anchorLocal = now;
|
||||
Log(LOG_I, @"Setting anchor point: anchorHost=%.3f == anchorLocal=%.3f",
|
||||
CMTimeGetSeconds(anchorHost), anchorLocal);
|
||||
}
|
||||
|
||||
//CMTime targetTime = CMTimeSubtract(frame.pts90, frameQueue.ptsCorrection); // total of all dropped frames
|
||||
CMTime targetTime = frame.pts90;
|
||||
CFTimeInterval targetLocal = CMTimeGetSeconds(targetTime);
|
||||
|
||||
#ifdef DISPLAYLINK_VERBOSE
|
||||
// iOS double-buffering renders the frame we queue here during the interval +2 from now
|
||||
CMTime hostDelta = CMTimeSubtract(frame.pts90, anchorHost);
|
||||
CFTimeInterval hostDeltaF = CMTimeGetSeconds(hostDelta);
|
||||
CMTime drift = CMTimeSubtract(hostDelta, CMTimeMakeWithSeconds(now - anchorLocal, NSEC_PER_SEC));
|
||||
Log(LOG_I, @"[%.3f] rendering frame %d / %.3f @ %.3f, drift %.3f, queue size %d",
|
||||
deadline - anchorLocal, frame.frameNumber, hostDeltaF, targetLocal,
|
||||
CMTimeGetSeconds(drift), [_frameQueue count]);
|
||||
#endif
|
||||
|
||||
// Schedule the frame at its original timestamp, adjusted by a correction factor (total duration of all dropped frames)
|
||||
[self renderFrame:frame atTime:targetTime];
|
||||
|
||||
// Update metrics
|
||||
if (lastTargetLocal != 0) {
|
||||
[self->_callbacks observeFloat:PLOT_FRAMETIME value:(targetLocal - lastTargetLocal) * 1000.0];
|
||||
}
|
||||
lastTargetLocal = targetLocal;
|
||||
|
||||
// weighted moving average of how much time displayLink needs after dequeuing a frame.
|
||||
// This is used to avoid overshooting a vsync by waiting too long.
|
||||
const double alpha = 0.1f;
|
||||
avgOverhead = ((CACurrentMediaTime() - dl0) * alpha) + (avgOverhead * (1.0 - alpha));
|
||||
}
|
||||
|
||||
// are we returning too late?
|
||||
CFTimeInterval dl2 = CACurrentMediaTime();
|
||||
if (dl2 > deadline) {
|
||||
Log(LOG_W, @"*** returned past deadline by %.3f ms (%f > %f)", (dl2 - deadline) * 1000.0, dl2, deadline);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -374,7 +374,7 @@
|
||||
FB290E2619B37A4E004C83CF /* Project object */ = {
|
||||
isa = PBXProject;
|
||||
attributes = {
|
||||
BuildIndependentTargetsInParallel = YES;
|
||||
BuildIndependentTargetsInParallel = NO;
|
||||
LastUpgradeCheck = 1510;
|
||||
ORGANIZATIONNAME = "Moonlight Stream";
|
||||
TargetAttributes = {
|
||||
|
||||
Executable
+268
@@ -0,0 +1,268 @@
|
||||
#!/usr/bin/env perl
|
||||
#
|
||||
# Don't try to follow the logic in here, XML::Twig allows for some real nonsense
|
||||
|
||||
use strict;
|
||||
use warnings;
|
||||
|
||||
use Getopt::Long qw(GetOptions);
|
||||
use Pod::Usage qw(pod2usage);
|
||||
use XML::Twig;
|
||||
|
||||
my $inSettings = 0;
|
||||
my @list = ();
|
||||
my @clipboard = ();
|
||||
my $help = 0;
|
||||
my $after;
|
||||
my $move;
|
||||
my $to;
|
||||
|
||||
GetOptions(
|
||||
'help|?' => \$help,
|
||||
'add-after=s' => \$after,
|
||||
'move=s' => \$move,
|
||||
'to=s' => \$to,
|
||||
) or pod2usage(2);
|
||||
pod2usage(1) if $help || ($move && !$to) || (!$move && $to);
|
||||
|
||||
my $xml = shift || pod2usage(2);
|
||||
|
||||
my $twig = XML::Twig->new(
|
||||
twig_handlers => {
|
||||
'#COMMENT' => sub {
|
||||
my ($t, $comment) = @_;
|
||||
if ($comment->{comment} eq 'Settings View Controller') {
|
||||
$inSettings = 1;
|
||||
}
|
||||
else {
|
||||
$inSettings = 0;
|
||||
}
|
||||
},
|
||||
label => sub {
|
||||
my ($t, $label) = @_;
|
||||
if ($inSettings) {
|
||||
push @list, [ label => $label ];
|
||||
}
|
||||
},
|
||||
segmentedControl => sub {
|
||||
my ($t, $control) = @_;
|
||||
if ($inSettings) {
|
||||
push @list, [ control => $control ];
|
||||
}
|
||||
},
|
||||
slider => sub {
|
||||
my ($t, $slider) = @_;
|
||||
if ($inSettings) {
|
||||
push @list, [ slider => $slider ];
|
||||
}
|
||||
},
|
||||
view => sub {
|
||||
my ($t, $view) = @_;
|
||||
if ($inSettings) {
|
||||
push @list, [ view => $view ];
|
||||
}
|
||||
},
|
||||
},
|
||||
comments => 'process',
|
||||
keep_atts_order => 1,
|
||||
pretty_print => 'indented',
|
||||
empty_tags => 'normal',
|
||||
);
|
||||
|
||||
$twig->parsefile($xml);
|
||||
|
||||
if ($after) {
|
||||
do_after($twig, $after);
|
||||
}
|
||||
elsif ($move && $to) {
|
||||
do_move($twig, $move, $to);
|
||||
}
|
||||
|
||||
do_output($twig);
|
||||
|
||||
sub do_after {
|
||||
my ($twig, $after) = @_;
|
||||
|
||||
my $LABEL_TEMPLATE = qq{
|
||||
<label opaque="NO" userInteractionEnabled="NO" contentMode="left" horizontalHuggingPriority="251" verticalHuggingPriority="251" fixedFrame="YES" text="New Label Template" textAlignment="natural" lineBreakMode="tailTruncation" baselineAdjustment="alignBaselines" adjustsFontSizeToFit="NO" translatesAutoresizingMaskIntoConstraints="NO" id="%s">
|
||||
<rect key="frame" x="16" y="999" width="35" height="21"></rect>
|
||||
<autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMaxY="YES"></autoresizingMask>
|
||||
<fontDescription key="fontDescription" type="system" pointSize="17"></fontDescription>
|
||||
<color key="textColor" red="0.93902439019999995" green="0.9625305918" blue="1" alpha="1" colorSpace="custom" customColorSpace="sRGB"></color>
|
||||
<nil key="highlightedColor"></nil>
|
||||
</label>};
|
||||
|
||||
my $CONTROL_TEMPLATE = qq{
|
||||
<segmentedControl opaque="NO" contentMode="scaleToFill" fixedFrame="YES" contentHorizontalAlignment="left" contentVerticalAlignment="top" segmentControlStyle="plain" selectedSegmentIndex="0" translatesAutoresizingMaskIntoConstraints="NO" id="%s" userLabel="New Control Template">
|
||||
<rect key="frame" x="16" y="999" width="459" height="28"></rect>
|
||||
<autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMaxY="YES"></autoresizingMask>
|
||||
<segments>
|
||||
<segment title="No"></segment>
|
||||
<segment title="Yes"></segment>
|
||||
</segments>
|
||||
<color key="tintColor" red="0.6716768742" green="0.61711704730000005" blue="0.99902987480000005" alpha="1" colorSpace="custom" customColorSpace="sRGB"></color>
|
||||
<color key="selectedSegmentTintColor" red="0.6716768742" green="0.61711704730000005" blue="0.99902987480000005" alpha="1" colorSpace="custom" customColorSpace="sRGB"></color>
|
||||
</segmentedControl>};
|
||||
|
||||
my @to_add = (
|
||||
[ label => XML::Twig::Elt->parse( sprintf($LABEL_TEMPLATE, generate_id()) ) ],
|
||||
[ control => XML::Twig::Elt->parse( sprintf($CONTROL_TEMPLATE, generate_id()) ) ],
|
||||
);
|
||||
|
||||
# Find where to insert them
|
||||
for (my $i = 0; $i < scalar @list; $i++) {
|
||||
my ($type, $e) = @{$list[$i]};
|
||||
if ($type eq 'label' && ($e->att('text') // q{}) eq $after) {
|
||||
($type, $e) = @{$list[$i + 1]};
|
||||
|
||||
# paste the block to a new location
|
||||
for my $tm (@to_add) {
|
||||
$tm->[1]->paste(after => $e);
|
||||
$e = $tm->[1];
|
||||
}
|
||||
|
||||
# also move the block in the ordering list, which is where the y values are adjusted
|
||||
$i += 2;
|
||||
splice(@list, $i, 0, @to_add);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sub do_move {
|
||||
my ($twig, $move, $to) = @_;
|
||||
|
||||
my @to_move = ();
|
||||
|
||||
# Pull out the bits to move
|
||||
for (my $i = 0; $i < scalar @list; $i++) {
|
||||
my ($type, $e) = @{$list[$i]};
|
||||
if ($type eq 'label' && ($e->att('text') // q{}) eq $move) {
|
||||
# move this item and the next one (the control)
|
||||
push @to_move, [ $type => $e->cut ];
|
||||
($type, $e) = @{$list[$i + 1]};
|
||||
push @to_move, [ $type => $e->cut ];
|
||||
splice(@list, $i, 2);
|
||||
last;
|
||||
}
|
||||
}
|
||||
|
||||
# Find where to insert them
|
||||
for (my $i = 0; $i < scalar @list; $i++) {
|
||||
my ($type, $e) = @{$list[$i]};
|
||||
if ($type eq 'label' && ($e->att('text') // q{}) eq $to) {
|
||||
($type, $e) = @{$list[$i + 1]};
|
||||
|
||||
# paste the block to a new location
|
||||
for my $tm (@to_move) {
|
||||
$tm->[1]->paste(after => $e);
|
||||
$e = $tm->[1];
|
||||
}
|
||||
|
||||
# also move the block in the ordering list, which is where the y values are adjusted
|
||||
$i += 2;
|
||||
splice(@list, $i, 0, @to_move);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sub do_output {
|
||||
my $twig = shift;
|
||||
|
||||
# first label
|
||||
my $x = 16;
|
||||
my $y = 20;
|
||||
|
||||
# spacing
|
||||
my $label_to_control = 29;
|
||||
my $control_to_label = 35;
|
||||
my $label_to_label = 43;
|
||||
my $slider_to_next = 29;
|
||||
my $view_to_next = 43;
|
||||
my $seen_rdv = 0;
|
||||
my $last = q{};
|
||||
|
||||
printf STDERR "%30s %s %s\n\n", q{}, "auto", "current";
|
||||
|
||||
for my $row (@list) {
|
||||
my ($type, $e) = @{$row};
|
||||
my $rect = $e->has_child('rect');
|
||||
my $orig_y = $rect->att('y');
|
||||
|
||||
$rect->set_att(x => $x);
|
||||
$rect->set_att(y => $y);
|
||||
|
||||
if ($type eq "label") {
|
||||
my $text = $e->att('userLabel') // $e->att('text') // "(unknown label)";
|
||||
printf STDERR "%30s %d %4d\n", $text, $y, $orig_y;
|
||||
|
||||
if ($text =~ /Caption/) { # space after captions is a bit larger
|
||||
$y += $label_to_label;
|
||||
}
|
||||
else {
|
||||
$y += $label_to_control;
|
||||
}
|
||||
}
|
||||
elsif ($type eq "control") {
|
||||
my ($userLabel) = $e->att('userLabel') // "(unknown control)";
|
||||
printf STDERR "%30s %d %4d\n", $userLabel, $y, $orig_y;
|
||||
|
||||
$y += $control_to_label;
|
||||
}
|
||||
elsif ($type eq "slider") {
|
||||
printf STDERR "%30s %d %4d\n", "slider", $y, $orig_y;
|
||||
$y += $slider_to_next;
|
||||
}
|
||||
elsif ($type eq "view") {
|
||||
my $customClass = $e->att('customClass') // q{};
|
||||
if ($customClass && $customClass eq "UIScrollView") {
|
||||
# This is the entire view, so we're done
|
||||
last;
|
||||
}
|
||||
|
||||
printf STDERR "%30s %d %4d\n", "view", $y, $orig_y;
|
||||
$y += $view_to_next;
|
||||
}
|
||||
$last = $type;
|
||||
}
|
||||
|
||||
# Output the entire XML with our changes
|
||||
$twig->print;
|
||||
}
|
||||
|
||||
sub generate_id {
|
||||
# generate an id of the form K4A-oH-Bs0
|
||||
my $chars = [ split //, 'abcdefghiklmnopqrstuvwxyzABCDEFGHIKLMNOPQRSTUVWXYZ0123456789' ];
|
||||
|
||||
my $random_fragment = sub {
|
||||
my $length = shift;
|
||||
return '' if $length < 1;
|
||||
my $fragment = '';
|
||||
$fragment .= $chars->[int(rand(scalar @{$chars}))] for (1..$length);
|
||||
return $fragment;
|
||||
};
|
||||
|
||||
return join '-', (
|
||||
$random_fragment->(3),
|
||||
$random_fragment->(2),
|
||||
$random_fragment->(3),
|
||||
);
|
||||
}
|
||||
|
||||
=pod
|
||||
|
||||
=head1 NAME
|
||||
|
||||
SettingsHelper.pl - Less painful storyboard settings elements
|
||||
|
||||
=head1 SYNOPSIS
|
||||
|
||||
SettingsHelper.pl [options] [iPad.storyboard] > New.storyboard
|
||||
|
||||
Run with no options to display the list of items in a storyboard.
|
||||
|
||||
Options:
|
||||
--add-after LABEL Insert a new empty label + control pair after the given label (and its control)
|
||||
--move FROM --to TO Move the first label and control after the "to" label
|
||||
--help This help message.
|
||||
|
||||
=cut
|
||||
+1
Submodule third_party/imgui added at d99ab9f903
+1
Submodule third_party/implot added at 3da8bd3429
Reference in New Issue
Block a user