Skip to content

SDK / exoplayer

Go to a project
Commit 5ac982b9 by rohks Committed by microkatz
Options

Make `MetadataRenderer` configurable to output metadata early. 

PiperOrigin-RevId: 457974611
(cherry picked from commit 621617f9)
parent 129cfb6e
Inline Side-by-side
Showing with 223 additions and 44 deletions
RELEASENOTES.md
Release notes Release notes
### 1.0.0-beta02 (2022-07-21) ### Unreleased changes
This release corresponds to the
[ExoPlayer 2.18.1 release](https://github.com/google/ExoPlayer/releases/tag/r2.18.1).
* Core library: * Core library:
* Ensure that changing the `ShuffleOrder` with `ExoPlayer.setShuffleOrder` * Ensure that changing the `ShuffleOrder` with `ExoPlayer.setShuffleOrder`
results in a call to `Player.Listener#onTimelineChanged` with results in a call to `Player.Listener#onTimelineChanged` with
`reason=Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED` `reason=Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED`
([#9889](https://github.com/google/ExoPlayer/issues/9889)). ([#9889](https://github.com/google/ExoPlayer/issues/9889)).
* For progressive media, only include selected tracks in buffered position
([#10361](https://github.com/google/ExoPlayer/issues/10361)).
* Allow custom logger for all ExoPlayer log output
([#9752](https://github.com/google/ExoPlayer/issues/9752)).
* Fix implementation of `setDataSourceFactory` in
`DefaultMediaSourceFactory`, which was non-functional in some cases
([#116](https://github.com/androidx/media/issues/116)).
* Extractors: * Extractors:
* Add support for AVI
([#2092](https://github.com/google/ExoPlayer/issues/2092)).
* Fix parsing of H265 short term reference picture sets * Fix parsing of H265 short term reference picture sets
([#10316](https://github.com/google/ExoPlayer/issues/10316)). ([#10316](https://github.com/google/ExoPlayer/issues/10316)).
* Fix parsing of bitrates from `esds` boxes * Metadata:
([#10381](https://github.com/google/ExoPlayer/issues/10381)). * `MetadataRenderer` can now be configured to render metadata as soon as
* DASH: they are available. Create an instance with
* Parse ClearKey license URL from manifests `MetadataRenderer(MetadataOutput, Looper,
([#10246](https://github.com/google/ExoPlayer/issues/10246)). MetadataDecoderFactory, boolean)` to specify whether the renderer will
* UI: output metadata early or in sync with the player position.
* Ensure TalkBack announces the currently active speed option in the
playback controls menu
([#10298](https://github.com/google/ExoPlayer/issues/10298)).
* RTSP: * RTSP:
* Add VP8 fragmented packet handling * Add RTP reader for H263
([#110](https://github.com/androidx/media/pull/110)). ([#63](https://github.com/androidx/media/pull/63)).
* Leanback extension:
* Listen to `playWhenReady` changes in `LeanbackAdapter`
([10420](https://github.com/google/ExoPlayer/issues/10420)).
* Cast:
* Use the `MediaItem` that has been passed to the playlist methods as
`Window.mediaItem` in `CastTimeline`
([#25](https://github.com/androidx/media/issues/25),
[#8212](https://github.com/google/ExoPlayer/issues/8212)).
* Support `Player.getMetadata()` and `Listener.onMediaMetadataChanged()`
with `CastPlayer` ([#25](https://github.com/androidx/media/issues/25)).
### 1.0.0-beta01 (2022-06-16) ### 1.0.0-beta01 (2022-06-16)
...@@ -136,8 +115,6 @@ This release corresponds to the ...@@ -136,8 +115,6 @@ This release corresponds to the
* Remove `RawCcExtractor`, which was only used to handle a Google-internal * Remove `RawCcExtractor`, which was only used to handle a Google-internal
subtitle format. subtitle format.
* Extractors: * Extractors:
* Add support for AVI
([#2092](https://github.com/google/ExoPlayer/issues/2092)).
* Matroska: Parse `DiscardPadding` for Opus tracks. * Matroska: Parse `DiscardPadding` for Opus tracks.
* MP4: Parse bitrates from `esds` boxes. * MP4: Parse bitrates from `esds` boxes.
* Ogg: Allow duplicate Opus ID and comment headers * Ogg: Allow duplicate Opus ID and comment headers
...@@ -187,8 +164,6 @@ This release corresponds to the ...@@ -187,8 +164,6 @@ This release corresponds to the
of `DefaultCompositeSequenceableLoaderFactory` can be passed explicitly of `DefaultCompositeSequenceableLoaderFactory` can be passed explicitly
if required. if required.
* RTSP: * RTSP:
* Add RTP reader for H263
([#63](https://github.com/androidx/media/pull/63)).
* Add RTP reader for MPEG4 * Add RTP reader for MPEG4
([#35](https://github.com/androidx/media/pull/35)). ([#35](https://github.com/androidx/media/pull/35)).
* Add RTP reader for HEVC * Add RTP reader for HEVC
......
libraries/exoplayer/src/main/java/androidx/media3/exoplayer/metadata/MetadataRenderer.java
...@@ -39,7 +39,12 @@ import java.util.ArrayList; ...@@ -39,7 +39,12 @@ import java.util.ArrayList;
import java.util.List; import java.util.List;
import org.checkerframework.dataflow.qual.SideEffectFree; import org.checkerframework.dataflow.qual.SideEffectFree;
/** A renderer for metadata. */ /**
* A renderer for metadata.
*
* <p>The renderer can be configured to render metadata as soon as they are available using {@link
* #MetadataRenderer(MetadataOutput, Looper, MetadataDecoderFactory, boolean)}.
*/
@UnstableApi @UnstableApi
public final class MetadataRenderer extends BaseRenderer implements Callback { public final class MetadataRenderer extends BaseRenderer implements Callback {
...@@ -50,6 +55,7 @@ public final class MetadataRenderer extends BaseRenderer implements Callback { ...@@ -50,6 +55,7 @@ public final class MetadataRenderer extends BaseRenderer implements Callback {
private final MetadataOutput output; private final MetadataOutput output;
@Nullable private final Handler outputHandler; @Nullable private final Handler outputHandler;
private final MetadataInputBuffer buffer; private final MetadataInputBuffer buffer;
private final boolean outputMetadataEarly;
@Nullable private MetadataDecoder decoder; @Nullable private MetadataDecoder decoder;
private boolean inputStreamEnded; private boolean inputStreamEnded;
...@@ -59,6 +65,9 @@ public final class MetadataRenderer extends BaseRenderer implements Callback { ...@@ -59,6 +65,9 @@ public final class MetadataRenderer extends BaseRenderer implements Callback {
private long outputStreamOffsetUs; private long outputStreamOffsetUs;
/** /**
* Creates an instance that uses {@link MetadataDecoderFactory#DEFAULT} to create {@link
* MetadataDecoder} instances.
*
* @param output The output. * @param output The output.
* @param outputLooper The looper associated with the thread on which the output should be called. * @param outputLooper The looper associated with the thread on which the output should be called.
* If the output makes use of standard Android UI components, then this should normally be the * If the output makes use of standard Android UI components, then this should normally be the
...@@ -71,6 +80,8 @@ public final class MetadataRenderer extends BaseRenderer implements Callback { ...@@ -71,6 +80,8 @@ public final class MetadataRenderer extends BaseRenderer implements Callback {
} }
/** /**
* Creates an instance.
*
* @param output The output. * @param output The output.
* @param outputLooper The looper associated with the thread on which the output should be called. * @param outputLooper The looper associated with the thread on which the output should be called.
* If the output makes use of standard Android UI components, then this should normally be the * If the output makes use of standard Android UI components, then this should normally be the
...@@ -81,11 +92,34 @@ public final class MetadataRenderer extends BaseRenderer implements Callback { ...@@ -81,11 +92,34 @@ public final class MetadataRenderer extends BaseRenderer implements Callback {
*/ */
public MetadataRenderer( public MetadataRenderer(
MetadataOutput output, @Nullable Looper outputLooper, MetadataDecoderFactory decoderFactory) { MetadataOutput output, @Nullable Looper outputLooper, MetadataDecoderFactory decoderFactory) {
this(output, outputLooper, decoderFactory, /* outputMetadataEarly= */ false);
}
/**
* Creates an instance.
*
* @param output The output.
* @param outputLooper The looper associated with the thread on which the output should be called.
* If the output makes use of standard Android UI components, then this should normally be the
* looper associated with the application's main thread, which can be obtained using {@link
* android.app.Activity#getMainLooper()}. Null may be passed if the output should be called
* directly on the player's internal rendering thread.
* @param decoderFactory A factory from which to obtain {@link MetadataDecoder} instances.
* @param outputMetadataEarly Whether the renderer outputs metadata early. When {@code true},
* {@link #render} will output metadata as soon as they are available to the renderer,
* otherwise {@link #render} will output metadata in sync with the rendering position.
*/
public MetadataRenderer(
MetadataOutput output,
@Nullable Looper outputLooper,
MetadataDecoderFactory decoderFactory,
boolean outputMetadataEarly) {
super(C.TRACK_TYPE_METADATA); super(C.TRACK_TYPE_METADATA);
this.output = Assertions.checkNotNull(output); this.output = Assertions.checkNotNull(output);
this.outputHandler = this.outputHandler =
outputLooper == null ? null : Util.createHandler(outputLooper, /* callback= */ this); outputLooper == null ? null : Util.createHandler(outputLooper, /* callback= */ this);
this.decoderFactory = Assertions.checkNotNull(decoderFactory); this.decoderFactory = Assertions.checkNotNull(decoderFactory);
this.outputMetadataEarly = outputMetadataEarly;
buffer = new MetadataInputBuffer(); buffer = new MetadataInputBuffer();
outputStreamOffsetUs = C.TIME_UNSET; outputStreamOffsetUs = C.TIME_UNSET;
} }
...@@ -217,7 +251,8 @@ public final class MetadataRenderer extends BaseRenderer implements Callback { ...@@ -217,7 +251,8 @@ public final class MetadataRenderer extends BaseRenderer implements Callback {
private boolean outputMetadata(long positionUs) { private boolean outputMetadata(long positionUs) {
boolean didOutput = false; boolean didOutput = false;
if (pendingMetadata != null if (pendingMetadata != null
&& pendingMetadata.presentationTimeUs <= getPresentationTimeUs(positionUs)) { && (outputMetadataEarly
|| pendingMetadata.presentationTimeUs <= getPresentationTimeUs(positionUs))) {
invokeRenderer(pendingMetadata); invokeRenderer(pendingMetadata);
pendingMetadata = null; pendingMetadata = null;
didOutput = true; didOutput = true;
......
libraries/exoplayer/src/test/java/androidx/media3/exoplayer/metadata/MetadataRendererTest.java
...@@ -145,16 +145,87 @@ public class MetadataRendererTest { ...@@ -145,16 +145,87 @@ public class MetadataRendererTest {
assertThat(metadata).isEmpty(); assertThat(metadata).isEmpty();
} }
@Test
public void renderMetadata_withTimelyOutput() throws Exception {
EventMessage emsg =
new EventMessage(
"urn:test-scheme-id",
/* value= */ "",
/* durationMs= */ 1,
/* id= */ 0,
"Test data".getBytes(UTF_8));
byte[] encodedEmsg = eventMessageEncoder.encode(emsg);
List<Metadata> metadata = new ArrayList<>();
MetadataRenderer renderer =
new MetadataRenderer(/* output= */ metadata::add, /* outputLooper= */ null);
FakeSampleStream fakeSampleStream =
createFakeSampleStream(
ImmutableList.of(
sample(/* timeUs= */ 100_000, C.BUFFER_FLAG_KEY_FRAME, encodedEmsg),
sample(/* timeUs= */ 1_000_000, C.BUFFER_FLAG_KEY_FRAME, encodedEmsg),
END_OF_STREAM_ITEM));
fakeSampleStream.writeData(/* startPositionUs= */ 0);
renderer.replaceStream(
new Format[] {EMSG_FORMAT},
fakeSampleStream,
/* startPositionUs= */ 0L,
/* offsetUs= */ 0L);
// Call render() twice, the first call is to read the format and the second call will read the
// metadata.
renderer.render(/* positionUs= */ 0, /* elapsedRealtimeUs= */ 0);
renderer.render(/* positionUs= */ 500_000, /* elapsedRealtimeUs= */ 0);
assertThat(metadata).hasSize(1);
assertThat(metadata.get(0).presentationTimeUs).isEqualTo(100_000);
}
@Test
public void renderMetadata_withEarlyOutput() throws Exception {
EventMessage emsg =
new EventMessage(
"urn:test-scheme-id",
/* value= */ "",
/* durationMs= */ 1,
/* id= */ 0,
"Test data".getBytes(UTF_8));
byte[] encodedEmsg = eventMessageEncoder.encode(emsg);
List<Metadata> metadata = new ArrayList<>();
MetadataRenderer renderer =
new MetadataRenderer(
/* output= */ metadata::add,
/* outputLooper= */ null,
MetadataDecoderFactory.DEFAULT,
/* outputMetadataEarly= */ true);
FakeSampleStream fakeSampleStream =
createFakeSampleStream(
ImmutableList.of(
sample(/* timeUs= */ 100_000, C.BUFFER_FLAG_KEY_FRAME, encodedEmsg),
sample(/* timeUs= */ 1_000_000, C.BUFFER_FLAG_KEY_FRAME, encodedEmsg),
END_OF_STREAM_ITEM));
fakeSampleStream.writeData(/* startPositionUs= */ 0);
renderer.replaceStream(
new Format[] {EMSG_FORMAT},
fakeSampleStream,
/* startPositionUs= */ 0L,
/* offsetUs= */ 0L);
// Call render() twice, the first call is to read the format and the second call will read the
// metadata.
renderer.render(/* positionUs= */ 0, /* elapsedRealtimeUs= */ 0);
renderer.render(/* positionUs= */ 500_000, /* elapsedRealtimeUs= */ 0);
// The renderer outputs metadata early.
assertThat(metadata).hasSize(2);
assertThat(metadata.get(0).presentationTimeUs).isEqualTo(100_000);
assertThat(metadata.get(1).presentationTimeUs).isEqualTo(1_000_000);
}
private static List<Metadata> runRenderer(byte[] input) throws ExoPlaybackException { private static List<Metadata> runRenderer(byte[] input) throws ExoPlaybackException {
List<Metadata> metadata = new ArrayList<>(); List<Metadata> metadata = new ArrayList<>();
MetadataRenderer renderer = new MetadataRenderer(metadata::add, /* outputLooper= */ null); MetadataRenderer renderer = new MetadataRenderer(metadata::add, /* outputLooper= */ null);
FakeSampleStream fakeSampleStream = FakeSampleStream fakeSampleStream =
new FakeSampleStream( createFakeSampleStream(
new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024),
/* mediaSourceEventDispatcher= */ null,
DrmSessionManager.DRM_UNSUPPORTED,
new DrmSessionEventListener.EventDispatcher(),
EMSG_FORMAT,
ImmutableList.of( ImmutableList.of(
sample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME, input), END_OF_STREAM_ITEM)); sample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME, input), END_OF_STREAM_ITEM));
fakeSampleStream.writeData(/* startPositionUs= */ 0); fakeSampleStream.writeData(/* startPositionUs= */ 0);
...@@ -169,6 +240,17 @@ public class MetadataRendererTest { ...@@ -169,6 +240,17 @@ public class MetadataRendererTest {
return Collections.unmodifiableList(metadata); return Collections.unmodifiableList(metadata);
} }
private static FakeSampleStream createFakeSampleStream(
ImmutableList<FakeSampleStream.FakeSampleStreamItem> samples) {
return new FakeSampleStream(
new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024),
/* mediaSourceEventDispatcher= */ null,
DrmSessionManager.DRM_UNSUPPORTED,
new DrmSessionEventListener.EventDispatcher(),
EMSG_FORMAT,
samples);
}
/** /**
* Builds an ID3v2 tag containing a single 'user defined text information frame' (id='TXXX') with * Builds an ID3v2 tag containing a single 'user defined text information frame' (id='TXXX') with
* {@code description} and {@code value}. * {@code description} and {@code value}.
......
libraries/exoplayer_dash/src/test/java/androidx/media3/exoplayer/dash/e2etest/DashPlaybackTest.java
...@@ -23,6 +23,10 @@ import android.view.Surface; ...@@ -23,6 +23,10 @@ import android.view.Surface;
import androidx.media3.common.MediaItem; import androidx.media3.common.MediaItem;
import androidx.media3.common.Player; import androidx.media3.common.Player;
import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.exoplayer.ExoPlayer;
import androidx.media3.exoplayer.Renderer;
import androidx.media3.exoplayer.RenderersFactory;
import androidx.media3.exoplayer.metadata.MetadataDecoderFactory;
import androidx.media3.exoplayer.metadata.MetadataRenderer;
import androidx.media3.exoplayer.trackselection.DefaultTrackSelector; import androidx.media3.exoplayer.trackselection.DefaultTrackSelector;
import androidx.media3.test.utils.CapturingRenderersFactory; import androidx.media3.test.utils.CapturingRenderersFactory;
import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.DumpFileAsserts;
...@@ -96,4 +100,70 @@ public final class DashPlaybackTest { ...@@ -96,4 +100,70 @@ public final class DashPlaybackTest {
DumpFileAsserts.assertOutput( DumpFileAsserts.assertOutput(
applicationContext, playbackOutput, "playbackdumps/dash/emsg.dump"); applicationContext, playbackOutput, "playbackdumps/dash/emsg.dump");
} }
@Test
public void renderMetadata_withTimelyOutput() throws Exception {
Context applicationContext = ApplicationProvider.getApplicationContext();
RenderersFactory renderersFactory =
(eventHandler,
videoRendererEventListener,
audioRendererEventListener,
textRendererOutput,
metadataRendererOutput) ->
new Renderer[] {new MetadataRenderer(metadataRendererOutput, eventHandler.getLooper())};
ExoPlayer player =
new ExoPlayer.Builder(applicationContext, renderersFactory)
.setClock(new FakeClock(/* isAutoAdvancing= */ true))
.build();
player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1)));
CapturingRenderersFactory capturingRenderersFactory =
new CapturingRenderersFactory(applicationContext);
PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory);
player.setMediaItem(MediaItem.fromUri("asset:///media/dash/emsg/sample.mpd"));
player.prepare();
player.play();
TestPlayerRunHelper.playUntilPosition(player, /* mediaItemIndex= */ 0, /* positionMs= */ 500);
player.release();
// Ensure output contains metadata up to the playback position.
DumpFileAsserts.assertOutput(
applicationContext, playbackOutput, "playbackdumps/dash/metadata_from_timely_output.dump");
}
@Test
public void renderMetadata_withEarlyOutput() throws Exception {
Context applicationContext = ApplicationProvider.getApplicationContext();
RenderersFactory renderersFactory =
(eventHandler,
videoRendererEventListener,
audioRendererEventListener,
textRendererOutput,
metadataRendererOutput) ->
new Renderer[] {
new MetadataRenderer(
metadataRendererOutput,
eventHandler.getLooper(),
MetadataDecoderFactory.DEFAULT,
/* outputMetadataEarly= */ true)
};
ExoPlayer player =
new ExoPlayer.Builder(applicationContext, renderersFactory)
.setClock(new FakeClock(/* isAutoAdvancing= */ true))
.build();
player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1)));
CapturingRenderersFactory capturingRenderersFactory =
new CapturingRenderersFactory(applicationContext);
PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory);
player.setMediaItem(MediaItem.fromUri("asset:///media/dash/emsg/sample.mpd"));
player.prepare();
player.play();
TestPlayerRunHelper.playUntilPosition(player, /* mediaItemIndex= */ 0, /* positionMs= */ 500);
player.release();
// Ensure output contains all metadata irrespective of the playback position.
DumpFileAsserts.assertOutput(
applicationContext, playbackOutput, "playbackdumps/dash/metadata_from_early_output.dump");
}
} }
libraries/test_data/src/test/assets/playbackdumps/dash/metadata_from_early_output.dump 0 → 100644
MetadataOutput:
Metadata[0]:
presentationTimeUs = 100000
entry[0] = EMSG: scheme=urn:mpeg:dash:event:callback:2015, id=0, durationMs=1000, value=1
Metadata[1]:
presentationTimeUs = 100000
entry[0] = EMSG: scheme=urn:mpeg:dash:event:callback:2015, id=1, durationMs=1000, value=1
Metadata[2]:
presentationTimeUs = 1000000
entry[0] = EMSG: scheme=urn:mpeg:dash:event:callback:2015, id=2, durationMs=1000, value=1
libraries/test_data/src/test/assets/playbackdumps/dash/metadata_from_timely_output.dump 0 → 100644
MetadataOutput:
Metadata[0]:
presentationTimeUs = 100000
entry[0] = EMSG: scheme=urn:mpeg:dash:event:callback:2015, id=0, durationMs=1000, value=1
Metadata[1]:
presentationTimeUs = 100000
entry[0] = EMSG: scheme=urn:mpeg:dash:event:callback:2015, id=1, durationMs=1000, value=1
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment