Membrane.H26x.ParsingEngine (Membrane H.264 and H.265 plugin v0.11.2)

Copy Markdown View Source

Membrane-agnostic H26x parsing engine.

Splits incoming payloads into NAL units, parses them and groups them into access units, optionally generating best-effort timestamps. The access units are returned interleaved with parameter set change notifications - see event/0. Access units that are not decodable - containing NALus that failed to parse or no VCL NALu - are dropped (their parameter sets are still taken into account). Depending on the configured output stream structure, the engine also repeats the active parameter sets on keyframes or strips them from the access units altogether (for the structures carrying them out-of-band). Codec-specific behaviour is selected via the :codec field of config/0.

Summary

Types

Configuration of the parsing engine

An item of the engine's output.

If set to a map, timestamps are generated based on the provided constant framerate (available only for the :bytestream input alignment). :add_dts_offset shifts DTS values so that they don't exceed PTS values, defaults to true.

Alignment of the payloads fed to the engine - an arbitrary stream of bytes (:bytestream), a single NAL unit per payload (:nalu) or a whole access unit per payload (:au).

The structure of the input stream.

Structure of the H26x stream in the raw form transferred out-of-band: either :annexb, or a codec tag along with a Decoder Configuration Record binary carrying the stream's parameter sets (nil if no SPS has been seen yet, so no DCR can be generated).

Structure of the H26x stream - either Annex B, where the NALus are separated by a start code (0x(00)000001), or length-prefixed (as described in ISO/IEC 14496-15), where each NALu is prefixed with its length.

t()

Functions

Drains all buffered data into access units. To be used on an input alignment change or end of stream.

Creates a parser for the given input stream structure and alignment.

Feeds a payload through the parser, returning the access units completed by it, interleaved with parameter set change notifications - see event/0.

Changes the input alignment and stream structure, keeping the accumulated parsing state. If the input configuration actually changed, the engine is flushed first (see flush/1) and the resulting events are returned. When the new structure carries a Decoder Configuration Record, the parameter sets it holds are scheduled to be parsed, skipping the ones already active in the stream.

Types

codec()

@type codec() :: :h264 | :h265

config()

@type config() :: %{
  :codec => codec(),
  :input_stream_structure => input_stream_structure(),
  :input_alignment => input_alignment(),
  optional(:output_stream_structure) => stream_structure() | nil,
  optional(:repeat_parameter_sets) => boolean(),
  optional(:initial_parameter_sets) => [binary()],
  optional(:generate_best_effort_timestamps) =>
    generate_best_effort_timestamps()
}

Configuration of the parsing engine:

  • :codec - the codec of the parsed stream, either :h264 or :h265.
  • :input_stream_structure - see input_stream_structure/0.
  • :input_alignment - see input_alignment/0.
  • :output_stream_structure - the stream structure the output access units are intended for. Determines how the parameter sets are handled: for the structures carrying them out-of-band (:avc1, :hvc1) they are stripped from the access units and carried solely by the DCRs of the :parameter_sets events. Defaults to the input stream structure.
  • :repeat_parameter_sets - if true, all the active parameter sets are attached to each keyframe access unit. Takes no effect for the :avc1 and :hvc1 output stream structures, where the parameter sets travel out-of-band. Defaults to false.
  • :initial_parameter_sets - raw (unprefixed) payloads of parameter sets absent from the stream, scheduled to be parsed before the first pushed payload. Defaults to [].
  • :generate_best_effort_timestamps - see generate_best_effort_timestamps/0. Defaults to false.

event()

@type event() ::
  {:access_unit, Membrane.H26x.AccessUnit.t()}
  | {:parameter_sets,
     %{
       output_raw_stream_structure: raw_stream_structure(),
       active: [Membrane.H26x.NALu.t()]
     }}

An item of the engine's output.

Access units (see Membrane.H26x.AccessUnit.t/0) are interleaved with parameter set change notifications: whenever the set of active parameter sets changes, a :parameter_sets event is emitted right before the access unit that introduced the change. The event carries all the parameter sets active at that point of the stream (the most recent set per parameter set type and id) and the raw output stream structure (see raw_stream_structure/0) - for the length-prefixed output stream structures its Decoder Configuration Record is generated out of the active parameter sets.

generate_best_effort_timestamps()

@type generate_best_effort_timestamps() ::
  false
  | %{
      :framerate => {frames :: pos_integer(), seconds :: pos_integer()},
      optional(:add_dts_offset) => boolean()
    }

If set to a map, timestamps are generated based on the provided constant framerate (available only for the :bytestream input alignment). :add_dts_offset shifts DTS values so that they don't exceed PTS values, defaults to true.

input_alignment()

@type input_alignment() :: :bytestream | :nalu | :au

Alignment of the payloads fed to the engine - an arbitrary stream of bytes (:bytestream), a single NAL unit per payload (:nalu) or a whole access unit per payload (:au).

input_stream_structure()

@type input_stream_structure() ::
  stream_structure()
  | {codec_tag :: :avc1 | :avc3 | :hvc1 | :hev1, dcr :: binary()}

The structure of the input stream.

For the length-prefixed structures (:avc1, :avc3, :hvc1, :hev1), instead of the NALu length size, a Decoder Configuration Record binary (e.g. coming from an MP4 container) may be provided - the NALu length size is then read from it and the parameter sets it carries are scheduled to be parsed before the first pushed payload.

raw_stream_structure()

@type raw_stream_structure() ::
  :annexb | {codec_tag :: :avc1 | :avc3 | :hvc1 | :hev1, dcr :: binary() | nil}

Structure of the H26x stream in the raw form transferred out-of-band: either :annexb, or a codec tag along with a Decoder Configuration Record binary carrying the stream's parameter sets (nil if no SPS has been seen yet, so no DCR can be generated).

stream_structure()

@type stream_structure() ::
  :annexb
  | {codec_tag :: :avc1 | :avc3 | :hvc1 | :hev1,
     nalu_length_size :: pos_integer()}

Structure of the H26x stream - either Annex B, where the NALus are separated by a start code (0x(00)000001), or length-prefixed (as described in ISO/IEC 14496-15), where each NALu is prefixed with its length.

t()

@opaque t()

Functions

flush(engine)

@spec flush(t()) :: {[event()], t()}

Drains all buffered data into access units. To be used on an input alignment change or end of stream.

new(config)

@spec new(config()) :: t()

Creates a parser for the given input stream structure and alignment.

Raises an ArgumentError if the configured codec is not supported.

push(engine, payload, timestamps \\ {nil, nil})

@spec push(t(), binary(), Membrane.H26x.NALu.timestamps()) :: {[event()], t()}

Feeds a payload through the parser, returning the access units completed by it, interleaved with parameter set change notifications - see event/0.

reconfigure_input(engine, input_alignment, input_stream_structure)

@spec reconfigure_input(t(), input_alignment(), input_stream_structure()) ::
  {[event()], t()} | no_return()

Changes the input alignment and stream structure, keeping the accumulated parsing state. If the input configuration actually changed, the engine is flushed first (see flush/1) and the resulting events are returned. When the new structure carries a Decoder Configuration Record, the parameter sets it holds are scheduled to be parsed, skipping the ones already active in the stream.

Raises if the input stream structure differs fundamentally from the current one - the only change allowed mid-stream is the NALu length size of a length-prefixed structure.