notes/vlc_decoder.md

decoder_owner

During playback, decoder behaviour is handled by a decoder_owner. This owner controls play/pause, flush, drain, waiting, and decoding.

The decoder behaviour with resources can be quite confusing because resources are sometimes allocated from vout memory and sometimes from decoder memory. Indeed, when the decoder behind the scene is a harware decoder, and especially when it is able to directly render the decoded picture with the correct timing on the video output, the picture is likely to be allocated by the API controlling this hardware decoder. This mecanism is called inflight pictures or direct rendering in VLC.

Some examples of this can be found in the omxil/mediacodec decoder.

decoder

General architecture

In VLC, a decoder is a special structure containing the following functions:

• int pf_decode (decoder_t *, block_t *): used by audio and video decoder to push the block into the actual decoder.
• block_t *pf_packetize (decoder_t *, block_t ***): used by the different packetizer, which are decoder_t objects too, to transform a list of blocks into new blocks.
• void pf_flush (decoder_t *): used by all decoder to perform a flush operation so as to clean up the real decoder module.
• block_t pf_get_cc (decoder_t *, decoder_cc_desc_t *): used by spu decoder so as to // TODO

When a block is decoded and direct rendering is disabled, the decoder_QueueVideo is called and allows the decoder_owner to collect the picture.

Push-event architecture

What we may want is a push-event architecture but some obstacles prevents an easy implementation:

First, in the situation where only one event queue exists and because of the direct rendering feature, if a pause order is made, it will be pushed after the decoding block order. This is great in the general case because you get full control of the pipeline behaviour but it leads to an unexpected playback of the queues block before pausing in the case of harware decoder.

Then, we could argue that the decoder might need to be paused as soon as possible or flushed as soon as possible, for instance because we seeked in the video. As a consequence, we would need a way to push out of band order with a higher priority than usual data.

Memory allocation

Date, tick and timestamp