rx-player

# The SegmentSinks | Consideration | Status | | ----------------------- | --------------------------------- | | Preferred import style | Directory-only _[1]_ | | Multithread environment | Should be runnable in a WebWorker | _[1]_ Only the `segment_sinks` directory itself should be imported and relied on by the rest of the code, not its inner files (thus `./index.ts` should export everything that may be imported by outside code). ## Overview The `core/segment_sinks` directory contains the part of the code directly related to the insertion and removal of media segments to a buffer for later decoding. This is either done through native `SourceBuffers`, which are JavaScript objects implemented by the browser or a custom buffer implementation entirely defined in the code of the RxPlayer. The interface through which the RxPlayer's code push segment is called the `SegmentSink`, it comes on top of the buffer implementation (regardless if it comes from native `SourceBuffer` objects or if it is a custom one). A `SegmentSink` is defined for a type of media (e.g. "video", "audio", "text"...) they are defined in the `src/core/segment_sinks/implementations` directory. Here's a simplified architecture schema of the code in that directory: ``` +--------------------------------------------------------------------------+ | Rest of the RxPlayer's code | +--------------------------------------------------------------------------+ | ^ Ask to get / create / remove | | Returns created SegmentSink or SegmentSink for a given type | | wanted information or get information about the | | available types | | | | V | +--------------------------------------------------------------------------+ | SegmentSinksStore | +--------------------------------------------------------------------------+ | | | | creates | creates | creates | (if needed) | (if needed) | (if needed) | | | V V V +-------------+ +-------------+ +-------------+ | AudioVideo | | AudioVideo | | | |SegmentSink| |SegmentSink| | Text | | (video) | | (audio) | |SegmentSink| | | | | | | +-------------+ +-------------+ +-------------+ Uses both Uses both Uses | | | | | | | | | | V V V V V +-----+ +-----+ +-----+ +-----+ +-----+ | SI* | | SB* | | SI* | | SB* | | SI* | +-----+ +-----+ +-----+ +-----+ +-----+ SI*: SegmentInventory. Store the metadata on every segments currently available in the associated media buffer. SB*: SourceBuffer (browser implementation of a media buffer). ``` ## SegmentSinksStore The `SegmentSinksStore` is the main export from there. It facilitates the creation and destruction of these `SegmentSinks`. Its roles are to: - announce which types of `SegmentSink` can be currently created on the HTMLMediaElement (example of a type of buffer would be "audio", "video" or "text"). For example, no "video" `SegmentSink` should be created on an `<audio>` element (though it wouldn't cause any problem, it would be useless as video cannot be rendered here). To give another example, you should not create a "text" `SegmentSink` if no text track parser has been added to the RxPlayer. - Create only one `SegmentSink` instance per type of buffer. Multiple `SegmentSink` for a single type could lead to browser issues and to conflicts in the RxPlayer code. - Provide a synchronization mechanism to announce when all `SourceBuffers` are ready to receive segments. I'll explain: `SourceBuffers` are browser implementations for media data buffers. They typically are used by the "video" and "audio" `SegmentSink`. Among several other constraints, all `SourceBuffers` needed to play a given content should be created before we can start pushing segments to any of them. This is a browser limitation. This is where this synchronization mechanism can become useful. The `SegmentSinksStore` will signal when all of the `SourceBuffers` needed for the given contents are created, so that the rest of the RxPlayer knows when it can begin to push segments to those. Note that this means that `SourceBuffers` for an un-needed type (e.g. an audio content won't need a video `SourceBuffer`) have to be explicitely "disabled" here, as the `SegmentSinksStore` cannot know whether it should wait until those `SourceBuffers` are created of if you just don't need it. ## SegmentSinks implementations A `SegmentSink` is an Object maintaining a media buffer for a given type (e.g. "audio", "video", "text" etc.) used for later decoding. There exists several `SegmentSink` implementations in the RxPlayer's code depending on the type concerned. An implementation takes the form of a class with a well-defined API shared with every other implementations. It allows to push segments, remove data and retrieve information about the data that is contained within it. At its core, it can either rely on a browser-defined `SourceBuffer` Object or can be entirely defined in the code of the RxPlayer. A `SegmentSink` also keeps an inventory containing the metadata of all segments currently contained in it, with the help of a `SegmentInventory` Object (see corresponding chapter). It is the main interface the rest of the RxPlayer code has with media buffers. ## BufferGarbageCollector The `BufferGarbageCollector` is a function used by the RxPlayer to periodically perform "garbage collection" manually on a given `SegmentSink`. It is based on the following building bricks: - A playback observer emitting the current time (in seconds) when the garbage collection task should be performed - The `SegmentSink` on which the garbage collection task should run - The maximum time margin authorized for the buffer behind the current position - The maximum time margin authorized for the buffer ahead of the current position Basically, each times the given playback observer emits, the BufferGarbageCollector will ensure that the volume of data before and ahead of the current position does not grow into a larger value than what is configured. For now, its code is completely decoupled for the rest of the code in that directory. This is why it is not included in the schema included on the top of this page. ## The SegmentInventory The SegmentInventory is a class which registers some information about every segments currently present in a `SegmentSink`. One of them is created for every new `SegmentSink`. This helps the RxPlayer to avoid re-downloading segments unnecessarily and know when old one have been garbage collected. For example, we could decide not to re-download a segment in any of the following cases: - The same segment is already completely present in the `SegmentSink` - The same segment is partially present in the `SegmentSink` (read: a part has been removed or garbage collected), but enough is still there for what we want to play - Another segment is in the `SegmentSink` at the wanted time, but it is the same content in a better or samey quality On the contrary, we need to download the segment in the following cases: - No segment has been pushed at the given time - A segment has been pushed, but at a lower quality than what we currently want - A segment has been pushed at a sufficient quality, but we miss to much of it for our needs (too much has been garbage collected, removed or the segment is just too short). - A segment has been pushed but is not exactly the content we want (example: it is in another language) Thanks to the SegmentInventory, we can infer on which situation we are at any time. ### Implementation The SegmentInventory is merely a "Store", meaning it will just store and process the data you give to it, without searching for the information itself. It contains in its state an array, the _inventory_, which stores every segments which should be present in the `SegmentSink` in a chronological order. To construct this inventory, three methods can be used: - one to add information about a new chunk (part of a segment or the whole segment), which should have been pushed to the `SegmentSink`. - one to indicate that every chunks from a given segment have been pushed. - one to synchronize the currently pushed segments with what the `SegmentSink` says it has buffered (which can be different for example after an automatic garbage collection). After calling the synchronization one, you should be able to tell which parts of which segments are currently _living_ in your `SegmentSink`.