- Generated by
1.8.14
|
My Project
|
MMAL (Multi-Media Abstraction Layer) is a framework which is used to provide a host-side, simple and relatively low-level interface to multimedia components running on VideoCore. It also provides a component interface so that new components can be easily created and integrated into the framework.
There is no requirement that all the components be running on VideoCore as MMAL doesn't put any restriction on where components live. The current implementation for instance provides some components which can be run on both host-side or VideoCore (e.g. the splitter component).
The MMAL API has been designed to support all the following features:
The MMAL API is based on the concept of components, ports and buffer headers. Clients create MMAL components which expose ports for each individual elementary stream of data they support (e.g. audio/video). Components expose input ports to receive data from the client, and expose output ports to return data to the client.
Data sent to or received from the component needs to be attached to a buffer header. Buffer headers are necessary because they contain buffer specific ancillary data which is necessary for the component and client to do their processing (e.g timestamps).
MMAL lets clients create multi-media components (video encoders, video decoders, camera, and so-on) using a common API. Clients exchange data with components using buffer headers. A buffer header has a pointer to the payload data. Buffer headers are sent to and received from ports that are provided by components.
A typical decoder component would have a single input port and a single output port, but the same architecture could also be used for components with different layouts (e.g. a camera with a capture and preview port, or a debugging component with just an input port).
Each component is identified by a unique name. To create a specific component the client needs to call mmal_component_create with the desired component's name as an argument. This call will return a context (MMAL_COMPONENT_T) to the component. This context will expose the input and output ports (MMAL_PORT_T) supported by this specific component.
A port (MMAL_PORT_T) is the entity which exposes an elementary stream (MMAL_ES_FORMAT_T) on a component. It is also the entity to which buffer headers (MMAL_BUFFER_HEADER_T) are sent or from which they are received.
Clients do not need to create ports. They are created automatically by the component when this one is created but the format of a port might need to be set by the client depending on the type of component the client is using.
For example, for a video decoding component, one input port and one output port will be available. The format of the input port must be set by the client (using mmal_port_format_commit) while the format of the output port will be automatically set by the component once the component has enough information to find out what its format should be.
If the input port format contains enough information for the component to determine the format of the output port straight away, then the output port will be set to the proper format when mmal_port_format_commit returns. Otherwise the output format will be set to MMAL_ENCODING_UNKNOWN until the component is fed enough data to determine the format of the output port. When this happens, the client will receive an event on the output port, signalling that its format has changed.
Buffer headers (MMAL_BUFFER_HEADER_T) are used to exchange data with components. They do not contain the data directly but instead contain a pointer to the data being transferred.
Separating the buffer headers from the payload means that the memory for the data can be allocated outside of MMAL (e.g. if it is supplied by an external library) while still providing a consistent way to exchange data between clients and components.
Buffer headers are allocated from pools and are reference counted. The refcount will drop when mmal_buffer_header_release is called and the buffer header will be recycled to its pool when it reaches zero. The client can be notified when the buffer header is recycled so that it can recycle the associated payload memory as well.
A pool of buffer headers should be created after committing the format of the port. When the format is changed, the minimum and recommended size and number of buffers may change.
Queues (MMAL_QUEUE_T) are a facility that allows thread-safe processing of buffer headers from the client. Callbacks triggered by a MMAL component when it sends a buffer header to the client can simply put the buffer in a queue and let the main processing thread of the client get its data from the queue.
Pools (MMAL_POOL_T) let clients allocate a fixed number of buffer headers, and a queue (MMAL_QUEUE_T). They are used for buffer header allocation. Optionally a pool can also allocate the payload memory for the client.
Pools can also be resized after creation, for example, if the port format is changed leading to a new number or size of buffers being required.
Components support setting and getting component specific parameters using mmal_port_parameter_set and mmal_port_parameter_get. Parameters are identified using an integer index; parameter data is binary. See the Pre-defined MMAL parameter IDs page for more information on the pre-defined parameters.
Components can generate events on their ports. Events are sent to clients as buffer headers and thus when the client receives a buffer header on one of the component's port it should check if the buffer header is an event and in which case process it and then release it (mmal_buffer_header_release). The reason for transmitting events in-band with the actual data is that it is often very valuable to know exactly when the event happens relative to the the actual data (e.g. with a focus event, from which video frame are we in focus).
Buffer headers used to transmit events are allocated internally by the framework so it is important to release the buffer headers with mmal_buffer_header_release so the buffer headers make it back to their actual owner.
Event buffer headers are allocated when the component is created, based on the minimum number and size of control port buffers set by the component. Component wide events (not port specific) are sent to the control port callback when that port is enabled. Port events are sent to the port callback, the same as data buffers, but the 'cmd' field is non-zero.
The API requires that the MMAL core be the same or more recent version as the components and clients. Clients and components can be older and the API will still work both at compile-time and run-time.
The following code is a simple example on how to do video decoding using MMAL. Note that the code is intended to be clear and illustrate how to use MMAL at its most fundamental level, not necessarily the most efficient way to achieve the same result. Use of opaque images, tunneling and zero-copy inter-processor buffers can all improve the performance or reduce the load.
The Port Connection Utility functions can also be used to replace much of the common "boilerplate" code, especially when a pipeline of several components needs to be processed.
1.8.14