2009-12-09 12:39:58 +01:00
|
|
|
Linux kernel media framework
|
|
|
|
============================
|
|
|
|
|
|
|
|
This document describes the Linux kernel media framework, its data structures,
|
|
|
|
functions and their usage.
|
|
|
|
|
|
|
|
|
|
|
|
Introduction
|
|
|
|
------------
|
|
|
|
|
|
|
|
The media controller API is documented in DocBook format in
|
|
|
|
Documentation/DocBook/v4l/media-controller.xml. This document will focus on
|
|
|
|
the kernel-side implementation of the media framework.
|
|
|
|
|
|
|
|
|
2009-12-09 12:40:00 +01:00
|
|
|
Abstract media device model
|
|
|
|
---------------------------
|
|
|
|
|
|
|
|
Discovering a device internal topology, and configuring it at runtime, is one
|
|
|
|
of the goals of the media framework. To achieve this, hardware devices are
|
|
|
|
modeled as an oriented graph of building blocks called entities connected
|
|
|
|
through pads.
|
|
|
|
|
|
|
|
An entity is a basic media hardware building block. It can correspond to
|
|
|
|
a large variety of logical blocks such as physical hardware devices
|
|
|
|
(CMOS sensor for instance), logical hardware devices (a building block
|
|
|
|
in a System-on-Chip image processing pipeline), DMA channels or physical
|
|
|
|
connectors.
|
|
|
|
|
|
|
|
A pad is a connection endpoint through which an entity can interact with
|
|
|
|
other entities. Data (not restricted to video) produced by an entity
|
|
|
|
flows from the entity's output to one or more entity inputs. Pads should
|
|
|
|
not be confused with physical pins at chip boundaries.
|
|
|
|
|
|
|
|
A link is a point-to-point oriented connection between two pads, either
|
|
|
|
on the same entity or on different entities. Data flows from a source
|
|
|
|
pad to a sink pad.
|
|
|
|
|
|
|
|
|
2009-12-09 12:39:58 +01:00
|
|
|
Media device
|
|
|
|
------------
|
|
|
|
|
|
|
|
A media device is represented by a struct media_device instance, defined in
|
|
|
|
include/media/media-device.h. Allocation of the structure is handled by the
|
|
|
|
media device driver, usually by embedding the media_device instance in a
|
|
|
|
larger driver-specific structure.
|
|
|
|
|
|
|
|
Drivers register media device instances by calling
|
|
|
|
|
|
|
|
media_device_register(struct media_device *mdev);
|
|
|
|
|
|
|
|
The caller is responsible for initializing the media_device structure before
|
|
|
|
registration. The following fields must be set:
|
|
|
|
|
|
|
|
- dev must point to the parent device (usually a pci_dev, usb_interface or
|
|
|
|
platform_device instance).
|
|
|
|
|
|
|
|
- model must be filled with the device model name as a NUL-terminated UTF-8
|
|
|
|
string. The device/model revision must not be stored in this field.
|
|
|
|
|
|
|
|
The following fields are optional:
|
|
|
|
|
|
|
|
- serial is a unique serial number stored as a NUL-terminated ASCII string.
|
|
|
|
The field is big enough to store a GUID in text form. If the hardware
|
|
|
|
doesn't provide a unique serial number this field must be left empty.
|
|
|
|
|
|
|
|
- bus_info represents the location of the device in the system as a
|
|
|
|
NUL-terminated ASCII string. For PCI/PCIe devices bus_info must be set to
|
|
|
|
"PCI:" (or "PCIe:") followed by the value of pci_name(). For USB devices,
|
|
|
|
the usb_make_path() function must be used. This field is used by
|
|
|
|
applications to distinguish between otherwise identical devices that don't
|
|
|
|
provide a serial number.
|
|
|
|
|
|
|
|
- hw_revision is the hardware device revision in a driver-specific format.
|
|
|
|
When possible the revision should be formatted with the KERNEL_VERSION
|
|
|
|
macro.
|
|
|
|
|
|
|
|
- driver_version is formatted with the KERNEL_VERSION macro. The version
|
|
|
|
minor must be incremented when new features are added to the userspace API
|
|
|
|
without breaking binary compatibility. The version major must be
|
|
|
|
incremented when binary compatibility is broken.
|
|
|
|
|
|
|
|
Upon successful registration a character device named media[0-9]+ is created.
|
|
|
|
The device major and minor numbers are dynamic. The model name is exported as
|
|
|
|
a sysfs attribute.
|
|
|
|
|
|
|
|
Drivers unregister media device instances by calling
|
|
|
|
|
|
|
|
media_device_unregister(struct media_device *mdev);
|
|
|
|
|
|
|
|
Unregistering a media device that hasn't been registered is *NOT* safe.
|
2009-12-09 12:40:00 +01:00
|
|
|
|
|
|
|
|
|
|
|
Entities, pads and links
|
|
|
|
------------------------
|
|
|
|
|
|
|
|
- Entities
|
|
|
|
|
|
|
|
Entities are represented by a struct media_entity instance, defined in
|
|
|
|
include/media/media-entity.h. The structure is usually embedded into a
|
|
|
|
higher-level structure, such as a v4l2_subdev or video_device instance,
|
|
|
|
although drivers can allocate entities directly.
|
|
|
|
|
|
|
|
Drivers initialize entities by calling
|
|
|
|
|
|
|
|
media_entity_init(struct media_entity *entity, u16 num_pads,
|
|
|
|
struct media_pad *pads, u16 extra_links);
|
|
|
|
|
|
|
|
The media_entity name, type, flags, revision and group_id fields can be
|
|
|
|
initialized before or after calling media_entity_init. Entities embedded in
|
|
|
|
higher-level standard structures can have some of those fields set by the
|
|
|
|
higher-level framework.
|
|
|
|
|
|
|
|
As the number of pads is known in advance, the pads array is not allocated
|
|
|
|
dynamically but is managed by the entity driver. Most drivers will embed the
|
|
|
|
pads array in a driver-specific structure, avoiding dynamic allocation.
|
|
|
|
|
|
|
|
Drivers must set the direction of every pad in the pads array before calling
|
|
|
|
media_entity_init. The function will initialize the other pads fields.
|
|
|
|
|
|
|
|
Unlike the number of pads, the total number of links isn't always known in
|
|
|
|
advance by the entity driver. As an initial estimate, media_entity_init
|
|
|
|
pre-allocates a number of links equal to the number of pads plus an optional
|
|
|
|
number of extra links. The links array will be reallocated if it grows beyond
|
|
|
|
the initial estimate.
|
|
|
|
|
|
|
|
Drivers register entities with a media device by calling
|
|
|
|
|
|
|
|
media_device_register_entity(struct media_device *mdev,
|
|
|
|
struct media_entity *entity);
|
|
|
|
|
|
|
|
Entities are identified by a unique positive integer ID. Drivers can provide an
|
|
|
|
ID by filling the media_entity id field prior to registration, or request the
|
|
|
|
media controller framework to assign an ID automatically. Drivers that provide
|
|
|
|
IDs manually must ensure that all IDs are unique. IDs are not guaranteed to be
|
|
|
|
contiguous even when they are all assigned automatically by the framework.
|
|
|
|
|
|
|
|
Drivers unregister entities by calling
|
|
|
|
|
|
|
|
media_device_unregister_entity(struct media_entity *entity);
|
|
|
|
|
|
|
|
Unregistering an entity will not change the IDs of the other entities, and the
|
|
|
|
ID will never be reused for a newly registered entity.
|
|
|
|
|
|
|
|
When a media device is unregistered, all its entities are unregistered
|
|
|
|
automatically. No manual entities unregistration is then required.
|
|
|
|
|
|
|
|
Drivers free resources associated with an entity by calling
|
|
|
|
|
|
|
|
media_entity_cleanup(struct media_entity *entity);
|
|
|
|
|
|
|
|
This function must be called during the cleanup phase after unregistering the
|
|
|
|
entity. Note that the media_entity instance itself must be freed explicitly by
|
|
|
|
the driver if required.
|
|
|
|
|
|
|
|
Entities have flags that describe the entity capabilities and state.
|
|
|
|
|
|
|
|
MEDIA_ENT_FL_DEFAULT indicates the default entity for a given type.
|
|
|
|
This can be used to report the default audio and video devices or the
|
|
|
|
default camera sensor.
|
|
|
|
|
|
|
|
Logical entity groups can be defined by setting the group ID of all member
|
|
|
|
entities to the same non-zero value. An entity group serves no purpose in the
|
|
|
|
kernel, but is reported to userspace during entities enumeration. The group_id
|
|
|
|
field belongs to the media device driver and must not by touched by entity
|
|
|
|
drivers.
|
|
|
|
|
|
|
|
Media device drivers should define groups if several entities are logically
|
|
|
|
bound together. Example usages include reporting
|
|
|
|
|
|
|
|
- ALSA, VBI and video nodes that carry the same media stream
|
|
|
|
- lens and flash controllers associated with a sensor
|
|
|
|
|
|
|
|
- Pads
|
|
|
|
|
|
|
|
Pads are represented by a struct media_pad instance, defined in
|
|
|
|
include/media/media-entity.h. Each entity stores its pads in a pads array
|
|
|
|
managed by the entity driver. Drivers usually embed the array in a
|
|
|
|
driver-specific structure.
|
|
|
|
|
|
|
|
Pads are identified by their entity and their 0-based index in the pads array.
|
|
|
|
Both information are stored in the media_pad structure, making the media_pad
|
|
|
|
pointer the canonical way to store and pass link references.
|
|
|
|
|
|
|
|
Pads have flags that describe the pad capabilities and state.
|
|
|
|
|
|
|
|
MEDIA_PAD_FL_SINK indicates that the pad supports sinking data.
|
|
|
|
MEDIA_PAD_FL_SOURCE indicates that the pad supports sourcing data.
|
|
|
|
|
|
|
|
One and only one of MEDIA_PAD_FL_SINK and MEDIA_PAD_FL_SOURCE must be set for
|
|
|
|
each pad.
|
|
|
|
|
|
|
|
- Links
|
|
|
|
|
|
|
|
Links are represented by a struct media_link instance, defined in
|
|
|
|
include/media/media-entity.h. Each entity stores all links originating at or
|
|
|
|
targetting any of its pads in a links array. A given link is thus stored
|
|
|
|
twice, once in the source entity and once in the target entity. The array is
|
|
|
|
pre-allocated and grows dynamically as needed.
|
|
|
|
|
|
|
|
Drivers create links by calling
|
|
|
|
|
|
|
|
media_entity_create_link(struct media_entity *source, u16 source_pad,
|
|
|
|
struct media_entity *sink, u16 sink_pad,
|
|
|
|
u32 flags);
|
|
|
|
|
|
|
|
An entry in the link array of each entity is allocated and stores pointers
|
|
|
|
to source and sink pads.
|
|
|
|
|
|
|
|
Links have flags that describe the link capabilities and state.
|
|
|
|
|
|
|
|
MEDIA_LNK_FL_ENABLED indicates that the link is enabled and can be used
|
|
|
|
to transfer media data. When two or more links target a sink pad, only
|
|
|
|
one of them can be enabled at a time.
|
|
|
|
MEDIA_LNK_FL_IMMUTABLE indicates that the link enabled state can't be
|
|
|
|
modified at runtime. If MEDIA_LNK_FL_IMMUTABLE is set, then
|
|
|
|
MEDIA_LNK_FL_ENABLED must also be set since an immutable link is always
|
|
|
|
enabled.
|