Gstreamer基本概念深入解析
我們先看看gstreamer結構體中的一些物件的繼承關係圖：
GObject
+----GstElement（包含GstBus,GstState, List of pad、GstClock等）
+----GstBin （元件的容器，包含GstClock，子bus等）
+----GstPipeline（包含stream_time等）
另外幾個類關係：
GObject
+---- GstPad
GObject
+---- GstBus
GstObject
+---- GstPluginFeature
+---- GstElementFactory（factory that the element was created from，_GstElementClass儲存有該結構的一個指標）
我們看一個pipeline的解釋和示例圖：
Pipeline
~~~~~~~~
A pipeline is a special bin subclass that provides the following features to its
children:
- Select and manage a global clock for all its children.
- Manage running_time based on the selected clock. Running_time is the elapsed
time the pipeline spent in the PLAYING state and is used for
synchronisation.
- Manage latency in the pipeline.
- Provide means for elements to comunicate with the application by the GstBus.
- Manage the global state of the elements such as Errors and end-of-stream.
Normally the application creates one pipeline that will manage all the elements
in the application.
+-----------------------------------------------------------+
| ----------> downstream -------------------> |
| |
| pipeline |
| +---------+ +----------+ +-----------+ +----------+ |
| | filesrc | | oggdemux | | vorbisdec | | alsasink | |
| | src-sink src-sink src-sink | |
| +---------+ +----------+ +-----------+ +----------+ |
| |
| <---------< upstream <-------------------< |
+-----------------------------------------------------------+
The application does not have to manage any of the complexities of the
actual dataflow/decoding/conversions/synchronisation etc. but only calls high
level functions on the pipeline object such as PLAY/PAUSE/STOP.
GStreamer supports two possible types of dataflow, the push and pull model. In the
push model, an upstream element sends data to a downstream element by calling a
method on a sinkpad. In the pull model, a downstream element requests data from
an upstream element by calling a method on a source pad.
The most common dataflow is the push model. The pull model can be used in specific
circumstances by demuxer elements. The pull model can also be used by low latency
audio applications.
The data passed between pads is encapsulated in Buffers. The buffer contains a
pointer to the actual data and also metadata describing the data.
可見，pipeline的主要佈局是elements, Bins 和pads.把上述幾個物件繼承關係牢記腦海中。PADs是聯結器。pipeline的各物件之間的通訊通過快取buffers、事件event、查詢query、訊息message機制實現， 排程通過時鐘和佇列實現。
下面我們開始深入解剖。本篇文章，專門為邊看程式碼，邊學gstreamer的初級開發者瞭解基本概念而寫。
1. 元件(Elements)
元件(element)是GStreamer中最重要的概念。它是整個gstreamer框架中最重要的一個物件，它定義了pipeline的結構。你可以通過建立一系列的元件(Elements),並把它們連線起來,從而讓資料流在這個被連線的各個元件(Elements)之間傳輸。每個元件(Elements)都有一個特殊的函式介面,對於有些元件(Elements)的函式介面它們是用於能夠讀取檔案的資料,解碼檔案資料的。而有些元件(Elements)的函式介面只是輸出相應的資料到具體的裝置上(例如,音效卡裝置)。你可以將若干個元件(Elements)連線在一起,從而建立一個管道(pipeline)來完成一個特殊的任務,例如,媒體播放或者錄音。GStreamer已經預設安裝了很多有用的元件(Elements),通過使用這些元件(Elements)你能夠構建一個具有多種功能的應用程式。
每個元件包含有多個gstPad(襯墊)。這些gstpad隨後用於元件（element）之間的連線和通訊，並最終構成一個可以傳遞和處理資料的管道pipeline.幾乎所有的decoder,encoder,demuxer,video or audio output都實際上是一個GstElement。（ 注：面向物件裡的is-a 關係）
gstelement例項的結構如下所示
struct _GstElement
{
GstObject object;、
/*< public >*/ /* with LOCK */
GStaticRecMutex *state_lock; //用於序列化執行gst_element_set_state()
/* element state */
GCond *state_cond; //用於狀態變化完成時發射通知訊號
guint32 state_cookie; //用於檢測gst_element_set_state()和gst_element_get_state()的併發執行
GstState current_state; //元件當前所處狀態
GstState next_state; //元件的下一個狀態。如果正處於正確的狀態，該值可以為GST_STATE_VOID_PENDING
GstState pending_state; //元件的應趨向的最終狀態。如果正處於正確的狀態，該值可以為GST_STATE_VOID_PENDING
GstStateChangeReturn last_return; //元件狀態改變時，最後的那個返回值
GstBus *bus; //包含的GstBus,該匯流排由父元素或應用程式提供。Gstpipeline有自己的bus
/* allocated clock */
GstClock *clock; //時鐘，通常由頂級的gstpipeline管道提供
GstClockTimeDiff base_time; /* NULL/READY: 0 - PAUSED: current time - PLAYING: difference to clock*///element狀態剛變為playing時的時鐘時間，播放狀態下時鐘時間減去base_time可以獲得流時間（即播放時間，執行時間）
/* element pads, these lists can only be iteated while holding
* the LOCK or checking the cookie after each LOCK. */
guint16 numpads;//number of pads of the element, includes both source and sink pads.
GList *pads; //List of pad,襯墊列表（每個元件都包含有pad）, 一個襯墊(Pads)可以被看作是一個元件(element)插座或者埠,元件(element)之間的連結就是依靠著襯墊(Pads)
guint16 numsrcpads;
GList *srcpads;
guint16 numsinkpads;
GList *sinkpads;
guint32 pads_cookie;
/*< private >*/
union {
struct {
/* state set by application */
GstState target_state;
} ABI;
/* adding + 0 to mark ABI change to be undone later */
gpointer _gst_reserved[GST_PADDING + 0];
} abidata;
};
新增和刪除pad的方法在gstelement的類結構體中有定義，這裡從略。讀者可從網上下載程式碼閱讀。
2. 箱櫃(Bins)和管道(pipelines)
箱櫃(Bins)是一個可裝載一系列元件(element)的容器。管道(pipelines)是箱櫃(Bins)的一個特殊的子型別,管道(pipelines)可以操作包含在它自身內部的所有元件(element)。gstbin的例項結構體定義如下：
A bin is an element subclass and acts as a container for other elements so that multiple
elements can be combined into one element.
A bin can have its own source and sinkpads by ghostpadding one or more of its children's
pads to itself.
struct _GstBin {
GstElement element;
/*< public >*/ /* with LOCK */
/* our children, subclass are supposed to update these
* fields to reflect their state with _iterate_*() */
gint numchildren; //the number of children in this bin
GList *children;//the list of children in this bin
guint32 children_cookie;//updated whenever @children changes
GstBus *child_bus;// internal bus for handling child messages
GList *messages;//queued and cached messages
gboolean polling;//the bin is currently calculating its state
gboolean state_dirty;//the bin needs to recalculate its state (deprecated)
gboolean clock_dirty;//the bin needs to select a new clock
GstClock *provided_clock;//the last clock selected
GstElement *clock_provider;//the element that provided @provided_clock
/*< private >*/
GstBinPrivate *priv;/
gpointer _gst_reserved[GST_PADDING - 1];
};
我們再來分析下GstPipeline的例項結構：
struct _GstPipeline {
GstBin bin;
/*< public >*/ /* with LOCK */
GstClock *fixed_clock;
//The fixed clock of the pipeline, used when GST_PIPELINE_FLAG_FIXED_CLOCK is set.
GstClockTime stream_time; //The stream time of the pipeline. A better name for this property would be the running_time, the total time spent in the PLAYING state without being flushed. (deprecated, use the start_time on GstElement).注意到gstelement例項結構裡有一個base_time.它們是什麼關係呢？當前時間與base_time相減可以得到running-time(執行時間)。在播放速率一致的情況下，這個執行時間與stream_time是相等的！ element可以提供一個時鐘，管道pipeline可以選擇一個時鐘，所有的接收器sinks都要與之同步！
GstClockTime delay;//Extra delay added to base_time to compensate for computing delays when setting elements to PLAYING
/*< private >*/
GstPipelinePrivate *priv;
gpointer _gst_reserved[GST_PADDING-1];
};
因為箱櫃(Bins)本身又是元件(element)的子集,所以你能夠象操作普通元件(element)一樣的操作一個箱櫃(Bins),通過這種方法可以降低你的應用程式的複雜度。你可以改變一個箱櫃(Bins)的狀態來改變箱櫃(Bins)內部所有元件(element)的狀態。箱櫃(Bins)可以傳送匯流排訊息(bus messages)給它的子集元件(element)(這些訊息包括:錯誤訊息(error messages),標籤訊息(tag messages),EOS訊息(EOS messages))。
struct _GstPipelineClass {
GstBinClass parent_class;
/*< private >*/
gpointer _gst_reserved[GST_PADDING];
};
管道(pipeline)是高階的箱櫃(Bins)。當你設定管道的暫停或者播放狀態的時候,資料流將開始流動,並且媒體資料處理也開始處理。一旦開始,管道將在一個單獨的執行緒中執行,直到被停止或者資料流播放完畢。
3.襯墊(Pads)
襯墊在GStreamer中被用於多個元件的連結,從而讓資料流能在這樣的連結中流動。一個襯墊(Pads)可以被看作是一個元件(element)插座或者埠,元件(element)之間的連結就是依靠著襯墊(Pads)。襯墊(Pads)有處理特殊資料的能力:一個襯墊(Pads)能夠限制資料流型別的通過。連結成功的條件是：只有在兩個襯墊(Pads)允許通過的資料型別一致的時候才被建立。資料型別的設定使用了一個叫做caps negotiation的方法。資料型別被為一個GstCaps變數所描述。
struct _GstCaps {
GType type;
/*< public >*/ /* with COW */
/* refcounting */
gint refcount;
/*< public >*/ /* read only */
GstCapsFlags flags;
/*< private >*/
GPtrArray *structs;//
/*註釋，方便理解-----------struct _GPtrArray
{
gpointer *pdata;
guint len;
};*/
/*< private >*/
gpointer _gst_reserved[GST_PADDING];
};
下面的這個比喻可能對你理解襯墊(Pads)有所幫助。一個襯墊(Pads)很象一個物理裝置上的插頭。例如一個家庭影院系統。一個家庭影院系統由一個功放(amplifier),一個DVD機,還有一個無聲的視訊投影組成。我們需要連線DVD機到功放(amplifier),因為兩個裝置都有音訊插口;我們還需要連線投影機到DVD機上,因為兩個裝置都有視訊處理插口。但我們很難將投影機與功放(amplifier)連線起來,因為他們之間處理的是不同的插口。GStreamer襯墊(Pads)的作用跟家庭影院系統中的插口是一樣的。
對於大部分情況,所有的資料流都是在連結好的元素之間流動。資料向元件(element)以外流出可以通過一個或者多個 source襯墊(Pads),元件(element)接受資料是通過一個或者多個sink襯墊(Pads)來完成的。Source元件(element)和sink元件(element)分別有且僅有一個 sink 襯墊(Pads)或者source襯墊(Pads)。資料在這裡代表的是緩衝區(buffers) (GstBuffer物件描述了資料的緩衝區(buffers)的資訊)和事件(events) (GstEvent物件描述了資料的事件(events)資訊)。
/**
* GstPad:
* @element_private: private data owned by the parent element
* @padtemplate: padtemplate for this pad
* @direction: the direction of the pad, cannot change after creating
* the pad.
* @stream_rec_lock: recursive stream lock of the pad, used to protect
* the data used in streaming.
* @task: task for this pad if the pad is actively driving dataflow.
* @preroll_lock: lock used when prerolling
* @preroll_cond: conf to signal preroll
* @block_cond: conditional to signal pad block
* @block_callback: callback for the pad block if any
* @block_data: user data for @block_callback
* @caps: the current caps of the pad
* @getcapsfunc: function to get caps of the pad
* @setcapsfunc: function to set caps on the pad
* @acceptcapsfunc: function to check if pad can accept caps
* @fixatecapsfunc: function to fixate caps
* @activatefunc: pad activation function
* @activatepushfunc: function to activate/deactivate pad in push mode
* @activatepullfunc: function to activate/deactivate pad in pull mode
* @linkfunc: function called when pad is linked
* @unlinkfunc: function called when pad is unlinked
* @peer: the pad this pad is linked to
* @sched_private: private storage for the scheduler
* @chainfunc: function to chain data to pad
* @checkgetrangefunc: function to check if pad can operate in pull mode
* @getrangefunc: function to get a range of data from a pad
* @eventfunc: function to send an event to a pad
* @mode: current activation mode of the pad
* @querytypefunc: get list of supported queries
* @queryfunc: perform a query on the pad
* @intlinkfunc: get the internal links of this pad
* @bufferallocfunc: function to allocate a buffer for this pad
* @do_buffer_signals: counter counting installed buffer signals
* @do_event_signals: counter counting installed event signals
* @iterintlinkfunc: get the internal links iterator of this pad
*
* The #GstPad structure. Use the functions to update the variables.
*/