gstreamer外掛編寫教程1-Introduction
官網：https://gstreamer.freedesktop.org/
文件教程：https://gstreamer.freedesktop.org/documentation/plugin-development/index.html
目錄
GstMiniObject, Buffers and Events
GStreamer是建立流媒體應用程式的一個非常強大和通用的框架。GStreamer框架的許多優點來自於它的模組化:GStreamer可以無縫地整合新的外掛模組。但是，由於模組化和功能強大的代價往往是更高的複雜性(考慮一下，例如CORBA)，編寫新的外掛並不總是容易的。
本指南旨在幫助您理解GStreamer框架，以便開發新的外掛來擴充套件現有功能。教程通過一個示例外掛(C語言實現的一個音訊過濾器外掛)來闡述接下來的大部分問題.然而,指南在之後的部分介紹編寫其他型別的外掛,指南的結束位置描述瞭如果通過Python來繫結GStreamer。
GStreamer is an extremely powerful and versatile framework for creating streaming media applications. Many of the virtues of the GStreamer framework come from its modularity: GStreamer can seamlessly incorporate new plugin modules. But because modularity and power often come at a cost of greater complexity (consider, for example,
This guide is intended to help you understand the GStreamer framework so you can develop new plugins to extend the existing functionality. The guide addresses most issues by following the development of an example plugin - an audio filter plugin - written in C. However, the later parts of the guide also present some issues involved in writing other types of plugins, and the end of the guide describes some of the Python bindings for GStreamer.
Preface
What is GStreamer?
GStreamer是一個用於建立流媒體應用程式的框架。基礎設計來自俄勒岡研究生院的視訊管道，以及DirectShow的一些想法。
GStreamer的開發框架可以編寫任何型別的流式多媒體應用程式。GStreamer框架旨在簡化編寫處理音訊或視訊或兩者的應用程式。它不僅限於音訊和視訊，還可以處理任何型別的資料流。管道設計的開銷小於所應用的過濾器引起的開銷。這使得GStreamer成為設計高階音訊應用程式的良好框架，這些應用程式對延遲或效能提出了很高的要求。
GStreamer最明顯的用途之一是使用它來構建媒體播放器。GStreamer已經包含用於構建媒體播放器的元件，可以支援各種格式， 包括MP3，Ogg / Vorbis，MPEG-1/2，AVI，Quicktime，mod等。然而，GStreamer不僅僅是另一個媒體播放器。它的主要優點是可插拔元件可以混合並匹配到任意管道中，因此可以編寫完整的視訊或音訊編輯應用程式。
該框架基於提供各種編解碼器和其他功能的外掛。外掛可以連結並佈置在管道中。此管道定義資料流。
GStreamer核心功能是為外掛，資料流，同步和媒體型別處理/協商提供了一個框架。它還提供了使用各種外掛編寫應用程式的API。
GStreamer is a framework for creating streaming media applications. The fundamental design comes from the video pipeline at Oregon Graduate Institute, as well as some ideas from DirectShow.
GStreamer's development framework makes it possible to write any type of streaming multimedia application. The GStreamer framework is designed to make it easy to write applications that handle audio or video or both. It isn't restricted to audio and video, and can process any kind of data flow. The pipeline design is made to have little overhead above what the applied filters induce. This makes GStreamer a good framework for designing even high-end audio applications which put high demands on latency or performance.
One of the most obvious uses of GStreamer is using it to build a media player. GStreamer already includes components for building a media player that can support a very wide variety of formats, including MP3, Ogg/Vorbis, MPEG-1/2, AVI, Quicktime, mod, and more. GStreamer, however, is much more than just another media player. Its main advantages are that the pluggable components can be mixed and matched into arbitrary pipelines so that it's possible to write a full-fledged video or audio editing application.
The framework is based on plugins that will provide the various codec and other functionality. The plugins can be linked and arranged in a pipeline. This pipeline defines the flow of the data.
The GStreamer core function is to provide a framework for plugins, data flow, synchronization and media type handling/negotiation. It also provides an API to write applications using the various plugins.
Who Should Read This Guide?
本指南介紹瞭如何為GStreamer編寫新模組。該指南與幾組人群相關：
-
任何想要對GStreamer新增新方法的人。例如，該組中的某個人可能想要建立新的資料格式轉換器，新的視覺化工具或新的解碼器或編碼器。
-
任何想要新增對新輸入和輸出裝置的支援的人。例如，該組中的人可能希望新增寫入新視訊輸出系統或從數碼相機或特殊麥克風讀取資料的功能。
-
任何想要以任何方式擴充套件GStreamer的人。在理解外掛系統對其餘程式碼的約束之前，您需要了解外掛系統的工作原理。此外，閱讀本文後，您可能會對使用外掛可以完成的工作感到驚訝。
如果您只想使用GStreamer的現有功能，或者您只想使用GStreamer的應用程式，則本指南與您無關。如果您只對使用現有外掛編寫新應用程式感興趣 - 而且已經有相當多的外掛 - 您可能需要檢視GStreamer應用程式開發手冊。如果您只是想獲得GStreamer應用程式的幫助，那麼您應該檢視該特定應用程式的使用者手冊。
This guide explains how to write new modules for GStreamer. The guide is relevant to several groups of people:
-
Anyone who wants to add support for new ways of processing data in GStreamer. For example, a person in this group might want to create a new data format converter, a new visualization tool, or a new decoder or encoder.
-
Anyone who wants to add support for new input and output devices. For example, people in this group might want to add the ability to write to a new video output system or read data from a digital camera or special microphone.
-
Anyone who wants to extend GStreamer in any way. You need to have an understanding of how the plugin system works before you can understand the constraints that the plugin system places on the rest of the code. Also, you might be surprised after reading this at how much can be done with plugins.
This guide is not relevant to you if you only want to use the existing functionality of GStreamer, or if you just want to use an application that uses GStreamer. If you are only interested in using existing plugins to write a new application - and there are quite a lot of plugins already - you might want to check the GStreamer Application Development Manual. If you are just trying to get help with a GStreamer application, then you should check with the user manual for that particular application.
Preliminary Reading
本指南假設您對GStreamer的基本工作方式有所瞭解。有關GStreamer中程式設計概念的簡要介紹，您可能應該首先閱讀GStreamer應用程式開發手冊。另請檢視GStreamer網站上提供的其他文件。
為了理解本手冊，您需要對C語言有基本的瞭解。由於GStreamer遵循GObject程式設計模型，本指南還假設您瞭解GObject 程式設計的基礎知識。您還可以參考Eric Harlow寫的書的書“ 使用GTK +和GDK開發Linux應用程式”。
This guide assumes that you are somewhat familiar with the basic workings of GStreamer. For a gentle introduction to programming concepts in GStreamer, you may wish to read the GStreamer Application Development Manual first. Also check out the other documentation available on the GStreamer web site.
In order to understand this manual, you will need to have a basic understanding of the C language. Since GStreamer adheres to the GObject programming model, this guide also assumes that you understand the basics of GObject programming. You may also want to have a look at Eric Harlow's book Developing Linux Applications with GTK+ and GDK.
Structure of This Guide
為了幫助您瀏覽本指南，它分為幾個大部分。每個部分都涉及有關GStreamer外掛開發的特定主題。本指南的各部分按以下順序排列：
To help you navigate through this guide, it is divided into several large parts. Each part addresses a particular broad topic concerning GStreamer plugin development. The parts of this guide are laid out in the following order:
本部分介紹了構建外掛通常需要執行的所有基本步驟，例如使用GStreamer註冊元素以及設定基礎知識，以便它可以從鄰居元素接收資料並將資料傳送到相鄰元素。討論首先給出了生成基本結構和在Constructing the Boilerplate中註冊元素 的示例。然後，您將學習如何編寫程式碼以獲得一個基本的過濾器外掛，這些外掛工作在特定pads，鏈函式 和那些狀態。
之後，我們將展示一些關於如何為應用程式配置元素以及如何在新增屬性和訊號中進行應用程式元素互動的GObject概念。接下來，您將學習如何構建一個快速測試應用程式來測試您在構建測試應用程式中學到的所有知識。我們將在這裡介紹基礎知識。對於成熟的應用程式開發，您應該檢視“應用程式開發手冊”。
-
Building a Plugin - Introduction to the structure of a plugin, using an example audio filter for illustration.
This part covers all the basic steps you generally need to perform to build a plugin, such as registering the element with GStreamer and setting up the basics so it can receive data from and send data to neighbour elements. The discussion begins by giving examples of generating the basic structures and registering an element in Constructing the Boilerplate. Then, you will learn how to write the code to get a basic filter plugin working in Specifying the pads, The chain function and What are states?.
After that, we will show some of the GObject concepts on how to make an element configurable for applications and how to do application-element interaction in Adding Properties and Signals. Next, you will learn to build a quick test application to test all that you've just learned in Building a Test Application. We will just touch upon basics here. For full-blown application development, you should look at the Application Development Manual.
-
高階過濾器概念 - 有關GStreamer外掛開發的高階功能的資訊。
在瞭解了基本步驟之後，您應該能夠建立具有一些不錯功能的功能音訊或視訊過濾器外掛。但是，GStreamer為外掛編寫者提供了更多功能。本指南的這一部分包括有關更高階主題的章節，例如排程，GStreamer中的媒體型別定義，時鐘，介面和標記。由於這些功能是針對特定目的的，因此您可以按任何順序閱讀它們，其中大多數不需要其他部分的知識。
第一章，名為不同的排程模式，解釋元素排程的一些基礎知識。它不是很深入，但主要是介紹其他事情為何如此起作用。如果您對GStreamer內部感興趣，請閱讀本章。接下來，我們將應用這些知識並討論另一種型別的資料傳輸不同的排程模式（而不是您在鏈函式中學到的）。基於迴圈的元素將使您更好地控制輸入速率。這在編寫複用器或解複用器時非常有用。
接下來，我們將在媒體型別和屬性中討論GStreamer中的媒體識別。您將學習如何定義新媒體型別並瞭解GStreamer中定義的標準媒體型別列表。
在下一章中，您將學習request pads和sometimes pads的概念，這些墊是動態建立的墊，因為應用程式要求它（請求）或因為媒體流需要它（有時）。
下一章Clocking將解釋GStreamer中時鐘的概念。當您想知道元素應該如何實現音訊/視訊同步時，您需要此資訊。
接下來的幾章將討論進行應用程式元素互動的高階方法。以前，我們在GObject中瞭解了新增屬性和 訊號的方法。我們將在支援動態引數中討論動態引數，這是一種預先定義元素行為的方法。接下來，您將瞭解Interfaces中的介面。基於GObject的GInterface，介面是應用程式元素互動的特定於目標的方式。最後，您將瞭解在GStreamer中元資料如何在被處理在標記（元資料和Streaminfo）中。
最後一章，活動：尋求，導航等，將討論GStreamer中事件的概念。事件是進行應用程式元素互動的另一種方式。例如，他們負責尋求。它們也是元素彼此互動的另一種方式，例如讓對方瞭解媒體流不連續性，在管道內轉發標籤等等。
-
Advanced Filter Concepts - Information on advanced features of GStreamer plugin development.
After learning about the basic steps, you should be able to create a functional audio or video filter plugin with some nice features. However, GStreamer offers more for plugin writers. This part of the guide includes chapters on more advanced topics, such as scheduling, media type definitions in GStreamer, clocks, interfaces and tagging. Since these features are purpose-specific, you can read them in any order, most of them don't require knowledge from other sections.
The first chapter, named Different scheduling modes, will explain some of the basics of element scheduling. It is not very in-depth, but is mostly some sort of an introduction on why other things work as they do. Read this chapter if you're interested in GStreamer internals. Next, we will apply this knowledge and discuss another type of data transmission than what you learned in The chain function: Different scheduling modes. Loop-based elements will give you more control over input rate. This is useful when writing, for example, muxers or demuxers.
Next, we will discuss media identification in GStreamer in Media Types and Properties. You will learn how to define new media types and get to know a list of standard media types defined in GStreamer.
In the next chapter, you will learn the concept of request- and sometimes-pads, which are pads that are created dynamically, either because the application asked for it (request) or because the media stream requires it (sometimes). This will be in Request and Sometimes pads.
The next chapter, Clocking, will explain the concept of clocks in GStreamer. You need this information when you want to know how elements should achieve audio/video synchronization.
The next few chapters will discuss advanced ways of doing application-element interaction. Previously, we learned on the GObject-ways of doing this in Adding Properties and Signals. We will discuss dynamic parameters, which are a way of defining element behaviour over time in advance, in Supporting Dynamic Parameters. Next, you will learn about interfaces in Interfaces. Interfaces are very target- specific ways of application-element interaction, based on GObject's GInterface. Lastly, you will learn about how metadata is handled in GStreamer in Tagging (Metadata and Streaminfo).
The last chapter, Events: Seeking, Navigation and More, will discuss the concept of events in GStreamer. Events are another way of doing application-element interaction. They take care of seeking, for example. They are also yet another a way in which elements interact with each other, such as letting each other know about media stream discontinuities, forwarding tags inside a pipeline and so on.
-
建立特殊元素型別 - 編寫其他外掛型別的說明。
由於本指南的前兩部分使用音訊過濾器作為示例，因此引入的概念適用於過濾器外掛。但是許多概念同樣適用於其他外掛型別，包括源，接收器和自動外掛。本指南的這一部分介紹了處理這些更專業的外掛型別時出現的問題。本章首先重點介紹可以使用基類（預製基類）編寫的元素，然後在寫入解複用器或解析器時編寫特殊型別的元素，編寫N對1元素或複用器 和編寫管理器。
-
Creating special element types - Explanation of writing other plugin types.
Because the first two parts of the guide use an audio filter as an example, the concepts introduced apply to filter plugins. But many of the concepts apply equally to other plugin types, including sources, sinks, and autopluggers. This part of the guide presents the issues that arise when working on these more specialized plugin types. The chapter starts with a special focus on elements that can be written using a base-class (Pre-made base classes), and later also goes into writing special types of elements in Writing a Demuxer or Parser, Writing a N-to-1 Element or Muxer and Writing a Manager.
-
附錄 - 外掛開發人員的更多資訊。這些附錄包含一些頑固地拒絕在指南的其他部分中乾淨利落的資訊。本節的大部分內容尚未完成。
-
Appendices - Further information for plugin developers.
The appendices contain some information that stubbornly refuses to fit cleanly in other sections of the guide. Most of this section is not yet finished.
本指南的前言部分的其餘部分簡要概述了GStreamer外掛開發中涉及的基本概念。涵蓋的主題包括元素和外掛， Pads，GstMiniObject，緩衝區和事件 以及媒體型別和屬性。如果您已經熟悉此資訊，可以使用此簡短概述來重新整理記憶體，或者跳轉到構建外掛。
如您所見，有很多東西需要學習，所以讓我們開始吧！
-
通過從GstBin擴充套件來建立複合元素和複雜元素。這將允許您建立插入其他外掛的外掛。
-
將新媒體型別與typedetect函式一起新增到登錄檔中。這將允許您的外掛在全新的媒體型別上執行。
The remainder of this introductory part of the guide presents a short overview of the basic concepts involved in GStreamer plugin development. Topics covered include Elements and Plugins, Pads, GstMiniObject, Buffers and Events and Media types and Properties. If you are already familiar with this information, you can use this short overview to refresh your memory, or you can skip to Building a Plugin.
As you can see, there a lot to learn, so let's get started!
-
Creating compound and complex elements by extending from a GstBin. This will allow you to create plugins that have other plugins embedded in them.
-
Adding new media types to the registry along with typedetect functions. This will allow your plugin to operate on a completely new media type.
Foundations
本指南的這一章介紹了GStreamer的基本概念。理解這些概念將有助於您解決擴充套件GStreamer所涉及的問題。許多這些概念在GStreamer應用程式開發手冊中有更詳細的解釋; 這裡介紹的基本概念主要用於重新整理你的記憶。
This chapter of the guide introduces the basic concepts of GStreamer. Understanding these concepts will help you grok the issues involved in extending GStreamer. Many of these concepts are explained in greater detail in the GStreamer Application Development Manual; the basic concepts presented here serve mainly to refresh your memory.
Elements and Plugins
元素是GStreamer的核心。在外掛開發的上下文中，元素是從GstElement類派生的物件。元素在與其他元素連結時提供某種功能：例如，源元素向流提供資料，過濾元素作用於流中的資料。沒有元素，GStreamer只是一堆沒有任何連結的概念管道配件。GStreamer附帶了大量元素，但也可以編寫額外的元素。
但是，僅僅編寫新元素是不行的，您還需要將元素封裝在外掛中以使GStreamer能夠使用它。外掛本質上是一個可載入的程式碼塊，通常稱為共享物件檔案或動態連結庫。單個外掛可能包含多個元素的實現，或者只包含一個元素。為簡單起見，本指南主要關注包含一個元素的外掛。
濾波器是元件的一個重要型別處理資料的流。資料的生產者和消費者分別稱為源和宿 元素。Bin元素包含其他元素。一種型別的bin負責同步它們包含的元素，以便資料流暢地流動。另一種型別的bin，稱為 autoplugger元素，自動將其他元素新增到bin並將它們連結在一起，以便它們充當兩個任意流型別之間的過濾器。
即使只使用標準軟體包，外掛機制也可在GStreamer中隨處使用。一些非常基本的函式駐留在核心庫中，所有其他函式都在外掛中實現。外掛登錄檔用於在二進位制登錄檔檔案中儲存外掛的詳細資訊。這樣，使用GStreamer的程式不必載入所有外掛來確定所需的外掛。只有在元素請求的時候才會載入外掛。
關於GStreamer更詳細資訊參考參考GstElement和 GstPlugin。
Elements are at the core of GStreamer. In the context of plugin development, an element is an object derived from the GstElement class. Elements provide some sort of functionality when linked with other elements: For example, a source element provides data to a stream, and a filter element acts on the data in a stream. Without elements, GStreamer is just a bunch of conceptual pipe fittings with nothing to link. A large number of elements ship with GStreamer, but extra elements can also be written.
Just writing a new element is not entirely enough, however: You will need to encapsulate your element in a pluginto enable GStreamer to use it. A plugin is essentially a loadable block of code, usually called a shared object file or a dynamically linked library. A single plugin may contain the implementation of several elements, or just a single one. For simplicity, this guide concentrates primarily on plugins containing one element.
A filter is an important type of element that processes a stream of data. Producers and consumers of data are called source and sink elements, respectively. Bin elements contain other elements. One type of bin is responsible for synchronization of the elements that they contain so that data flows smoothly. Another type of bin, calledautoplugger elements, automatically add other elements to the bin and links them together so that they act as a filter between two arbitrary stream types.
The plugin mechanism is used everywhere in GStreamer, even if only the standard packages are being used. A few very basic functions reside in the core library, and all others are implemented in plugins. A plugin registry is used to store the details of the plugins in an binary registry file. This way, a program using GStreamer does not have to load all plugins to determine which are needed. Plugins are only loaded when their provided elements are requested.
See the GStreamer Library Reference for the current implementation details of GstElement and GstPlugin.
Pads
Pads用於協商GStreamer中元素之間的連結和資料流。可以將墊視為元素上的“位置”或“埠”，其中可以與其他元素建立連結，並且資料可以通過該元素流入或流出這些元素。Pads具有特定的資料處理功能：pad可以限制流經它的資料型別。當兩個pad的允許資料型別相容時，才允許在兩個pad之間建立連結。
這裡的比喻可能會有所幫助。pad類似於物理裝置上的插頭或插孔。例如，考慮由放大器，DVD播放器和（靜音）視訊投影儀組成的家庭影院系統。允許將DVD播放器連線到放大器，因為兩個裝置都有音訊插孔，並且允許將投影儀連線到DVD播放器，因為兩個裝置都具有相容的視訊插孔。由於投影儀和放大器具有不同型別的插孔，因此可能無法建立投影儀和放大器之間的連結。GStreamer中的墊子與家庭影院系統中的插孔具有相同的用途。
在大多數情況下，GStreamer中的所有資料都通過元素之間的連結流動。資料通過一個或多個source pad流出，通過一個或多個sink pad接收資料。source element僅僅含有src pad，sink element 僅僅含有sink pad。
請參閱GStreamer庫參考GstPad以獲得關於這些的詳細資訊，
Pads are used to negotiate links and data flow between elements in GStreamer. A pad can be viewed as a “place” or “port” on an element where links may be made with other elements, and through which data can flow to or from those elements. Pads have specific data handling capabilities: A pad can restrict the type of data that flows through it. Links are only allowed between two pads when the allowed data types of the two pads are compatible.
An analogy may be helpful here. A pad is similar to a plug or jack on a physical device. Consider, for example, a home theater system consisting of an amplifier, a DVD player, and a (silent) video projector. Linking the DVD player to the amplifier is allowed because both devices have audio jacks, and linking the projector to the DVD player is allowed because both devices have compatible video jacks. Links between the projector and the amplifier may not be made because the projector and amplifier have different types of jacks. Pads in GStreamer serve the same purpose as the jacks in the home theater system.
For the most part, all data in GStreamer flows one way through a link between elements. Data flows out of one element through one or more source pads, and elements accept incoming data through one or more sink pads. Source and sink elements have only source and sink pads, respectively.
See the GStreamer Library Reference for the current implementation details of a GstPad.
GstMiniObject, Buffers and Events
GStreamer中的所有資料流都被切割成塊，這些塊從一個元素上的src pad傳遞到另一個元素上的sink pad。GstMiniObject是用於儲存這些資料塊的結構。
GstMiniObject包含以下重要型別：
-
一個確切的型別，指示此GstMiniObject是什麼型別的資料（事件，緩衝區......）。
-
引用計數，指示當前持有對miniobject的引用的元素數。當引用計數降至零時，此物件會被處理，並且在某種意義上將釋放其儲存器（有關詳細資訊，請參見下文）。
對於資料傳輸，定義了兩種型別的GstMiniObject：事件（控制）和緩衝區（內容）。
緩衝區可能包含兩個連結的打擊墊知道如何處理的任何型別的資料。通常，緩衝區包含從一個元素流向另一個元素的某種音訊或視訊資料。
緩衝區還包含描述緩衝區內容的元資料。一些重要的元資料型別是：
-
指向一個或多個GstMemory物件的指標。GstMemory物件是封裝記憶體區域的refcounted物件。
-
一個時間戳，指示緩衝區中內容的首選顯示時間戳。
事件包含有關在兩個連結的pads之間流動的流的狀態的資訊。僅當元素明確支援它們時才會傳送事件，否則核心將（嘗試）自動處理事件。事件用於指示例如媒體型別，媒體流的結尾或者應該重新整理快取記憶體。
事件可能包含以下幾項：
-
指示所包含事件型別的子型別。
-
事件的其他內容取決於特定的事件型別。
活動將在活動中廣泛討論：尋求，導航等。在此之前，將使用的唯一事件是EOS事件，該事件用於指示流末尾（通常是檔案結束）。
更詳細的內容參考GStreamer庫索引 GstMiniObject， GstBuffer和 GstEvent。
All streams of data in GStreamer are chopped up into chunks that are passed from a source pad on one element to a sink pad on another element. GstMiniObject is the structure used to hold these chunks of data.
GstMiniObject contains the following important types:
-
An exact type indicating what type of data (event, buffer, ...) this GstMiniObject is.
-
A reference count indicating the number of elements currently holding a reference to the miniobject. When the reference count falls to zero, the miniobject will be disposed, and its memory will be freed in some sense (see below for more details).
For data transport, there are two types of GstMiniObject defined: events (control) and buffers (content).
Buffers may contain any sort of data that the two linked pads know how to handle. Normally, a buffer contains a chunk of some sort of audio or video data that flows from one element to another.
Buffers also contain metadata describing the buffer's contents. Some of the important types of metadata are:
-
Pointers to one or more GstMemory objects. GstMemory objects are refcounted objects that encapsulate a region of memory.
-
A timestamp indicating the preferred display timestamp of the content in the buffer.
Events contain information on the state of the stream flowing between the two linked pads. Events will only be sent if the element explicitly supports them, else the core will (try to) handle the events automatically. Events are used to indicate, for example, a media type, the end of a media stream or that the cache should be flushed.
Events may contain several of the following items:
-
A subtype indicating the type of the contained event.
-
The other contents of the event depend on the specific event type.
Events will be discussed extensively in Events: Seeking, Navigation and More. Until then, the only event that will be used is the EOS event, which is used to indicate the end-of-stream (usually end-of-file).
See the GStreamer Library Reference for the current implementation details of a GstMiniObject, GstBuffer andGstEvent.
Buffer Allocation
緩衝區能夠儲存幾種不同型別的記憶體塊。最通用的緩衝區包含malloc（）分配的記憶體。這些緩衝區雖然方便，但並不總是非常快，因為資料通常需要專門複製到緩衝區中。
許多專用元素建立指向特殊記憶體的緩衝區。例如，file src元素通常將檔案對映到應用程式的地址空間（使用mmap（）），並建立指向該地址範圍的緩衝區。file src建立的這些緩衝區與通用緩衝區完全相同，只是它們是隻讀的。緩衝區釋放程式碼自動確定釋放底層記憶體的正確方法。接收這些緩衝區的下游元素不需要做任何特殊的處理或不引用它。
元素可能獲得專用緩衝區的另一種方法是通過GstBufferPool或GstAllocator從下游對等方請求它們。元素可以從下游對等元素中請求GstBufferPool或GstAllocator。如果下游能夠提供這些物件，則上游可以使用它們來分配緩衝區。檢視更多的記憶體分配。
許多接收器元件具有將資料複製到硬體或直接訪問硬體的加速方法。這些元素通常能夠為其上游對等體建立GstBufferPool或GstAllocator。一個這樣的例子是ximage sink。它建立包含XImages的緩衝區。因此，當上遊對等體將資料複製到緩衝區時，它會直接複製到XImage中，使ximage sink能夠將影象直接繪製到螢幕上，而不必先將資料複製到XImage中。
過濾元素通常有機會在就地處理緩衝區，或者在從源緩衝區複製到目標緩衝區時工作。實現這兩種演算法是最佳的，因為GStreamer框架可以根據需要選擇最快的演算法。當然，這隻適用於嚴格的過濾器 - 在源和接收器墊上具有完全相同格式的元素。
Buffers are able to store chunks of memory of several different types. The most generic type of buffer contains memory allocated by malloc(). Such buffers, although convenient, are not always very fast, since data often needs to be specifically copied into the buffer.
Many specialized elements create buffers that point to special memory. For example, the filesrc element usually maps a file into the address space of the application (using mmap()), and creates buffers that point into that address range. These buffers created by filesrc act exactly like generic buffers, except that they are read-only. The buffer freeing code automatically determines the correct method of freeing the underlying memory. Downstream elements that receive these kinds of buffers do not need to do anything special to handle or unreference it.
Another way an element might get specialized buffers is to request them from a downstream peer through a GstBufferPool or GstAllocator. Elements can ask a GstBufferPool or GstAllocator from the downstream peer element. If downstream is able to provide these objects, upstream can use them to allocate buffers. See more in Memory allocation.
Many sink elements have accelerated methods for copying data to hardware, or have direct access to hardware. It is common for these elements to be able to create a GstBufferPool or GstAllocator for their upstream peers. One such example is ximagesink. It creates buffers that contain XImages. Thus, when an upstream peer copies data into the buffer, it is copying directly into the XImage, enabling ximagesink to draw the image directly to the screen instead of having to copy data into an XImage first.
Filter elements often have the opportunity to either work on a buffer in-place, or work while copying from a source buffer to a destination buffer. It is optimal to implement both algorithms, since the GStreamer framework can choose the fastest algorithm as appropriate. Naturally, this only makes sense for strict filters -- elements that have exactly the same format on source and sink pads.
Media types and Properties
GStreamer使用型別系統來確保元素之間傳遞的資料採用可識別的格式。型別系統對於確保在連結元素之間的cap時完全指定格式所需的引數正確匹配也很重要。元素之間建立的每個連結都具有指定的型別和可選的一組屬性。檢視更多關於caps協商上限的談判。
GStreamer uses a type system to ensure that the data passed between elements is in a recognized format. The type system is also important for ensuring that the parameters required to fully specify a format match up correctly when linking pads between elements. Each link that is made between elements has a specified type and optionally a set of properties. See more about caps negotiation in Caps negotiation.
The Basic Types
GStreamer已經支援許多基本媒體型別。以下是GStreamer中用於緩衝區的一些基本型別的表格。該表包含名稱（“媒體型別”）以及型別的描述，與型別關聯的屬性以及每個屬性的含義。已定義型別列表中包含受支援型別的完整列表。
GStreamer already supports many basic media types. Following is a table of a few of the basic types used for buffers in GStreamer. The table contains the name ("media type") and a description of the type, the properties associated with the type, and the meaning of each property. A full list of supported types is included in List of Defined Types.
| Media Type | Description | Property | Property Type | Property Values | Property Description |
|---|---|---|---|---|---|
| audio/* | All audio types | rate | integer | greater than 0 | The sample rate of the data, in samples (per channel) per second. |
| channels | integer | greater than 0 | The number of channels of audio data. | ||
| audio/x-raw | Unstructured and uncompressed raw integer audio data. | format | string | S8 U8 S16LE S16BE U16LE U16BE S24_32LE S24_32BE U24_32LE U24_32BE S32LE S32BE U32LE U32BE S24LE S24BE U24LE U24BE S20LE S20BE U20LE U20BE S18LE S18BE U18LE U18BE F32LE F32BE F64LE F64BE | The format of the sample data. |
| audio/mpeg | Audio data compressed using the MPEG audio encoding scheme. | mpegversion | integer | 1, 2 or 4 | The MPEG-version used for encoding the data. The value 1 refers to MPEG-1, -2 and -2.5 layer 1, 2 or 3. The values 2 and 4 refer to the MPEG-AAC audio encoding schemes. |
| framed | boolean | 0 or 1 | A true value indicates that each buffer contains exactly one frame. A false value indicates that frames and buffers do not necessarily match up. | ||
| layer | integer | 1, 2, or 3 | The compression scheme layer used to compress the data (only if mpegversion=1). | ||
| bitrate | integer | greater than 0 | The bitrate, in bits per second. For VBR (variable bitrate) MPEG data, this is the average bitrate. | ||
| audio/x-vorbis | Vorbis audio data | There are currently no specific properties defined for this type. |