<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>OpenMAX AL for Android</title>
</head>
<body>
<b>Note:</b>
<a href="http://developer.android.com/reference/android/media/MediaCodec.html">android.media.MediaCodec</a>
is recommended for new applications at API level 16 and above. The NDK
equivalent (in <media/NdkMedia*.h>) is recommended for new native
applications at API level 21 and above.
<h1>OpenMAX AL for Android</h1>
This article describes the Android native multimedia APIs based on the
Khronos Group OpenMAX AL™ 1.0.1 standard, as of Android API level 14 (Android
platform version 4.0) and higher.
<p>
OpenMAX AL is a companion API to OpenSL ES, but for multimedia (video
and audio) rather than audio only.
<p>
Android 4.0 provides a direct, efficient path for low-level streaming multimedia. The path is
ideal for applications that need to maintain complete control over media data before passing it to
the platform for presentation. For example, media applications can retrieve data from any
source, apply proprietary encryption/decryption, and then send the data to the platform for display.
<p>
Applications can send processed data to the platform as a multiplexed stream of audio/video
content in MPEG-2 transport stream format. The platform de-muxes, decodes, and renders the content.
The audio track is rendered to the active audio device, while the video track is rendered to either
a Surface or a SurfaceTexture. When rendering to a SurfaceTexture, the application can apply
subsequent graphics effects to each frame using OpenGL.
<p>
OpenMAX AL provides a C language interface that is also callable from C++, and
exposes features similar to these Android APIs
callable from Java programming language code:
<ul>
<li><a href="http://developer.android.com/reference/android/media/MediaPlayer.html">
android.media.MediaPlayer</a>
</ul>
As with all of the Android Native Development Kit (NDK), the primary
purpose of OpenMAX AL for Android is to facilitate the implementation
of shared libraries to be called from Java programming language code via Java Native
Interface (JNI). NDK is not intended for writing pure C/C++
applications.
<p>
Note: though based on OpenMAX AL, the Android native multimedia API
is <i>not</i> a conforming implementation of either OpenMAX AL 1.0.1
profile (media player or media player / recorder). This is because Android does not
implement all of the features required by either of the profiles.
Any known cases where Android behaves differently than the specification
are described in section "Android extensions" below.
The Android OpenMAX AL implementation has limited features, and is
intended primarily for certain performance-sensitive native streaming
multimedia applications such as video players.
<p>
The major feature is the ability to play an MPEG-2 transport stream
containing a single program stream made up of one H.264 video elementary
stream and one AAC audio elementary stream. The application provides
the stream via an Android buffer queue data source, which is based on
the OpenSL ES buffer queue concept and Android-specific extensions.
<p>
The video sink is an <code>ANativeWindow *</code> abstract handle,
derived from an <code>android.view.Surface</code> ("surface").
A Surface from <code>SurfaceHolder.getSurface()</code> should be used when displaying
an unaltered video within a fixed SurfaceView frame. A Surface from
<code>new Surface(SurfaceTexture)</code> allows streaming the decoded
video frames to an OpenGL ES 2.0 texture, where the frames can be used
as input to a shader algorithm in the Graphics Processing Unit (GPU).
Be sure to <code>release()</code> the Surface as soon as possible after
calling <code>setSurface</code> or ANativeWindow_fromSurface.
<p>
The audio sink is always an output mix, a device-independent mixer object
similar to that of OpenSL ES.
<h2>Getting started</h2>
<h3>Example code</h3>
<h4>Recommended</h4>
Supported and tested example code, usable as a model
for your own code, is located in NDK folder
<code>platforms/android-14/samples/native-media/</code>.
<h4>Not recommended</h4>
The OpenMAX AL 1.0.1 specification contains example code in the
appendices (see section "References" below for the link to this
specification). However, the examples in Appendix D: Sample Code
use features
not supported by Android. Some examples also contain
typographical errors, or use APIs that are likely to change.
Proceed with caution in referring to these;
though the code may be helpful in understanding the full OpenMAX AL
standard, it should not be used as is with Android.
<h3>Adding OpenMAX AL to your application source code</h3>
OpenMAX AL is a C API, but is callable from both C and C++ code.
<p>
Add the following lines to your code:
<pre>
#include <OMXAL/OpenMAXAL.h>
#include <OMXAL/OpenMAXAL_Android.h>
</pre>
<h3>Makefile</h3>
Modify your Android.mk as follows:
<pre>
LOCAL_LDLIBS += libOpenMAXAL
</pre>
<h3>Multimedia content</h3>
The only supported way to supply multimedia content is via an MPEG-2
transport stream.
<p>
Finding or creating useful multimedia content for your application is
beyond the scope of this article.
<p>
Note that it is your responsibility to ensure that you are legally
permitted to play the content.
<h3>Debugging</h3>
For robustness, we recommend that you examine the <code>XAresult</code>
value which is returned by most APIs. Use of <code>assert</code>
vs. more advanced error handling logic is a matter of coding style
and the particular API; see the Wikipedia article on
<a href="http://en.wikipedia.org/wiki/Assertion_(computing)">assert</a>
for more information. In the supplied example, we have used <code>assert</code>
for "impossible" conditions which would indicate a coding error, and
explicit error handling for others which are more likely to occur
in production.
<p>
Many API errors result in a log entry, in addition to the non-zero
result code. These log entries provide additional detail which can
be especially useful for the more complex APIs such as
<code>Engine::CreateMediaPlayer</code>.
<p>
Use <a href="http://developer.android.com/guide/developing/tools/adb.html">
adb logcat</a>, the
<a href="http://developer.android.com/guide/developing/eclipse-adt.html">
Eclipse ADT plugin</a> LogCat pane, or
<a href="http://developer.android.com/guide/developing/tools/ddms.html#logcat">
ddms logcat</a> to see the log.
<h2>Supported features from OpenMAX AL 1.0.1</h2>
This section summarizes available features. In some
cases, there are limitations which are described in the next
sub-section.
<h3>Global entry points</h3>
Supported global entry points:
<ul>
<li><code>xaCreateEngine</code>
<li><code>xaQueryNumSupportedEngineInterfaces</code>
<li><code>xaQuerySupportedEngineInterfaces</code>
</ul>
<h3>Objects and interfaces</h3>
The following figure indicates objects and interfaces supported by
Android's OpenMAX AL implementation. A green cell means the feature
is supported.
<p>
<img src="chart3.png" alt="Supported objects and interfaces">
<h3>Limitations</h3>
This section details limitations with respect to the supported
objects and interfaces from the previous section.
<h4>Audio</h4>
The audio stream type cannot be configured; it is always <code>AudioManager.STREAM_MUSIC</code>.
<p>
Effects are not supported.
<h4>Dynamic interface management</h4>
<code>RemoveInterface</code> and <code>ResumeInterface</code> are not supported.
<h4>Engine</h4>
Supported:
<ul>
<li><code>CreateMediaPlayer</code>
</ul>
Not supported:
<ul>
<li><code>CreateCameraDevice</code>
<li><code>CreateRadioDevice</code>
<li><code>CreateLEDDevice</code>
<li><code>CreateVibraDevice</code>
<li><code>CreateMetadataExtractor</code>
<li><code>CreateExtensionObject</code>
<li><code>GetImplementationInfo</code>
</ul>
For <code>CreateMediaPlayer</code>, these restrictions apply:
<ul>
<li>audio sink is an output mix data locator
<li>video sink is a native display data locator
<li>soundbank, LED array, and vibra sinks must be <code>NULL</code>
</ul>
<h4>MIME data format</h4>
In the current Android implementation of OpenMAX AL, a media player
receives its source data as an MPEG-2 transport stream via a
buffer queue.
<p>
The source data locator must be <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code>
(see "Android extensions" below).
<p>
The source data format must be <code>XADataFormat_MIME</code>.
Initialize <code>mimeType</code> to <code>XA_ANDROID_MIME_MP2TS</code>,
and <code>containerType</code> to <code>XA_CONTAINERTYPE_MPEG_TS</code>.
<p>
The contained transport stream must have a single program with one H.264
video elementary stream and one AAC audio elementary stream.
<h4>Object</h4>
<code>Resume</code>, <code>RegisterCallback</code>,
<code>AbortAsyncOperation</code>, <code>SetPriority</code>,
<code>GetPriority</code>, and <code>SetLossOfControlInterfaces</code>
are not supported.
<h4>StreamInformation</h4>
Use the StreamInformation interface on a media player object to discover
when the video metrics (height/width or aspect ratio) are available or
changed, and to then get the sizes.
<p>
Supported:
<ul>
<li><code>RegisterStreamChangeCallback</code>
<li><code>QueryMediaContainerInformation</code>
<li><code>QueryStreamInformation</code>
<li><code>QueryStreamType</code>
</ul>
Not supported:
<ul>
<li><code>QueryActiveStreams</code>
<li><code>QueryStreamName</code>
<li><code>SetActiveStream</code>
</ul>
<h4>VideoDecoderCapabilities</h4>
This interface on the engine object reports video decoder capabilities
without interpretation, exactly as claimed by the underlying OpenMAX IL
implementation.
<p>
These fields in <code>XAVideoCodecDescriptor</code> are filled:
<ul>
<li><code>codecId</code>
<li><code>profileSetting</code>
<li><code>levelSetting</code>
</ul>
The other fields are not filled and should be ignored.
<p>
Applications should rely on the capabilities documented at
<a href="http://developer.android.com/guide/appendix/media-formats.html">Android Supported Media Formats</a>,
not the information reported by this interface.
<h3>Data structures</h3>
Android API level 14 supports these OpenMAX AL 1.0.1 data structures:
<ul>
<li>XADataFormat_MIME
<li>XADataLocator_NativeDisplay
<li>XADataLocator_OutputMix
<li>XADataSink
<li>XADataSource
<li>XAEngineOption
<li>XAInterfaceID
<li>XAMediaContainerInformation
<li>XANativeHandle
<li>XA*StreamInformation
<li>XAVideoCodecDescriptor
</ul>
<h4>XADataLocator_NativeDisplay</h4>
The native display data locator is used to specify the video sink:
<pre>
typedef struct XADataLocator_NativeDisplay_ {
XAuint32 locatorType; // XA_DATALOCATOR_NATIVEDISPLAY
XANativeHandle hWindow; // ANativeWindow *
XANativeHandle hDisplay; // NULL
} XADataLocator_NativeDisplay;
</pre>
Set the <code>hWindow</code> field to an
<code>ANativeWindow *</code> and set <code>hDisplay</code> to <code>NULL</code>.
You can get a <code>ANativeWindow *</code> handle from an <code>android.view.Surface</code>,
using this NDK function:
<pre>
#include <android/native_window_jni.h>
ANativeWindow* ANativeWindow_fromSurface(JNIEnv* env, jobject surface);
</pre>
Don't forget to free this handle in your shutdown code with <code>ANativeWindow_release</code>.
<h3>Platform configuration</h3>
OpenMAX AL for Android is designed for multi-threaded applications,
and is thread-safe.
<p>
OpenMAX AL for Android supports a single engine per application, and
up to 32 objects. Available device memory and CPU may further
restrict the usable number of objects.
<p>
<code>xaCreateEngine</code> recognizes, but ignores, these engine options:
<ul>
<li><code>XA_ENGINEOPTION_THREADSAFE</code>
<li><code>XA_ENGINEOPTION_LOSSOFCONTROL</code>
</ul>
OpenMAX AL and OpenSL ES may be used together in the same application.
In this case, there is internally a single shared engine object,
and the 32 object limit is shared between OpenMAX AL and OpenSL ES.
The application should first create both engines, then use both engines,
and finally destroy both engines. The implementation maintains a
reference count on the shared engine, so that it is correctly destroyed
at the second destroy.
<h2>Planning for future versions of OpenMAX AL</h2>
The Android native multimedia APIs at level 14 are based on Khronos
Group OpenMAX AL 1.0.1 (see section "References" below).
Khronos has released
a revised version 1.1 of the standard. The revised version
includes new features, clarifications, correction of
typographical errors, and some incompatibilities. Most of the
incompatibilities are relatively minor, or are in areas of OpenMAX AL
not supported by Android. However, even a small change
can be significant for an application developer, so it is important
to prepare for this.
<p>
The Android team is committed to preserving future API binary
compatibility for developers to the extent feasible. It is our
intention to continue to support future binary compatibility of the
1.0.1-based API, even as we add support for later versions of the
standard. An application developed with this version should
work on future versions of the Android platform, provided that
you follow the guidelines listed in section "Planning for
binary compatibility" below.
<p>
Note that future source compatibility will <i>not</i> be a goal. That is,
if you upgrade to a newer version of the NDK, you may need to modify
your application source code to conform to the new API. We expect
that most such changes will be minor; see details below.
<h3>Planning for binary compatibility</h3>
We recommend that your application follow these guidelines,
to improve future binary compatibility:
<ul>
<li>
Use only the documented subset of Android-supported features from
OpenMAX AL 1.0.1.
<li>
Do not depend on a particular result code for an unsuccessful
operation; be prepared to deal with a different result code.
<li>
Application callback handlers generally run in a restricted context,
and should be written to perform their work quickly and then return
as soon as possible. Do not do complex operations within a callback
handler. For example, within a buffer queue completion callback,
you can enqueue another buffer, but do not create a media player.
<li>
Callback handlers should be prepared to be called more or less
frequently, to receive additional event types, and should ignore
event types that they do not recognize. Callbacks that are configured
with an event mask of enabled event types should be prepared to be
called with multiple event type bits set simultaneously.
Use "&" to test for each event bit rather than a switch case.
<li>
Use prefetch status and callbacks as a general indication of progress, but do
not depend on specific hard-coded fill levels or callback sequence.
The meaning of the prefetch status fill level, and the behavior for
errors that are detected during prefetch, may change.
</ul>
<h3>Planning for source compatibility</h3>
As mentioned, source code incompatibilities are expected in the next
version of OpenMAX AL from Khronos Group. Likely areas of change include:
<ul>
<li>Addition of <code>const</code> to input parameters passed by reference,
and to <code>XAchar *</code> struct fields used as input values.
This should not require any changes to your code.
<li>Substitution of unsigned types for some parameters that are
currently signed. You may need to change a parameter type from
<code>XAint32</code> to <code>XAuint32</code> or similar, or add a cast.
<li>Additional fields in struct types. For output parameters, these
new fields can be ignored, but for input parameters the new fields
will need to be initialized. Fortunately, these are expected to all
be in areas not supported by Android.
<li>Interface
<a href="http://en.wikipedia.org/wiki/Globally_unique_identifier">
GUIDs</a> will change. Refer to interfaces by symbolic name rather than GUID
to avoid a dependency.
<li><code>XAchar</code> will change from <code>unsigned char</code>
to <code>char</code>. This primarily affects the MIME data format.
<li><code>XADataFormat_MIME.mimeType</code> will be renamed to <code>pMimeType</code>.
We recommend that you initialize the <code>XADataFormat_MIME</code>
data structure using a brace-enclosed comma-separated list of values,
rather than by field name, to isolate your code from this change.
In the example code we have used this technique.
</ul>
<h2>Android extensions</h2>
The API for Android extensions is defined in <code>OMXAL/OpenMAXAL_Android.h</code>
.
Consult that file for details on these extensions. Unless otherwise
noted, all interfaces are "explicit".
<p>
Note that use these extensions will limit your application's
portability to other OpenMAX AL implementations. If this is a concern,
we advise that you avoid using them, or isolate your use of these
with <code>#ifdef</code> etc.
<p>
The following figure shows which Android-specific interfaces and
data locators are available for each object type.
<p>
<img src="chart4.png" alt="Android extensions">
<h3>Android buffer queue data locator and interface</h3>
<h4>Comparison with OpenSL ES buffer queue</h4>
The Android buffer queue data locator and interface are based on
similar concepts from OpenSL ES 1.0.1, with these differences:
<ul>
<li>Though currently usable with only a media player and MPEG-2 transport
stream data, the Android buffer queue API is designed for flexibility
so that the API can also apply to other use cases in the future.
<li>Commands may be
optionally specified by the application at time of <code>Enqueue</code>.
Each command consists of an item key and optional item value.
Command key/value pairs are carried alongside the corresponding buffer in the queue,
and thus are processed in synchrony with the buffer.
<li>To enqueue command(s) without associated data, specify
a buffer address of NULL and buffer size of zero, along
with at least one command.
<li>Status may be
provided by the implementation during a completion callback.
Each status consists of an item key and optional item value.
Status key/value pairs are carried alongside
the corresponding buffer in the queue, and thus are received by the
application in synchrony with the completion callback.
<li>The completion callback receives additional parameters:
buffer address, buffer maximum data size, buffer actual size consumed (or filled by a future
recorder object), and a <code>void *</code> for application.
<li>The callback returns a value, which must be <code>XA_RESULT_SUCCESS</code>.
</ul>
The data locator type code is <code>XA_DATALOCATOR_ANDROIDBUFFERQUEUE</code> and
the associated structure is <code>XADataLocator_AndroidBufferQueue</code>.
<p>
The interface ID is <code>XA_IID_ANDROIDBUFFERQUEUESOURCE</code>.
<h4>Usage</h4>
A typical buffer queue configuration is 8 buffers of 1880 bytes each.
<p>
The application enqueues filled buffers of data in MPEG-2 transport
stream format. The buffer size must be a multiple of 188 bytes,
the size of an MPEG-2 transport stream packet. The buffer data must
be properly aligned on a packet boundary, and formatted per the MPEG-2
Part 1 specification.
<p>
An application may supply zero or one of these item codes
(command key/value pairs) at <code>Enqueue</code>:
<dl>
<dt>XA_ANDROID_ITEMKEY_EOS</dt>
<dd>End of stream. Informs the decode and rendering components that playback is complete.
The application must not call <code>Enqueue</code> again.
There is no associated value, so <code>itemSize</code> must be zero.
There must be no data buffer alongside the EOS command.
</dd>
<dt>XA_ANDROID_ITEMKEY_DISCONTINUITY</dt>
<dd>Discontinuity. This and following buffers have a new presentation time.
The new presentation time may be optionally specified as a parameter,
expressed in <code>itemData</code> as a 64-bit unsigned integer in units of 90 kHz clock ticks.
The <code>itemSize</code> should be either zero or 8.
The discontinuity command is intended for seeking to a new point in
the stream. The application should flush its internal data, then send
the discontinuity command prior to, or alongside of, the first buffer
corresponding to the new stream position.
The initial packets in the video elementary stream
should describe an IDR (Instantaneous Decoding Refresh) frame.
Note that the discontinuity
command is not intended for stream configuration / format changes;
for these use <code>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</code>.
</dd>
<dt>XA_ANDROID_ITEMKEY_FORMAT_CHANGE</dt>
<dd>Format change. This and following buffers have a new format,
for example for MBR (Multiple Bit Rate) or resolution switching.
</dd>
</dl>
<p>
Upon notification of completion via a registered callback, the now
consumed buffer is available for the application to re-fill.
<p>
The implementation may supply zero or more of these item codes
(status key/value pairs) to the callback handler:
<dl>
<dt>XA_ANDROID_ITEMKEY_BUFFERQUEUEEVENT</dt>
<dd>Buffer queue event mask. The <code>itemSize</code> is 4, and <code>itemData</code> contains
the bit-wise "or" of zero or more of the <code>XA_ANDROIDBUFFERQUEUEEVENT_*</code>
symbols. This event mask explains the reason(s) why the callback handler
was called.</dd>
</dl>
<h3>Reporting of extensions</h3>
<code>Engine::QueryNumSupportedExtensions</code>,
<code>Engine::QuerySupportedExtension</code>,
<code>Engine::IsExtensionSupported</code> report these extensions:
<ul>
<li><code>ANDROID_SDK_LEVEL_#</code>
where # is the platform API level, 14 or higher
</ul>
<h2>Programming notes</h2>
These notes supplement the OpenMAX AL 1.0.1 specification,
available in the "References" section below.
<h3>Objects and interface initialization</h3>
Two aspects of the OpenMAX AL programming model that may be unfamiliar
to new developers are the distinction between objects and interfaces,
and the initialization sequence.
<p>
Briefly, an OpenMAX AL object is similar to the object concept
in programming languages such as Java and C++, except an OpenMAX AL
object is <i>only</i> visible via its associated interfaces. This
includes the initial interface for all objects, called
<code>XAObjectItf</code>. There is no handle for an object itself,
only a handle to the <code>XAObjectItf</code> interface of the object.
<p>
An OpenMAX AL object is first "created", which returns an
<code>XAObjectItf</code>, then "realized". This is similar to the
common programming pattern of first constructing an object (which
should never fail other than for lack of memory or invalid parameters),
and then completing initialization (which may fail due to lack of
resources). The realize step gives the implementation a
logical place to allocate additional resources if needed.
<p>
As part of the API to create an object, an application specifies
an array of desired interfaces that it plans to acquire later. Note
that this array does <i>not</i> automatically acquire the interfaces;
it merely indicates a future intention to acquire them. Interfaces
are distinguished as "implicit" or "explicit". An explicit interface
<i>must</i> be listed in the array if it will be acquired later.
An implicit interface need not be listed in the object create array,
but there is no harm in listing it there. OpenMAX AL has one more
kind of interface called "dynamic", which does not need to be
specified in the object create array, and can be added later after
the object is created. The Android implementation provides a
convenience feature to avoid this complexity; see section "Dynamic
interfaces at object creation" above.
<p>
After the object is created and realized, the application should
acquire interfaces for each feature it needs, using
<code>GetInterface</code> on the initial <code>XAObjectItf</code>.
<p>
Finally, the object is available for use via its interfaces, though
note that some objects require further setup.
Some use cases needs a bit more preparation in
order to detect connection errors. See the next section
"Media player prefetch" for details.
<p>
After your application is done with the object, you should explicitly
destroy it; see section "Destroy" below.
<h3>Media player prefetch</h3>
For a media player, <code>Object::Realize</code> allocates resources
but does not connect to the data source (i.e. "prepare") or begin
pre-fetching data. These occur once the player state is set to
either <code>XA_PLAYSTATE_PAUSED</code> or <code>XA_PLAYSTATE_PLAYING</code>.
<p>
The prefetch status interface is useful for detecting errors.
Register a callback and enable at least the
<code>XA_PREFETCHEVENT_FILLLEVELCHANGE</code> and
<code>XA_PREFETCHEVENT_STATUSCHANGE</code> events. If both of these
events are delivered simultaneously, and
<code>PrefetchStatus::GetFillLevel</code> reports a zero level, and
<code>PrefetchStatus::GetPrefetchStatus</code> reports
<code>XA_PREFETCHSTATUS_UNDERFLOW</code>, then this indicates a
non-recoverable error in the data source or in rendering to the video sink.
<p>
The next version of OpenMAX AL is expected to add more explicit
support for handling errors in the data source. However, for future
binary compatibility, we intend to continue to support the current
method for reporting a non-recoverable error.
<p>
In summary, a recommended code sequence is:
<ul>
<li>Engine::CreateMediaPlayer
<li>Object:Realize
<li>Object::GetInterface for XA_IID_PREFETCHSTATUS
<li>PrefetchStatus::SetCallbackEventsMask
<li>PrefetchStatus::RegisterCallback
<li>Object::GetInterface for XA_IID_PLAY
<li>Play::SetPlayState to XA_PLAYSTATE_PAUSED or XA_PLAYSTATE_PLAYING
<li>preparation and prefetching occur here; during this time your
callback will be called with periodic status updates and
error notifications
</ul>
<h3>Destroy</h3>
Be sure to destroy all objects on exit from your application. Objects
should be destroyed in reverse order of their creation, as it is
not safe to destroy an object that has any dependent objects.
For example, destroy in this order: audio players and recorders,
output mix, then finally the engine.
<p>
OpenMAX AL does not support automatic garbage collection or
<a href="http://en.wikipedia.org/wiki/Reference_counting">reference counting</a>
of interfaces. After you call <code>Object::Destroy</code>, all extant
interfaces derived from the associated object become <i>undefined</i>.
<p>
The Android OpenMAX AL implementation does not detect the incorrect
use of such interfaces.
Continuing to use such interfaces after the object is destroyed will
cause your application to crash or behave in unpredictable ways.
<p>
We recommend that you explicitly set both the primary object interface
and all associated interfaces to NULL as part of your object
destruction sequence, to prevent the accidental misuse of a stale
interface handle.
<h3>Callbacks and threads</h3>
Callback handlers are generally called <i>synchronously</i> with
respect to the event, that is, at the moment and location where the
event is detected by the implementation. But this point is
<i>asynchronous</i> with respect to the application. Thus you should
use a mutex or other synchronization mechanism to control access
to any variables shared between the application and the callback
handler. In the example code, such as for buffer queues, we have
omitted this synchronization in the interest of simplicity. However,
proper mutual exclusion would be critical for any production code.
<p>
Callback handlers are called from internal
non-application thread(s) which are not attached to the Dalvik virtual machine and thus
are ineligible to use JNI. Because these internal threads are
critical to the integrity of the OpenMAX AL implementation, a callback
handler should also not block or perform excessive work. Therefore,
if your callback handler needs to use JNI or do anything significant
(e.g. beyond an <code>Enqueue</code> or something else simple such as the "Get"
family), the handler should instead post an event for another thread
to process.
<p>
Note that the converse is safe: a Dalvik application thread which has
entered JNI is allowed to directly call OpenMAX AL APIs, including
those which block. However, blocking calls are not recommended from
the main thread, as they may result in the dreaded "Application Not
Responding" (ANR).
<p>
(Throughout this document, "Dalvik" can be considered to also refer to "ART").
<h3>Performance</h3>
As OpenMAX AL is a native C API, non-Dalvik application threads which
call OpenMAX AL have no Dalvik-related overhead such as garbage
collection pauses. However, there is no additional performance
benefit to the use of OpenMAX AL other than this. In particular, use
of OpenMAX AL does not result in lower audio or video latency, higher scheduling
priority, etc. than what the platform generally provides.
On the other hand, as the Android platform and specific device
implementations continue to evolve, an OpenMAX AL application can
expect to benefit from any future system performance improvements.
<h3>Security and permissions</h3>
As far as who can do what, security in Android is done at the
process level. Java programming language code can't do anything more than native code, nor
can native code do anything more than Java programming language code. The only differences
between them are what APIs are available that provide functionality
that the platform promises to support in the future and across
different devices.
<p>
Applications using OpenMAX AL must request whatever permissions they
would need for similar non-native APIs.
Applications that play
network resources need <code>android.permission.NETWORK</code>.
Note that the current Android implementation does not directly access the network,
but many applications that play multimedia receive their data via the network.
<h2>Platform issues</h2>
This section describes known issues in the initial platform
release which supports these APIs.
<h3>Dynamic interface management</h3>
<code>DynamicInterfaceManagement::AddInterface</code> does not work.
Instead, specify the interface in the array passed to Create.
<h2>References and resources</h2>
Android:
<ul>
<li><a href="http://developer.android.com/resources/index.html">
Android developer resources</a>
<li><a href="http://groups.google.com/group/android-developers">
Android developers discussion group</a>
<li><a href="http://developer.android.com/sdk/ndk/index.html">Android NDK</a>
<li><a href="http://groups.google.com/group/android-ndk">
Android NDK discussion group</a> (for developers of native code, including OpenMAX AL)
<li><a href="http://code.google.com/p/android/issues/">
Android open source bug database</a>
</ul>
Khronos Group:
<ul>
<li><a href="http://www.khronos.org/openmax/al/">
Khronos Group OpenMAX AL Overview</a>
<li><a href="http://www.khronos.org/registry/omxal/">
Khronos Group OpenMAX AL 1.0.1 specification</a>
<li><a href="http://www.khronos.org/message_boards/viewforum.php?f=30">
Khronos Group public message board for OpenMAX AL</a>
(please limit to non-Android questions)
</ul>
For convenience, we have included a copy of the OpenMAX AL 1.0.1
specification with the NDK in
<code>docs/openmaxal/OpenMAX_AL_1_0_1_Specification.pdf</code>.
<p>
Miscellaneous:
<ul>
<li><a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a>
<li><a href="http://en.wikipedia.org/wiki/MPEG-2">MPEG-2</a>
<li><a href="http://en.wikipedia.org/wiki/MPEG_transport_stream">MPEG-2 transport stream</a>
<li><a href="http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC">H.264</a>
<li><a href="http://en.wikipedia.org/wiki/Advanced_Audio_Coding">AAC</a>
</ul>
</body>
</html>