UPDATE: Newer tutorial here: http://www.raspberrypi.org/phpBB3/viewt ... 33&t=57721
Others have helped me get this far so I figured I'd return the favor to help the next person.
Here is sample code showing how to use the Raspberry Pi's OpenMAX API to decode a JPEG image to a buffer (not displaying it on screen). It does not use the ilclient (because I felt the ilclient code was a bit too confusing) but instead is in C++ and uses a few classes I wrote to make the whole process more object oriented and friendly. It should be extracted to the same place as the rest of the samples (/opt/vc/src/hello_pi) and then you can build it just by typing "make".
http://www.rulecity.com/browse/rpi/hello_jpeg.tar.gz
I hope this can help someone get started with OpenMAX.
OpenMAX: How to decode a JPEG (example source code)
Last edited by MattOwnby on Tue Oct 08, 2013 8:10 pm, edited 1 time in total.
Re: OpenMAX: How to decode a JPEG (example source code)
Matt
This is a really nice example and I've found it very useful. Thanks for making it available.
I've been playing with your example to try and decode some frames from my web cam. It works really well, except the output format is YUV420 packed. Is there a way we can change the output format that is created by the decoder or is that dependant on the hardware? If we cannot change the output is I have to find a way to convert the output to RGB on the GPU?
As a complete aside I've noticed that you code does not seem to work, for me anyway, if the source image requires more than one buffer to load it into the decoder. Another example on the forums only seems to work if more than one buffer is required, but not if only one buffer is needed (hence it did not work for me). Any idea what could be causing this?
This is a really nice example and I've found it very useful. Thanks for making it available.
I've been playing with your example to try and decode some frames from my web cam. It works really well, except the output format is YUV420 packed. Is there a way we can change the output format that is created by the decoder or is that dependant on the hardware? If we cannot change the output is I have to find a way to convert the output to RGB on the GPU?
As a complete aside I've noticed that you code does not seem to work, for me anyway, if the source image requires more than one buffer to load it into the decoder. Another example on the forums only seems to work if more than one buffer is required, but not if only one buffer is needed (hence it did not work for me). Any idea what could be causing this?
Re: OpenMAX: How to decode a JPEG (example source code)
To make it output to a format other than YUV420, I believe you need to tunnel it into the resizer; the resizer will output in RGB format. To see an example of how to do this, check out the XBMC source code at:
https://github.com/xbmc/xbmc-rbp/blob/m ... XImage.cpp
and
https://github.com/xbmc/xbmc-rbp/blob/m ... MXCore.cpp
Yesterday I tried to modify my "hello_jpeg" program to do this but it didn't work and I decided I could deal with YUV420 so I abandoned my efforts. If you are successful, please post your work
As for the buffer, I have a bug in my code where it does not "wraparound" if the buffer index exceeds the max (I believe it wants 3 buffers allocated so if it needs more than 3 this could cause the crash). Also I have only tested it with very small jpegs (obviously) so for a big jpeg, you'd probably need to start emptying the output buffer while still feeding it new input (unverified, that would just be my speculation). Another bug in my code which I am hoping someone else can step up and fix hehehehe.
https://github.com/xbmc/xbmc-rbp/blob/m ... XImage.cpp
and
https://github.com/xbmc/xbmc-rbp/blob/m ... MXCore.cpp
Yesterday I tried to modify my "hello_jpeg" program to do this but it didn't work and I decided I could deal with YUV420 so I abandoned my efforts. If you are successful, please post your work

As for the buffer, I have a bug in my code where it does not "wraparound" if the buffer index exceeds the max (I believe it wants 3 buffers allocated so if it needs more than 3 this could cause the crash). Also I have only tested it with very small jpegs (obviously) so for a big jpeg, you'd probably need to start emptying the output buffer while still feeding it new input (unverified, that would just be my speculation). Another bug in my code which I am hoping someone else can step up and fix hehehehe.
Re: OpenMAX: How to decode a JPEG (example source code)
Matt
Thanks I'll let you know how I get on.
Thanks I'll let you know how I get on.
Re: OpenMAX: How to decode a JPEG (example source code)
hi Matt
thanks for sharing!
I'd like to encode an image into the JPEG format using OpenMax, and I think your code might give me some clues...
Thanks again
Jojo
thanks for sharing!
I'd like to encode an image into the JPEG format using OpenMax, and I think your code might give me some clues...
Thanks again
Jojo
Re: OpenMAX: How to decode a JPEG (example source code)
Just an update,
I am really close to solving the two previous issues that we have discussed.
My latest code currently decodes JPEGs and does colorspace conversion (via the resizer) to RGBA, and even benchmarks how fast the process is (I am getting 80-150 FPS depending on which test JPEG I use).
The only thing not working on it is decoding the large JPEGs; I've put some effort into making this work and _almost_ have it working. I know of one (big) bug in my code which is likely causing this to not work but my pi suddenly crashed and I am not physically near it so I can't power cycle it. So I hope to have something "final" soon. As an added bonus I also threw in proper clean-up code.
I am really close to solving the two previous issues that we have discussed.
My latest code currently decodes JPEGs and does colorspace conversion (via the resizer) to RGBA, and even benchmarks how fast the process is (I am getting 80-150 FPS depending on which test JPEG I use).
The only thing not working on it is decoding the large JPEGs; I've put some effort into making this work and _almost_ have it working. I know of one (big) bug in my code which is likely causing this to not work but my pi suddenly crashed and I am not physically near it so I can't power cycle it. So I hope to have something "final" soon. As an added bonus I also threw in proper clean-up code.
Re: OpenMAX: How to decode a JPEG (example source code)
Sounds really exciting!
I'm looking forward to seeing that
I'm looking forward to seeing that

Re: OpenMAX: How to decode a JPEG (example source code)
And as promised, here is my new and improved "version 2".
Improvements over original:
- decodes to RGBA* instead of YV12
- decodes larger JPEGs than v1
- benchmarks decoding speed
- can trivially be modified to optionally resize (see line 420 and 421 of JPEG.cpp)
See README.txt file for more information.
Download link: http://www.rulecity.com/browse/rpi/hello_jpeg_v2.tar.gz
* EDIT : This message until recently implied that the resizer could be used for arbitrary colorspace conversion (for example, to other formats besides YV12 and RGBA). It turns out that the resizer will only support YV12 (aka YUV420), RGBA, and RGB565 as output (and probably input too). See the documentation here:
https://github.com/raspberrypi/firmware ... esize.html
Improvements over original:
- decodes to RGBA* instead of YV12
- decodes larger JPEGs than v1
- benchmarks decoding speed
- can trivially be modified to optionally resize (see line 420 and 421 of JPEG.cpp)
See README.txt file for more information.
Download link: http://www.rulecity.com/browse/rpi/hello_jpeg_v2.tar.gz
* EDIT : This message until recently implied that the resizer could be used for arbitrary colorspace conversion (for example, to other formats besides YV12 and RGBA). It turns out that the resizer will only support YV12 (aka YUV420), RGBA, and RGB565 as output (and probably input too). See the documentation here:
https://github.com/raspberrypi/firmware ... esize.html
MattOwnby wrote:Others have helped me get this far so I figured I'd return the favor to help the next person.
Here is sample code showing how to use the Raspberry Pi's OpenMAX API to decode a JPEG image to a buffer (not displaying it on screen). It does not use the ilclient (because I felt the ilclient code was a bit too confusing) but instead is in C++ and uses a few classes I wrote to make the whole process more object oriented and friendly. It should be extracted to the same place as the rest of the samples (/opt/vc/src/hello_pi) and then you can build it just by typing "make".
http://www.rulecity.com/browse/rpi/hello_jpeg.tar.gz
I hope this can help someone get started with OpenMAX.
Last edited by MattOwnby on Thu Feb 28, 2013 7:23 pm, edited 1 time in total.
Re: OpenMAX: How to decode a JPEG (example source code)
Matt
You've been busy. I've been away on holiday. Again many thanks for posting your efforts on line. I have been using your previous example to get an ilclient version of the decoder working. I look forward to playing with this version to get the resizer to work, I'll post my results once working.
Again thanks for letting us all see how you are getting on.
You've been busy. I've been away on holiday. Again many thanks for posting your efforts on line. I have been using your previous example to get an ilclient version of the decoder working. I look forward to playing with this version to get the resizer to work, I'll post my results once working.
Again thanks for letting us all see how you are getting on.
Re: OpenMAX: How to decode a JPEG (example source code)
Ok so I finall got a version of the JPEG decoder working using the ilclient library. There is one problem with the first frame is always failing to return an eos event apart from that it works ok. This is based entirely on Matt's earlier examples, so same rules this implementation is for educational/non-commercial use only.
I've currently got this code plugged into a program that receives frames from a network CCTV camera and projects onto the cube from the hello triangles sample.
Code as follows
Openmax.h
openmax.c
I've currently got this code plugged into a program that receives frames from a network CCTV camera and projects onto the cube from the hello triangles sample.
Code as follows
Openmax.h
Code: Select all
/***************************************************************************
* openmax.h
*
* Written by Anthong Sale Sept 2012
* Based on an original by Matt Ownby, August 2012
* You are free to use this for educational/non-commercial use
****************************************************************************/
#ifndef _OPTION_H_
#define _OPTION_H_
/*
Defines the methods for interacting with openmax il and ilclient to decode
jpeg images from the camera
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include "bcm_host.h"
#include "ilclient.h"
#define OMXJPEG_OK 0
#define OMXJPEG_ERROR_ILCLIENT_INIT -1024
#define OMXJPEG_ERROR_OMX_INIT -1025
#define OMXJPEG_ERROR_MEMORY -1026
#define OMXJPEG_ERROR_CREATING_COMP -1027
#define OMXJPEG_ERROR_WRONG_NO_PORTS -1028
#define OMXJPEG_ERROR_EXECUTING -1029
#define OMXJPEG_ERROR_NOSETTINGS -1030
typedef struct _OPENMAX_JPEG_DECODER OPENMAX_JPEG_DECODER;
//this function run the boilerplate to setup the openmax components;
int setupOpenMaxJpegDecoder(OPENMAX_JPEG_DECODER** decoder);
//this function passed the jpeg image buffer in, and returns the decoded image
int decodeImage(OPENMAX_JPEG_DECODER* decoder,
char* sourceImage, size_t imageSize);
//this function cleans up the decoder.
void cleanup(OPENMAX_JPEG_DECODER* decoder);
#endif
Code: Select all
/***************************************************************************
* openmax.c
*
* Written by Anthong Sale Sept 2012
* Based on an original by Matt Ownby, August 2012
* You are free to use this for educational/non-commercial use
****************************************************************************/
#include "openmax.h"
#define TIMEOUT_MS 2000
typedef struct _COMPONENT_DETAILS{
COMPONENT_T* component;
OMX_HANDLETYPE handle;
int inPort;
int outPort;
} COMPONENT_DETAILS;
struct _OPENMAX_JPEG_DECODER{
ILCLIENT_T* client;
COMPONENT_DETAILS* imageDecoder;
COMPONENT_DETAILS* imageResizer;
OMX_BUFFERHEADERTYPE** ppInputBufferHeader;
int inputBufferHeaderCount;
OMX_BUFFERHEADERTYPE* pOutputBufferHeader;
};
int bufferIndex = 0; //index to buffer array
int portSettingsChanged(OPENMAX_JPEG_DECODER* decoder)
{
OMX_PARAM_PORTDEFINITIONTYPE portdef;
// need to setup the input for the resizer with the output of the decoder
portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
portdef.nVersion.nVersion = OMX_VERSION;
portdef.nPortIndex = decoder->imageDecoder->outPort;
OMX_GetParameter(decoder->imageDecoder->handle,
OMX_IndexParamPortDefinition, &portdef);
unsigned int uWidth = (unsigned int) portdef.format.image.nFrameWidth;
unsigned int uHeight = (unsigned int) portdef.format.image.nFrameHeight;
// tell resizer input what the decoder output will be providing
portdef.nPortIndex = decoder->imageResizer->inPort;
OMX_SetParameter(decoder->imageResizer->handle,
OMX_IndexParamPortDefinition, &portdef);
// establish tunnel between decoder output and resizer input
OMX_SetupTunnel(decoder->imageDecoder->handle,
decoder->imageDecoder->outPort,
decoder->imageResizer->handle,
decoder->imageResizer->inPort);
// enable ports
OMX_SendCommand(decoder->imageDecoder->handle,
OMX_CommandPortEnable,
decoder->imageDecoder->outPort, NULL);
OMX_SendCommand(decoder->imageResizer->handle,
OMX_CommandPortEnable,
decoder->imageResizer->inPort, NULL);
// put resizer in idle state (this allows the outport of the decoder to become enabled)
OMX_SendCommand(decoder->imageResizer->handle,
OMX_CommandStateSet, OMX_StateIdle, NULL);
// wait for state change complete
ilclient_wait_for_event(decoder->imageResizer->component,
OMX_EventCmdComplete,
OMX_CommandStateSet, 1,
OMX_StateIdle, 1, 0, TIMEOUT_MS);
// once the state changes, both ports should become enabled and the resizer
// output should generate a settings changed event
ilclient_wait_for_event(decoder->imageDecoder->component,
OMX_EventCmdComplete,
OMX_CommandPortEnable, 1,
decoder->imageDecoder->outPort, 1,
0, TIMEOUT_MS);
ilclient_wait_for_event(decoder->imageResizer->component,
OMX_EventCmdComplete,
OMX_CommandPortEnable, 1,
decoder->imageResizer->inPort, 1,
0, TIMEOUT_MS);
ilclient_wait_for_event(decoder->imageResizer->component,
OMX_EventPortSettingsChanged,
decoder->imageResizer->outPort, 1,
0, 1, 0, TIMEOUT_MS);
ilclient_disable_port(decoder->imageResizer->component,
decoder->imageResizer->outPort);
// query output buffer requirements for resizer
portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
portdef.nVersion.nVersion = OMX_VERSION;
portdef.nPortIndex = decoder->imageResizer->outPort;
OMX_GetParameter(decoder->imageResizer->handle,
OMX_IndexParamPortDefinition, &portdef);
// change output color format and dimensions to match input
portdef.format.image.eCompressionFormat = OMX_IMAGE_CodingUnused;
portdef.format.image.eColorFormat = OMX_COLOR_Format32bitABGR8888;
portdef.format.image.nFrameWidth = uWidth;
portdef.format.image.nFrameHeight = uHeight;
portdef.format.image.nStride = 0;
portdef.format.image.nSliceHeight = 0;
portdef.format.image.bFlagErrorConcealment = OMX_FALSE;
OMX_SetParameter(decoder->imageResizer->handle,
OMX_IndexParamPortDefinition, &portdef);
// grab output requirements again to get actual buffer size requirement (and buffer count requirement!)
OMX_GetParameter(decoder->imageResizer->handle,
OMX_IndexParamPortDefinition, &portdef);
// move resizer into executing state
ilclient_change_component_state(decoder->imageResizer->component,
OMX_StateExecuting);
// show some logging so user knows it's working
printf("Width: %u Height: %u Output Color Format: 0x%x Buffer Size: %u\n",
(unsigned int) portdef.format.image.nFrameWidth,
(unsigned int) portdef.format.image.nFrameHeight,
(unsigned int) portdef.format.image.eColorFormat,
(unsigned int) portdef.nBufferSize);
fflush(stdout);
// enable output port of resizer
OMX_SendCommand(decoder->imageResizer->handle,
OMX_CommandPortEnable,
decoder->imageResizer->outPort, NULL);
//allocate the buffer
//void* outputBuffer = 0;
//if (posix_memalign(&outputBuffer, portdef.nBufferAlignment, portdef.nBufferSize) != 0)
//{
// perror("Allocating output buffer");
// return OMXJPEG_ERROR_MEMORY;
//}
//set the buffer
//int ret = OMX_UseBuffer(decoder->imageResizer->handle,
// &decoder->pOutputBufferHeader,
// decoder->imageResizer->outPort, NULL,
// portdef.nBufferSize,
// (OMX_U8 *) outputBuffer);
int ret = OMX_AllocateBuffer(decoder->imageResizer->handle,
&decoder->pOutputBufferHeader,
decoder->imageResizer->outPort,
NULL,
portdef.nBufferSize);
if(ret != OMX_ErrorNone){
perror("Eror allocating buffer");
return OMXJPEG_ERROR_MEMORY;
}
ilclient_wait_for_event(decoder->imageResizer->component,
OMX_EventCmdComplete,
OMX_CommandPortEnable, 1,
decoder->imageResizer->outPort, 1,
0, TIMEOUT_MS);
return OMXJPEG_OK;
}
int portSettingsChangedAgain(OPENMAX_JPEG_DECODER* decoder)
{
ilclient_disable_port(decoder->imageDecoder->component,
decoder->imageDecoder->outPort);
ilclient_disable_port(decoder->imageResizer->component,
decoder->imageResizer->inPort);
OMX_PARAM_PORTDEFINITIONTYPE portdef;
// need to setup the input for the resizer with the output of the decoder
portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
portdef.nVersion.nVersion = OMX_VERSION;
portdef.nPortIndex = decoder->imageDecoder->outPort;
OMX_GetParameter(decoder->imageDecoder->handle,
OMX_IndexParamPortDefinition, &portdef);
// tell resizer input what the decoder output will be providing
portdef.nPortIndex = decoder->imageResizer->inPort;
OMX_SetParameter(decoder->imageResizer->handle,
OMX_IndexParamPortDefinition, &portdef);
// enable output of decoder and input of resizer (ie enable tunnel)
ilclient_enable_port(decoder->imageDecoder->component,
decoder->imageDecoder->outPort);
ilclient_enable_port(decoder->imageResizer->component,
decoder->imageResizer->inPort);
// need to wait for this event
ilclient_wait_for_event(decoder->imageResizer->component,
OMX_EventPortSettingsChanged,
decoder->imageResizer->outPort, 1,
0, 0, 0, TIMEOUT_MS);
return OMXJPEG_OK;
}
int prepareResizer(OPENMAX_JPEG_DECODER* decoder)
{
decoder->imageResizer = malloc(sizeof(COMPONENT_DETAILS));
if(decoder->imageResizer == NULL)
{
perror("malloc image resizer");
return OMXJPEG_ERROR_MEMORY;
}
int ret = ilclient_create_component(decoder->client,
&decoder->imageResizer->component,
"resize",
ILCLIENT_DISABLE_ALL_PORTS |
ILCLIENT_ENABLE_INPUT_BUFFERS |
ILCLIENT_ENABLE_OUTPUT_BUFFERS);
if(ret != 0)
{
perror("image resizer");
return OMXJPEG_ERROR_CREATING_COMP;
}
//grab the handle for later use
decoder->imageResizer->handle = ILC_GET_HANDLE(decoder->imageResizer->component);
//get and store the ports
OMX_PORT_PARAM_TYPE port;
port.nSize = sizeof(OMX_PORT_PARAM_TYPE);
port.nVersion.nVersion = OMX_VERSION;
OMX_GetParameter(ILC_GET_HANDLE(decoder->imageResizer->component),
OMX_IndexParamImageInit, &port);
if(port.nPorts != 2){
return OMXJPEG_ERROR_WRONG_NO_PORTS;
}
decoder->imageResizer->inPort = port.nStartPortNumber;
decoder->imageResizer->outPort = port.nStartPortNumber + 1;
decoder->pOutputBufferHeader = NULL;
return OMXJPEG_OK;
}
int prepareImageDecoder(OPENMAX_JPEG_DECODER* decoder)
{
decoder->imageDecoder = malloc(sizeof(COMPONENT_DETAILS));
if(decoder->imageDecoder == NULL)
{
perror("malloc image decoder");
return OMXJPEG_ERROR_MEMORY;
}
int ret = ilclient_create_component(decoder->client,
&decoder->imageDecoder->component,
"image_decode",
ILCLIENT_DISABLE_ALL_PORTS |
ILCLIENT_ENABLE_INPUT_BUFFERS);
if(ret != 0)
{
perror("image decode");
return OMXJPEG_ERROR_CREATING_COMP;
}
//grab the handle for later use in OMX calls directly
decoder->imageDecoder->handle = ILC_GET_HANDLE(decoder->imageDecoder->component);
//get and store the ports
OMX_PORT_PARAM_TYPE port;
port.nSize = sizeof(OMX_PORT_PARAM_TYPE);
port.nVersion.nVersion = OMX_VERSION;
OMX_GetParameter(decoder->imageDecoder->handle,
OMX_IndexParamImageInit, &port);
if(port.nPorts != 2){
return OMXJPEG_ERROR_WRONG_NO_PORTS;
}
decoder->imageDecoder->inPort = port.nStartPortNumber;
decoder->imageDecoder->outPort = port.nStartPortNumber + 1;
return OMXJPEG_OK;
}
int startupImageDecoder(OPENMAX_JPEG_DECODER* decoder)
{
//move to idle
ilclient_change_component_state(decoder->imageDecoder->component,
OMX_StateIdle);
//set input image format
OMX_IMAGE_PARAM_PORTFORMATTYPE imagePortFormat;
memset(&imagePortFormat, 0, sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE));
imagePortFormat.nSize = sizeof(OMX_IMAGE_PARAM_PORTFORMATTYPE);
imagePortFormat.nVersion.nVersion = OMX_VERSION;
imagePortFormat.nPortIndex = decoder->imageDecoder->inPort;
imagePortFormat.eCompressionFormat = OMX_IMAGE_CodingJPEG;
OMX_SetParameter(decoder->imageDecoder->handle,
OMX_IndexParamImagePortFormat,
&imagePortFormat);
//get buffer requirements
OMX_PARAM_PORTDEFINITIONTYPE portdef;
portdef.nSize = sizeof(OMX_PARAM_PORTDEFINITIONTYPE);
portdef.nVersion.nVersion = OMX_VERSION;
portdef.nPortIndex = decoder->imageDecoder->inPort;
OMX_GetParameter(decoder->imageDecoder->handle,
OMX_IndexParamPortDefinition,
&portdef);
//enable the port and setup the buffers
OMX_SendCommand(decoder->imageDecoder->handle,
OMX_CommandPortEnable,
decoder->imageDecoder->inPort,
NULL);
decoder->inputBufferHeaderCount = portdef.nBufferCountActual;
//allocate pointer array
decoder->ppInputBufferHeader = (OMX_BUFFERHEADERTYPE **)malloc(
sizeof(void) * decoder->inputBufferHeaderCount);
//allocate each buffer
int i;
for(i = 0; i < decoder->inputBufferHeaderCount; i ++)
{
if(OMX_AllocateBuffer(decoder->imageDecoder->handle,
&decoder->ppInputBufferHeader[i],
decoder->imageDecoder->inPort,
(void *)i,
portdef.nBufferSize) != OMX_ErrorNone)
{
perror("Allocate decode buffer");
return OMXJPEG_ERROR_MEMORY;
}
}
//wait for port enable to complete - which it should once buffers are assigned
int ret = ilclient_wait_for_event(decoder->imageDecoder->component,
OMX_EventCmdComplete,
OMX_CommandPortEnable, 0,
decoder->imageDecoder->inPort, 0,
0, TIMEOUT_MS);
if(ret != 0)
{
fprintf(stderr, "Did not get port enable %d\n", ret);
return OMXJPEG_ERROR_EXECUTING;
}
//start executing the decoder
ret = OMX_SendCommand(decoder->imageDecoder->handle,
OMX_CommandStateSet,
OMX_StateExecuting, NULL);
if(ret != 0)
{
fprintf(stderr, "Error starting image decoder %x\n", ret);
return OMXJPEG_ERROR_EXECUTING;
}
ret = ilclient_wait_for_event(decoder->imageDecoder->component,
OMX_EventCmdComplete,
OMX_StateExecuting, 0,
0, 1,
0, TIMEOUT_MS);
if(ret != 0)
{
fprintf(stderr, "Did not receive executing stat %d\n", ret);
//return OMXJPEG_ERROR_EXECUTING;
}
return OMXJPEG_OK;
}
//this function run the boilerplate to setup the openmax components;
int setupOpenMaxJpegDecoder(OPENMAX_JPEG_DECODER** pDecoder)
{
*pDecoder = malloc(sizeof(OPENMAX_JPEG_DECODER));
if(pDecoder[0] == NULL)
{
perror("malloc decoder");
return OMXJPEG_ERROR_MEMORY;
}
memset(*pDecoder, 0, sizeof(OPENMAX_JPEG_DECODER));
if((pDecoder[0]->client = ilclient_init()) == NULL) {
perror("ilclient_init");
return OMXJPEG_ERROR_ILCLIENT_INIT;
}
if(OMX_Init() != OMX_ErrorNone) {
ilclient_destroy(pDecoder[0]->client);
perror("OMX_Init");
return OMXJPEG_ERROR_OMX_INIT;
}
//prepare the image decoder
int ret = prepareImageDecoder(pDecoder[0]);
if(ret != OMXJPEG_OK)
return ret;
ret = prepareResizer(pDecoder[0]);
if(ret != OMXJPEG_OK)
return ret;
ret = startupImageDecoder(pDecoder[0]);
if(ret != OMXJPEG_OK)
return ret;
return OMXJPEG_OK;
}
//this function passed the jpeg image buffer in, and returns the decoded image
int decodeImage(OPENMAX_JPEG_DECODER* decoder,
char* sourceImage, size_t imageSize)
{
char* sourceOffset = sourceImage; //we store a seperate buffer ot image so we can offset it
size_t toread = 0; //bytes left to read from buffer
toread += imageSize;
int bFilled = 0; //have we filled our output buffer
bufferIndex = 0;
while(toread > 0)
{
//get next buffer from array
OMX_BUFFERHEADERTYPE* pBufHeader = decoder->ppInputBufferHeader[bufferIndex];
//step index and reset to 0 if required
bufferIndex++;
if(bufferIndex >= decoder->inputBufferHeaderCount)
bufferIndex = 0;
//work out the next chunk to load into the decoder
if(toread > pBufHeader->nAllocLen)
pBufHeader->nFilledLen = pBufHeader->nAllocLen;
else
pBufHeader->nFilledLen = toread;
toread = toread - pBufHeader->nFilledLen;
//pass the bytes to the buffer
memcpy(pBufHeader->pBuffer, sourceOffset, pBufHeader->nFilledLen);
//update the buffer pointer and set the input flags
sourceOffset = sourceOffset + pBufHeader->nFilledLen;
pBufHeader->nOffset=0;
pBufHeader->nFlags =0;
if(toread <= 0)
{
pBufHeader->nFlags = OMX_BUFFERFLAG_EOS;
}
//empty the current buffer
int ret = OMX_EmptyThisBuffer(decoder->imageDecoder->handle, pBufHeader);
if(ret != OMX_ErrorNone)
{
perror("Empty input buffer");
fprintf(stderr, "return code %x\n", ret);
return OMXJPEG_ERROR_MEMORY;
}
//wait for buffer to empty or port changed event
int done = 0;
while((done == 0) || (decoder->pOutputBufferHeader == NULL))
{
if(decoder->pOutputBufferHeader == NULL)
{
ret = ilclient_wait_for_event(decoder->imageDecoder->component,
OMX_EventPortSettingsChanged,
decoder->imageDecoder->outPort, 0,
0, 1,
0, 5);
if(ret == 0)
{
portSettingsChanged(decoder);
}
}
else
{
ret = ilclient_remove_event(decoder->imageDecoder->component,
OMX_EventPortSettingsChanged,
decoder->imageDecoder->outPort, 0,
0, 1);
if(ret == 0)
portSettingsChangedAgain(decoder);
}
//check to see if buffer is now empty
if(pBufHeader->nFilledLen == 0)
done = 1;
if((done == 0) || (decoder->pOutputBufferHeader == NULL))
sleep(1);
}
//fill the buffer if we have created the buffer
if(bFilled == 0)
{
if((decoder->pOutputBufferHeader == NULL))
{
portSettingsChanged(decoder);
}
ret = OMX_FillThisBuffer(decoder->imageResizer->handle,
decoder->pOutputBufferHeader);
if(ret != OMX_ErrorNone)
{
perror("Filling output buffer");
fprintf(stderr, "Error code %x\n", ret);
return OMXJPEG_ERROR_MEMORY;
}
bFilled = 1;
}
}
//wait for buffer to fill
while(decoder->pOutputBufferHeader->nFilledLen > 0)
{
sleep(5);
}
//wait for end of stream events
int ret = ilclient_wait_for_event(decoder->imageDecoder->component,
OMX_EventBufferFlag,
decoder->imageDecoder->outPort, 1,
OMX_BUFFERFLAG_EOS, 1,
0, 2);
if(ret != 0)
{
fprintf(stderr,"No EOS event on image decoder %d\n", ret);
}
ret = ilclient_wait_for_event(decoder->imageResizer->component,
OMX_EventBufferFlag,
decoder->imageResizer->outPort, 1,
OMX_BUFFERFLAG_EOS, 1,
0, 2);
if(ret != 0)
{
fprintf(stderr, "No EOS event on image resizer %d\n", ret);
}
return OMXJPEG_OK;
}
//this function cleans up the decoder.
void cleanup(OPENMAX_JPEG_DECODER* decoder)
{
//flush everything through
OMX_SendCommand(decoder->imageDecoder->handle,
OMX_CommandFlush,
decoder->imageDecoder->outPort,
NULL);
ilclient_wait_for_event(decoder->imageDecoder->component,
OMX_EventCmdComplete,
OMX_CommandFlush, 0,
decoder->imageDecoder->outPort, 0,
0, TIMEOUT_MS);
OMX_SendCommand(decoder->imageResizer->handle,
OMX_CommandFlush,
decoder->imageResizer->inPort,
NULL);
ilclient_wait_for_event(decoder->imageResizer->component,
OMX_EventCmdComplete,
OMX_CommandFlush, 0,
decoder->imageResizer->inPort, 1,
0, TIMEOUT_MS);
OMX_SendCommand(decoder->imageDecoder->handle, OMX_CommandPortDisable,
decoder->imageDecoder->inPort, NULL);
int i = 0;
for(i = 0; i < decoder->inputBufferHeaderCount; i ++){
OMX_BUFFERHEADERTYPE *vpBufHeader = decoder->ppInputBufferHeader[i];
OMX_FreeBuffer(decoder->imageDecoder->handle,
decoder->imageDecoder->inPort, vpBufHeader);
}
ilclient_wait_for_event(decoder->imageDecoder->component, OMX_EventCmdComplete,
OMX_CommandPortDisable, 0,
decoder->imageDecoder->inPort, 0,
0, TIMEOUT_MS);
OMX_SendCommand(decoder->imageResizer->handle, OMX_CommandPortDisable,
decoder->imageResizer->outPort, NULL);
OMX_FreeBuffer(decoder->imageResizer->handle,
decoder->imageResizer->outPort,
decoder->pOutputBufferHeader);
ilclient_wait_for_event(decoder->imageResizer->component, OMX_EventCmdComplete,
OMX_CommandPortDisable, 0,
decoder->imageResizer->outPort, 0,
0, TIMEOUT_MS);
OMX_SendCommand(decoder->imageDecoder->handle, OMX_CommandPortDisable,
decoder->imageDecoder->outPort, NULL);
ilclient_wait_for_event(decoder->imageDecoder->component, OMX_EventCmdComplete,
OMX_CommandPortDisable, 0,
decoder->imageDecoder->outPort, 0,
0, TIMEOUT_MS);
OMX_SendCommand(decoder->imageResizer->handle, OMX_CommandPortDisable,
decoder->imageResizer->inPort, NULL);
ilclient_wait_for_event(decoder->imageResizer->component, OMX_EventCmdComplete,
OMX_CommandPortDisable, 0,
decoder->imageResizer->inPort, 0,
0, TIMEOUT_MS);
OMX_SetupTunnel(decoder->imageDecoder->handle, decoder->imageDecoder->outPort, NULL, 0);
OMX_SetupTunnel(decoder->imageResizer->handle, decoder->imageResizer->inPort, NULL, 0);
ilclient_change_component_state(decoder->imageDecoder->component, OMX_StateIdle);
ilclient_change_component_state(decoder->imageResizer->component, OMX_StateIdle);
ilclient_wait_for_event(decoder->imageDecoder->component, OMX_EventCmdComplete,
OMX_CommandStateSet, 0,
OMX_StateIdle, 0,
0, TIMEOUT_MS);
ilclient_wait_for_event(decoder->imageResizer->component, OMX_EventCmdComplete,
OMX_CommandStateSet, 0,
OMX_StateIdle, 0,
0, TIMEOUT_MS);
ilclient_change_component_state(decoder->imageDecoder->component, OMX_StateLoaded);
ilclient_change_component_state(decoder->imageResizer->component, OMX_StateLoaded);
ilclient_wait_for_event(decoder->imageDecoder->component, OMX_EventCmdComplete,
OMX_CommandStateSet, 0,
OMX_StateLoaded, 0,
0, TIMEOUT_MS);
ilclient_wait_for_event(decoder->imageResizer->component, OMX_EventCmdComplete,
OMX_CommandStateSet, 0,
OMX_StateLoaded, 0,
0, TIMEOUT_MS);
OMX_Deinit();
if(decoder->client != NULL)
{
ilclient_destroy(decoder->client);
}
}
Re: OpenMAX: How to decode a JPEG (example source code)
I'm trying to do the same. To use hello_triangle sample and your openmax.o to get jpeg image projected onto the cube.
But program got stuck in function decodeImage in
where the nFilledLen isn't changing anymore and has still value == BufferSize (3145728)
The std output is :
Did not receive executing stat -1
Width: 1024 Height: 768 Output Color Format: 0x7f000001 Buffer Size: 3145728
Have you also tried to use eglImage to write output of resizer directly to texture ?
But program got stuck in function decodeImage in
Code: Select all
while(decoder->pOutputBufferHeader->nFilledLen > 0)
{
sleep(5)
}
The std output is :
Did not receive executing stat -1
Width: 1024 Height: 768 Output Color Format: 0x7f000001 Buffer Size: 3145728
Have you also tried to use eglImage to write output of resizer directly to texture ?
Re: OpenMAX: How to decode a JPEG (example source code)
Ok so you need to put a guard around the fill logic so it eventually times out. I did have to do some work to get the jpeg output from my camera into a format that was supported by the openmax logic. I know it does not support progressive JPEGs. A good tool to use if jpegsnoop. Its a windows tool but it appears to have similar restrictions to the openmax decoder.
I'm in the process of looking at eglCreateImage. There is not much available online other than people asking how it works.
If I have any joy I'll let you know.
I'm in the process of looking at eglCreateImage. There is not much available online other than people asking how it works.
If I have any joy I'll let you know.
Re: OpenMAX: How to decode a JPEG (example source code)
Sorry just checked again, that condition should be
Code: Select all
while(pOutputBufferHeader->nFilledLen == 0)
sleep(5);
Re: OpenMAX: How to decode a JPEG (example source code)
Or even better remove the wait logic on pOutputBufferHeader and rely on the EOS events to let you know hen the image has been filled.
Re: OpenMAX: How to decode a JPEG (example source code)
MattOwnby, thanks for sharing your code. I learned a lot with it.MattOwnby wrote:(I am getting 80-150 FPS depending on which test JPEG I use).
You mentioned a very high FPS rate. But when trying your code, with a 640x480 images, I got only ~12 FPS.
Would you mind to inform the JPEG size / resolution you've tested with?
Cheers
Re: OpenMAX: How to decode a JPEG (example source code)
Hi,
Can you please tell me how to get decoded RAW jpeg from Your code? I've analyzed your code but i have no ide which buffer contains decodec data (from openmax.c).
Greetings,
Marcin
Can you please tell me how to get decoded RAW jpeg from Your code? I've analyzed your code but i have no ide which buffer contains decodec data (from openmax.c).
Greetings,
Marcin
Re: OpenMAX: How to decode a JPEG (example source code)
Hi, I am getting 85 FPS on a file that is 31k in size. See below for details.hajjar wrote:MattOwnby, thanks for sharing your code. I learned a lot with it.MattOwnby wrote:(I am getting 80-150 FPS depending on which test JPEG I use).
You mentioned a very high FPS rate. But when trying your code, with a 640x480 images, I got only ~12 FPS.
Would you mind to inform the JPEG size / resolution you've tested with?
Cheers
Code: Select all
pi@raspberrypi /opt/vc/src/hello_pi/hello_jpeg $ ./hello_jpeg.bin /mnt/lair1.jpgWidth: 640 Height: 480 Output Color Format: 0x7f000001 Buffer Size: 1228800
Total elapsed milliseconds: 3531
Total frames decoded: 300
Total frames / second is 84.961767
Write outraw.raw
Code: Select all
pi@raspberrypi /opt/vc/src/hello_pi/hello_jpeg $ ls -l /mnt/lair1.jpg
-rw-r--r-- 1 nobody nogroup 31510 Aug 22 15:41 /mnt/lair1.jpg
Re: OpenMAX: How to decode a JPEG (example source code)
cool!
jumble wrote:Ok so I finall got a version of the JPEG decoder working using the ilclient library. There is one problem with the first frame is always failing to return an eos event apart from that it works ok. This is based entirely on Matt's earlier examples, so same rules this implementation is for educational/non-commercial use only.
I've currently got this code plugged into a program that receives frames from a network CCTV camera and projects onto the cube from the hello triangles sample.
Re: OpenMAX: How to decode a JPEG (example source code)
It writes out the decoded buffer to a file ("output.raw"). Look for the "fwrite" command to see where the buffer is.Speed666 wrote:Hi,
Can you please tell me how to get decoded RAW jpeg from Your code? I've analyzed your code but i have no ide which buffer contains decodec data (from openmax.c).
Greetings,
Marcin
Re: OpenMAX: How to decode a JPEG (example source code)
I have been working on an OpenVG based YouTube client and have attempted to implement this code (the hello_jpeg_v2) in my project.
Essentially, I just modified the JPEG::DoIT function (renamed to OMXJPEG::CreateImageFromBuf)to read data directly from memory and return a VGImage handle. Everything worked except that the images are upside down. For the most part at tried to leave the c++ code intact and modify as little as possible.
It there a simple way to fix that?
Right now I'm manually inverting the image with a CPU loop which does seem like a kludge and kind of defeats the purpose. My loop is down around line 250.
https://github.com/bbond007/raspytube/b ... MXJPEG.cpp
Also, are there any limitations with the re-sizer. It seems like not all horizontal resolutions are supported and some lead to a corrupt image.
Thanks for the nice work!
Essentially, I just modified the JPEG::DoIT function (renamed to OMXJPEG::CreateImageFromBuf)to read data directly from memory and return a VGImage handle. Everything worked except that the images are upside down. For the most part at tried to leave the c++ code intact and modify as little as possible.
It there a simple way to fix that?
Right now I'm manually inverting the image with a CPU loop which does seem like a kludge and kind of defeats the purpose. My loop is down around line 250.
https://github.com/bbond007/raspytube/b ... MXJPEG.cpp
Also, are there any limitations with the re-sizer. It seems like not all horizontal resolutions are supported and some lead to a corrupt image.
Thanks for the nice work!
Re: OpenMAX: How to decode a JPEG (example source code)
Looks promising
It is probably a newby error, but I cannot get it to compile. I downloaded
http://www.rulecity.com/browse/rpi/hello_jpeg_v2.tar.gz
unpacked and put it in /opt/vc/src/hello_pi
But upon make I get:
g++ -c -o OMXComponent.o OMXComponent.cpp
In file included from OMXComponent.cpp:4:0:
OMXComponent.h:8:30: fatal error: IL/OMX_Component.h: No such file or directory
compilation terminated.
make: *** [OMXComponent.o] Error 1
Somehow the path to the headers is not found
It is probably a newby error, but I cannot get it to compile. I downloaded
http://www.rulecity.com/browse/rpi/hello_jpeg_v2.tar.gz
unpacked and put it in /opt/vc/src/hello_pi
But upon make I get:
g++ -c -o OMXComponent.o OMXComponent.cpp
In file included from OMXComponent.cpp:4:0:
OMXComponent.h:8:30: fatal error: IL/OMX_Component.h: No such file or directory
compilation terminated.
make: *** [OMXComponent.o] Error 1
Somehow the path to the headers is not found
Re: OpenMAX: How to decode a JPEG (example source code)
Yout g++ is being called with no include paths (or compile options).Beach wrote:Looks promising
It is probably a newby error, but I cannot get it to compile. I downloaded
http://www.rulecity.com/browse/rpi/hello_jpeg_v2.tar.gz
unpacked and put it in /opt/vc/src/hello_pi
But upon make I get:
g++ -c -o OMXComponent.o OMXComponent.cpp
In file included from OMXComponent.cpp:4:0:
OMXComponent.h:8:30: fatal error: IL/OMX_Component.h: No such file or directory
compilation terminated.
make: *** [OMXComponent.o] Error 1
Somehow the path to the headers is not found
g++ -DSTANDALONE -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_POSIX -D_LINUX -fPIC -DPIC -D_REENTRANT -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -U_FORTIFY_SOURCE -Wall -g -DHAVE_LIBOPENMAX=2 -DOMX -DOMX_SKIP64BIT -ftree-vectorize -pipe -DUSE_EXTERNAL_OMX -DHAVE_LIBBCM_HOST -DUSE_EXTERNAL_LIBBCM_HOST -DUSE_VCHIQ_ARM -Wno-psabi -I/opt/vc/include/ -I/opt/vc/include/interface/vcos/pthreads -I./ -I../libs/ilclient -I../libs/vgfont -g -c OMXComponent.cpp -o OMXComponent.o -Wno-deprecated-declarations
(see you are missing a bunch of stuff)
Makefile calls ../ Makedile.include (/opt/vc/src/hello_pi/Makefile.include) so the problem must be in there... I have included mine for you to compare. I'm not shure if I changed something to get it to compile. Hope it helps.
CFLAGS+=-DSTANDALONE -D__STDC_CONSTANT_MACROS -D__STDC_LIMIT_MACROS -DTARGET_POSIX -D_LINUX -fPIC -DPIC -D_REE
LDFLAGS+=-L$(SDKSTAGE)/opt/vc/lib/ -lGLESv2 -lEGL -lopenmaxil -lbcm_host -lvcos -lvchiq_arm -lpthread -lrt -L.
INCLUDES+=-I$(SDKSTAGE)/opt/vc/include/ -I$(SDKSTAGE)/opt/vc/include/interface/vcos/pthreads -I./ -I../libs/il
all: $(BIN) $(LIB)
%.o: %.c
@rm -f $@
$(CC) $(CFLAGS) $(INCLUDES) -g -c $< -o $@ -Wno-deprecated-declarations
%.o: %.cpp
@rm -f $@
$(CXX) $(CFLAGS) $(INCLUDES) -g -c $< -o $@ -Wno-deprecated-declarations
%.bin: $(OBJS)
$(CC) -o $@ -Wl,--whole-archive $(OBJS) $(LDFLAGS) -Wl,--no-whole-archive -rdynamic
%.a: $(OBJS)
$(AR) r $@ $^
clean:
for i in $(OBJS); do (if test -e "$$i"; then ( rm $$i ); fi ); done
@rm -f $(BIN) $(LIB)
Re: OpenMAX: How to decode a JPEG (example source code)
Apparantly I was missing the 'cpp' bit in the Makefile.include.
However I am still getting errors.
The headers are now found, but I am getting a lot, and I mean a lot, of 'undefinded reference to .....' listings. It results in an error.
I downloaded and copied the latest version of the /opt directory from Git, but the same errors occur. Might be missing some packages?
Appreciate your help
However I am still getting errors.
The headers are now found, but I am getting a lot, and I mean a lot, of 'undefinded reference to .....' listings. It results in an error.
I downloaded and copied the latest version of the /opt directory from Git, but the same errors occur. Might be missing some packages?
Appreciate your help
Re: OpenMAX: How to decode a JPEG (example source code)
I have the same problem.
I actually never got the v2 of this source to compile on its own. I'm not sure I ever tried until now. I did get the code to compile in my Youtube client project somehow.
Anyway, just now, I did get it to compile with this Makefile.
Just replace the Makefile in hello_jpeg with this one and see if it works for you.
If not, there is not much more I can suggest....
I actually never got the v2 of this source to compile on its own. I'm not sure I ever tried until now. I did get the code to compile in my Youtube client project somehow.
Anyway, just now, I did get it to compile with this Makefile.
Just replace the Makefile in hello_jpeg with this one and see if it works for you.
If not, there is not much more I can suggest....
- Attachments
-
- Makefile.zip
- (819 Bytes) Downloaded 999 times
Re: OpenMAX: How to decode a JPEG (example source code)
Thanks for the makefile
It now compiles. One step at a time, because I am getting a runtime error now:
error while loading shared libraries: LibGLesv2.so: cannot open shared object file: No such file or directory
It now compiles. One step at a time, because I am getting a runtime error now:
error while loading shared libraries: LibGLesv2.so: cannot open shared object file: No such file or directory