Develop Reference

Converting specialized NV12 video frames to RGB

I have an H264 stream that's decoded using an Android MediaCodec. When I query the output MediaFormat, the color format is 2141391875. Apparently, that's a specialized NV12 variant known as HAL_PIXEL_FORMAT_NV12_ADRENO_TILED. This is on a Nexus 7 (2013).
I want to take this data and convert it to RGB so I can create a Bitmap. I've found StackOverflow posts for converting other formats to RGB, not this format. I've tried code from those other posts, the result is just streaks of color. (To view the Bitmap, I draw on the Canvas associated with a Surface, as well as write it out as a JPEG -- it looks the same in both cases.)
How can I convert this particular data to RGB?

2141391875 decimal is 0x7FA30C03 in hex, which according to this header file is OMX_QCOM_COLOR_FormatYUV420PackedSemiPlanar64x32Tile2m8ka. Which amounts to the same thing as the constant you found: this is a proprietary Qualcomm color format.
The easiest (and fastest) way to convert it is to let OpenGL ES do the work for you. See for example ExtractMpegFramesTest, which decodes video frames to a SurfaceTexture, renders the texture to an off-screen surface, and then reads the pixels out with glReadPixels(). The GLES driver will handle the RGB conversion for you.
If you want to do the conversion yourself, you will need to reverse-engineer the color format, or find someone who has done so already and is willing to share.

Android processing a video to CMYK

I am working on a project and in that project I need to be able to:
get a frame of a video
change it to a CMYK
try to render that frame
Do you guys have any suggestions?
To get the frames I think I can use ffmpeg.
I think the person that wrote
Android processing a video, YCrCb frames to video
can help me but I'm open to any suggestions.
I appreciate all the help I can get, thank you

Assuming you know how to convert YUV to RGB, the remaining problem is RGB to CMYK. This can be very simple (C = 1 - R, M = 1 - G, Y = 1 - B) or very complex (3D LUT implementing a complex gamut mapping based on tone curves and ink coverages, etc.). You can assume video has standard Rec.709 gamma but you'll need to better define what kind of CMYK you're looking for and if they are linear or some other mapping.

Optimize an image for text recognition using tesseract

I have user tesseract ocr for my android project to recognize text from an image taken from the camera. But the results are not accurate. I want to optimize the image using opencv. I want to achieve the following for the captured image which is decoded in Bitmap.Config.ARGB_8888 format:
Detect the objects in the resized image.
Once the object is identified, compute its border w.r.t original image. (This is for removing the camera angle effect)
Extract the object from original image, by applying perspective transform.
Apply white balance to remove lightening effects.
In the example provided by with the tess_two api, they are using Leptonica for the image manipulations like drawing the bounding boxes around the words..But in my case I want to use OpenCV...Your guidance will be highly appreciated...

That's a lot you are asking for, and depending on the object may be impossible. You should check out the tutorials on 2D feature detection and object detection (http://docs.opencv.org/doc/tutorials/features2d/table_of_content_features2d/table_of_content_features2d.html and http://docs.opencv.org/doc/tutorials/objdetect/table_of_content_objdetect/table_of_content_objdetect.html) to see if there is something you can use.
White balance does not do anything to lighting, you should do adaptive thresholding or some kind of high pass filtering instead.

Android - show processed camera preview [duplicate]

I want to use the onPreviewFrame to post-process the image before displaying it to the user (i.e. apply a color tint, sepia, etc). As I understand, the byte[] data returned to the callback is encoded in YUV420sp. Have people been decoding this to RGB in Java or using NDK (native code)? Does anyone have an example of a function that decodes this to RGB and how the RGB values are used afterwards?
Thanks.

I found a sample application that translates the YUV420 into RGB and displays (sort of) real time histograms over the preview image.
http://www.stanford.edu/class/ee368/Android/index.html

This helps?
Parameters params = mCamera.getParameters();
param.setPreviewFormat(ImageFormat.RGB_565);
mCamera.setParameters(param);
First check if rgb is supported
http://developer.android.com/reference/android/hardware/Camera.Parameters.html#getPreviewFormat%28%29
and then set preview format to rgb
http://developer.android.com/reference/android/hardware/Camera.Parameters.html#setPreviewFormat%28int%29

Image processing with Android Camera

I want to use the onPreviewFrame to post-process the image before displaying it to the user (i.e. apply a color tint, sepia, etc). As I understand, the byte[] data returned to the callback is encoded in YUV420sp. Have people been decoding this to RGB in Java or using NDK (native code)? Does anyone have an example of a function that decodes this to RGB and how the RGB values are used afterwards?
Thanks.

I found a sample application that translates the YUV420 into RGB and displays (sort of) real time histograms over the preview image.
http://www.stanford.edu/class/ee368/Android/index.html

This helps?
Parameters params = mCamera.getParameters();
param.setPreviewFormat(ImageFormat.RGB_565);
mCamera.setParameters(param);
First check if rgb is supported
http://developer.android.com/reference/android/hardware/Camera.Parameters.html#getPreviewFormat%28%29
and then set preview format to rgb
http://developer.android.com/reference/android/hardware/Camera.Parameters.html#setPreviewFormat%28int%29