VideoSurfaceView - render to file - android

I'm using VideoSurfaceView to render filtered video. I'm doing it buy changing the fragment shader according to my needs. Now I would like to save/render the video after the changes to a file of the same format(Ex. mp4 - h264) but couldn't find how to do it.
PS - saving texture as bitmap and the bitmap to a file is easy but I could find how to do it with videos..
Any experts here?

As you already found out and said in the comments, OpenGL can't export multiple frames as a video.
Though if you simply want to filter/process each frame of a video, then you don't need OpenGL at all, and you don't need a Fragment Shader, you can simply loop through all the pixels yourself.
Now let's say that you process your video one frame at a time, and each frame is a BufferedImage, you can of course use whatever you want or get provided with, as long as you have the option to get and set pixels.
I'm simply supplying you with a way of calculating and applying a filter, you will have to do the decoding and encoding of the video file yourself.
But back to the BufferedImage, first we want to get all the pixels in our BufferedImage, we do that using the following.
BufferedImage bi = ...; // Here you would get a frame from the video
int width = bi.getWidth();
int height = bi.getHeight();
int[] pixels = ((DataBufferInt) bi.getRaster().getDataBuffer()).getData();
Be aware that depending on the type of image and if the image contains transparency, the DataBuffer might vary between a DataBufferInt to DataBufferByte, etc. You can read about the different DataBuffers in the Oracle Docs, click here.
Now simply by looping through the pixels from the image, then we can apply and create any kind of effect and filtering.
Let's say we want to create a grayscale effect also called a black-and-white effect, you would then do that by the following.
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
final int index = x + y * width;
final int pixel = pixels[index];
final int alpha = (pixel >> 24) & 0xFF;
final int red = (pixel >> 16) & 0xFF;
final int green = (pixel >> 8) & 0xFF;
final int blue = pixel & 0xFF;
final int gray = (red + green + blue) / 3;
pixels[index] = alpha << 24 | gray << 16 | gray << 8 | gray;
}
}
Now you can simply save the image again, or do anything else you would like to do. Though you can also use and draw the BufferedImage, because the pixel array provided by the BufferedImage will of course change the BufferedImage as well.
Important if you want to perform a blur effect, then after you calculate each pixel store it into another array, because performing a blur effect, requires the surrounding pixels. Therefore it you replace the old once while you calculate all the pixels, some of the pixels will use the calculated values instead of the actual value.
The above code also works for images as well of course.
Extra
If you want to get RGBA values which is stored in a single int then you can do the following.
int pixel = 0xFFFF8040; // This is a random testing value
int alpha = (pixel >> 24) & 0xFF; // Would equal 255 using the testing value
int red = (pixel >> 16) & 0xFF; // ... 255 ...
int green = (pixel >> 8) & 0xFF; // ... 128 ...
int blue = pixel & 0xFF; // ... 64 ...
Then if you have the RGBA values and want to combine them to a single int then you can do the following.
int alpha = 255;
int red = 255;
int green = 128;
int blue = 64;
int pixel = alpha << 24 | red << 16 | green << 8 | blue;
If you only have the RGB values then you just do, either red << 16 | green << 8 | blue or you do 255 << 24 | red << 16 | green << 8 | blue

Related

What are these bright yellow/red/pink colors I get from subtracting two images?

I'm making an Android app, in which the user takes two images and the first is "subtracted" from the second on a pixel-by-pixel basis.
Essentially, the two Bitmaps are converted to 2D int arrays, and the image subtraction is performed using the following method:
private int[][] pixelmapDifference(int[][] subtrahend, int[][] minuend) {
int[][] diff = new int[subtrahend.length][subtrahend[0].length];
for (int x = 0; x < diff.length; x++) {
for (int y = 0; y < diff[0].length; y++) {
diff[x][y] = minuend[x][y] - subtrahend[x][y];
}
}
return diff;
}
The resultant 2D array is then converted to a Bitmap. This is what the 3 images look like (first, second, and difference).
How do I account for this? I'd like to just get the difference between the two, in this case just the water.
You are subtracting always second from the first. What happens when second one is brighter? Value returned is below zero. I'm not one hundred percent sure what will happen, but documentation says that color is
int color = (A & 0xff) << 24 | (R & 0xff) << 16 | (G & 0xff) << 8 | (B & 0xff);
So as you substract lighter from darker in some situations outcome are these strange dots.

Huge negative values extracted by using getPixel() method

I am having a problem with an image processing app I am developing (newbie here). I am trying to extract the value of specific pixels by using the getPixel() method.
I am having a problem though. The number I get from this method is a huge negative number, something like -1298383. Is this normal? How can I fix it?
Thanks.
I'm not an expert, but to me it looks like you are getting the hexadecimal value. Perhaps you want something more understandable like the value of each RGB layer.
To unpack a pixel into its RGB values you should do something like:
private short[][] red;
private short[][] green;
private short[][] blue;
/**
* Map each intensity of an RGB colour into its respective colour channel
*/
private void unpackPixel(int pixel, int row, int col) {
red[row][col] = (short) ((pixel >> 16) & 0xFF);
green[row][col] = (short) ((pixel >> 8) & 0xFF);
blue[row][col] = (short) ((pixel >> 0) & 0xFF);
}
And after changes in each channel you can pack the pixel back.
/**
* Create an RGB colour pixel.
*/
private int packPixel(int red, int green, int blue) {
return (red << 16) | (green << 8) | blue;
}
Sorry if it is not what you are looking for.
You can get the pixel from the view like this:
ImageView imageView = ((ImageView)v);
Bitmap bitmap = ((BitmapDrawable)imageView.getDrawable()).getBitmap();
int pixel = bitmap.getPixel(x,y);
Now you can get each channel with:
int redValue = Color.red(pixel);
int blueValue = Color.blue(pixel);
int greenValue = Color.green(pixel);
getPixel() returns the Color at the specified location. Throws an exception if x or y are out of bounds (negative or >= to the width or height respectively).
The returned color is a non-premultiplied ARGB value.

glReadPixels without loss of precision (Android)

I'm having a lot of trouble converting unique integers (index numbers) into unique float colours that are interpretable by an RGB565 OpenGL surface. When I assign unique colours, more often than not they are drawn as slightly different values due to loss of precision, so when I read the colour with glReadPixels and try to convert it back into a float for comparison, they are not equal.
I posted a similar question here OpenGL ES 2.0 solid colour & colour value precision issue but failed to implement the answer I was given, can anyone give me specifics (code and explanation) for this?
If you only need 605 unique values, then 10 bits of precision (up to 1024 values) should be enough.
RGB565 has 16 bits of precision, so you can use the 6 extra bits of precision as a form of error-correction by spacing the values out so that if there is a small adjustment of the values through rounding or dithering or whatever, you can set it to the closest valid value.
So, assign 3 of your 10 bits to R, 4 to G and 3 to B.
For example red and blue have a range of 0-31, but you only need 8 possible values (3 bits), so you only store the values 2, 6, 10, 14, 18, 22, 26, 30. When scaled up to 8 bits, these values will be 16, 48, 80, 112, 144, 176, 208, 240. Then when you reconstruct the index, any value in the range of 0-31 is interpreted as a 0, 32-63 is a 1, 64-95 is a 2 and so on (this can be done with a simple bit-shift). That way small errors of +/- a small amount won't matter.
void assignID(int regionnumber)
{
int UBR=31; //Upper boundary for blue and red
int UG=63; //Upper boundary for green
// split regionnumber into 3/4/3 bits:
int R = (regionnumber >> 7) & 7;
int G = (regionnumber >> 3) & 15;
int B = regionnumber & 7;
// space out the values by multiplying by 4 and adding 2:
R = R * 4 + 2;
G = G * 4 + 2;
B = B * 4 + 2;
// combine into an RGB565 value if you need it:
int RGB565 = (R << 11) | (G << 5) | B;
// assign the colors
regions[regionnumber].mColorID[0] = ((float)R)/UBR;
regions[regionnumber].mColorID[1] = ((float)G)/UG; // careful, there was a bug here
regions[regionnumber].mColorID[2] = ((float)B)/UBR;
}
Then at the other end, when you read a value from the screen, convert the RGB values back to integers with 3, 4 and 3 bits each and reconstruct the region:
int R = (b[0] & 0xFF) >> 5;
int G = (b[1] & 0xFF) >> 4;
int B = (b[2] & 0xFF) >> 5;
int regionnumber = (R << 7) | (G << 3) | B;

Convert RGB565 to Greyscale (8 bits)

I am trying to convert a rgb565 image (video stream from the Android phone camera) into a greyscale (8 bits) image.
So far I got to the following code (the conversion is computed in native code using the Android NDK). Note that my input image is 640*480 and I want to crop it to make it fit in a 128*128 buffer.
#define RED(a) ((((a) & 0xf800) >> 11) << 3)
#define GREEN(a) ((((a) & 0x07e0) >> 5) << 2)
#define BLUE(a) (((a) & 0x001f) << 3)
typedef unsigned char byte;
void toGreyscale(byte *rgbs, int widthIn, int heightIn, byte *greyscales)
{
const int textSize = 128;
int x,y;
short* rgbPtr = (short*)rgbs;
byte *greyPtr = greyscales;
// rgbs arrives in RGB565 (16 bits) format
for (y=0; y<textSize; y++)
{
for (x=0; x<textSize; x++)
{
short pixel = *(rgbPtr++);
int r = RED(pixel);
int g = GREEN(pixel);
int b = BLUE(pixel);
*(greyPtr++) = (byte)((r+g+b) / 3);
}
rgbPtr += widthIn - textSize;
}
}
The image is sent to the function like this
jbyte* cImageIn = env->GetByteArrayElements(imageIn, &b);
jbyte* cImageOut = (jbyte*)env->GetDirectBufferAddress(imageOut);
toGreyscale((byte*)cImageIn, widthIn, heightIn, (byte*)cImageOut);
The result I get is a horizontally-reversed image (no idea why...the UVs to display the result are correct...), but the biggest problem is that only the red channel is actually correct when I display them separately. The green and blue channels are all messed up and I have no idea why. I checked on the Internet and all the resources I found showed that the masks I am using are correct. Any idea where the mistake could be?
Thanks!
May be an endianess issue?
You could check quickly by reversing the two bytes of your 16 bits word before shifting out the RGB components.

How to apply, converting image from colored to grayscale algorithm to Android?

I am trying to use one of these algorithms to convert a RGB image to grayscale:
The lightness method averages the most prominent and least prominent colors:
(max(R, G, B) + min(R, G, B)) / 2.
The average method simply averages the values: (R + G + B) / 3.
The formula for luminosity is 0.21 R + 0.71 G + 0.07 B.
But I get very weird results! I know there are other ways to acheive this but is it possible to do this way?
Here is the code:
for(int i = 0 ; i < eWidth*eHeight;i++){
int R = (pixels[i] >> 16) ; //bitwise shifting
int G = (pixels[i] >> 8) ;
int B = pixels[i] ;
int gray = (R + G + B )/ 3 ;
pixels[i] = (gray << 16) | (gray << 8) | gray ;
}
You need to strip off the bits that aren't part of the component you're getting, especially if there's any sign extension going on in the shifts.
int R = (q[i] >> 16) & 0xff ; //bitwise shifting
int G = (q[i] >> 8) & 0xff ;
int B = q[i] & 0xff ;
What you made looks allright to me..
I once did this, in java, in much the same way.
Getting the average of the 0-255 color values of RGB, to get grayscale, and it looks alot like yours.
public int getGray(int row, int col) throws Exception
{
checkInImage(row,col);
int[] rgb = this.getRGB(row,col);
return (int) (rgb[0]+rgb[1]+rgb[2])/3;
}
I understand you are not asking for hoe to code this, but for algorithm?
There is no "correct" algorithm as per http://www.dfanning.com/ip_tips/color2gray.html
They use
Y = 0.3*R + 0.59*G + 0.11*B
You can certainly modify each pixel in Java, but that's very inefficient. If you have the option, I would use a ColorMatrix. See the Android documentation for details: http://developer.android.com/resources/samples/ApiDemos/src/com/example/android/apis/graphics/ColorMatrixSample.html
You could set the matrix' saturation to 0 to make it grayscale.
IF you really want to do it in Java, you can do it the way you did it, but you'll need to mask out each element first, i.e. apply & 0xff to it.

Categories

Resources