On Android,
Anyone have any idea what trick snapchat pulls to get such high fps on their camera preview? I have tried various methods:
using a textureview instead of surface view
forcing hardware acceleration
using lower resolutions
using different preview formats (YV12 , NV21 drops frames)
changing focusing mode
None have left me anywhere close to the constant 30fps or maybe even above that snapchat seems to get. I can just about get to the same fps as the stock google camera app, but this isn't great, and mine displays at much lower resolution.
EDIT:
The method used is the same as that used by the official android video recording app. The preview there is of the same image quality and is locked to 30fps.
try this it works
public void takeSnapPhoto() {
camera.setOneShotPreviewCallback(new Camera.PreviewCallback() {
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
Camera.Parameters parameters = camera.getParameters();
int format = parameters.getPreviewFormat();
//YUV formats require more conversion
if (format == ImageFormat.NV21 || format == ImageFormat.YUY2 || format == ImageFormat.NV16) {
int w = parameters.getPreviewSize().width;
int h = parameters.getPreviewSize().height;
// Get the YuV image
YuvImage yuv_image = new YuvImage(data, format, w, h, null);
// Convert YuV to Jpeg
Rect rect = new Rect(0, 0, w, h);
ByteArrayOutputStream output_stream = new ByteArrayOutputStream();
yuv_image.compressToJpeg(rect, 100, output_stream);
byte[] byt = output_stream.toByteArray();
FileOutputStream outStream = null;
try {
// Write to SD Card
File file = createFileInSDCard(FOLDER_PATH, "Image_"+System.currentTimeMillis()+".jpg");
//Uri uriSavedImage = Uri.fromFile(file);
outStream = new FileOutputStream(file);
outStream.write(byt);
outStream.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
}
}
}
});}
I believed they used the android NDK. You can find more information in android developer.
Using pure C/C++ is faster than JAVA code in performance critical tasks such as image and video processing.
You can also try to improve the performance by compiling the application with another compiler, like the Intel compiler.
Related
Is there any way to acquire the preview frame directly in portrait inside onPreviewFrame method?
I've tried:
camera.setDisplayOrientation(90);
but this seems to work only for the display. The doc reports:
Set the clockwise rotation of preview display in degrees. This affects
the preview frames and the picture displayed after snapshot. This
method is useful for portrait mode applications.
This does not affect the order of byte array passed in onPreviewFrame(byte[], Camera), JPEG pictures, or recorded videos.
This method is not allowed to be called during preview.
I'm targetting API level >= 8, and I've a portrait locked app. I want to avoid manually rotating byte array of data passed as frame.
Many thanks in advance.
Try this it will work
public void takeSnapPhoto() {
camera.setOneShotPreviewCallback(new Camera.PreviewCallback() {
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
Camera.Parameters parameters = camera.getParameters();
int format = parameters.getPreviewFormat();
//YUV formats require more conversion
if (format == ImageFormat.NV21 || format == ImageFormat.YUY2 || format == ImageFormat.NV16) {
int w = parameters.getPreviewSize().width;
int h = parameters.getPreviewSize().height;
// Get the YuV image
YuvImage yuv_image = new YuvImage(data, format, w, h, null);
// Convert YuV to Jpeg
Rect rect = new Rect(0, 0, w, h);
ByteArrayOutputStream output_stream = new ByteArrayOutputStream();
yuv_image.compressToJpeg(rect, 100, output_stream);
byte[] byt = output_stream.toByteArray();
FileOutputStream outStream = null;
try {
// Write to SD Card
File file = createFileInSDCard(FOLDER_PATH, "Image_"+System.currentTimeMillis()+".jpg");
//Uri uriSavedImage = Uri.fromFile(file);
outStream = new FileOutputStream(file);
outStream.write(byt);
outStream.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
}
}
}
});}
I tried several things to try to get the camera preview to show up in portrait on a SurfaceView. Nothing worked. I am testing on a Droid that has 2.0.1. I tried:
1) forcing the layout to be portrait by: this.setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
2) using
Camera.Parameters parameters = camera.getParameters();
parameters.set("orientation", "portrait");
parameters.setRotation(90);
camera.setParameters(parameters);
Is there something else I can try? If this a bug in Android or the phone how can I make sure that this is the case so that I have proof to inform the client?
Thanks,
Prasanna
As of API lvl 8, this is available:
public final void setDisplayOrientation (int degrees)
i.e. with portait in the manifest:
public void surfaceCreated(SurfaceHolder holder) {
mCamera = Camera.open();
mCamera.setDisplayOrientation(90);
i have a working solution for portrait mode working in 2.1 (tested on Desire) maybe less.
Activity screen orientation is set to portrait.
(android:screenOrientation="portrait")
the camera parameters:
Camera.Parameters p = mCamera.getParameters();
p.set("jpeg-quality", 100);
p.set("orientation", "landscape");
p.set("rotation", 90);
p.setPictureFormat(PixelFormat.JPEG);
p.setPreviewSize(h, w);// here w h are reversed
mCamera.setParameters(p);
and the image will be portrait.
SurfaceHolder you use for camera must be at a size compatible with phone preview size
usualy screen resolution.
Funny on Desire 2.2 is not working...
Here is the fix:
At surfaceCreated(..) or when you have this line
camera = Camera.open();
add
camera.setDisplayOrientation(90);//only 2.2>
Camera.Parameters p = camera.getParameters();
p.set("jpeg-quality", 100);
p.setRotation(90);
p.setPictureFormat(PixelFormat.JPEG);
p.setPreviewSize(h, w);
camera.setParameters(p);
You can try this (good for 2.2 or below). Here I rotate the image before saving it to sd card. But it is only for portrait mode. If you had to make it for both mode then you should check camera orientation and put some check before capturing image.
PictureCallback jpegCallback = new PictureCallback() {
public void onPictureTaken(byte[] data, Camera camera) {
FileOutputStream outStream = null;
try {
imageFilePath = getFilename();
InputStream is = new ByteArrayInputStream(data);
Bitmap bmp = BitmapFactory.decodeStream(is);
// Getting width & height of the given image.
if (bmp != null){
int w = bmp.getWidth();
int h = bmp.getHeight();
// Setting post rotate to 90
Matrix mtx = new Matrix();
mtx.postRotate(90);
// Rotating Bitmap
Bitmap rotatedBMP = Bitmap.createBitmap(bmp, 0, 0, w, h, mtx, true);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
rotatedBMP.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] byteArray = stream.toByteArray();
outStream = new FileOutputStream
(String.format(imageFilePath,System.currentTimeMillis()));
outStream.write(byteArray);
outStream.close();
} else {
outStream = new FileOutputStream
(String.format(imageFilePath,System.currentTimeMillis()));
outStream.write(data);
outStream.close();
}
preview.camera.startPreview();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
}
}
};
There's no way to do this on many current devices, including the G1 and Droid. Take a look at the relevant bug report here:
http://code.google.com/p/android/issues/detail?id=1193
Also see a comment from one of the Android engineers (Dave) here:
http://groups.google.com/group/android-developers/browse_thread/thread/24dfa452ffc0e049
The link Roman gave of the issue thread has a workable solution that I'm using now.
Find it here:
http://code.google.com/p/android/issues/detail?id=1193#c26
There is no need you have to set any parameters for the orientation until you need to do that explicitly. By Default, it supports this facility. In my case, i have a Activity and above that activity i have a camera view, so i didn't set any orientation for the camera properties, instead for the activity i set the orientation as portrait in the Manifest file. now the app looks and works good. Might be helpful for some one..
Thanks.
When trying to convert the byte[] of Camera.onPreviewFrame to Bitamp using BitmapFactory.decodeByteArray gives me an error SkImageDecoder::Factory returned null
Following is my code:
public void onPreviewFrame(byte[] data, Camera camera) {
Bitmap bmp=BitmapFactory.decodeByteArray(data, 0, data.length);
}
This has been hard to find! But since API 8, there is a YuvImage class in android.graphics. It's not an Image descendent, so all you can do with it is save it to Jpeg, but you could save it to memory stream and then load into Bitmap Image if that's what you need.
import android.graphics.YuvImage;
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
try {
Camera.Parameters parameters = camera.getParameters();
Size size = parameters.getPreviewSize();
YuvImage image = new YuvImage(data, parameters.getPreviewFormat(),
size.width, size.height, null);
File file = new File(Environment.getExternalStorageDirectory()
.getPath() + "/out.jpg");
FileOutputStream filecon = new FileOutputStream(file);
image.compressToJpeg(
new Rect(0, 0, image.getWidth(), image.getHeight()), 90,
filecon);
} catch (FileNotFoundException e) {
Toast toast = Toast
.makeText(getBaseContext(), e.getMessage(), 1000);
toast.show();
}
}
Since Android 3.0 you can use a TextureView and TextureSurface to display the camera, and then use mTextureView.getBitmap() to retrieve a friendly RGB preview frame.
A very skeletal example of how to do this is given in the TextureView docs. Note that you'll have to set your application or activity to be hardware accelerated by putting android:hardwareAccelerated="true" in the manifest.
I found the answer after a long time. Here it is...
Instead of using BitmapFactory, I used my custom method to decode this byte[] data to a valid image format. To decode the image to a valid image format, one need to know what picture format is being used by the camera by calling camera.getParameters().getPictureFormat(). This returns a constant defined by ImageFormat. After knowing the format, use the appropriate encoder to encode the image.
In my case, the byte[] data was in the YUV format, so I looked for YUV to BMP conversion and that solved my problem.
you can try this:
This example send camera frames to server
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
try {
byte[] baos = convertYuvToJpeg(data, camera);
StringBuilder dataBuilder = new StringBuilder();
dataBuilder.append("data:image/jpeg;base64,").append(Base64.encodeToString(baos, Base64.DEFAULT));
mSocket.emit("newFrame", dataBuilder.toString());
} catch (Exception e) {
Log.d("########", "ERROR");
}
}
};
public byte[] convertYuvToJpeg(byte[] data, Camera camera) {
YuvImage image = new YuvImage(data, ImageFormat.NV21,
camera.getParameters().getPreviewSize().width, camera.getParameters().getPreviewSize().height, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int quality = 20; //set quality
image.compressToJpeg(new Rect(0, 0, camera.getParameters().getPreviewSize().width, camera.getParameters().getPreviewSize().height), quality, baos);//this line decreases the image quality
return baos.toByteArray();
}
I am writing an app to capture the camera preview frames and convert it to bitmap in Android. Here is my code:
Camera.PreviewCallback previewCallback = new Camera.PreviewCallback()
{
public void onPreviewFrame(byte[] data, Camera camera)
{
try
{
BitmapFactory.Options opts = new BitmapFactory.Options();
Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);//,opts);
}
catch(Exception e)
{
}
}
};
mCamera = Camera.open();
mCamera.setPreviewCallback(previewCallback);
After I start preview, the callback got called with data, but the bitmap is null.
What did I do wrong when convert the byte array to BitMap?
In the onPreviewFrame() function, you should check the image format first.
This the NV21 example.
public void onPreviewFrame(byte[] data, Camera camera)
{
Parameters parameters = camera.getParameters();
imageFormat = parameters.getPreviewFormat();
if (imageFormat == ImageFormat.NV21)
{
Rect rect = new Rect(0, 0, PreviewSizeWidth, PreviewSizeHeight);
YuvImage img = new YuvImage(data, ImageFormat.NV21, PreviewSizeWidth, PreviewSizeHeight, null);
OutputStream outStream = null;
File file = new File(NowPictureFileName);
try
{
outStream = new FileOutputStream(file);
img.compressToJpeg(rect, 100, outStream);
outStream.flush();
outStream.close();
}
catch (FileNotFoundException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
For another way to take pictures, check out this article: how to use camera in android
Have you tried decoding the preview frame data to RGB before you use BitmapFactory? The default format is YUV which I'm not sure is compatible with BitmapFactory. Dave Manpearl's decode method can be found here:
Getting frames from Video Image in Android
Let me know if it works.
Cheers,
Paul
I tried several things to try to get the camera preview to show up in portrait on a SurfaceView. Nothing worked. I am testing on a Droid that has 2.0.1. I tried:
1) forcing the layout to be portrait by: this.setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
2) using
Camera.Parameters parameters = camera.getParameters();
parameters.set("orientation", "portrait");
parameters.setRotation(90);
camera.setParameters(parameters);
Is there something else I can try? If this a bug in Android or the phone how can I make sure that this is the case so that I have proof to inform the client?
Thanks,
Prasanna
As of API lvl 8, this is available:
public final void setDisplayOrientation (int degrees)
i.e. with portait in the manifest:
public void surfaceCreated(SurfaceHolder holder) {
mCamera = Camera.open();
mCamera.setDisplayOrientation(90);
i have a working solution for portrait mode working in 2.1 (tested on Desire) maybe less.
Activity screen orientation is set to portrait.
(android:screenOrientation="portrait")
the camera parameters:
Camera.Parameters p = mCamera.getParameters();
p.set("jpeg-quality", 100);
p.set("orientation", "landscape");
p.set("rotation", 90);
p.setPictureFormat(PixelFormat.JPEG);
p.setPreviewSize(h, w);// here w h are reversed
mCamera.setParameters(p);
and the image will be portrait.
SurfaceHolder you use for camera must be at a size compatible with phone preview size
usualy screen resolution.
Funny on Desire 2.2 is not working...
Here is the fix:
At surfaceCreated(..) or when you have this line
camera = Camera.open();
add
camera.setDisplayOrientation(90);//only 2.2>
Camera.Parameters p = camera.getParameters();
p.set("jpeg-quality", 100);
p.setRotation(90);
p.setPictureFormat(PixelFormat.JPEG);
p.setPreviewSize(h, w);
camera.setParameters(p);
You can try this (good for 2.2 or below). Here I rotate the image before saving it to sd card. But it is only for portrait mode. If you had to make it for both mode then you should check camera orientation and put some check before capturing image.
PictureCallback jpegCallback = new PictureCallback() {
public void onPictureTaken(byte[] data, Camera camera) {
FileOutputStream outStream = null;
try {
imageFilePath = getFilename();
InputStream is = new ByteArrayInputStream(data);
Bitmap bmp = BitmapFactory.decodeStream(is);
// Getting width & height of the given image.
if (bmp != null){
int w = bmp.getWidth();
int h = bmp.getHeight();
// Setting post rotate to 90
Matrix mtx = new Matrix();
mtx.postRotate(90);
// Rotating Bitmap
Bitmap rotatedBMP = Bitmap.createBitmap(bmp, 0, 0, w, h, mtx, true);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
rotatedBMP.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] byteArray = stream.toByteArray();
outStream = new FileOutputStream
(String.format(imageFilePath,System.currentTimeMillis()));
outStream.write(byteArray);
outStream.close();
} else {
outStream = new FileOutputStream
(String.format(imageFilePath,System.currentTimeMillis()));
outStream.write(data);
outStream.close();
}
preview.camera.startPreview();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
}
}
};
There's no way to do this on many current devices, including the G1 and Droid. Take a look at the relevant bug report here:
http://code.google.com/p/android/issues/detail?id=1193
Also see a comment from one of the Android engineers (Dave) here:
http://groups.google.com/group/android-developers/browse_thread/thread/24dfa452ffc0e049
The link Roman gave of the issue thread has a workable solution that I'm using now.
Find it here:
http://code.google.com/p/android/issues/detail?id=1193#c26
There is no need you have to set any parameters for the orientation until you need to do that explicitly. By Default, it supports this facility. In my case, i have a Activity and above that activity i have a camera view, so i didn't set any orientation for the camera properties, instead for the activity i set the orientation as portrait in the Manifest file. now the app looks and works good. Might be helpful for some one..
Thanks.