My goal is to add an overlay on the camera preview that will find book edges. For that, I override the onPreviewFrame where I do the following:
public void onPreviewFrame(byte[] data, Camera camera) {
Camera.Parameters parameters = camera.getParameters();
int width = parameters.getPreviewSize().width;
int height = parameters.getPreviewSize().height;
Mat mat = new Mat((int) (height*1.5), width, CvType.CV_8UC1);
mat.put(0,0,data);
byte[] bytes = new byte[(int) (height*width*1.5)];
mat.get(0,0,bytes);
if (!test) { //to only do once
File pictureFile = getOutputMediaFile();
try {
FileOutputStream fos = new FileOutputStream(pictureFile);
fos.write(bytes);
fos.close();
Uri picUri = Uri.fromFile(pictureFile);
updateGallery(picUri);
test = true;
} catch (IOException e) {
e.printStackTrace();
}
}
}
For now I simply want to take one of the previews and save it after the conversion to mat.
After spending countless hours getting the above to look right, the saved picture cannot be seen on my testing phone (LG Leon). I can't seem to find the issue. Am I mixing the height/width because I'm taking pictures in portrait mode? I tried switching them and still doesn't work. Where is the problem?
The fastest method I managed to find is described HERE in my recently asked question. You can find the method to extract the image in the answer I wrote in my question below. The thing is that the image you get through onPreviewFrame() is NV21. After receiving this image it may be that you need to convert it to RGB (depends on what do you want to achieve; this is also done in the answer I gave you previously).
Seems quite inefficient but it works for me (for now):
//get the camera parameters
Camera.Parameters parameters = camera.getParameters();
int width = parameters.getPreviewSize().width;
int height = parameters.getPreviewSize().height;
//convert the byte[] to Bitmap through YuvImage;
//make sure the previewFormat is NV21 (I set it so somewhere before)
YuvImage yuv = new YuvImage(data, parameters.getPreviewFormat(), width, height, null);
ByteArrayOutputStream out = new ByteArrayOutputStream();
yuv.compressToJpeg(new Rect(0, 0, width, height), 70, out);
Bitmap bmp = BitmapFactory.decodeByteArray(out.toByteArray(), 0, out.size());
//convert Bitmap to Mat; note the bitmap config ARGB_8888 conversion that
//allows you to use other image processing methods and still save at the end
Mat orig = new Mat();
bmp = bmp.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(bmp, orig);
//here you do whatever you want with the Mat
//Mat to Bitmap to OutputStream to byte[] to File
Utils.matToBitmap(orig, bmp);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.JPEG, 70, stream);
byte[] bytes = stream.toByteArray();
File pictureFile = getOutputMediaFile();
try {
FileOutputStream fos = new FileOutputStream(pictureFile);
fos.write(bytes);
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
Related
I am trying face detection and adding mask(graphic overlay) using google vision api ,the problem is i could not get the ouptut from camera after detecting and adding mask.so far I have tried this solution from github , https://github.com/googlesamples/android-vision/issues/24 ,based on this issue i have added a custom detector class,
Mobile Vision API - concatenate new detector object to continue frame processing . and added this on mydetector class How to create Bitmap from grayscaled byte buffer image? .
MyDetectorClass
class MyFaceDetector extends Detector<Face>
{
private Detector<Face> mDelegate;
MyFaceDetector(Detector<Face> delegate) {
mDelegate = delegate;
}
public SparseArray<Face> detect(Frame frame) {
// *** add your custom frame processing code here
ByteBuffer byteBuffer = frame.getGrayscaleImageData();
byte[] bytes = byteBuffer.array();
int w = frame.getMetadata().getWidth();
int h = frame.getMetadata().getHeight();
YuvImage yuvimage=new YuvImage(bytes, ImageFormat.NV21, w, h, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
yuvimage.compressToJpeg(new Rect(0, 0, w, h), 100, baos); // Where 100 is the quality of the generated jpeg
byte[] jpegArray = baos.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Log.e("got bitmap","bitmap val " + bitmap);
return mDelegate.detect(frame);
}
public boolean isOperational() {
return mDelegate.isOperational();
}
public boolean setFocus(int id) {
return mDelegate.setFocus(id);
}
}
frame processing
public SparseArray<Face> detect(Frame frame)
{
// *** add your custom frame processing code here
ByteBuffer byteBuffer = frame.getGrayscaleImageData();
byte[] bytes = byteBuffer.array();
int w = frame.getMetadata().getWidth();
int h = frame.getMetadata().getHeight();
YuvImage yuvimage=new YuvImage(bytes, ImageFormat.NV21, w, h, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
yuvimage.compressToJpeg(new Rect(0, 0, w, h), 100, baos); // Where 100 is the quality of the generated jpeg
byte[] jpegArray = baos.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Log.e("got bitmap","bitmap val " + bitmap);
return mDelegate.detect(frame);
}
i am getting a rotated bitmap ,that is without the mask (graphic overlay) i have added .How can i get the camera output with mask .
Thanks in advance.
The simple answer is: You can't.
Why? Android camera output frames in NV21 ByteBuffer. And you must generate your masks based on the landmarks points in a separated Bitmap, then join them.
Sorry but, that's how the Android Camera API work. Nothing can be done. You must do it manually.
Also, I wouldn't get the camera preview then convert it to YuvImage then to Bitmap. That process consumes a lot of resources and makes preview very very slow. Instead I would use this method which will be a lot faster and rotates your preview internally so you don't loose time doing it:
outputFrame = new Frame.Builder().setImageData(mPendingFrameData, mPreviewSize.getWidth(), mPreviewSize.getHeight(), ImageFormat.NV21)
.setId(mPendingFrameId)
.setTimestampMillis(mPendingTimeMillis)
.setRotation(mRotation)
.build();
mDetector.receiveFrame(outputFrame);
All the code can be found in CameraSource.java
I know, this is an old question, but I've got problems with encoding a byte[] into a bitmap...
Background: I'm writing an Andoid-App which receives picturebytes via UDP, encodes them into a bitmap and displays the picture in an image view.
Since my functions didn't work, I cancelled the UDP-Connection for testing and wrote all the image-bytes in a huge variable. So they're all correct...
The function returns "null".
The function I'm using:
public Bitmap ByteArrayToImage(byte[] imageData)
{
var bmpOutput = BitmapFactory.DecodeByteArray(imageData, 0, imageData.Length);
return bmpOutput;
}
another function I tried out:
public Bitmap ByteArrayToImage2(byte[] imageData)
{
Bitmap bmpReturn;
bmpReturn = (Android.Graphics.Bitmap) Android.Graphics.Bitmap.FromArray<byte>(imageData);
return bmpReturn;
}
A function I found in the internet:
public static Bitmap bytesToUIImage (byte[] bytes)
{
if (bytes == null)
return null;
Bitmap bitmap;
var documentsFolder = Environment.GetFolderPath (Environment.SpecialFolder.Personal);
//Create a folder for the images if not exists
System.IO.Directory.CreateDirectory(System.IO.Path.Combine (documentsFolder, "images"));
string imatge = System.IO.Path.Combine (documents, "images", "image.jpg");
System.IO.File.WriteAllBytes(imatge, bytes.Concat(new Byte[]{(byte)0xD9}).ToArray());
bitmap = BitmapFactory.DecodeFile(imatge);
return bitmap;
}
Most unfortunately, the last function didn't work as well, but here I have do admit, that I was a bit confused about the 'documents' in
string imatge = System.IO.Path.Combine (documents, "images", "image.jpg");
I got an error and changed it into documentsFolder since i guess, that should (or could) be right....
Thank you in advance for your help
it seems, I found the error...
I stored the public Bitmap ByteArrayToImage(byte[] imageData) in another class. I don't know why, but when I decode the Bytearray in the class that also receives the array, all works fine...
If someone knows the reason, feel welcome to let me know, but for now I'm happy ;-)
I did something similar
On sender side:
Camera.Parameters parameters = camera.getParameters();
if (parameters.getPreviewFormat() == ImageFormat.NV21) {
Rect rect = new Rect(0, 0, parameters.getPreviewSize().width, parameters.getPreviewSize().height);
YuvImage yuvimage = new YuvImage(data, ImageFormat.NV21, parameters.getPreviewSize().width, parameters.getPreviewSize().height, null);
ByteArrayOutputStream os = new ByteArrayOutputStream();
yuvimage.compressToJpeg(rect, 75, os);
byte[] videoFrame = os.toByteArray();
//send the video frame to reciever
}
On receiving side:
DataInputStream dIn = new DataInputStream(socket.getInputStream());
int length = 0;
length = dIn.readInt();
if (length > 0) {
byte[] message = new byte[length];
dIn.readFully(message, 0, message.length);
BitmapFactory.Options options = new BitmapFactory.Options();
options.inSampleSize = 4;
final Bitmap bitmap = BitmapFactory.decodeByteArray(message, 0, message.length, options);
ReceiverActivity.this.runOnUiThread(new Runnable() {
#Override
public void run() {
imgPreview.setImageBitmap(bitmap);
}
});
There is a built in method to decode a byte array into a bitmap. The problem comes when we are talking of big images. With small ones you can use:
Bitmap bmp = BitmapFactory.DecodeByteArray (data, 0, data.length);
Be aware. Those bitmaps are not mutable, so you will not be able to use canvases on those. To make them mutable go to: BitmapFactory.decodeResource returns a mutable Bitmap in Android 2.2 and an immutable Bitmap in Android 1.6
Is there any way to acquire the preview frame directly in portrait inside onPreviewFrame method?
I've tried:
camera.setDisplayOrientation(90);
but this seems to work only for the display. The doc reports:
Set the clockwise rotation of preview display in degrees. This affects
the preview frames and the picture displayed after snapshot. This
method is useful for portrait mode applications.
This does not affect the order of byte array passed in onPreviewFrame(byte[], Camera), JPEG pictures, or recorded videos.
This method is not allowed to be called during preview.
I'm targetting API level >= 8, and I've a portrait locked app. I want to avoid manually rotating byte array of data passed as frame.
Many thanks in advance.
Try this it will work
public void takeSnapPhoto() {
camera.setOneShotPreviewCallback(new Camera.PreviewCallback() {
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
Camera.Parameters parameters = camera.getParameters();
int format = parameters.getPreviewFormat();
//YUV formats require more conversion
if (format == ImageFormat.NV21 || format == ImageFormat.YUY2 || format == ImageFormat.NV16) {
int w = parameters.getPreviewSize().width;
int h = parameters.getPreviewSize().height;
// Get the YuV image
YuvImage yuv_image = new YuvImage(data, format, w, h, null);
// Convert YuV to Jpeg
Rect rect = new Rect(0, 0, w, h);
ByteArrayOutputStream output_stream = new ByteArrayOutputStream();
yuv_image.compressToJpeg(rect, 100, output_stream);
byte[] byt = output_stream.toByteArray();
FileOutputStream outStream = null;
try {
// Write to SD Card
File file = createFileInSDCard(FOLDER_PATH, "Image_"+System.currentTimeMillis()+".jpg");
//Uri uriSavedImage = Uri.fromFile(file);
outStream = new FileOutputStream(file);
outStream.write(byt);
outStream.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
}
}
}
});}
Is there a way to rotate byte array without decoding it to Bitmap?
Currently in jpeg PictureCallback I just write byte array directly to file. But pictures are rotated. I would like to rotate them without decoding to bitmap with hope that this will conserve my memory.
BitmapFactory.Options o = new BitmapFactory.Options();
o.inJustDecodeBounds = true;
BitmapFactory.decodeByteArray(data, 0, data.length, o);
int orientation;
if (o.outHeight < o.outWidth) {
orientation = 90;
} else {
orientation = 0;
}
File photo = new File(tmp, "demo.jpeg");
FileOutputStream fos;
BufferedOutputStream bos = null;
try {
fos = new FileOutputStream(photo);
bos = new BufferedOutputStream(fos);
bos.write(data);
bos.flush();
} catch (IOException e) {
Log.e(TAG, "Failed to save photo", e);
} finally {
IOUtils.closeQuietly(bos);
}
Try this. It will solve the purpose.
Bitmap storedBitmap = BitmapFactory.decodeByteArray(data, 0, data.length, null);
Matrix mat = new Matrix();
mat.postRotate("angle"); // angle is the desired angle you wish to rotate
storedBitmap = Bitmap.createBitmap(storedBitmap, 0, 0, storedBitmap.getWidth(), storedBitmap.getHeight(), mat, true);
You can set JPEG rotation via Exif header without decoding it. This is the most efficient method, but some viewers may still show a rotated image.
Alternatively, you can use JPEG lossless rotation. Unfortunately, I am not aware of free Java implementations of this algorithm.
Update on SourceForge, there is a Java open source class LLJTran. The Android port is on GitHub.
I don't think that there is such possibility. Bytes order depends from picture encoding (png, jpeg). So you are forced to decode image to do something with it.
Try like this,
private byte[] rotateImage(byte[] data, int angle) {
Log.d("labot_log_info","CameraActivity: Inside rotateImage");
Bitmap bmp = BitmapFactory.decodeByteArray(data, 0, data.length, null);
Matrix mat = new Matrix();
mat.postRotate(angle);
bmp = Bitmap.createBitmap(bmp, 0, 0, bmp.getWidth(), bmp.getHeight(), mat, true);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.JPEG, 100, stream);
return stream.toByteArray();
}
You can call the rotateImage by providing the image data which is getting from onPictureTaken method and an angle for rotation.
Eg: rotateImage(data, 90);
When trying to convert the byte[] of Camera.onPreviewFrame to Bitamp using BitmapFactory.decodeByteArray gives me an error SkImageDecoder::Factory returned null
Following is my code:
public void onPreviewFrame(byte[] data, Camera camera) {
Bitmap bmp=BitmapFactory.decodeByteArray(data, 0, data.length);
}
This has been hard to find! But since API 8, there is a YuvImage class in android.graphics. It's not an Image descendent, so all you can do with it is save it to Jpeg, but you could save it to memory stream and then load into Bitmap Image if that's what you need.
import android.graphics.YuvImage;
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
try {
Camera.Parameters parameters = camera.getParameters();
Size size = parameters.getPreviewSize();
YuvImage image = new YuvImage(data, parameters.getPreviewFormat(),
size.width, size.height, null);
File file = new File(Environment.getExternalStorageDirectory()
.getPath() + "/out.jpg");
FileOutputStream filecon = new FileOutputStream(file);
image.compressToJpeg(
new Rect(0, 0, image.getWidth(), image.getHeight()), 90,
filecon);
} catch (FileNotFoundException e) {
Toast toast = Toast
.makeText(getBaseContext(), e.getMessage(), 1000);
toast.show();
}
}
Since Android 3.0 you can use a TextureView and TextureSurface to display the camera, and then use mTextureView.getBitmap() to retrieve a friendly RGB preview frame.
A very skeletal example of how to do this is given in the TextureView docs. Note that you'll have to set your application or activity to be hardware accelerated by putting android:hardwareAccelerated="true" in the manifest.
I found the answer after a long time. Here it is...
Instead of using BitmapFactory, I used my custom method to decode this byte[] data to a valid image format. To decode the image to a valid image format, one need to know what picture format is being used by the camera by calling camera.getParameters().getPictureFormat(). This returns a constant defined by ImageFormat. After knowing the format, use the appropriate encoder to encode the image.
In my case, the byte[] data was in the YUV format, so I looked for YUV to BMP conversion and that solved my problem.
you can try this:
This example send camera frames to server
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
try {
byte[] baos = convertYuvToJpeg(data, camera);
StringBuilder dataBuilder = new StringBuilder();
dataBuilder.append("data:image/jpeg;base64,").append(Base64.encodeToString(baos, Base64.DEFAULT));
mSocket.emit("newFrame", dataBuilder.toString());
} catch (Exception e) {
Log.d("########", "ERROR");
}
}
};
public byte[] convertYuvToJpeg(byte[] data, Camera camera) {
YuvImage image = new YuvImage(data, ImageFormat.NV21,
camera.getParameters().getPreviewSize().width, camera.getParameters().getPreviewSize().height, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int quality = 20; //set quality
image.compressToJpeg(new Rect(0, 0, camera.getParameters().getPreviewSize().width, camera.getParameters().getPreviewSize().height), quality, baos);//this line decreases the image quality
return baos.toByteArray();
}