I have a text recognition app and I only want the app to read text within a certain box on my screen. My approach is to crop AN image from my camera before I send it to text recognition. I am having an issue cropping my image first.
If I try to convert the byte to a bitmap then crop it using -
Bitmap bmp = BitmapFactory.decodeByteArray(data, 0, data.length);
Bitmap resizedbitmap=Bitmap.createBitmap(bmp, 100, 100, 200, 200);
and then use create bitmap to create a new bitmap that is cropped with a RECT - but since it is null it is failing.
I am receiving null for the bitmap - I have tried changing the options, I have added read write permissions to the manifest - but it is still coming up null.
I am now trying to convert the byte array to a yuvimage but I think data is being lost in the compression so when I scan my image - the text recognition is giving me distorted text blocks and it is not very accurate.
int correctRotation = RNCameraViewHelper.getCorrectCameraRotation(rotation, getFacing(),
getCameraOrientation());
if (data.length < (1.5 * width * height)) {
return;
}
if (willCallTextTask) {
textRecognizerTaskLock = true;
TextRecognizerAsyncTaskDelegate delegate = (TextRecognizerAsyncTaskDelegate) cameraView;
final byte[] compressedImage;
final YuvImage yuvImage = new YuvImage(data, ImageFormat.NV21, width, height, null);
final ByteArrayOutputStream imageStream = new ByteArrayOutputStream();
Log.d("Tag", "Image Stream" + getWidth());
Log.d("Tag", "WIDTH" + width);
Log.d("Tag", "HEIGHT" + height);
yuvImage.compressToJpeg(new Rect(0, 0, width, height), 100, imageStream);
compressedImage = imageStream.toByteArray();
new TextRecognizerAsyncTask(delegate, mThemedReactContext, compressedImage, width, height, correctRotation,
getResources().getDisplayMetrics().density, getFacing(), getWidth(), getHeight(), mPaddingX, mPaddingY)
.execute();
}
}
});
}
Any suggestions would be great - or if theres a better way to do this that would be amazing!
Related
In Android Studio I can use the following lines of code to get an array from Bitmap
Bitmap bitmap = get_Image();
Bitmap resizeBitmap = getResizedBitmap(bitmap);
int[] iArr = new int[(resizeBitmap.getWidth() * resizeBitmap.getHeight())];
resizeBitmap.getPixels(iArr, 0, resizeBitmap.getWidth(), 0, 0, resizeBitmap.getWidth(), resizeBitmap.getHeight());
Here is the resizer function:
public Bitmap getResizedBitmap(Bitmap bitmap) {
if (576 == bitmap.getWidth() && 1024 == bitmap.getHeight()) {
return bitmap;
}
Bitmap createBitmap = Bitmap.createBitmap(576, 1024, Config.RGB_565);
Matrix scaledMatrix = scaledMatrix(bitmap.getWidth(), bitmap.getHeight(), createBitmap.getWidth(), createBitmap.getHeight());
new Canvas(createBitmap).drawBitmap(bitmap, scaledMatrix, null);
return createBitmap;
}
However when I convert that int array back to bitmap the image is wrong
Bitmap bmp = Bitmap.createBitmap(1024, 576, Config.RGB_565);
bmp.setPixels(iArr, 0, 1024, 0, 0, 1024, 576);
textViewToChange.setText("Bitmap created");
This is the image when i save it to a file or display it https://imgur.com/a/ZQ0DJAy The above image is how it looks like and below is how it should had looked like
If i read the incorrect image in python, the RGB pixels are placed exactly where they should be like in the correct image but the image dosen't look the same. Why? How can i fix this?
This question already has answers here:
Android Camera2 API YUV_420_888 to JPEG
(3 answers)
Closed 4 years ago.
I am working on AR project where i need to capture the current frame and save it to gallery. I am able to get the image using Frame class in AR core , but the format of image is YUV_420_888. I have already tried lots of solutions to covert this to bitmap but couldn't able to solve it.
This is how I convert to jpeg.
public Bitmap imageToBitmap(Image image, float rotationDegrees) {
assert (image.getFormat() == ImageFormat.NV21);
// NV21 is a plane of 8 bit Y values followed by interleaved Cb Cr
ByteBuffer ib = ByteBuffer.allocate(image.getHeight() * image.getWidth() * 2);
ByteBuffer y = image.getPlanes()[0].getBuffer();
ByteBuffer cr = image.getPlanes()[1].getBuffer();
ByteBuffer cb = image.getPlanes()[2].getBuffer();
ib.put(y);
ib.put(cb);
ib.put(cr);
YuvImage yuvImage = new YuvImage(ib.array(),
ImageFormat.NV21, image.getWidth(), image.getHeight(), null);
ByteArrayOutputStream out = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(0, 0,
image.getWidth(), image.getHeight()), 50, out);
byte[] imageBytes = out.toByteArray();
Bitmap bm = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
Bitmap bitmap = bm;
// On android the camera rotation and the screen rotation
// are off by 90 degrees, so if you are capturing an image
// in "portrait" orientation, you'll need to rotate the image.
if (rotationDegrees != 0) {
Matrix matrix = new Matrix();
matrix.postRotate(rotationDegrees);
Bitmap scaledBitmap = Bitmap.createScaledBitmap(bm,
bm.getWidth(), bm.getHeight(), true);
bitmap = Bitmap.createBitmap(scaledBitmap, 0, 0,
scaledBitmap.getWidth(), scaledBitmap.getHeight(), matrix, true);
}
return bitmap;
}
I am trying face detection and adding mask(graphic overlay) using google vision api ,the problem is i could not get the ouptut from camera after detecting and adding mask.so far I have tried this solution from github , https://github.com/googlesamples/android-vision/issues/24 ,based on this issue i have added a custom detector class,
Mobile Vision API - concatenate new detector object to continue frame processing . and added this on mydetector class How to create Bitmap from grayscaled byte buffer image? .
MyDetectorClass
class MyFaceDetector extends Detector<Face>
{
private Detector<Face> mDelegate;
MyFaceDetector(Detector<Face> delegate) {
mDelegate = delegate;
}
public SparseArray<Face> detect(Frame frame) {
// *** add your custom frame processing code here
ByteBuffer byteBuffer = frame.getGrayscaleImageData();
byte[] bytes = byteBuffer.array();
int w = frame.getMetadata().getWidth();
int h = frame.getMetadata().getHeight();
YuvImage yuvimage=new YuvImage(bytes, ImageFormat.NV21, w, h, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
yuvimage.compressToJpeg(new Rect(0, 0, w, h), 100, baos); // Where 100 is the quality of the generated jpeg
byte[] jpegArray = baos.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Log.e("got bitmap","bitmap val " + bitmap);
return mDelegate.detect(frame);
}
public boolean isOperational() {
return mDelegate.isOperational();
}
public boolean setFocus(int id) {
return mDelegate.setFocus(id);
}
}
frame processing
public SparseArray<Face> detect(Frame frame)
{
// *** add your custom frame processing code here
ByteBuffer byteBuffer = frame.getGrayscaleImageData();
byte[] bytes = byteBuffer.array();
int w = frame.getMetadata().getWidth();
int h = frame.getMetadata().getHeight();
YuvImage yuvimage=new YuvImage(bytes, ImageFormat.NV21, w, h, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
yuvimage.compressToJpeg(new Rect(0, 0, w, h), 100, baos); // Where 100 is the quality of the generated jpeg
byte[] jpegArray = baos.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Log.e("got bitmap","bitmap val " + bitmap);
return mDelegate.detect(frame);
}
i am getting a rotated bitmap ,that is without the mask (graphic overlay) i have added .How can i get the camera output with mask .
Thanks in advance.
The simple answer is: You can't.
Why? Android camera output frames in NV21 ByteBuffer. And you must generate your masks based on the landmarks points in a separated Bitmap, then join them.
Sorry but, that's how the Android Camera API work. Nothing can be done. You must do it manually.
Also, I wouldn't get the camera preview then convert it to YuvImage then to Bitmap. That process consumes a lot of resources and makes preview very very slow. Instead I would use this method which will be a lot faster and rotates your preview internally so you don't loose time doing it:
outputFrame = new Frame.Builder().setImageData(mPendingFrameData, mPreviewSize.getWidth(), mPreviewSize.getHeight(), ImageFormat.NV21)
.setId(mPendingFrameId)
.setTimestampMillis(mPendingTimeMillis)
.setRotation(mRotation)
.build();
mDetector.receiveFrame(outputFrame);
All the code can be found in CameraSource.java
For my application, I have create two bitmap as per my requirements.
One for actual image that goes through some image processing and second logo bitmap that only display application logo on top left corner.
Now at saving time I want to combine these bitmaps and want to generate single JPEG file as output.
To accomplish this task I have write following code.
orignalbitmap = orignalbitmap.copy(Config.ARGB_8888, true);
Canvas savedCanvas = new Canvas(orignalbitmap);
savedCanvas.setBitmap(logoBitmap);
savedCanvas.drawBitmap(orignalbitmap, 0, 0, transPaint);
savedCanvas.drawBitmap(logoBitmap, 0, 0, transPaint);
try {
orignalbitmap.compress(CompressFormat.JPEG, 100, new FileOutputStream(new File("/mnt/sdcard/original.jpg")));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
But at present I only got my original image as output not attached with logo. I want image with logo also that data available in logo bitmap. So how to combine both bitmaps data that I can't understand so please provide some guidance in this.
use this function for combine two bitmap in single bitmap
public static Bitmap combineImages(Bitmap c, Bitmap s)
{
Bitmap cs = null;
int width, height = 0;
if(c.getWidth() > s.getWidth()) {
width = c.getWidth() + s.getWidth();
height = c.getHeight();
} else {
width = s.getWidth() + s.getWidth();
height = c.getHeight();
}
cs = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
Canvas comboImage = new Canvas(cs);
comboImage.drawBitmap(c, 0f, 0f, null);
comboImage.drawBitmap(s, c.getWidth(), 0f, null);
return cs;
}
This is a code I have written to run android's face detector. Unfortunately, it doesn't find any. I have put this in a onPreviewFrame(data, camera).
Camera.Parameters parameters = camera.getParameters();
Camera.Size size = parameters.getPreviewSize();
YuvImage image = new YuvImage(data, ImageFormat.NV21, size.width, size.height, null);
Rect rectangle = new Rect(0, 0, size.width, size.height);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
int quality = 100;
image.compressToJpeg(rectangle, quality, stream);
Bitmap bitmap = BitmapFactory.decodeByteArray(stream.toByteArray(), 0, stream.size());
FaceDetector detector = new FaceDetector(size.width, size.height, 5);
FaceDetector.Face[] faces = new FaceDetector.Face[5];
int numFaces = detector.findFaces(bitmap, faces);
textView.setText("numFaces = " + numFaces);
Any ideas? fixes?
I think you should check whether your bitmap data is correct firstly. You must setPreviewSize before startPreview. check whether the size is same as your preview data. or you can hard code it to (640, 480) for a test. If your bitmap is correct. maybe the API cannot work due to the input data quality. you can dump some data, and try other app or lib, such as OpenCV, to verify. Good luck.