This is a code I have written to run android's face detector. Unfortunately, it doesn't find any. I have put this in a onPreviewFrame(data, camera).
Camera.Parameters parameters = camera.getParameters();
Camera.Size size = parameters.getPreviewSize();
YuvImage image = new YuvImage(data, ImageFormat.NV21, size.width, size.height, null);
Rect rectangle = new Rect(0, 0, size.width, size.height);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
int quality = 100;
image.compressToJpeg(rectangle, quality, stream);
Bitmap bitmap = BitmapFactory.decodeByteArray(stream.toByteArray(), 0, stream.size());
FaceDetector detector = new FaceDetector(size.width, size.height, 5);
FaceDetector.Face[] faces = new FaceDetector.Face[5];
int numFaces = detector.findFaces(bitmap, faces);
textView.setText("numFaces = " + numFaces);
Any ideas? fixes?
I think you should check whether your bitmap data is correct firstly. You must setPreviewSize before startPreview. check whether the size is same as your preview data. or you can hard code it to (640, 480) for a test. If your bitmap is correct. maybe the API cannot work due to the input data quality. you can dump some data, and try other app or lib, such as OpenCV, to verify. Good luck.
Related
I have a text recognition app and I only want the app to read text within a certain box on my screen. My approach is to crop AN image from my camera before I send it to text recognition. I am having an issue cropping my image first.
If I try to convert the byte to a bitmap then crop it using -
Bitmap bmp = BitmapFactory.decodeByteArray(data, 0, data.length);
Bitmap resizedbitmap=Bitmap.createBitmap(bmp, 100, 100, 200, 200);
and then use create bitmap to create a new bitmap that is cropped with a RECT - but since it is null it is failing.
I am receiving null for the bitmap - I have tried changing the options, I have added read write permissions to the manifest - but it is still coming up null.
I am now trying to convert the byte array to a yuvimage but I think data is being lost in the compression so when I scan my image - the text recognition is giving me distorted text blocks and it is not very accurate.
int correctRotation = RNCameraViewHelper.getCorrectCameraRotation(rotation, getFacing(),
getCameraOrientation());
if (data.length < (1.5 * width * height)) {
return;
}
if (willCallTextTask) {
textRecognizerTaskLock = true;
TextRecognizerAsyncTaskDelegate delegate = (TextRecognizerAsyncTaskDelegate) cameraView;
final byte[] compressedImage;
final YuvImage yuvImage = new YuvImage(data, ImageFormat.NV21, width, height, null);
final ByteArrayOutputStream imageStream = new ByteArrayOutputStream();
Log.d("Tag", "Image Stream" + getWidth());
Log.d("Tag", "WIDTH" + width);
Log.d("Tag", "HEIGHT" + height);
yuvImage.compressToJpeg(new Rect(0, 0, width, height), 100, imageStream);
compressedImage = imageStream.toByteArray();
new TextRecognizerAsyncTask(delegate, mThemedReactContext, compressedImage, width, height, correctRotation,
getResources().getDisplayMetrics().density, getFacing(), getWidth(), getHeight(), mPaddingX, mPaddingY)
.execute();
}
}
});
}
Any suggestions would be great - or if theres a better way to do this that would be amazing!
Is there a way to get an image of the current live wallpaper using the WallpaperManager API? I tried the following code, but it simply returned the the icon of the app that was used to set the wallpaper.
PackageManager pm = getApplicationContext().getPackageManager();
// Check if user has set a live wallpaper
if (WallpaperManager.getInstance(this).getWallpaperInfo() != null) {
Drawable wallpaperDrawable = WallpaperManager.getInstance(this).getWallpaperInfo().loadThumbnail(pm);
}
Have you tried to use MediaProjection API to achieve this. I suggest to look along the following code. For more information about the API refer this.
//You have to start the Screen Capture based on an action.
//mWidth, mHeight depends on device screen width, height
mImageReader = ImageReader.newInstance(mWidth, mHeight, PixelFormat.RGBA_8888, 2);
mVirtualDisplay = sMediaProjection.createVirtualDisplay(SCREENCAP_NAME, mWidth, mHeight, mDensity, VIRTUAL_DISPLAY_FLAGS, mImageReader.getSurface(), null, mHandler);
mImageReader.setOnImageAvailableListener(new ImageAvailableListener(), mHandler)
startActivityForResult(mProjectionManager.createScreenCaptureIntent(), REQUEST_CODE);
//Then convert the media captured to an Image
sMediaProjection.stop();
image = mImageReader.acquireLatestImage();
Image.Plane[] planes = image.getPlanes();
ByteBuffer buffer = planes[0].getBuffer();
bitmap = Bitmap.createBitmap(mWidth, mHeight, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(buffer);
File file = new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), "SNAPSHOT.jpg");
fos = new FileOutputStream(file);
bitmap.compress(CompressFormat.JPEG, 100, fos);
fos.close();
I am trying face detection and adding mask(graphic overlay) using google vision api ,the problem is i could not get the ouptut from camera after detecting and adding mask.so far I have tried this solution from github , https://github.com/googlesamples/android-vision/issues/24 ,based on this issue i have added a custom detector class,
Mobile Vision API - concatenate new detector object to continue frame processing . and added this on mydetector class How to create Bitmap from grayscaled byte buffer image? .
MyDetectorClass
class MyFaceDetector extends Detector<Face>
{
private Detector<Face> mDelegate;
MyFaceDetector(Detector<Face> delegate) {
mDelegate = delegate;
}
public SparseArray<Face> detect(Frame frame) {
// *** add your custom frame processing code here
ByteBuffer byteBuffer = frame.getGrayscaleImageData();
byte[] bytes = byteBuffer.array();
int w = frame.getMetadata().getWidth();
int h = frame.getMetadata().getHeight();
YuvImage yuvimage=new YuvImage(bytes, ImageFormat.NV21, w, h, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
yuvimage.compressToJpeg(new Rect(0, 0, w, h), 100, baos); // Where 100 is the quality of the generated jpeg
byte[] jpegArray = baos.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Log.e("got bitmap","bitmap val " + bitmap);
return mDelegate.detect(frame);
}
public boolean isOperational() {
return mDelegate.isOperational();
}
public boolean setFocus(int id) {
return mDelegate.setFocus(id);
}
}
frame processing
public SparseArray<Face> detect(Frame frame)
{
// *** add your custom frame processing code here
ByteBuffer byteBuffer = frame.getGrayscaleImageData();
byte[] bytes = byteBuffer.array();
int w = frame.getMetadata().getWidth();
int h = frame.getMetadata().getHeight();
YuvImage yuvimage=new YuvImage(bytes, ImageFormat.NV21, w, h, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
yuvimage.compressToJpeg(new Rect(0, 0, w, h), 100, baos); // Where 100 is the quality of the generated jpeg
byte[] jpegArray = baos.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Log.e("got bitmap","bitmap val " + bitmap);
return mDelegate.detect(frame);
}
i am getting a rotated bitmap ,that is without the mask (graphic overlay) i have added .How can i get the camera output with mask .
Thanks in advance.
The simple answer is: You can't.
Why? Android camera output frames in NV21 ByteBuffer. And you must generate your masks based on the landmarks points in a separated Bitmap, then join them.
Sorry but, that's how the Android Camera API work. Nothing can be done. You must do it manually.
Also, I wouldn't get the camera preview then convert it to YuvImage then to Bitmap. That process consumes a lot of resources and makes preview very very slow. Instead I would use this method which will be a lot faster and rotates your preview internally so you don't loose time doing it:
outputFrame = new Frame.Builder().setImageData(mPendingFrameData, mPreviewSize.getWidth(), mPreviewSize.getHeight(), ImageFormat.NV21)
.setId(mPendingFrameId)
.setTimestampMillis(mPendingTimeMillis)
.setRotation(mRotation)
.build();
mDetector.receiveFrame(outputFrame);
All the code can be found in CameraSource.java
My goal is to add an overlay on the camera preview that will find book edges. For that, I override the onPreviewFrame where I do the following:
public void onPreviewFrame(byte[] data, Camera camera) {
Camera.Parameters parameters = camera.getParameters();
int width = parameters.getPreviewSize().width;
int height = parameters.getPreviewSize().height;
Mat mat = new Mat((int) (height*1.5), width, CvType.CV_8UC1);
mat.put(0,0,data);
byte[] bytes = new byte[(int) (height*width*1.5)];
mat.get(0,0,bytes);
if (!test) { //to only do once
File pictureFile = getOutputMediaFile();
try {
FileOutputStream fos = new FileOutputStream(pictureFile);
fos.write(bytes);
fos.close();
Uri picUri = Uri.fromFile(pictureFile);
updateGallery(picUri);
test = true;
} catch (IOException e) {
e.printStackTrace();
}
}
}
For now I simply want to take one of the previews and save it after the conversion to mat.
After spending countless hours getting the above to look right, the saved picture cannot be seen on my testing phone (LG Leon). I can't seem to find the issue. Am I mixing the height/width because I'm taking pictures in portrait mode? I tried switching them and still doesn't work. Where is the problem?
The fastest method I managed to find is described HERE in my recently asked question. You can find the method to extract the image in the answer I wrote in my question below. The thing is that the image you get through onPreviewFrame() is NV21. After receiving this image it may be that you need to convert it to RGB (depends on what do you want to achieve; this is also done in the answer I gave you previously).
Seems quite inefficient but it works for me (for now):
//get the camera parameters
Camera.Parameters parameters = camera.getParameters();
int width = parameters.getPreviewSize().width;
int height = parameters.getPreviewSize().height;
//convert the byte[] to Bitmap through YuvImage;
//make sure the previewFormat is NV21 (I set it so somewhere before)
YuvImage yuv = new YuvImage(data, parameters.getPreviewFormat(), width, height, null);
ByteArrayOutputStream out = new ByteArrayOutputStream();
yuv.compressToJpeg(new Rect(0, 0, width, height), 70, out);
Bitmap bmp = BitmapFactory.decodeByteArray(out.toByteArray(), 0, out.size());
//convert Bitmap to Mat; note the bitmap config ARGB_8888 conversion that
//allows you to use other image processing methods and still save at the end
Mat orig = new Mat();
bmp = bmp.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(bmp, orig);
//here you do whatever you want with the Mat
//Mat to Bitmap to OutputStream to byte[] to File
Utils.matToBitmap(orig, bmp);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.JPEG, 70, stream);
byte[] bytes = stream.toByteArray();
File pictureFile = getOutputMediaFile();
try {
FileOutputStream fos = new FileOutputStream(pictureFile);
fos.write(bytes);
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
When you take a picture with the front facing camera in Android the preview is reflected along the Y axis to make the image seen appear as if the user was looking in the mirror. I want to undo this effect (apply a second reflection) or just stop the one thats done automatically.
I though to use this:
Camera mCamera;
....
mCamera.setPreviewCallback(...);
But I dont really know what to do with the overriding of
onPreviewFrame(byte[] data, Camera camera){...}
Whats the best way I can achieve what I've described?
Note I am trying to apply this effect to the live preview, not images that are already taken.
First when you open your camera instance with Camera.open() you should open front camera with Camera.open(getSpecialFacingCamera())
private int getSpecialFacingCamera() {
int cameraId = -1;
// Search for the front facing camera
int numberOfCameras = Camera.getNumberOfCameras();
for (int i = 0; i < numberOfCameras; i++) {
Camera.CameraInfo info = new Camera.CameraInfo();
Camera.getCameraInfo(i, info);
if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
cameraId = i;
break;
}
}
return cameraId;
}
Then in your callback method where camera data is converting in to image
you can use this code to keep it normal
public void onPictureTaken(byte[] data, Camera camera){
Bitmap newImage = null;
Bitmap cameraBitmap;
if (data != null) {
cameraBitmap = BitmapFactory.decodeByteArray(data, 0, (data != null) ? data.length : 0);
if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT) {
// use matrix to reverse image data and keep it normal
Matrix mtx = new Matrix();
//this will prevent mirror effect
mtx.preScale(-1.0f, 1.0f);
// Setting post rotate to 90 because image will be possibly in landscape
mtx.postRotate(90.f);
// Rotating Bitmap , create real image that we want
newImage = Bitmap.createBitmap(cameraBitmap, 0, 0, cameraBitmap.getWidth(), cameraBitmap.getHeight(), mtx, true);
}else{// LANDSCAPE MODE
//No need to reverse width and height
newImage = Bitmap.createScaledBitmap(cameraBitmap, screenWidth, screenHeight, true);
cameraBitmap = newImage;
}
}
}
you can pass newImage in canvas and create jpeg image and save it on device.
Do not forgot Camera is deprecated in Api level 21...
You can use Matrix to flip the image data, something like:
byte[] baImage = null;
Size size = camera.getParameters().getPreviewSize();
ByteArrayOutputStream os = new ByteArrayOutputStream();
YuvImage yuv = new YuvImage(data, ImageFormat.NV21, size.width, size.height, null);
yuv.compressToJpeg(new Rect(0, 0, size.width, size.height), 100, os);
baImage = os.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(rawImage, 0, rawImage.length);
Matrix matrix = new Matrix();
matrix.preScale(-1.0f, 1.0f);
Bitmap mirroredBitmap = Bitmap.createBitmap(bitmap, 0, 0, size.width, size.height, matrix, false);