I'm wanting to write a whiteboard app. I have a beginning that renders a bitmap (the drawing page) and then copies that bitmap to the surfaceView. It works perfectly in the emulator, but when I run it on my Samsung Galaxy Ace, it unexpectedly closes. This code:
public void surfaceCreated(SurfaceHolder holder) {
Log.d(TAG, "Create surface");
mo_paper = BitmapFactory.decodeResource(getResources(), R.drawable.paper);
Log.d(TAG, "Created paper");
mo_easel = new Canvas();
Log.d(TAG, "Created easel");
mo_easel.setBitmap(mo_paper);
Log.d(TAG, "Set easel");
mo_matrix = new Matrix();
Log.d(TAG, "Assets loaded");
mainThread.setRunning(true);
mainThread.start();
Log.d(TAG, "Threads started");
}
outputs 'Created easel' but not 'Set easel', so it appears the .setBitmap() method is causing the error.
Bitmaps loaded from resources are immutable. You need to pass a BitmapFactory.Options that tell BitmapFactory that you want the resulting bitmap to be immutable.
Romain Guy was correct. On the phone the bitmap was being loaded as immutable, but in the emulator and on another phone, it was being loaded as mutable (contrary to the documentation!). Setting the option inMutable isn't possible on API's before 11, so in my case, the simple solution was to create an empty bitmap with
mo_paper = Bitmap.createBitmap(paperWidth, paperHeight, Bitmap.Config.ARGB_8888);
and just draw on whatever other bitmaps I want
Related
Background
Android got a new API on Kitkat and Lollipop, to video capture the screen. You can do it either via the ADB tool or via code (starting from Lollipop).
Ever since the new API was out, many apps came to that use this feature, allowing to record the screen, and Microsoft even made its own Google-Now-On-tap competitor app.
Using ADB, you can use:
adb shell screenrecord /sdcard/video.mp4
You can even do it from within Android Studio itself.
The problem
I can't find any tutorial or explanation about how to do it using the API, meaning in code.
What I've found
The only place I've found is the documentations (here, under "Screen capturing and sharing"), telling me this:
Android 5.0 lets you add screen capturing and screen sharing
capabilities to your app with the new android.media.projection APIs.
This functionality is useful, for example, if you want to enable
screen sharing in a video conferencing app.
The new createVirtualDisplay() method allows your app to capture the
contents of the main screen (the default display) into a Surface
object, which your app can then send across the network. The API only
allows capturing non-secure screen content, and not system audio. To
begin screen capturing, your app must first request the user’s
permission by launching a screen capture dialog using an Intent
obtained through the createScreenCaptureIntent() method.
For an example of how to use the new APIs, see the MediaProjectionDemo
class in the sample project.
Thing is, I can't find any "MediaProjectionDemo" sample. Instead, I've found "Screen Capture" sample, but I don't understand how it works, as when I've run it, all I've seen is a blinking screen and I don't think it saves the video to a file. The sample seems very buggy.
The questions
How do I perform those actions using the new API:
start recording, optionally including audio (mic/speaker/both).
stop recording
take a screenshot instead of video.
Also, how do I customize it (resolution, requested fps, colors, time...)?
First step and the one which Ken White rightly suggested & which you may have already covered is the Example Code provided officially.
I have used their API earlier. I agree screenshot is pretty straight forward. But, screen recording is also under similar lines.
I will answer your questions in 3 sections and will wrap it up with a link. :)
1. Start Video Recording
private void startScreenRecord(final Intent intent) {
if (DEBUG) Log.v(TAG, "startScreenRecord:sMuxer=" + sMuxer);
synchronized(sSync) {
if (sMuxer == null) {
final int resultCode = intent.getIntExtra(EXTRA_RESULT_CODE, 0);
// get MediaProjection
final MediaProjection projection = mMediaProjectionManager.getMediaProjection(resultCode, intent);
if (projection != null) {
final DisplayMetrics metrics = getResources().getDisplayMetrics();
final int density = metrics.densityDpi;
if (DEBUG) Log.v(TAG, "startRecording:");
try {
sMuxer = new MediaMuxerWrapper(".mp4"); // if you record audio only, ".m4a" is also OK.
if (true) {
// for screen capturing
new MediaScreenEncoder(sMuxer, mMediaEncoderListener,
projection, metrics.widthPixels, metrics.heightPixels, density);
}
if (true) {
// for audio capturing
new MediaAudioEncoder(sMuxer, mMediaEncoderListener);
}
sMuxer.prepare();
sMuxer.startRecording();
} catch (final IOException e) {
Log.e(TAG, "startScreenRecord:", e);
}
}
}
}
}
2. Stop Video Recording
private void stopScreenRecord() {
if (DEBUG) Log.v(TAG, "stopScreenRecord:sMuxer=" + sMuxer);
synchronized(sSync) {
if (sMuxer != null) {
sMuxer.stopRecording();
sMuxer = null;
// you should not wait here
}
}
}
2.5. Pause and Resume Video Recording
private void pauseScreenRecord() {
synchronized(sSync) {
if (sMuxer != null) {
sMuxer.pauseRecording();
}
}
}
private void resumeScreenRecord() {
synchronized(sSync) {
if (sMuxer != null) {
sMuxer.resumeRecording();
}
}
}
Hope the code helps. Here is the original link to the code that I referred to and from which this implementation(Video recording) is also derived from.
3. Take screenshot Instead of Video
I think by default its easy to capture the image in bitmap format. You can still go ahead with MediaProjectionDemo example to capture screenshot.
[EDIT] : Code encrypt for screenshot
a. To create virtual display depending on device width / height
mImageReader = ImageReader.newInstance(mWidth, mHeight, PixelFormat.RGBA_8888, 2);
mVirtualDisplay = sMediaProjection.createVirtualDisplay(SCREENCAP_NAME, mWidth, mHeight, mDensity, VIRTUAL_DISPLAY_FLAGS, mImageReader.getSurface(), null, mHandler);
mImageReader.setOnImageAvailableListener(new ImageAvailableListener(), mHandler);
b. Then start the Screen Capture based on an intent or action-
startActivityForResult(mProjectionManager.createScreenCaptureIntent(), REQUEST_CODE);
Stop Media projection-
sMediaProjection.stop();
c. Then convert to image-
//Process the media capture
image = mImageReader.acquireLatestImage();
Image.Plane[] planes = image.getPlanes();
ByteBuffer buffer = planes[0].getBuffer();
int pixelStride = planes[0].getPixelStride();
int rowStride = planes[0].getRowStride();
int rowPadding = rowStride - pixelStride * mWidth;
//Create bitmap
bitmap = Bitmap.createBitmap(mWidth + rowPadding / pixelStride, mHeight, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(buffer);
//Write Bitmap to file in some path on the phone
fos = new FileOutputStream(STORE_DIRECTORY + "/myscreen_" + IMAGES_PRODUCED + ".png");
bitmap.compress(CompressFormat.PNG, 100, fos);
fos.close();
There are several implementations (full code) of Media Projection API available.
Some other links that can help you in your development-
Video Recording with MediaProjectionManager - website
android-ScreenCapture - github as per android developer's observations :)
screenrecorder - github
Capture and Record Android Screen using MediaProjection APIs - website
Hope it helps :) Happy coding and screen recording!
PS: Can you please tell me the Microsoft app you are talking about? I have not used it. Would like to try it :)
Here's another question for you :)
Basically i made a realtime streaming service, sending multiple jpegs to my android app, that decodes them as soon as he receives them.
// dIn is DataInputStream
// videoFeed is an ImageView
// bitmap is Bitmap
// hand is an Handler of the main thread
//CODE EXECUTED IN ANOTHER THERAD
byte[] inBuff = new byte[8];
byte[] imgBuff;
String inMsg;
while(socket.isConnected()) {
dIn.readFully(inBuff);
inMsg = new String(inBuff, "ASCII").trim();
int size = Integer.parseInt(inMsg);
imgBuff = new byte[size];
dIn.readFully(imgBuff);
out.write("SEND-NEXT-JPEG".getBytes("ASCII"));
bitmap = BitmapFactory.decodeByteArray(imgBuff, 0, size);
hand.post(setImage);
}
}
private Runnable setImage = new Runnable() {
#Override
public void run() {
videoFeed.setImageBitmap(bitmap);
}
};
The problem is that after about 10 or 20 jpegs are perfectly decoded in realtime, the app freezes for 400ms or so and then it continues to decode other 10/20 jpegs before another freeze...
I know that sending multiple jpegs it's not a good way for streaming video but i can only change the client (android app), not the server.
Do you have any idea for get a fluid video and avoid freezes? thanks!
Right now, you are using the three-parameter version of decodeByteArray(). Instead, switch to the four-parameter version, passing in a BitmapFactory.Options as the last value. On there, set inBitmap to be a Bitmap object that can be reused.
This requires you to maintain a small Bitmap object pool. It could be as simple as two Bitmap instances: the one that is presently being displayed and the one that you are preparing for the next "frame" of the video.
The catch is that, for API Level 18 and below, the Bitmap needs to be the same resolution (height and width in pixels). In your case, that's probably not a problem, as I would imagine that each of your bitmaps have the same resolution.
I am trying to create a thumbnail from video. I use the following line:
Bitmap thumb = ThumbnailUtils.createVideoThumbnail(selectedVideoPath, MediaStore.Images.Thumbnails.MICRO_KIND);
It works great when I select an existing video from the gallery, but returns NULL when recording a new video and then trying to get the thumbnail, although the path is valid (/storage/emulated/0/airImagePicker/1394007123308.3gp).
I am on HTC One Android 4.2.2.
Thanks!
I faced the same problem and noticed that it worked when there was a delay between taking the video and creating the bitmap. A workaround that worked for me was to retry creating the bitmap with busy waiting until it wasn't null( it took a few seconds). It's clearly not a clean solution but it seems to do the job.
example of use (in c# xamarin android)
try {
Bitmap bitmap = null;
for (int time = 0; time < 6000; time += timeInterval) {
bitmap = ThumbnailUtils.CreateVideoThumbnail (videoFile.Path, ThumbnailKind.MiniKind);
if (bitmap != null)
break;
await Task.Delay (timeInterval);
}
return bitmap;
I hope it helps.
I have asked similar questions a few times and still have not resolved my issue, so I thought I'd come at it another way and see if anyone can help me out.
I am writing a game for Android; this is my first attempt at a program this large/complex. The game is a 2d action/puzzler, and I am using Canvas rather than OpenGL ES for drawing.
Everything is going well, except when I try to draw pngs/bmps/jpgs, any images I imported myself. I can draw shapes and animations using the built-in Android canvas drawShape methods (including many Paints with Alpha content) till the cows come home, and maintain over 60fps, but as soon as I try to add my own image (say, a 60kb png saved from Photoshop) I immediately see a major performance hit. The bigger the size of the png on the screen, the bigger the hit (which makes sense).
I have a simple question that may help me understand if I'm doing something wrong here:
If you use the Canvas draw method to draw a red rectangle on the screen, should I expect it to be possible to instead import and display a red rectangle of the same dimensions without a loss in performance? I have done a lot of research on this issue but it is still not clear to me why Android (or Nexus 7) would have such a hard time with my images.
Is Canvas the problem? Do I need to port to Libdgx or AndEngine (that will be a process I think...)?
If it helps, this is how I'm loading my assets:
#Override
public Image newImage(String fileName, ImageFormat format) {
Config config = null;
if (format == ImageFormat.RGB565)
config = Config.RGB_565;
else if (format == ImageFormat.ARGB4444)
config = Config.ARGB_4444;
else
config = Config.ARGB_8888;
Options options = new Options();
options.inPreferredConfig = config;
InputStream in = null;
Bitmap bitmap = null;
try {
in = assets.open(fileName);
bitmap = BitmapFactory.decodeStream(in, null, options);
if (bitmap == null)
throw new RuntimeException("Couldn't load bitmap from asset '"
+ fileName + "'");
} catch (IOException e) {
throw new RuntimeException("Couldn't load bitmap from asset '"
+ fileName + "'");
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
}
}
}
if (bitmap.getConfig() == Config.RGB_565)
format = ImageFormat.RGB565;
else if (bitmap.getConfig() == Config.ARGB_4444)
format = ImageFormat.ARGB4444;
else
format = ImageFormat.ARGB8888;
return new AndroidImage(bitmap, format);
}
Canvas are slower than GLSurfaceView, but on Nexus 7 should handle 300 bitmap on one screen with no sweat.
Search your performence problem in onDraw.
Watch this for tips and tricks in real time game design:
http://www.google.com/events/io/2010/sessions/writing-real-time-games-android.html
My question is how to handle an Out of Memory error when decoding a byte array into a bitmap so I can do a rotation on it. My code is as follows and before you say its a duplicate, I have tried using BitmapFactory.Options and setting the sample size to 2. However the quality loss was far too bad to be acceptable. Also it appears to only be happening on one device so maybe its a one off thing, however I'm inclined to believe if it affects one, there will be 25 more like it later. Also this is happening on the FIRST photo taken and this is the only work that this activity does with regards to bitmaps. And while I'm working in Monodroid, Java answers are welcome too as I can usually translate them to C# fairly easily.
public void GotImage(byte[] image)
{
try
{
Android.Graphics.Bitmap thePicture = Android.Graphics.BitmapFactory.DecodeByteArray(image, 0, image.Length);
Array.Clear(image, 0, image.Length);
image = null;
GC.Collect();
Android.Graphics.Matrix m = new Android.Graphics.Matrix();
m.PostRotate(90);
Android.Graphics.Bitmap rotatedPicture = Android.Graphics.Bitmap.CreateBitmap(thePicture, 0, 0, thePicture.Width, thePicture.Height, m, true);
thePicture.Dispose();
thePicture = null;
GC.Collect();
using (MemoryStream ms = new MemoryStream())
{
rotatedPicture.Compress(Android.Graphics.Bitmap.CompressFormat.Jpeg, 100, ms);
image = ms.ToArray();
}
rotatedPicture.Dispose();
rotatedPicture = null;
GC.Collect();
listOfImages.Add(image);
storeButton.Text = " Store " + listOfImages.Count + " Pages ";
storeButton.Enabled = true;
takePicButton.Enabled = true;
gotImage = false;
cameraPreviewArea.camera.StartPreview();
}
catch (Exception ex)
{
AlertDialog.Builder alertDialog = new AlertDialog.Builder(this);
alertDialog.SetTitle("Error Taking Picture");
alertDialog.SetMessage(ex.ToString());
alertDialog.SetPositiveButton("OK", delegate { });
alertDialog.Show();
}
}
What's rotatedPicture.Dispose()? Does this just set the reference to null? The best and quickest way to get rid of a Bitmap's memory is via the recycle() method.
Well after a long day of learning, I discovered a fix/workaround. This involved setting the resolution of the picture being taken by the camera before the picture was taken instead of trying to scale it after the fact. I also set the option in settings for the user to try different resolutions till they get one that works best for them.
Camera.Parameters parameters = camera.GetParameters();
parameters.SetPictureSize(parameters.SupportedPictureSizes[parameters.SupportedPictureSizes.Count - 1].Width,
parameters.SupportedPictureSizes[parameters.SupportedPictureSizes.Count - 1].Height);
camera.SetParameters(parameters);
camera.StartPreview();