I'm trying to port an emulator that i have written in java to android. Things have been going nicely, I was able to port most of my codes with minor changes however due to how emulation works, I need to render image at pixel level.
As for desktop java I use
int[] pixelsA = ((DataBufferInt) src.getRaster().getDataBuffer()).getData();
which allow me to get the reference to the pixel buffer and update it on the fly(minimize object creations)
Currently this is what my emulator for android does for every frame
#Override
public void onDraw(Canvas canvas)
{
buffer = Bitmap.createBitmap(pixelsA, 256, 192, Bitmap.Config.RGB_565);
canvas.drawBitmap(buffer, 0, 0, null);
}
pixelsA is an array int[], pixelsA contains all the colour informations, so every frame it will have to create a bitmap object by doing
buffer = Bitmap.createBitmap(pixelsA, 256, 192, Bitmap.Config.RGB_565);
which I believe is quite expensive and slow.
Is there any way to draw pixels efficiently with canvas?
One quite low-level method, but working fine for me (with native code):
Create Bitmap object, as big as your visible screen.
Also create a View object and implement onDraw method.
Then in native code you'd load libjnigraphics.so native library, lookup functions AndroidBitmap_lockPixels and AndroidBitmap_unlockPixels.
These functions are defined in Android source in bitmap.h.
Then you'd call lock/unlock on a bitmap, receiving address to raw pixels. You must interpret RGB format of pixels accordingly to what it really is (16-bit 565 or 32-bit 8888).
After changing content of the bitmap, you want to present this on screen.
Call View.invalidate() on your View. In its onDraw, blit your bitmap into given Canvas.
This method is very low level and dependent on actual implementation of Android, however it's very fast, you may get 60fps no problem.
bitmap.h is part of Android NDK since platform version 8, so this IS official way to do this from Android 2.2.
You can use the drawBitmap method that avoids creating a Bitmap each time, or even as a last resort, draw the pixels one by one with drawPoint.
Don't recreate the bitmap every single time. Try something like this:
Bitmap buffer = null;
#Override
public void onDraw(Canvas canvas)
{
if(buffer == null) buffer = Bitmap.createBitmap(256, 192, Bitmap.Config.RGB_565);
buffer.copyPixelsFromBuffer(pixelsA);
canvas.drawBitmap(buffer, 0, 0, null);
}
EDIT: as pointed out, you need to update the pixel buffer. And the bitmap must be mutable for that to happen.
if pixelsA is already an array of pixels (which is what I would infer from your statement about containing colors) then you can just render them directly without converting with:
canvas.drawBitmap(pixelsA, 0, 256, 0, 0, 256, 192, false, null);
Related
I am creating an Android Tile that is meant to display custom and dynamically created graphics, i.e. a chart.
However, due to several limitations I have yet to find a way to do so. Tiles seem to work fundamentally different than Activities do and the Tiles' API only allows for several, predefined UI elements to be created. The only usable one for me seems to be the Image LayoutElement.
The Image can be created by either passing a resource or a ByteArray. Former is not possible when dealing with dynamically created graphs.
Thus, my only hope (I think) is to create an Image in the form of a ByteArray myself.
How can I do this? Is there any Java framework to draw graphics directly?
I have considered the following:
Using the provided UI elements: wouldn't work since the placement is way to imprecise and the exact position of an element cannot be controlled. Also, these elements are not meant for drawing.
Using AWT: doesn't work on Android. Thus, almost any drawing and/or charting library is out of the game.
JavaFX: would probably work but there seems to be now way to draw directly on ByteArrays/BufferedImages as the application needs to be rendered first. Rendering JavaFX doesn't seem possible for Tiles.
Using Android's Canvas: again, an Activity is needed.
Turns out I was wrong: you can very well use the Canvas within a Tile. Converting it to a resource is, however, a little tricky, so here's some code:
final Bitmap bitmap = Bitmap.createBitmap(chart.getWidth(), chart.getHeight(),
Bitmap.Config.RGB_565);
final Canvas canvas = new Canvas(bitmap);
// Sets the background color
final Color background = Color.valueOf(chart.getBackgroundColor());
canvas.drawRGB(
Math.round(background.red() * 255),
Math.round(background.green() * 255),
Math.round(background.blue() * 255)
);
// YOUR DRAWING OPERATIONS: e.g. canvas.drawRect
final ByteBuffer byteBuffer = ByteBuffer.allocate(bitmap.getByteCount());
bitmap.copyPixelsToBuffer(byteBuffer);
final byte[] bytes = byteBuffer.array();
return new ResourceBuilders.ImageResource.Builder()
.setInlineResource(
new ResourceBuilders.InlineImageResource.Builder()
.setData(bytes)
.setWidthPx(chart.getWidth())
.setHeightPx(chart.getHeight())
.setFormat(ResourceBuilders.IMAGE_FORMAT_RGB_565)
.build()
)
.build();
This example shows using Compose Canvas to render charts for Tiles.
https://github.com/google/horologist/pull/249
Also you can encode to PDF
Remove
setFormat(ResourceBuilders.IMAGE_FORMAT_RGB_565)
and use
val bytes = ByteArrayOutputStream().apply {
compress(Bitmap.CompressFormat.PNG, 100, this)
}.toByteArray()
I have a basic openGL ES 20 application running with on a GLSurfaceView that has been added:
GLSurfaceView view = new GLSurfaceView(this);
view.setRenderer(new OpenGLRenderer());
setContentView(view);
Basically I am trying get a screenshot with the following method:
private static Bitmap getScreenshot(View v)
{
Bitmap b = Bitmap.createBitmap(v.getWidth(), v.getHeight(),
Bitmap.Config.ARGB_8888);
Canvas c = new Canvas(b);
v.draw(c);
return b;
}
But it seems the bitmap is transparent. The view I am passing in is:
View content = m_rootActivity.getWindow().getDecorView().getRootView();
Anyone has a solution on how to get screenshot on openGL ES without resorting into going into the DrawFrame method which I have seen in other solutions.
Maybe pass in the reference of the renderer? Any help would be appreciated.
Update:
I was exploring in rendering the bitmap from the onDrawFrame (Display black screen while capture screenshot of GLSurfaceView)
However, I was wondering if there is a better solution since I won't have access to the renderer nor the surfaceview. I can pass in their reference but would like a solution where we can just capture the entire view like what was mentioned earlier.
See this question.
You can get a screenshot with:
#Override
public void onDrawFrame(GL10 gl) {
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
// draw ...
if (takeScreenshot) {
int screenshotSize = width * height;
ByteBuffer bb = ByteBuffer.allocateDirect(screenshotSize * 4);
bb.order(ByteOrder.nativeOrder());
GLES20.glReadPixels(0, 0, width, height, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, bb);
int pixelsBuffer[] = new int[screenshotSize];
bb.asIntBuffer().get(pixelsBuffer);
bb = null;
for (int i = 0; i < screenshotSize; ++i) {
// The alpha and green channels' positions are preserved while the red and blue are swapped
pixelsBuffer[i] = ((pixelsBuffer[i] & 0xff00ff00)) | ((pixelsBuffer[i] & 0x000000ff) << 16) | ((pixelsBuffer[i] & 0x00ff0000) >> 16);
}
Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
bitmap.setPixels(pixelsBuffer, screenshotSize-width, -width, 0, 0, width, height);
// save bitmap...
}
}
You can not by any chance at all get the buffer data to the CPU from the GPU without reading the pixels. You should understand that this is not the same pipeline as is with views, the data in the buffer are filled on the GPU and are then sent directly to the display or nowhere.
So that being said the answer is no. You can not simply get a screenshot as a concept of screenshot does not even exist in this matter. There are only raw (usually RGBA) data on the GPU buffer. And those data must be filled with what you draw to get all you have drawn, if you were to simply read those data at any time the buffer might just be cleared, it might be half drawn or if you are lucky fully drawn.
So that is the reason why you make those screenshot in the drawing pipeline as you must assure the buffer is filled with the data.
There are generally 2 smart ways of intercepting the drawing pipeline best done just before presenting the buffer. One is to pass a certain flag that a screenshot should be done where then the engine itself creates a screenshot which is nice since it has all the data of the buffer on the fly. The second is to create a callback handle where the engine will notify the owner on every frame being fully drawn, in this case the owner can do some additional drawing or creating a screenshot or count frames per second... this again has many bonuses but you do need to at least pass the buffer dimensions to do anything with the buffer.
Also note that reading the pixels is extremely slow and in some cases the image you will receive will be upside-down.
I am new at this - asking questions, android developement and NDK. I hope I am clear enough.
I need to be able to create multiple surfaces/bitmaps.
e.g.
Surface s = new Surface (width, height)
they can copy between each other
s->copy (s2) copy surface s to s2 (including format conversion between RGBA and alpha-text surface and resizing/scaling)
use fill (x, u, w, h, color) - fill rectangle with color (something like glClear)
As far as I understand you have only one ANativeWindow which is supplied to you by android_app->window variable and if I use EGL I can create upto 1 EGLSurface. I need to be able to create many surfaces (~ 100 for instance). How is this possible? And then blit all of them to the window framebuffer
There is also android/bitmap.h But I am not getting it exactly how to work with it. But it does not offer me API to create surface, just to get already created or something like this?
You can create bitmap through JNI calls:
// setup bitmap class
jclass bitmap_class = (jclass)env->FindClass ("android/graphics/Bitmap");
// setup create method
jmethodID bitmap_create_method = env->GetStaticMethodID (bitmap_class, "createBitmap", "(IILandroid/graphics/Bitmap$Config;)Landroid/graphics/Bitmap;");
// get_enum_value return jobject corresponding in our case to Bitmap.Config.ARGB_8888. (the implentation is irrelevant here)
jobject bitmap_config_ARGB = get_enum_value ("android/graphics/Bitmap$Config", "ARGB_8888");
// Do not forget to call DeleteLocalRef where appropriate
// create the bitmap by calling the CreateBitmap method
// Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
jobject bitmap = env->CallStaticObjectMethod (bitmap_class, bitmap_create_method, width, height, bconfig);
// at the end of course clean-up must be done
env->DeleteLocalRef (bitmap);
You can access some bitmap properties and the raw pixels through the API in android/bitmap.h
AndroidBitmap_getInfo gives information about format (ARGB_8888 or alpha-only), dimensions, stride or pitch.
AndroidBitmap_lockPixels give the raw pixels. After finished manipulating the pixels one MUST call AndroidBitmap_unlockPixels
To make fill (color, dimension)
JNI can help. This can be written through JNI calls (I will use java because it is easier for me to write and clearer to read).
canvas.save ();
canvas.setBitmap (bitmap);
canvas.clipRect (left, top, right, bottom, Region.Op.REPLACE);
canvas.drawColor (color, PorterDuff.Mode.SRC);
canvas.restore ();
To copy one bitmap over another one - copy (src_bitmap, src_rect, dest_rect)
canvas.save ();
canvas.setBitmap (dest_bitmap);
canvas.clipRect (left, top, right, bottom, Region.Op.REPLACE);
canvas.drawBitmap (src_bitmap, src_rect, dest_rect, null);
canvas.restore ();
You can create Bitmaps and use the jnigraphics library (android/bitmap.h) or you can use multiple EGL textures.
Using Bitmaps you'll have to implement fill yourself, because Bitmap does only have pixel-based getters and setters (see setPixels(..))
I have a rather big number of small bitmaps (100+, size about 40x40) each one have some opaque and some transparent parts and i need to paint them respecting these areas.
Bitmaps are in ARGB format, 888(rgb) plus a 256bit alpha channel, standard like in PNG format.
The only (working) way i found to draw them is the following approach:
create a bitmap (ARGB_8888)
fill the bitmap with the raw data
extract the alpha layer from the bitmap
create a BitmapShader (RGB_565) based on the original bitmap
Create a paint for the bitmap which uses the created shader
Then paint the alpha mask using the paint with the special BitmapShader.
The initialization code is run only once, of course:
void initializeTile( int t ){
// Allocate the bitmap:
Bitmap original_data = Bitmap.createBitmap( tile_w, tile_h, Bitmap.Config.ARGB_8888);
// Fill with raw data (this is actually native C++ code):
populateBitmap( original_data );
// Get the alpha mask:
tile_mask[ t ] = original_data.extractAlpha();
// Create the bitmap shader:
tile_data = original_data.copy( Bitmap.Config.RGB_565, false);
// Create the shader:
BitmapShader shader = new BitmapShader(tile_data, CLAMP, CLAMP);
// Create the paint:
tile_paint[ t ] = new Paint();
tile_paint[ t ].setDither(true);
tile_paint[ t ].setAntiAlias(true);
tile_paint[ t ].setFilterBitmap(true);
tile_paint[ t ].setShader( shader );
}
And the paint code is the most simple possible, and it's in the main draw loop:
void paintTile(t){
canvas.drawBitmap( tile_mask[ t ], tile_x[ t], tile_y[ t], tile_paint[ t] );
}
Now, on phones like the Ideos (Android 2.2) it run smooth and fine, but on other phones like the top-end Samsung Galaxy SII (Android 2.3) it's crappy and slow. This does not make much sense to me...
So, what do you think of this approach? Are there better, faster, ways to achieve the same result?
And, why do you think it's so slow on modern, fast hardware? Is there any ways to improve it?
Ok, after some work i found out a better solution. I cannot answer my own questions, so please do if you know more than me.
But, in case more people needs this, i am posting my new solution, which is much faster albeit a bit more complicated. The key idea is to use the shader approach ONLY during initialization and not for painting.
To do this, i create a new bitmap which will contain the "clipped" bitmap (with all the transparent areas cleared) using the shader approach, then paint that clipped bitmap without any shader in the draw code.
void initializeTile( int t ){
// Allocate the bitmap:
Bitmap original_data = Bitmap.createBitmap( tile_w, tile_h, Bitmap.Config.ARGB_8888);
// Fill with raw data (this is actually native C++ code):
populateBitmap( original_data );
// Now make a new bitmap to be clipped:
Bitmap clipped_data = Bitmap.createBitmap( tile_w, tile_h, Bitmap.Config.ARGB_8888);
Canvas canvas = new Canvas(clipped_data);
Paint clip_paint = new Paint();
clip_paint.setDither(true);
clip_paint.setAntiAlias(true);
clip_paint.setFilterBitmap(true);
clip_paint.setShader( new BitmapShader(original_data, CLAMP, CLAMP));
// Paint the clipped bitmap:
canvas.drawBitmap( tile_mask[ t ], 0, 0, clip_paint );
//Use the clipped bitmap as original bitmap:
tile_data[ t ] = clipped_data;
}
And also drawing code:
void paintTile(t){
canvas.drawBitmap( tile_data[ t ], tile_x[ t], tile_y[ t], null );
}
Overall, this is much faster.
Still it's unclear to me WHY Android would not paint my alpha-channelled bitmaps properly without all this mess!
I come from the Qt world and i am porting an application to Android. I am bit confused, i am banging my head for a few days now on something that must be so trivial that i cannot find why it's not working.
Some background: i have a C++ engine which i use trough NDK and JNI. This engine creates some bitmaps and passes them to the Java side, the Java side must display them on a View and let the user interact with them (drag and such).
The engine works properly, because i use it under Qt with full success. This is the workflow:
1- Java loads a big Bitmap from a custom data file (the C++ engine expects it to be in ARGB format, but it's compressed JPG data)
Bitmap.Config fmt = Bitmap.Config.ARGB_8888;
Bitmap bitmap = BitmapFactory.decodeByteArray(buffer, 0, size).copy( fmt , false);
2- initialize the C++ engine passing the bitmap. The C++ engine "breaks" the bitmap in smaller tiles. For tile it builds a rather complex alpha mask and stores it into the first byte of the bitmap (the "a" byte). This alpha mask only uses two values: 0xFF for opaque and 0x00 for transparent.
init_C_engine( this.fullImage );
3- The Java side then allocates all the tiles bitmaps, i do in two steps because before init i dont know which size will the tiles be. The engine will populate the tile_width and tile_height arrays:
Bitmap.Config fmt = Bitmap.Config.ARGB_8888;
for (int t = 0; t < this.puzzle_size; t++ ){
tile_data[ t ] = Bitmap.createBitmap( tile_width[t], tile_height[t], fmt);
4- Last step,inside the C++ engine, all the tiles bitmaps are filled:
for ( int n = 0; n < nBitmaps; n++ )
{
jobject bitmap = env->GetObjectArrayElement( bitmaps, n );
AndroidBitmap_getInfo(env, bitmap, &info);
AndroidBitmap_lockPixels(env, bitmap, reinterpret_cast<void **>(&pixels));
game->getTileBitmap( n, (unsigned char*)pixels );
AndroidBitmap_unlockPixels(env, bitmap);
env->SetObjectArrayElement( bitmaps, n, bitmap );
}
}
Now, in my custom View:
protected void onDraw(Canvas canvas) {
super.onDraw(canvas);
canvas.drawColor(Color.BLACK);
for ( int tile = 0; tile < board.nTiles; tile++ ){
canvas.drawBitmap( tile_data[tile],
tile_x[tile],
tile_y[tile], paint);
}
}
What i expect is that on my View i see my tiles with transparent areas. what i get instead is a weird behaviour so that on the black background i see the ENTIRE tile like the alpha bytes are all set to opaque, but when i move the tiles one of top of the other, the "transparent" areas get combined in some strange way, like colors are "xor"ed or multiplied in some way! When i move one tile on the other i can see the areas where the alpha bytes are set to transparent but colors gets mangled instead of being transparend!
Basically i expect that pixels having alpha set to 0 are drawn as transparent... i looked on internet but i could not find anything usefull to help me out....
Does somebody have ideas? Anything will be appreciated!
thanks.
Shouldn't you use the index t instead of tile inside the for loop inside onDraw? Like this:
canvas.drawBitmap(tile_data[t], tile_x[t], tile_y[t], paint);