I am creating an Android Tile that is meant to display custom and dynamically created graphics, i.e. a chart.
However, due to several limitations I have yet to find a way to do so. Tiles seem to work fundamentally different than Activities do and the Tiles' API only allows for several, predefined UI elements to be created. The only usable one for me seems to be the Image LayoutElement.
The Image can be created by either passing a resource or a ByteArray. Former is not possible when dealing with dynamically created graphs.
Thus, my only hope (I think) is to create an Image in the form of a ByteArray myself.
How can I do this? Is there any Java framework to draw graphics directly?
I have considered the following:
Using the provided UI elements: wouldn't work since the placement is way to imprecise and the exact position of an element cannot be controlled. Also, these elements are not meant for drawing.
Using AWT: doesn't work on Android. Thus, almost any drawing and/or charting library is out of the game.
JavaFX: would probably work but there seems to be now way to draw directly on ByteArrays/BufferedImages as the application needs to be rendered first. Rendering JavaFX doesn't seem possible for Tiles.
Using Android's Canvas: again, an Activity is needed.
Turns out I was wrong: you can very well use the Canvas within a Tile. Converting it to a resource is, however, a little tricky, so here's some code:
final Bitmap bitmap = Bitmap.createBitmap(chart.getWidth(), chart.getHeight(),
Bitmap.Config.RGB_565);
final Canvas canvas = new Canvas(bitmap);
// Sets the background color
final Color background = Color.valueOf(chart.getBackgroundColor());
canvas.drawRGB(
Math.round(background.red() * 255),
Math.round(background.green() * 255),
Math.round(background.blue() * 255)
);
// YOUR DRAWING OPERATIONS: e.g. canvas.drawRect
final ByteBuffer byteBuffer = ByteBuffer.allocate(bitmap.getByteCount());
bitmap.copyPixelsToBuffer(byteBuffer);
final byte[] bytes = byteBuffer.array();
return new ResourceBuilders.ImageResource.Builder()
.setInlineResource(
new ResourceBuilders.InlineImageResource.Builder()
.setData(bytes)
.setWidthPx(chart.getWidth())
.setHeightPx(chart.getHeight())
.setFormat(ResourceBuilders.IMAGE_FORMAT_RGB_565)
.build()
)
.build();
This example shows using Compose Canvas to render charts for Tiles.
https://github.com/google/horologist/pull/249
Also you can encode to PDF
Remove
setFormat(ResourceBuilders.IMAGE_FORMAT_RGB_565)
and use
val bytes = ByteArrayOutputStream().apply {
compress(Bitmap.CompressFormat.PNG, 100, this)
}.toByteArray()
Related
The Problem
I have been working on implementing a super resolution model with Tensorflow Lite. I have an empty bitmap 4x the size of the input bitmap (which is bmp):
Bitmap out = Bitmap.createBitmap(bmp.getWidth() * 4, bmp.getHeight() * 4, Bitmap.Config.ARGB_8888);
And I converted both bitmaps to TensorImages
TensorImage originalImage = TensorImage.fromBitmap(bmp);
TensorImage superImage = TensorImage.fromBitmap(out);
However, when I run the model (InterpreterApi tflite):
tflite.run(originalImage.getBuffer(), superImage.getBuffer());
The bitmap from superImage has not changed, and it holds the blank bitmap I made at the start.
superImage.getBitmap();
Things I've tried
I looked at basic examples and documentation, most are geared toward classification but they all seemed to do it this way.
I fed the input bitmap to the output, and my app showed the input, so I know that the file picking and preview works.
I tested with different datatypes to store the output, and they either left it blank or weren't compatible with Tensorflow.
What I think
I suspect the problem has something to do with tflite.run() changing a separate instance of superImage, and I get left with the old one. I may also need a different data format that I haven't tried yet.
Thank you for your time.
the keywords about this topic:
CustomSurfaceView: three custom surfaceview for three different levels.
Canvas: lock/unlockAndPost method (i'm not using custom bitmap )
Multi thread ( each surface is a separate thread )
Shapes ( shapes on canvas )
Client/Server ( architecture )
Flickering ( IS WHY I'M HERE )
We are developing a client/server application and I'm working on the client side. I'm receiving messages from the server containing general data (coordinates, color, width [...] ) about paths like, circle, rectangle, line and other shapes. The web application allows the user to send these data drawing on HTML5 Canvas, to an android device that receiving this messages and parsing it, will be able to redraw all the shapes. From my own experience on this subjects, I learned that the best way to keep in control all the things you draw on the canvas, is saving everything into a buffer, array, list or something like that, then reuse it when you want (for example, you can use the older path for show, hide, move or simply change something on the canvas). In my opinion, the android application follows the best practice of android development and OOP paradigm so I'm not assuming errors related to the bad architecture. In this case, I'm saving the messages on web client side. When the user draws on HTML5 Canvas, the messages which contain shape info are perfectly reported to the android canvas, but the problem appears when:
[example]
Consider you draw 10 objects (10 messages) and you want delete only one object on web app canvas, so the only way is clearing all the canvas, and redraw all the previous shapes without the deleted shape (so resend to the client 9 messages by loop the messages buffer ). This method works perfectly for the web app but cause flickering problem on android client. So after too many experiments I found a workaround, using a Thread.sleep(100)(Whooo! 100ms is too much), in order to parse slowly the messages and let the surfaceview thread to read correctly the data (data access through singleton pattern) and write on the double-buffer of the canvas.Well, it's slow and ugly but it works ! Actually I don't like this “horrible” workaround so please help me to see an exit strategy.
This is a piece of code where the canvas get data from shapes containers and draw if data are present. The data of each containers came from server messages.
#Override
public void run() {
Canvas canvas = null;
while (running) {
//this is the surface's canvas
try {
canvas = shapesSurfaceHolder.lockCanvas();
synchronized (shapesSurfaceHolder) {
if (shapesSurfaceHolder.getSurface().isValid()) {
if(!Parser.cmdClear){
//draw all the data present
canvas.drawPath(PencilData.getInstance().getPencilPath(),
PencilData.getInstance().getPaint());
canvas.drawPath(RectData.getInstance().getRectPath(),
RectData.getInstance().getPaint());
canvas.drawPath(CircleData.getInstance().getCirclePath(),
CircleData.getInstance().getPaint());
canvas.drawPath(LineData.getInstance().getLinePath(),
LineData.getInstance().getPaint());
canvas.drawText(TextData.getInstance().getText(),
TextData.getInstance().getX(),
TextData.getInstance().getY(),
TextData.getInstance().getPaint());
} else {
//remove all canvas content and clear data.
canvas.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR);
for (int i = 0; i < AbstractFactory.SHAPE_NUM; i++) {
abstracFactory.getShape(i).clearData();
}
}
}
}
} finally {
if (canvas != null) {
shapesSurfaceHolder.unlockCanvasAndPost(canvas);
}
}
}
}//end_run()
I can summarize that, apparently, my problem is to draw too quickly
Note:
Similar concept: Android thread controlling multiple texture views causes strange flickering
Hardware acceleration is enabled.
minSdkVersion 17
Tested on
Tablet Samsung SM-T113
Google Nexus 5
The TextureView issue was due to a bug specific to TextureView. You're using SurfaceView, so that does not apply here.
When drawing on a SurfaceView's Surface, you must update every pixel inside the dirty rect (i.e. the optional arg passed to lockCanvas()) every time. If you don't provide a dirty rect, that means the entire screen must be updated. This is because the Surface is double- or triple-buffered, and swapped when you call unlockCanvasAndPost(). If you lock / clear / unlock, then the next time you lock / draw / unlock, you will not be drawing into the buffer you previously cleared.
If you want to do incremental rendering, you should point your Canvas at an off-screen Bitmap and do all your rendering there. Then just blit the entire bitmap between lock and unlock. The alternative is to store up the drawing commands, starting with the initial clear, and play them all back between lock/unlock.
The phrase "three custom surfaceview" is somewhat concerning if they're all on screen at once. If you have them all at different Z depths (default, media overlay, top) then they will behave correctly, but the system is generally more efficient if you can put everything on one.
I'm trying to port an emulator that i have written in java to android. Things have been going nicely, I was able to port most of my codes with minor changes however due to how emulation works, I need to render image at pixel level.
As for desktop java I use
int[] pixelsA = ((DataBufferInt) src.getRaster().getDataBuffer()).getData();
which allow me to get the reference to the pixel buffer and update it on the fly(minimize object creations)
Currently this is what my emulator for android does for every frame
#Override
public void onDraw(Canvas canvas)
{
buffer = Bitmap.createBitmap(pixelsA, 256, 192, Bitmap.Config.RGB_565);
canvas.drawBitmap(buffer, 0, 0, null);
}
pixelsA is an array int[], pixelsA contains all the colour informations, so every frame it will have to create a bitmap object by doing
buffer = Bitmap.createBitmap(pixelsA, 256, 192, Bitmap.Config.RGB_565);
which I believe is quite expensive and slow.
Is there any way to draw pixels efficiently with canvas?
One quite low-level method, but working fine for me (with native code):
Create Bitmap object, as big as your visible screen.
Also create a View object and implement onDraw method.
Then in native code you'd load libjnigraphics.so native library, lookup functions AndroidBitmap_lockPixels and AndroidBitmap_unlockPixels.
These functions are defined in Android source in bitmap.h.
Then you'd call lock/unlock on a bitmap, receiving address to raw pixels. You must interpret RGB format of pixels accordingly to what it really is (16-bit 565 or 32-bit 8888).
After changing content of the bitmap, you want to present this on screen.
Call View.invalidate() on your View. In its onDraw, blit your bitmap into given Canvas.
This method is very low level and dependent on actual implementation of Android, however it's very fast, you may get 60fps no problem.
bitmap.h is part of Android NDK since platform version 8, so this IS official way to do this from Android 2.2.
You can use the drawBitmap method that avoids creating a Bitmap each time, or even as a last resort, draw the pixels one by one with drawPoint.
Don't recreate the bitmap every single time. Try something like this:
Bitmap buffer = null;
#Override
public void onDraw(Canvas canvas)
{
if(buffer == null) buffer = Bitmap.createBitmap(256, 192, Bitmap.Config.RGB_565);
buffer.copyPixelsFromBuffer(pixelsA);
canvas.drawBitmap(buffer, 0, 0, null);
}
EDIT: as pointed out, you need to update the pixel buffer. And the bitmap must be mutable for that to happen.
if pixelsA is already an array of pixels (which is what I would infer from your statement about containing colors) then you can just render them directly without converting with:
canvas.drawBitmap(pixelsA, 0, 256, 0, 0, 256, 192, false, null);
I am trying to build an app that tracks touchpoints and draws circles at those points using Flash Builder. The following works perfectly, but after a while, it begins to lag and the touch will be well ahead of the drawn circles. Is there a way of drawing the circles that does not produce lag as more and more of them are added?
In declarations, I have:
<fx:Component className="Circle">
<s:Ellipse>
<s:stroke>
<s:SolidColorStroke alpha="0"/>
</s:stroke>
</s:Ellipse>
</fx:Component>
And this is the drawing function:
var c:Circle = new Circle();
c.x = somex;
c.y = somey;
c.fill = new SolidColor(somecolorint);
c.height = somesize;
c.width = somesize;
c.alpha = 1;
addElement(c);
c = null;
Try taking a look at doing a fullscreen Bitmap created with a BitmapData class. As the touch points are moved, update the bitmap data at the coordinates where the touch occured. Modifying and blitting a screen-sized bitmap is extremely fast and will probably work great for what you're trying to do.
Another performance trade off often done is to make a series of lines instead of continuous circles. You create a new line segment only when a certain distance has been traveled, this lets you limit the number of nodes in the segment thereby keeping performance high.
I have a big spritesheet (3808x1632) composed by 42 frames.
I would present an animation with these frames and I use a thread to load a bitmap array with all the frames, with a splash screen waiting for its end.
I'm not using a SurfaceView (and a draw function of a canvas), I just load frame by frame in an ImageView in my main layout.
My approach is similar to Loading a large number of images from a spritesheet
The completion actually takes almost 15 seconds, not acceptable.
I use this kind of function:
for (int i=0; i<TotalFramesTeapotBG; i++) {
xStartTeapotBG = (i % framesInRowsTeapotBG) * frameWidthTeapotBG;
yStartTeapotBG = (i / framesInRowsTeapotBG) * frameHeightTeapotBG;
mVectorTeapotBG.add(Bitmap.createBitmap(framesBitmapTeapotBG, xStartTeapotBG, yStartTeapotBG, frameWidthTeapotBG, frameHeightTeapotBG));
}
framesBitmapTeapotBG is the big spritesheet.
Looking more deeply, I've read in the logcat that the createBitmap function takes a lot of time, maybe because the spritesheet is too big.
I found somewhere that I could make a window on the big spritesheet, using the rect function and canvas, creating small bitmaps to be loaded in the array, but it was not really clear. I'm talking about that post: cut the portion of bitmap
My question is: how can I speed the spritesheet cut?
Edit:
I'm trying to use this approach but I cannot see the final animation:
for (int i=0; i<TotalFramesTeapotBG; i++) {
xStartTeapotBG = (i % framesInRowsTeapotBG) * frameWidthTeapotBG;
yStartTeapotBG = (i / framesInRowsTeapotBG) * frameHeightTeapotBG;
Bitmap bmFrame = Bitmap.createBitmap(frameWidthTeapotBG, frameHeightTeapotBG, Bitmap.Config.ARGB_8888);
Canvas c = new Canvas(bmFrame);
Rect src = new Rect(xStartTeapotBG, yStartTeapotBG, frameWidthTeapotBG, frameHeightTeapotBG);
Rect dst = new Rect(0, 0, frameWidthTeapotBG, frameHeightTeapotBG);
c.drawBitmap(framesBitmapTeapotBG, src, dst, null);
mVectorTeapotBG.add(bmFrame);
}
Probably, the Bitmap bmFrame is not correctly managed.
The short answer is better memory management.
The sprite sheet you're loading is huge, and then you're making a copy of it into a bunch of little bitmaps. Supposing the sprite sheet can't be any smaller, I'd suggest taking one of two approaches:
Use individual bitmaps. This will reduce the memory copies as well as the number of times Dalvik will have to grow the heap. However, these benefits may be limited by the need to load many images off the filesystem instead of just one. This would be the case in a normal computer, but Android systems may get different results since they're run off flash memory.
Blit directly from your sprite sheet. When drawing, just draw straight from sprite sheet using something like Canvas.drawBitmap(Bitmap bitmap, Rect src, Rect dst, Paint paint). This will reduce your file loads to one large allocation that probably only needs to happen once in the lifetime of your activity.
I think the second option is probably the better of the two since it will be easier on the memory system and be less work for the GC.
Thanks to stevehb for the suggestion, I finally got it:
for (int i = 0; i < TotalFramesTeapotBG; i++) {
xStartTeapotBG = (i % framesInRowsTeapotBG) * frameWidthTeapotBG;
yStartTeapotBG = (i / framesInRowsTeapotBG) * frameHeightTeapotBG;
Bitmap bmFrame = Bitmap.createBitmap(frameWidthTeapotBG, frameHeightTeapotBG, Bitmap.Config.ARGB_8888);
Canvas c = new Canvas(bmFrame);
Rect src = new Rect(xStartTeapotBG, yStartTeapotBG, xStartTeapotBG+frameWidthTeapotBG, yStartTeapotBG+frameHeightTeapotBG);
Rect dst = new Rect(0, 0, frameWidthTeapotBG, frameHeightTeapotBG);
c.drawBitmap(framesBitmapTeapotBG, src, dst, null);
mVectorTeapotBG.add(bmFrame);
}
The computation time falls incredibly! :)
Use a LevelListDrawable. Cut the sprites into individual frames and drop them in your drawable resource directory. Either programmatically or through an xml based level-list drawable create your drawable. Then use ImageView.setImageLevel() to pick your frame.
I use a method of slicing based on rows and columns. However your sprite sheet is rather huge. You have to think you are putting that whole sheet into memory. 3808x1632x4 is the size of the image in memory.
Anyway, what I do is I take an image (lets say a 128x128) and then tell it there are 4 columns and 2 rows in the Sprite(bitmap, 4, 2) constructor. Then you can slice and dice based on that. bitmap.getWidth() / 4 etc... pretty simple stuff. However if you want to do some real stuff use OpenGL and use textures.
Oh I also forgot to mention there are some onDraw stuff that needs to happen. Basically you keep an index counter and slice a rectangle from the bitmap and draw that from a source rectangle to a destination rectangle on the canvas.