I want to load .png file via asset manager which is provided by android sdk. AssetManager manager; /........./ BitmapFactory.decodeStream(manager.open(path));
It returns BGR format data but opengl es 2.0 uses RGB format so , Blue seems red , red seems blue, how odd.
Is there any solution for it?
I use Nvıdia Tegra 2 (Android 2.2) device for test the application along with c++ via JNI.
You must know the number of bits for colors, let's say n bit is for a color, so the first n bit represents BLUE, the second n bits represent GREEN and the final n bits represent RED in the input. You need to swap these bit groups into the correct order, like this:
output = (input << (2 * n)) + (input << n >> n) + (input >> (2 * n));
To be able to use this solution you need to find out how much is n.
Recent versions of OpenGL, also provide BGR input formats; OpenGL-ES not, unfortunatly. Since you're on Android you have to deal with OpenGL-ES.
If you're using a fragment shader it is also trivial to apply a rgb→bgr swizzle, which if probably the easiest way to overcome this problem.
Related
I am following this blog post and GitHub almost exactly:
Blog
Github
But when I run, take a picture and call this line:
var outputs = new float[tfLabels.Count];
tfInterface.Feed("Placeholder", floatValues, 1, 227, 227, 3);
tfInterface.Run(new[] { "loss" });
tfInterface.Fetch("loss", outputs);
The app actually crashes and generates the error below on the .Run line.
I get this error in the output window (and the app crashes):
04-04 17:39:12.575 E/TensorFlowInferenceInterface( 8017): Failed to
run TensorFlow inference with inputs:[Placeholder], outputs:[loss]
Unhandled Exception:
Java.Lang.IllegalArgumentException: Input to reshape is a tensor with
97556 values, but the requested shape requires a multiple of 90944
[[Node: block0_0_reshape0 = Reshape[T=DT_FLOAT, Tshape=DT_INT32,
_device="/job:localhost/replica:0/task:0/device:CPU:0"](block0_0_concat,
block0_0_reshape0/shape)]]
According to the posts I am reading from the searching I am doing on this error, I sort of understand this is due to the image not fitting the expected size exactly but in the example I am following, this is resized to fit 227x227 everytime and converted to float like in these lines:
var resizedBitmap = Bitmap.CreateScaledBitmap(bitmap, 227, 227, false).Copy(Bitmap.Config.Argb8888, false);
var floatValues = new float[227 * 227 * 3];
var intValues = new int[227 * 227];
resizedBitmap.GetPixels(intValues, 0, 227, 0, 0, 227, 227);
for(int i = 0; i < intValues.Length; i++)
{
var val = intValues[i];
floatValues[i * 3 + 0] = ((val & 0xFF) - 104);
floatValues[i * 3 + 1] = (((val >> 8) & 0xFF) - 117);
floatValues[i * 3 + 2] = (((val >> 16) & 0xFF) - 123);
}
So, I don't understand what is causing this or how to fix it. Please help!
UPDATE: I found out the issue is with my model or my labels. I found this out by simply swapping in the model and label file from the sample/github above while leaving all my code the same. When I did this, I no longer get the error. HOWEVER, this still doesn't tell me much. The error is not very explanatory to point me in a direction of what could be wrong with my model. I assume it is the model because the labels file is simply just a text file with labels on each line. I used Custom Vision Service on Azure to create my model. It trained fine and tests just fine on the web portal. I then exported it as TensorFlow. So, I am not sure what I could have done wrong or how to fix it.
Thanks!
After no answers here and several days of searching and trial and error, I have found the issue. In general, I guess this reshape error I was getting you can get if you are feeding the model with an image size other that it is expecting or setup to receive.
The issue is that, everything I have read says that typically you must feed the model with a 227 x 227 x 3 image. Then, I started noticing that size varies on some posts. Some people say 225 x 225 x 3, others say 250 x 250 x 3 and so on. I had tried those sizes as well with no luck.
As you can see in my edit in the question, I did have a clue. When using somebody else's pretrained model, my code works fine. However, when I use my custom model which I created on the Microsoft Azure CustomVision.ai site, I was getting this error.
So, I decided I would try to inspect the models to see what was different. I followed this post: Inspect a pre trained model
When I inspected the model that works using TensorBoard, I see that the input is 227 x 227 x 3 which is what I expected. However, when I viewed my model, I noticed that it was 224 x 224 x 3! I changed my code to resize the image to that size and it works! Problem went away.
So, to summarize, for some reason Microsoft Custom Vision service model generated a model to expect an image size of 224 x 224 x 3. I didn't see any documentation or setting for this. I also don't know if that number will change with each model. If you get a similar shape error, the first place I would check is the size of the image you are feeding your model and what it expects as an input. The good news is you can check your model, even if pre-trained, using TensorBoard and the post I linked above. Look at the input section, it should look something like this:
Hope this helps!
As the title said, anyone know what is RGBX_8888 pixel format? and what is the difference with RGBA_8888? Is RGBA_8888 offers an alpha channel but RGBX_8888 does not?
The android documentation does not give much information on this unfortunately.
Thanks.
RGBX means, that the pixel format still has an alpha channel, but it is ignored, and is always set to 255.
Some reference:
Blackberry PixelFormat
(It is not android, however I guess that the naming conventions stay same across platforms.)
The RGBX 32 bit RGB format is stored in memory as 8 red bits, 8 green bits, 8 blue bits, and 8 ignored bits.
Android 4.1.2 source code (texture.cpp) Line 80
There is a function called PointSample, where it samples based on a template format, and the passed parameters. You can see, that at pixelformat RGBX_8888, the alpha channel is ignored and set to 255, while at RGBA_8888, it is normally sampled.
if (GGL_PIXEL_FORMAT_RGBA_8888 == format)
*sample = *(data + index);
else if (GGL_PIXEL_FORMAT_RGBX_8888 == format)
{
*sample = *(data + index);
*sample |= 0xff000000;
}
for an Android Sequencer App, we gave a try to the Superpowered audio engine, which uses the Android NDK and C++. It's a very young project and there are some bugs in it, for example if a sample is too short, the buffers aren't cleared correctly, so I like to clear them by myself. For these, the memory is aligned this way:
bufferA = (float *)memalign(16, (buffersize + 16) * sizeof(float) * 2);
Because I'm a C++ beginner, I don't know how I could iterate through this pointer bufferA to overwrite its values.
Can anybody help? Thanks!
In order to minimize the memory usage of bitmaps, yet still try to maximize the quality of them, I would like to ask a simple question:
Is there a way for me to check if a given image file (.png file) has transparency using the API, without checking every pixel in it?
If the image doesn't have any transparency, it would be the best to use a different bitmap format that uses only the RGB values.
The problem is that Android also doesn't have a format for just the 3 colors. Only RGB_565, which they say that degrade the quality of the image and that should have dithering feature enabled.
Is there also a way to read only the RGB values and be able to show them?
For me bitmap.hasAlpha() works fine to check first if the bitmap has alpha values. Afterwards you have to run through the pixels and create a second bitmap with no alpha I would suggest.
Let's start a bit off-topic
the problem is that android also doesn't have a format for just the 3 colors . only RGB_565 , which they say that degrade the quality of the image and that should have dithering feature enabled.
The reason for that problem is not really Android specific. It's about performance while drawing images. You get the best performance if the pixeldata fits exactly in 1 32bit memory cell.
So the most obvious good pixel format is the ARGB_8888 format which uses exactly 32bit (24 for the color 8 for alpha). While drawing you don't need to do anything but to loop over the image data and each cell you read can be drawn directly. The only downside is the required memory to work with such images, both when they just sit in memory and while displaying them since the graphic hardware has to transfer more data.
The second best option is to use a format where several pixels fit into 1 cell. Using 2 pixels in 32bit you have 16bit per pixel left and one of the formats using 16bit is the 565 format. 5bit red, 6bit green, 5bit blue. While drawing this you can still work on memory cells separately and all you have to do is to split 1 cell in parts. Due to the smaller memory size required for images, drawing can sometimes be even faster than using 32bit colors. Since in the beginning of android memory was a much bigger problem they chose this format to be the default.
And the worst category of formats are those where pixels don't fit into those cells. If you take just the 3 colors you get 24 bit and those need to be distributed over 2 cells in 3 out of 4 cases. For example the second pixel would use the remaining 8 bit from the first cell & the first 16bit of the next cell. The extra work required to work with 24bit colors is so big that it is not used. And when drawing images you usually have alpha at some point anyways and if not you simply use 32bit but ignore the alpha bits.
So the 16bit approach looks ugly & the 24 bit approach does not make sense. And since the memory limitations of Android are not as tight as they were and the hardware got faster, Android has switched it's default to 32bit (explained in even more details in http://www.curious-creature.org/2010/12/08/bitmap-quality-banding-and-dithering/)
Back to your real question
is there a way for me to check if a given image file (png file) has transparency using the API , without checking every pixel in it?
I don't know. But JPEG images don't support alpha and PNG images usually have alpha. You could simply abuse the file extension to get it right in most cases.
But I would suggest you don't bother with all that and simply use ARGB_8888 and apply the nice image loading techniques detailed in the Android Training documentation about Displaying Bitmaps Efficiently.
The reason people run into memory problems is usually either that they have way more images loaded in memory than they currently display or they use giant images that can't be displayed on the small screen of a phone. And in my opinion it makes more sense to add good memory management than complicating your code to downgrade the image quality.
There is a way to check if a PNG file has transparency, or at least if it supports it:
public final static int COLOR_GREY = 0;
public final static int COLOR_TRUE = 2;
public final static int COLOR_INDEX = 3;
public final static int COLOR_GREY_ALPHA = 4;
public final static int COLOR_TRUE_ALPHA = 6;
private final static int DECODE_BUFFER_SIZE = 16 * 1024;
private final static int HEADER_DECODE_BUFFER_SIZE = 1024;
/** given an inputStream of a png file , returns true iff found that it has transparency (in its header) */
private static boolean isPngInputStreamContainTransparency(final InputStream pngInputStream) {
try {
// skip: png signature,header chunk declaration,width,height,bitDepth :
pngInputStream.skip(12 + 4 + 4 + 4 + 1);
final byte colorType = (byte) pngInputStream.read();
switch (colorType) {
case COLOR_GREY_ALPHA:
case COLOR_TRUE_ALPHA:
return true;
case COLOR_INDEX:
case COLOR_GREY:
case COLOR_TRUE:
return false;
}
return true;
} catch (final Exception e) {
}
return false;
}
Other than that, I don't know if such a thing is possible.
i've found the next links which could be helpful for checking if the png file has transparency . sadly, it's a solution only for png files . rest of the files (like webP , bmp, ...) need to have a different parser .
links:
http://www.java2s.com/Code/Java/2D-Graphics-GUI/PNGDecoder.htm
http://hg.l33tlabs.org/twl/file/tip/src/de/matthiasmann/twl/utils/PNGDecoder.java
http://www.java-gaming.org/index.php/topic,24202
I have played for a while with OpenGL on Android on various devices. And unless I'm wrong, the default rendering is always performed with the RGB565 pixel format.
I would however like to render more accurate colors using RGB888.
The GLSurfaceView documentation mentions two methods which relate to pixel formats:
the setFormat() method exposed by SurfaceHolder, as returned by SurfaceView.getHolder()
the GLSurfaceView.setEGLConfigChooser() family of methods
Unless I'm wrong, I think I only need to use the later. Or is using SurfaceHolder.setFormat() relevant here?
The documentation of the EGLConfigChooser class mentions EGL10.eglChooseConfig(), to discover which configurations are available.
In my case it is ok to fallback to RGB565 if RGB888 isn't available, but I would prefer this to be quite rare.
So, is it possible to use RGB888 on most devices?
Are there any compatibility problems or weird bugs with this?
Do you have an example of a correct and reliable way to setup the GLSurfaceView for rendering RGB888?
On newer devices, most of them should support RGBA8888 as a native format. One way to force RGBA color format is to set the translucency of the surface, you'd still want to pick the EGLConfig to best guess the config for the channels in addition to the depth and stencil buffers.
setEGLConfigChooser(8, 8, 8, 8, 0, 0);
getHolder().setFormat(PixelFormat.RGBA_8888);
However, if I read your question correctly you're asking for RGB888 support (alpha don't care) in other words, RGBX8888 which might not be supported by all devices (driver vendor limitation).
Something to keep in mind about performance though, since RGBA8888 is the color format natively supported by most GPUs hardware it's best to avoid any other color format (non natively supported) since that usually translate into a color conversion underneath adding non necessary work load to the GPU.
This is how I do it;
{
window = [[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]];
// cocos2d will inherit these values
[window setUserInteractionEnabled:YES];
[window setMultipleTouchEnabled:NO];
// must be called before any othe call to the director
[Director useFastDirector];
[[Director sharedDirector] setDisplayFPS:YES];
// create an openGL view inside a window
[[Director sharedDirector] attachInView:window];
// Default texture format for PNG/BMP/TIFF/JPEG/GIF images
// It can be RGBA8888, RGBA4444, RGB5_A1, RGB565
// You can change anytime.
[Texture2D setDefaultAlphaPixelFormat:kTexture2DPixelFormat_];
glClearColor(0.7f,0.7f,0.6f,1.0f);
//glClearColor(1.0f,1.0f,1.0f,1.0f);
[window makeKeyAndVisible];
[[Director sharedDirector] runWithScene:[GameLayer node]];
}
I hope that helps!