Develop Reference

Get amplitude from MediaPlayer using Visualizer

i've been reading another posts about calculate the amplitude in real time from a Mediaplayer, but i have no clear how to get a value useful for me. What i need is a linear amplitude value normalize between 0-100, but as i've watched in another posts they are performing a db calculation which has not much sense, cause they are not normalized to max 0dB value (from How to calculate the audio amplitude in real time (android)):
double amplitude = 0;
for (int i = 0; i < audioData.length/2; i++) {
double y = (audioData[i*2] | audioData[i*2+1] << 8) / 32768.0
// depending on your endianness:
// double y = (audioData[i*2]<<8 | audioData[i*2+1]) / 32768.0
amplitude += Math.abs(y);
}
amplitude = amplitude / audioData.length / 2;
I've watched that for calculate de dB, i should do as below (from How to compute decibel (dB) of Amplitude from Media Player?
)
double sum=0;
for (int i = 0; i < audioData.length/2; i++) {
double y = (audioData[i*2] | audioData[i*2+1] << 8) / 32768.0;
sum += y * y;
}
double rms = Math.sqrt(sum / audioData.length/2);
dbAmp = 20.0*Math.log10(rms);
I've tried for that solution but the real time values are near to 0 but sometimes are over than 0, i mean, something between -Inifinit (no sound) to 1.2 (if i avoid 20.0* multiply) or anything else from than order. Anyway, i'd like to obtain a normalized value [0-100], not a dB value.

How to efficiently get and set bitmap pixels

I am currently making an app that involves altering the RGB values of pixels in a bitmap and creating a new bitmap after.
My problem is I need help increasing speed of this process. (It can take minutes to process a bitmap with inSampleSize = 2 and forever to process an inSampleSize = 1) Right now, I am using the getPixel and setPixel methods to alter the pixels and believe these two methods are the root of the problem as they are very inefficient. The getPixels method isn't suitable as I am not altering each pixel in order (ex. getting a pixel and changing a radius of 5 pixels around it to the same colour) unless anyone knows of a way to use getPixels (perhaps be able to put the pixels in a 2D array).
This is part of my code:
public static final alteredBitmp(Bitmap bp)
{
//initialize variables
// ..................
Bitmap bitmap = Bitmap.createBitmap(width, height,
Bitmap.Config.ARGB_8888);
for (int x = 0; x < width; x++) {
int left = Math.max(0, x - RADIUS);
int right = Math.min(x + RADIUS, width - 1);
for (int y = 0; y < height; ++y) {
int top = Math.max(0, y - RADIUS);
int bottom = Math.min(y + RADIUS, height - 1);
int maxIndex = -1;
for (int j = top; j <= bottom; j++) {
for (int i = left; i <= right; i++) {
pixelColor = bitmap.getPixel(i, j);
//get rgb values
//make changes to those values
}
}
}
}
//set new rgb values
bitmap.setPixel(x, y, Color.rgb(r, g, b));
//return new bitmap
Much thanks in advance!

Consider looking at RenderScript, which is Android's high performance compute framework. As you are iterating over width x height number of pixels and altering each one which in a modern device could be around a million pixels or higher, doing it in a single thread can take minutes. RenderScript can parallelize operations over CPU or the GPU where possible.
http://android-developers.blogspot.com/2012/01/levels-in-renderscript.html
http://developer.android.com/guide/topics/renderscript/index.html
Google IO 2013 session:
https://youtu.be/uzBw6AWCBpU
RenderScript compatibility library: http://android-developers.blogspot.com/2013/09/renderscript-in-android-support-library.html

Color Image in the Google Tango Leibniz API

I am trying to capture the image data in the onFrameAvailable method from a Google Tango. I am using the Leibniz release. In the header file it is said that the buffer contains HAL_PIXEL_FORMAT_YV12 pixel data. In the release notes they say the buffer contains YUV420SP. But in the documentation it is said the pixels are RGBA8888 format (). I am a little confused and additionally. I don't really get image data but a lot of magenta and green. Right now I am trying to convert from YUV to RGB similar to this one. I guess there is something wrong with the stride, too. Here eís the code of the onFrameAvailable method:
int size = (int)(buffer->width * buffer->height);
for (int i = 0; i < buffer->height; ++i)
{
for (int j = 0; j < buffer->width; ++j)
{
float y = buffer->data[i * buffer->stride + j];
float v = buffer->data[(i / 2) * (buffer->stride / 2) + (j / 2) + size];
float u = buffer->data[(i / 2) * (buffer->stride / 2) + (j / 2) + size + (size / 4)];
const float Umax = 0.436f;
const float Vmax = 0.615f;
y = y / 255.0f;
u = (u / 255.0f - 0.5f) ;
v = (v / 255.0f - 0.5f) ;
TangoData::GetInstance().color_buffer[3*(i*width+j)]=y;
TangoData::GetInstance().color_buffer[3*(i*width+j)+1]=u;
TangoData::GetInstance().color_buffer[3*(i*width+j)+2]=v;
}
}
I am doing the yuv to rgb conversion in the fragment shader.
Has anyone ever obtained an RGB image for the Google Tango Leibniz release? Or had someone similar problems when converting from YUV to RGB?

YUV420SP (aka NV21) is correct for the time being. An explanation is here. In this format you have a width x height array where each element is a Y byte, followed by a width/2 x height/2 array where each element is a V byte and a U byte. Your code is implementing YV21, which has separate arrays for V and U instead of interleaving them in one array.
You mention that you are doing YUV to RGB conversion in a fragment shader. If all you want to do with the camera images is draw then you can use TangoService_connectTextureId() and TangoService_updateTexture() instead of TangoService_connectOnFrameAvailable(). This approach delivers the camera image to you already in an OpenGL texture that gives your fragment shader RGB values without bothering with the pixel format details. You will need to bind to GL_TEXTURE_EXTERNAL_OES (instead of GL_TEXTURE_2D), and your fragment shader would look something like this:
#extension GL_OES_EGL_image_external : require
precision mediump float;
varying vec4 v_t;
uniform samplerExternalOES colorTexture;
void main() {
gl_FragColor = texture2D(colorTexture, v_t.xy);
}
If you really do want to pass YUV data to a fragment shader for some reason, you can do so without preprocessing it into floats. In fact, you don't need to unpack it at all - for NV21 just define a 1-byte texture for Y and a 2-byte texture for VU, and load the data as-is. Your fragment shader will use the same texture coordinates for both.

By the way, if someone experienced problems with capturing the image data on the Leibniz release, too: One of the developers told me that there is a bug concerning the camera and that it should be fixed with the Nash release.
The bug caused my buffer to be null but when I used the Nash update I got data again. However, right now the problem is that the data I am using doesn't make sense. I guess/hope the cause is that the Tablet didn't get the OTA update yet (there can be a gap between the actual release date and the OTA software update).

Just try code following:
//C#
public bool YV12ToPhoto(byte[] data, int width, int height, out Texture2D photo)
{
photo = new Texture2D(width, height);
int uv_buffer_offset = width * height;
for (int i = 0; i < height; i++)
{
for (int j = 0; j < width; j++)
{
int x_index = j;
if (j % 2 != 0)
{
x_index = j - 1;
}
// Get the YUV color for this pixel.
int yValue = data[(i * width) + j];
int uValue = data[uv_buffer_offset + ((i / 2) * width) + x_index + 1];
int vValue = data[uv_buffer_offset + ((i / 2) * width) + x_index];
// Convert the YUV value to RGB.
float r = yValue + (1.370705f * (vValue - 128));
float g = yValue - (0.689001f * (vValue - 128)) - (0.337633f * (uValue - 128));
float b = yValue + (1.732446f * (uValue - 128));
Color co = new Color();
co.b = b < 0 ? 0 : (b > 255 ? 1 : b / 255.0f);
co.g = g < 0 ? 0 : (g > 255 ? 1 : g / 255.0f);
co.r = r < 0 ? 0 : (r > 255 ? 1 : r / 255.0f);
co.a = 1.0f;
photo.SetPixel(width - j - 1, height - i - 1, co);
}
}
return true;
}
I have succeeded.

Get quaternion from Android gyroscope?

The official development documentation suggests the following way of obtaining the quaternion from the 3D rotation rate vector (wx, wy, wz).
// Create a constant to convert nanoseconds to seconds.
private static final float NS2S = 1.0f / 1000000000.0f;
private final float[] deltaRotationVector = new float[4]();
private float timestamp;
public void onSensorChanged(SensorEvent event) {
// This timestep's delta rotation to be multiplied by the current rotation
// after computing it from the gyro sample data.
if (timestamp != 0) {
final float dT = (event.timestamp - timestamp) * NS2S;
// Axis of the rotation sample, not normalized yet.
float axisX = event.values[0];
float axisY = event.values[1];
float axisZ = event.values[2];
// Calculate the angular speed of the sample
float omegaMagnitude = sqrt(axisX*axisX + axisY*axisY + axisZ*axisZ);
// Normalize the rotation vector if it's big enough to get the axis
// (that is, EPSILON should represent your maximum allowable margin of error)
if (omegaMagnitude > EPSILON) {
axisX /= omegaMagnitude;
axisY /= omegaMagnitude;
axisZ /= omegaMagnitude;
}
// Integrate around this axis with the angular speed by the timestep
// in order to get a delta rotation from this sample over the timestep
// We will convert this axis-angle representation of the delta rotation
// into a quaternion before turning it into the rotation matrix.
float thetaOverTwo = omegaMagnitude * dT / 2.0f;
float sinThetaOverTwo = sin(thetaOverTwo);
float cosThetaOverTwo = cos(thetaOverTwo);
deltaRotationVector[0] = sinThetaOverTwo * axisX;
deltaRotationVector[1] = sinThetaOverTwo * axisY;
deltaRotationVector[2] = sinThetaOverTwo * axisZ;
deltaRotationVector[3] = cosThetaOverTwo;
}
timestamp = event.timestamp;
float[] deltaRotationMatrix = new float[9];
SensorManager.getRotationMatrixFromVector(deltaRotationMatrix, deltaRotationVector);
// User code should concatenate the delta rotation we computed with the current rotation
// in order to get the updated rotation.
// rotationCurrent = rotationCurrent * deltaRotationMatrix;
}
}
My question is:
It is quite different from the acceleration case, where computing the resultant acceleration using the accelerations ALONG the 3 axes makes sense.
I am really confused why the resultant rotation rate can also be computed with the sub-rotation rates AROUND the 3 axes. It does not make sense to me.
Why would this method - finding the composite rotation rate magnitude - even work?

Since your title does not really match your questions, I'm trying to answer as much as I can.
Gyroscopes don't give an absolute orientation (as the ROTATION_VECTOR) but only rotational velocities around those axis they are built to 'rotate' around. This is due to the design and construction of a gyroscope. Imagine the construction below. The golden thing is rotating and due to the laws of physics it does not want to change its rotation. Now you can rotate the frame and measure these rotations.
Now if you want to obtain something as the 'current rotational state' from the Gyroscope, you will have to start with an initial rotation, call it q0 and constantly add those tiny little rotational differences that the gyroscope is measuring around the axis to it: q1 = q0 + gyro0, q2 = q1 + gyro1, ...
In other words: The Gyroscope gives you the difference it has rotated around the three constructed axis, so you are not composing absolute values but small deltas.
Now this is very general and leaves a couple of questions unanswered:
Where do I get an initial position from? Answer: Have a look at the Rotation Vector Sensor - you can use the Quaternion obtained from there as an initialisation
How to 'sum' q and gyro?
Depending on the current representation of a rotation: If you use a rotation matrix, a simple matrix multiplication should do the job, as suggested in the comments (note that this matrix-multiplication implementation is not efficient!):
/**
* Performs naiv n^3 matrix multiplication and returns C = A * B
*
* #param A Matrix in the array form (e.g. 3x3 => 9 values)
* #param B Matrix in the array form (e.g. 3x3 => 9 values)
* #return A * B
*/
public float[] naivMatrixMultiply(float[] B, float[] A) {
int mA, nA, mB, nB;
mA = nA = (int) Math.sqrt(A.length);
mB = nB = (int) Math.sqrt(B.length);
if (nA != mB)
throw new RuntimeException("Illegal matrix dimensions.");
float[] C = new float[mA * nB];
for (int i = 0; i < mA; i++)
for (int j = 0; j < nB; j++)
for (int k = 0; k < nA; k++)
C[i + nA * j] += (A[i + nA * k] * B[k + nB * j]);
return C;
}
To use this method, imagine that mRotationMatrix holds the current state, these two lines do the job:
SensorManager.getRotationMatrixFromVector(deltaRotationMatrix, deltaRotationVector);
mRotationMatrix = naivMatrixMultiply(mRotationMatrix, deltaRotationMatrix);
// Apply rotation matrix in OpenGL
gl.glMultMatrixf(mRotationMatrix, 0);
If you chose to use Quaternions, imagine again that mQuaternion contains the current state:
// Perform Quaternion multiplication
mQuaternion.multiplyByQuat(deltaRotationVector);
// Apply Quaternion in OpenGL
gl.glRotatef((float) (2.0f * Math.acos(mQuaternion.getW()) * 180.0f / Math.PI),mQuaternion.getX(),mQuaternion.getY(), mQuaternion.getZ());
Quaternion multiplication is described here - equation (23). Make sure, you apply the multiplication correctly, since it is not commutative!
If you want to simply know rotation of your device (I assume this is what you ultimately want) I strongly recommend the ROTATION_VECTOR-Sensor. On the other hand Gyroscopes are quite precise for measuring rotational velocity and have a very good dynamic response, but suffer from drift and don't give you an absolute orientation (to magnetic north or according to gravity).
UPDATE: If you want to see a full example, you can download the source-code for a simple demo-app from https://bitbucket.org/apacha/sensor-fusion-demo.

Makes sense to me. Acceleration sensors typically work by having some measurable quantity change when force is applied to the axis being measured. E.g. if gravity is pulling down on the sensor measuring that axis, it conducts electricity better. So now you can tell how hard gravity, or acceleration in some direction, is pulling. Easy.
Meanwhile gyros are things that spin (OK, or bounce back and forth in a straight line like a tweaked diving board). The gyro is spinning, now you spin, the gyro is going to look like it is spinning faster or slower depending on the direction you spun. Or if you try to move it, it will resist and try to keep going the way it is going. So you just get a rotation change out of measuring it. Then you have to figure out the force from the change by integrating all the changes over the amount of time.
Typically none of these things are one sensor either. They are often 3 different sensors all arranged perpendicular to each other, and measuring a different axis. Sometimes all the sensors are on the same chip, but they are still different things on the chip measured separately.

Average-Function of Sensor-Data in Android

I'm trying to build the weighted-average from the sensor-data I get by the SensorManager.
My problem is, that bearing, pitch and roll have a maximum value and when I'm exactly at this point, the values swap from 0 to 359 or backwards.
My average is at the moment simply an addition of all values and one division by the number of values.
Let's say I get the values: 1, 359, 350, 10
In this case, I want to have an average of 0. How do I have to change my equation to get this functionality?
Do I have to check for the "nearest" distance to 0/360 and using this value instead of the real value?
This would also make some troubles if I have values around 180:
160, 200 -> the average has to be 180, but with my nearest-distance idea, it would be 160, because 200 + 160 = 360.
How can I solve this?
Edit: This are the values I get from the SensorManager.
0 <= azimuth<360
180<=pitch<=180
90<=roll<=90
Edit2: Sorry, I forgot to mention that I'm using a weighted average:
double sum = 0;
for int i = 0; i < max; i++
sum += value[i] * (i / (triangular_number(max))
return sum

To calculate average of angle use the following
public static final float averageAngle(float[] terms, int totalTerm)
{
float sumSin = 0;
float sumCos = 0;
for (int i = 0; i < totalTerm; i++)
{
sumSin += Math.sin(terms[i]);
sumCos += Math.cos(terms[i]);
}
return (float) Math.atan2(sumSin / totalTerm, sumCos / totalTerm);
}

I found a blog post about this.
To summarize it shortly: You have to calculate the average of the sines of all your aizmuth values and the average of the cosines of the aizmuth values and then put these average values in the atan2 function, then if necessary make the result positive by adding 2 * PI. Don't forget to convert degrees values to radians and vice versa.