Is there a property or method that can be used to access the field of view ("FoV", "angle of view") of the camera when working with arcore?
From some experimentation it appears the FoV is typically about 60 degrees, but presumably this will vary depending on the device hardware.
If it cannot be directly accessed, is there a way to instead calculate the FoV angle from any of the Camera object properties e.g. the view matrix?
ARCore library v1.8.0 doesn't return FoV value. Instead you can calculate it using Camera parameters:
val frame = session.update()
val camera = frame.camera
val imageIntrinsics = camera.imageIntrinsics
val focalLength = imageIntrinsics.focalLength[0]
val size = imageIntrinsics.imageDimensions
val w = size[0]
val h = size[1]
val fovW = Math.toDegrees(2 * Math.atan(w / (focalLength * 2.0)))
val fovH = Math.toDegrees(2 * Math.atan(h / (focalLength * 2.0)))
Another solution with Camera2 API:
val cameraId = session.cameraConfig.cameraId
val cameraManager = context.getSystemService(Context.CAMERA_SERVICE) as CameraManager
val characteristics = cameraManager.getCameraCharacteristics(cameraId)
val maxFocus = characteristics.get(CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS)
val size = characteristics.get(CameraCharacteristics.SENSOR_INFO_PHYSICAL_SIZE)
val w = size.width
val h = size.height
val fovW = Math.toDegrees(2 * Math.atan(w / (maxFocus[0] * 2.0)))
val fovH = Math.toDegrees(2 * Math.atan(h / (maxFocus[0] * 2.0)))
Its available now within CameraIntrinsics class as getFocalLength()
The output is focal length in pixels which can be converted to degrees if image size is known.
Related
I'm using camera2 API, and a TextureView to capture photo.
As the TextureView size changed by myself and to preventing stretching result photo, I used these functions:
private fun showPreview(camera: CameraDevice) {
cameraDevice = camera
val surfaceTexture: SurfaceTexture? = mainBinding.textureView.surfaceTexture
surfaceTexture?.setDefaultBufferSize(previewSize.width, previewSize.height)
val previewSurface = Surface(surfaceTexture)
val viewFinderSize =
Size(mainBinding.textureView.measuredWidth, mainBinding.textureView.measuredHeight)
matrix = calculateTransform(viewFinderSize, previewSize)
mainBinding.textureView.setTransform(matrix)
captureRequestBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
captureRequestBuilder.addTarget(previewSurface)
cameraDevice.createCaptureSession(
listOf(previewSurface, imageReader.surface), captureStateCallback, null
)
}
private fun calculateTransform(viewFinderSize: Size, previewSize: Size): Matrix {
val matrix = Matrix()
val scaleFactors = if (viewFinderSize.height <= viewFinderSize.width) {
val previewRatio = previewSize.width / previewSize.height.toFloat()
val viewFinderRatio = viewFinderSize.width / viewFinderSize.height.toFloat()
val scaling = viewFinderRatio * previewRatio
PointF(1f, scaling)
} else {
val previewRatio = previewSize.height / previewSize.width.toFloat()
val viewFinderRatio = viewFinderSize.height / viewFinderSize.width.toFloat()
val scaling = viewFinderRatio * previewRatio
PointF(scaling, 1f)
}
matrix.preScale(scaleFactors.x, scaleFactors.y)
return matrix
}
These functions work correctly, But when I want to save result photo, bitmap data is more than just selected dimension in previewView.
I am able to run my custom tflite model in android but the output is totally wrong. I suspect it is due to my model needs input shape [1, 3, 640, 640] but the code makes channel last ByteBuffer. I have created tensor buffer like this TensorBuffer.createFixedSize(intArrayOf(1, 3, 640, 640), DataType.FLOAT32) but I still suspect inside the for loop, the channel is not properly set in the flat input (ByteBuffer).
I have copied this code from example where the required model shape was [1,32,32,3] (channel last). This is the reason for my doubt.
Below is my code:-
val model = YoloxPlate.newInstance(applicationContext)
val inputFeature0 = TensorBuffer.createFixedSize(intArrayOf(1, 3, 640, 640), DataType.FLOAT32)
val input = ByteBuffer.allocateDirect(640*640*3*4).order(ByteOrder.nativeOrder())
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
// Get channel values from the pixel value.
val r = Color.red(px)
val g = Color.green(px)
val b = Color.blue(px)
// Normalize channel values to [-1.0, 1.0]. This requirement depends on the model.
// For example, some models might require values to be normalized to the range
// [0.0, 1.0] instead.
val rf = r/ 1f
val gf = g/ 1f
val bf = b/ 1f
input.putFloat(bf)
input.putFloat(gf)
input.putFloat(rf)
}
}
inputFeature0.loadBuffer(input)
val outputs = model.process(inputFeature0)
val outputFeature0 = outputs.outputFeature0AsTensorBuffer
val flvals = outputFeature0.getFloatArray();
After using whiteboard and making and setting dim manually of the matrix, I figured it out.
It also used BGR instead of RGB as required by the model.
Working Perfectly now, here is the code (need to optimize multiple loop):-
val model = YoloxPlate.newInstance(applicationContext)
val inputFeature0 = TensorBuffer.createFixedSize(intArrayOf(1, 3, 640, 640), DataType.FLOAT32)
val input = ByteBuffer.allocateDirect(640*640*3*4).order(ByteOrder.nativeOrder())
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
val b = Color.blue(px)
val bf = b/ 1f
input.putFloat(bf)
}
}
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
val g = Color.green(px)
val gf = g/ 1f
input.putFloat(gf)
}
}
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
val r = Color.red(px)
val rf = r/ 1f
input.putFloat(rf)
}
}
inputFeature0.loadBuffer(input)
val outputs = model.process(inputFeature0)
val outputFeature0 = outputs.outputFeature0AsTensorBuffer
val flvals = outputFeature0.getFloatArray();
The old Sensor.TYPE_ORIENTATION sensor returned a pitch between -180° and 180°. This was a nice API which included filtering and worked great. Sadly Sensor.TYPE_ORIENTATION was deprecated and is not available on modern phones.
The blessed replacement for Sensor.TYPE_ORIENTATION is a complicated combination of Context.SENSOR_SERVICE and TYPE_MAGNETIC_FIELD and the SensorManager.getRotationMatrix() and SensorManager.getOrientation() functions. You're on your own when it comes to filtering. (As an aside I used iirj - the trivial low pass filters I found on Stackoverflow did not work as well as whatever Sensor.TYPE_ORIENTATION did)
The documentation for getOrientation claims that it returns a pitch between -π to π. This can't be true since the implementation is values[1] = (float) Math.asin(-R[7]); (asin returns values between -π/2 and π/2)
Is there any way to get the full 360° of pitch and roll from the rotation matrix?
This is a known issue that Google won't fix. I created my own getOrientation function based on Gregory G. Slabaugh's paper
// Based on pseudo code from http://www.close-range.com/docs/Computing_Euler_angles_from_a_rotation_matrix.pdf
object EulerAngleHelper {
private const val R11 = 0
private const val R12 = 1
private const val R13 = 2
private const val R21 = 3
private const val R22 = 4
private const val R23 = 5
private const val R31 = 6
private const val R32 = 7
private const val R33 = 8
private const val AZIMUTH = 0
private const val PITCH = 1
private const val ROLL = 2
private const val PHI_Z = AZIMUTH
private const val PSI_X = PITCH
private const val THETA_Y = ROLL
fun getOrientation(r: DoubleArray, values: DoubleArray): DoubleArray {
when {
r[R31] < -0.98 -> {
values[PHI_Z] = 0.0 // Anything; can set to 0
values[THETA_Y] = Math.PI / 2
values[PSI_X] = values[PHI_Z] + atan2(r[R12], r[R13])
}
r[R31] > 0.98 -> {
values[PHI_Z] = 0.0 // Anything; can set to 0
values[THETA_Y] = -Math.PI / 2
values[PSI_X] = values[PHI_Z] + atan2(-r[R12], -r[R13])
}
else -> {
values[THETA_Y] = -asin(r[R31])
val cosTheta = cos(values[THETA_Y])
values[PSI_X] = atan2(r[R32] / cosTheta, r[R33] / cosTheta)
values[PHI_Z] = atan2(r[R21] / cosTheta, r[R11] / cosTheta)
}
}
return values
}
}
I've only tested the pitch and roll.
I'm trying to draw the label lines as in picture using MPAndroidChart with a pie chart. I can't figure out how to
decouple the lines from the chart
draw that little circle at the beginning of the line.
Thank you.
This is by no means easy to achieve. To decouple the lines from the chart, you can use valueLinePart1OffsetPercentage and play with line part lengths. But to get the chart to draw dots at the end of lines, you need a custom renderer. Here's one:
class CustomPieChartRenderer(pieChart: PieChart, val circleRadius: Float)
: PieChartRenderer(pieChart, pieChart.animator, pieChart.viewPortHandler) {
override fun drawValues(c: Canvas) {
super.drawValues(c)
val center = mChart.centerCircleBox
val radius = mChart.radius
var rotationAngle = mChart.rotationAngle
val drawAngles = mChart.drawAngles
val absoluteAngles = mChart.absoluteAngles
val phaseX = mAnimator.phaseX
val phaseY = mAnimator.phaseY
val roundedRadius = (radius - radius * mChart.holeRadius / 100f) / 2f
val holeRadiusPercent = mChart.holeRadius / 100f
var labelRadiusOffset = radius / 10f * 3.6f
if (mChart.isDrawHoleEnabled) {
labelRadiusOffset = (radius - radius * holeRadiusPercent) / 2f
if (!mChart.isDrawSlicesUnderHoleEnabled && mChart.isDrawRoundedSlicesEnabled) {
rotationAngle += roundedRadius * 360 / (Math.PI * 2 * radius).toFloat()
}
}
val labelRadius = radius - labelRadiusOffset
val dataSets = mChart.data.dataSets
var angle: Float
var xIndex = 0
c.save()
for (i in dataSets.indices) {
val dataSet = dataSets[i]
val sliceSpace = getSliceSpace(dataSet)
for (j in 0 until dataSet.entryCount) {
angle = if (xIndex == 0) 0f else absoluteAngles[xIndex - 1] * phaseX
val sliceAngle = drawAngles[xIndex]
val sliceSpaceMiddleAngle = sliceSpace / (Utils.FDEG2RAD * labelRadius)
angle += (sliceAngle - sliceSpaceMiddleAngle / 2f) / 2f
if (dataSet.valueLineColor != ColorTemplate.COLOR_NONE) {
val transformedAngle = rotationAngle + angle * phaseY
val sliceXBase = cos(transformedAngle * Utils.FDEG2RAD.toDouble()).toFloat()
val sliceYBase = sin(transformedAngle * Utils.FDEG2RAD.toDouble()).toFloat()
val valueLinePart1OffsetPercentage = dataSet.valueLinePart1OffsetPercentage / 100f
val line1Radius = if (mChart.isDrawHoleEnabled) {
(radius - radius * holeRadiusPercent) * valueLinePart1OffsetPercentage + radius * holeRadiusPercent
} else {
radius * valueLinePart1OffsetPercentage
}
val px = line1Radius * sliceXBase + center.x
val py = line1Radius * sliceYBase + center.y
if (dataSet.isUsingSliceColorAsValueLineColor) {
mRenderPaint.color = dataSet.getColor(j)
}
c.drawCircle(px, py, circleRadius, mRenderPaint)
}
xIndex++
}
}
MPPointF.recycleInstance(center)
c.restore()
}
}
This custom renderer extends the default pie chart renderer. I basically just copied the code from PieChartRenderer.drawValues method, converted it to Kotlin, and removed everything that wasn't needed. I only kept the logic needed to determine the position of the points at the end of lines.
I tried to reproduce the image you showed:
val chart: PieChart = view.findViewById(R.id.pie_chart)
chart.setExtraOffsets(40f, 0f, 40f, 0f)
// Custom renderer used to add dots at the end of value lines.
chart.renderer = CustomPieChartRenderer(chart, 10f)
val dataSet = PieDataSet(listOf(
PieEntry(40f),
PieEntry(10f),
PieEntry(10f),
PieEntry(15f),
PieEntry(10f),
PieEntry(5f),
PieEntry(5f),
PieEntry(5f)
), "Pie chart")
// Chart colors
val colors = listOf(
Color.parseColor("#4777c0"),
Color.parseColor("#a374c6"),
Color.parseColor("#4fb3e8"),
Color.parseColor("#99cf43"),
Color.parseColor("#fdc135"),
Color.parseColor("#fd9a47"),
Color.parseColor("#eb6e7a"),
Color.parseColor("#6785c2"))
dataSet.colors = colors
dataSet.setValueTextColors(colors)
// Value lines
dataSet.valueLinePart1Length = 0.6f
dataSet.valueLinePart2Length = 0.3f
dataSet.valueLineWidth = 2f
dataSet.valueLinePart1OffsetPercentage = 115f // Line starts outside of chart
dataSet.isUsingSliceColorAsValueLineColor = true
// Value text appearance
dataSet.yValuePosition = PieDataSet.ValuePosition.OUTSIDE_SLICE
dataSet.valueTextSize = 16f
dataSet.valueTypeface = Typeface.DEFAULT_BOLD
// Value formatting
dataSet.valueFormatter = object : ValueFormatter() {
private val formatter = NumberFormat.getPercentInstance()
override fun getFormattedValue(value: Float) =
formatter.format(value / 100f)
}
chart.setUsePercentValues(true)
dataSet.selectionShift = 3f
// Hole
chart.isDrawHoleEnabled = true
chart.holeRadius = 50f
// Center text
chart.setDrawCenterText(true)
chart.setCenterTextSize(20f)
chart.setCenterTextTypeface(Typeface.DEFAULT_BOLD)
chart.setCenterTextColor(Color.parseColor("#222222"))
chart.centerText = "Center\ntext"
// Disable legend & description
chart.legend.isEnabled = false
chart.description = null
chart.data = PieData(dataSet)
Again, not very straightforward. I hope you like Kotlin! You can move most of that configuration code to a subclass if you need it often. Here's the result:
I'm not a MPAndroidChart expert. In fact, I've used it only once, and that was 2 years ago. But if you do your research, you can find a solution most of the time. Luckily, MPAndroidChart is a very customizable.
I want to build application which uses phone sensor fusion to rotate 3D object in OpenGL. But I want the Z axis to be locked therefore I want basically to apply 2D rotation to my model.
In order to build 2D rotation from the 3D matrix I get from SensorManager.getRotationMatrixFromVector() I build rotation matrix for each axis as explained on the picture:
I want to apply 2D rotation matrix which would be R=Ry*Rx however this seems not working. But applying R=Rz*Ry works as expected. My guess is that Rx values are not correct.
To build the Rz, Ry, Rx matrices I looked up values used by SensorManager.getOrientation() to calculate angles:
values[0] = (float) Math.atan2(R[1], R[4]);
values[1] = (float) Math.asin(-R[7]);
values[2] = (float) Math.atan2(-R[6], R[8]);
So here is how I build matrices for each axis:
private val degConst = 180/Math.PI
private var mTempRotationMatrix = MatrixCalculations.createUnit(3)
override fun onSensorChanged(event: SensorEvent?) {
val sensor = event?.sensor ?: return
when (sensor.type) {
Sensor.TYPE_ROTATION_VECTOR -> {
SensorManager.getRotationMatrixFromVector(mTempRotationMatrix, event.values)
val zSinAlpha = mTempRotationMatrix[1]
val zCosAlpha = mTempRotationMatrix[4]
val ySinAlpha = -mTempRotationMatrix[6]
val yCosAlpha = mTempRotationMatrix[8]
val xSinAlpha = -mTempRotationMatrix[7]
val xCosAlpha = mTempRotationMatrix[4]
val rx = MatrixCalculations.createUnit(3)
val ry = MatrixCalculations.createUnit(3)
val rz = MatrixCalculations.createUnit(3)
val sina = xSinAlpha
val cosa = xCosAlpha
val sinb = ySinAlpha
val cosb = yCosAlpha
val siny = zSinAlpha
val cosy = zCosAlpha
rx[4] = cosa
rx[5] = -sina
rx[7] = sina
rx[8] = cosa
ry[0] = cosb
ry[2] = sinb
ry[6] = -sinb
ry[8] = cosb
rz[0] = cosy
rz[1] = -siny
rz[3] = siny
rz[4] = cosy
val ryx = MatrixCalculations.multiply(ry, rx)
mTempRotationMatrix = ryx
MatrixCalculations.copy(mTempRotationMatrix, mRenderer.rotationMatrix)
LOG.info("product: [" + mRenderer.rotationMatrix.joinToString(" ") + "]")
val orientation = FloatArray(3)
SensorManager.getOrientation(mTempRotationMatrix, orientation)
LOG.info("yaw: " + orientation[0] * degConst + "\n\tpitch: " + orientation[1] * degConst + "\n\troll: " + orientation[2] * degConst)
}
The question is what I am doing wrong and what values to use for the Rx matrix. Is my math applied to this problem broken? Also interesting would be to know how value of event.values[3] is related to build rotation matrix in SensorManager.getRotationMatrixFromVector().
In theory operation R=Ry*Rx should give me correct rotation but it is not the case.