Develop Reference

How to detect button release after button press in Android Jetpack Compose?

I find listeners for onClick and onLongClick and even onPress but there is no event/listener for something like buttonDown and buttonUp, or onPress and onRelease.
Am I missing something? My current use case is that when a user presses a button I increment a count and when the user releases it I decrease the count. But in general I want something to start happening as soon as the user presses the button and stop when the user releases it. (For a real life example, see how Facebook Messenger records a video, you keep the button pressed to start and it stops when you release it.
I am using Jetpack Compose on Android.

You can use the InteractionSource.collectIsPressedAsState to know if the Button is pressed.
You can add a side effect to know when the Button is released.
Something like:
val interactionSource = remember { MutableInteractionSource() }
val isPressed by interactionSource.collectIsPressedAsState()
var currentStateTxt by remember { mutableStateOf("Not Pressed") }
var currentCount by remember { mutableStateOf(0) }
if (isPressed){
//Pressed
currentStateTxt = "Pressed"
currentCount += 1
//Use if + DisposableEffect to wait for the press action is completed
DisposableEffect(Unit) {
onDispose {
//released
currentStateTxt = "Released"
}
}
}
Button(onClick={},
interactionSource = interactionSource
){
Text("Current state = $currentStateTxt")
Text("Count = $currentCount")
}

Use .pointerInput modifier:
.pointerInput(Unit) {
forEachGesture {
awaitPointerEventScope {
awaitFirstDown()
//onPress actions here
do {
val event = awaitPointerEvent()
//Track other pointer evenst, like Drag etc...
} while (event.changes.any { it.pressed })
//onRelease actions here
}
}
}

these codes may be helpful for you
var isPressed by remember {
mutableStateOf(false)
}
.pointerInput(Unit) {
detectTapGestures(
onPress = {
try {
isPressed = true
isPlaying = true
sampleSong.start()
awaitRelease()
}
finally {
isPressed = false
isPlaying = false
sampleSong.pause()
}

I think you use Touch Listener on button , its easily detect button touch or untouch example
override fun onTouchEvent(e: MotionEvent): Boolean {
val x: Float = e.x
val y: Float = e.y
when (e.action) {
MotionEvent.ACTION_MOVE -> {
var dx: Float = x - previousX
var dy: Float = y - previousY
// reverse direction of rotation above the mid-line
if (y > height / 2) {
dx *= -1
}
// reverse direction of rotation to left of the mid-line
if (x < width / 2) {
dy *= -1
}
renderer.angle += (dx + dy) * TOUCH_SCALE_FACTOR
requestRender()
}
}
previousX = x
previousY = y
return true
}
More info of Touch Listener in this link Android Touch Listener

Modifier.pointerInput(Unit) {
detectTapGestures(
onPress = {
//start
val released = try {
tryAwaitRelease()
} catch (c: CancellationException) {
false
}
if (released) {
//ACTION_UP
} else {
//CANCELED
}
},
onTap = {
// onTap
},
onDoubleTap = {
//onDoubleTap
},
onLongPress = {
//onLongPress
}
)
}

How can I get onTouchEvent in Jetpack Compose?

In normal view, we can have onTouchEvent
override fun onTouchEvent(event: MotionEvent?): Boolean {
when (event?.action) {
MotionEvent.ACTION_DOWN -> {}
MotionEvent.ACTION_MOVE -> {}
MotionEvent.ACTION_UP -> {}
else -> return false
}
invalidate()
return true
}
In Jetpack Compose, I can only find we have the tapGestureFilter in the modifier, which only takes the action from the ACTION_UP only.
Modifier
.tapGestureFilter { Log.d("Track", "Tap ${it.x} | ${it.y}") }
.doubleTapGestureFilter { Log.d("Track", "DoubleTap ${it.x} | ${it.y}") }
Is there an equivalent onTouchEvent for Jetpack Compose?

We have a separate package for that, which is pretty useful.
There are two main extension functions that would be suitable for you:
pointerInput - docs
pointerInteropFilter - docs
If you want to handle and process the event I recommend using pointerInteropFilter which is the analogue of View.onTouchEvent. It's used along with modifier:
Column(modifier = Modifier.pointerInteropFilter {
when (it.action) {
MotionEvent.ACTION_DOWN -> {}
MotionEvent.ACTION_MOVE -> {}
MotionEvent.ACTION_UP -> {}
else -> false
}
true
})
That will be Compose adjusted code to your specified View.onTouchEvent sample.
P.S. Don't forget about #ExperimentalPointerInput annotation.

pointerInteropFilter is not described as preferred way to use if you are not using touch api with interoperation with existing View code.
A special PointerInputModifier that provides access to the underlying
MotionEvents originally dispatched to Compose. Prefer pointerInput and
use this only for interoperation with existing code that consumes
MotionEvents. While the main intent of this Modifier is to allow
arbitrary code to access the original MotionEvent dispatched to
Compose, for completeness, analogs are provided to allow arbitrary
code to interact with the system as if it were an Android View.
You can use pointerInput , awaitTouchDown for MotionEvent.ACTION_DOWN, and awaitPointerEvent for MotionEvent.ACTION_MOVE and MotionEvent.ACTION_UP
val pointerModifier = Modifier
.pointerInput(Unit) {
forEachGesture {
awaitPointerEventScope {
awaitFirstDown()
// ACTION_DOWN here
do {
//This PointerEvent contains details including
// event, id, position and more
val event: PointerEvent = awaitPointerEvent()
// ACTION_MOVE loop
// Consuming event prevents other gestures or scroll to intercept
event.changes.forEach { pointerInputChange: PointerInputChange ->
pointerInputChange.consumePositionChange()
}
} while (event.changes.any { it.pressed })
// ACTION_UP is here
}
}
}
Some key notes about gestures
pointerInput propagation is when you have more than one goes from
bottom one to top one.
If children and parent are listening for input changes it propagates
from inner children to outer then parent. Unlike touch events from
parent to children
If you don't consume events other events like scroll drag can
interfere or consume events, most of the events check if it's
consumed before propagating to them
detectDragGestures source code for instance
val down = awaitFirstDown(requireUnconsumed = false)
var drag: PointerInputChange?
var overSlop = Offset.Zero
do {
drag = awaitPointerSlopOrCancellation(
down.id,
down.type
) { change, over ->
change.consumePositionChange()
overSlop = over
}
} while (drag != null && !drag.positionChangeConsumed())
So when you need to prevent other events to intercept
call pointerInputChange.consumeDown() after awaitFirstDown, call
pointerInputChange.consumePositionChange() after awaitPointerEvent
and awaitFirstDown() has requireUnconsumed parameter which is
true by default. If you set it to false even if a pointerInput consumes
down before your gesture you still get it. This is also how events like drag use it to get first down no matter what.
Every available event you see detectDragGestures,
detectTapGestures even awaitFirstDown use awaitPointerEvent
for implementation, so using awaitFirstDown, awaitPointerEvent
and consuming changes you can configure your own gestures.
For instance, this is a function i customized from original detectTransformGestures only to be invoked with specific number of pointers down.
suspend fun PointerInputScope.detectMultiplePointerTransformGestures(
panZoomLock: Boolean = false,
numberOfPointersRequired: Int = 2,
onGesture: (centroid: Offset, pan: Offset, zoom: Float, rotation: Float) -> Unit,
) {
forEachGesture {
awaitPointerEventScope {
var rotation = 0f
var zoom = 1f
var pan = Offset.Zero
var pastTouchSlop = false
val touchSlop = viewConfiguration.touchSlop
var lockedToPanZoom = false
awaitFirstDown(requireUnconsumed = false)
do {
val event = awaitPointerEvent()
val downPointerCount = event.changes.size
// If any position change is consumed from another pointer or pointer
// count that is pressed is not equal to pointerCount cancel this gesture
val canceled = event.changes.any { it.positionChangeConsumed() } || (
downPointerCount != numberOfPointersRequired)
if (!canceled) {
val zoomChange = event.calculateZoom()
val rotationChange = event.calculateRotation()
val panChange = event.calculatePan()
if (!pastTouchSlop) {
zoom *= zoomChange
rotation += rotationChange
pan += panChange
val centroidSize = event.calculateCentroidSize(useCurrent = false)
val zoomMotion = abs(1 - zoom) * centroidSize
val rotationMotion =
abs(rotation * PI.toFloat() * centroidSize / 180f)
val panMotion = pan.getDistance()
if (zoomMotion > touchSlop ||
rotationMotion > touchSlop ||
panMotion > touchSlop
) {
pastTouchSlop = true
lockedToPanZoom = panZoomLock && rotationMotion < touchSlop
}
}
if (pastTouchSlop) {
val centroid = event.calculateCentroid(useCurrent = false)
val effectiveRotation = if (lockedToPanZoom) 0f else rotationChange
if (effectiveRotation != 0f ||
zoomChange != 1f ||
panChange != Offset.Zero
) {
onGesture(centroid, panChange, zoomChange, effectiveRotation)
}
event.changes.forEach {
if (it.positionChanged()) {
it.consumeAllChanges()
}
}
}
}
} while (!canceled && event.changes.any { it.pressed })
}
}
}
Edit
As of 1.2.0-beta01, partial consumes like
PointerInputChange.consemePositionChange(),
PointerInputChange.consumeDownChange(), and one for consuming all changes PointerInputChange.consumeAllChanges() are deprecated
PointerInputChange.consume()
is the only one to be used preventing other gestures/event.
Also i have a tutorial here that covers gestures in detail

Maybe a bit late, but since compose is constantly updating, this is how I do it as of today:
Modifier
.pointerInput(Unit) {
detectTapGestures {...}
}
.pointerInput(Unit) {
detectDragGestures { change, dragAmount -> ...}
})
We also have detectHorizontalDragGestures and detectVerticalDragGestures among others to help us.
ps: 1.0.0-beta03

After did some research, looks like can use dragGestureFilter, mixed with tapGestureFilter
Modifier
.dragGestureFilter(object: DragObserver {
override fun onDrag(dragDistance: Offset): Offset {
Log.d("Track", "onActionMove ${dragDistance.x} | ${dragDistance.y}")
return super.onDrag(dragDistance)
}
override fun onStart(downPosition: Offset) {
Log.d("Track", "onActionDown ${downPosition.x} | ${downPosition.y}")
super.onStart(downPosition)
}
override fun onStop(velocity: Offset) {
Log.d("Track", "onStop ${velocity.x} | ${velocity.y}")
super.onStop(velocity)
}
}, { true })
.tapGestureFilter {
Log.d("NGVL", "onActionUp ${it.x} | ${it.y}")
}
The reason still use tagGestureFilter, is because the onStop doesn't provide the position, but just the velocity, hence the tapGestureFilter does help provide the last position (if needed)

MPChart won't display hourly weather data

I'm creating a Weather app and i'm trying to display an MPChart which will use the hourly temps as Y points & the Hour (in "HH" format) as the XAxisLabels/ X points.
I'm creating the LineData for the chart in my Fragment's viewModel and expose it to the Fragment via liveData. I also create the XAxisLabels for the chart on my Fragment. The LineData seem to contain the right entries (viewModel: [Entry, x: 0.0 y: 39.07, Entry, x: 1.0 y: 38.68, Entry, x: 2.0 y: 38.16, Entry, x: 3.0 y: 37.29, Entry, x: 4.0 y: 36.2, Entry, x: 5.0 y: 35.22,........]), but the chart isn't even displaying any of the data. What am I missing here?
Here's what is rendered:
Here's the code:
ViewModel.kt
class WeatherViewModel(
private val forecastRepository: ForecastRepository,
private val weatherUnitConverter: WeatherUnitConverter,
context: Context
) : ViewModel() {
val hourlyWeatherEntries = forecastRepository.getHourlyWeather()
val hourlyChartData = hourlyWeatherEntries.switchMap { hourlyData ->
liveData {
Log.d("ViewModel hourly data", "$hourlyData")
val task = viewModelScope.async(Dispatchers.Default) {
createChartData(hourlyData)
}
emit(task.await())
}
}
/**
* Creates the line chart's data and returns them.
* #return The line chart's data (x,y) value pairs
*/
private fun createChartData(hourlyWeather: List<HourWeatherEntry>?): LineData {
if (hourlyWeather == null) return LineData()
Log.d("ViewModel class", hourlyWeather.toString())
val unitSystemValue = preferences.getString(UNIT_SYSTEM_KEY, "si")!!
val values = arrayListOf<Entry>()
for (i in hourlyWeather.indices) { // init data points
// format the temperature appropriately based on the unit system selected
val hourTempFormatted = when (unitSystemValue) {
UnitSystem.SI.name.toLowerCase(Locale.ROOT) -> hourlyWeather[i].temperature
UnitSystem.US.name.toLowerCase(Locale.ROOT) -> weatherUnitConverter.convertToFahrenheit(
hourlyWeather[i].temperature
)
else -> hourlyWeather[i].temperature
}
// Create the data point
values.add(
Entry(
i.toFloat(),
hourTempFormatted.toFloat(),
appContext.resources.getDrawable(
determineSummaryIcon(hourlyWeather[i].icon),
null
)
)
)
}
Log.d("MainFragment viewModel", "$values")
// create a data set and customize it
val lineDataSet = LineDataSet(values, "")
val color = appContext.resources.getColor(R.color.black, null)
val offset = MPPointF.getInstance()
offset.y = -35f
lineDataSet.apply {
valueFormatter = YValueFormatter()
setDrawValues(true)
fillDrawable = appContext.resources.getDrawable(R.drawable.gradient_night_chart, null)
setDrawFilled(true)
setDrawIcons(true)
setCircleColor(color)
mode = LineDataSet.Mode.HORIZONTAL_BEZIER
this.color = color // line color
iconsOffset = offset
lineWidth = 3f
valueTextSize = 9f
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
valueTypeface = appContext.resources.getFont(R.font.work_sans_medium)
}
}
// create a LineData object using our LineDataSet
val data = LineData(lineDataSet)
data.apply {
setValueTextColor(R.color.colorPrimary)
setValueTextSize(15f)
}
return data
}
}
MainFragment.kt
class MainFragment : Fragment() {
// Lazy inject the view model
private val viewModel: WeatherViewModel by viewModel()
private lateinit var hourlyChart: LineChart
override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
super.onViewCreated(view, savedInstanceState)
setUpChart()
lifecycleScope.launch {
// Shows a lottie animation while the data is being loaded
//scrollView.visibility = View.GONE
//lottieAnimView.visibility = View.VISIBLE
bindUIAsync().await()
// Stops the animation and reveals the layout with the data loaded
//scrollView.visibility = View.VISIBLE
//lottieAnimView.visibility = View.GONE
}
}
#SuppressLint("SimpleDateFormat")
private fun bindUIAsync() = lifecycleScope.async(Dispatchers.Main) {
// fetch hourly chart line data
val hourlyChartData = viewModel.hourlyChartData
hourlyChartData.observe(viewLifecycleOwner, Observer { lineData ->
if(lineData == null) {
Log.d("HOURLY CHART DATA", "Line data = null")
return#Observer
}
hourlyChart.data = lineData
hourlyChart.invalidate()
})
// fetch hourly weather
val hourlyWeather = viewModel.hourlyWeatherEntries//viewModel.hourlyWeatherEntries
// Observe the hourly weather live data
hourlyWeather.observe(viewLifecycleOwner, Observer { hourly ->
if (hourly == null) return#Observer
val xAxisLabels = arrayListOf<String>()
val sdf = SimpleDateFormat("HH")
for (i in hourly.indices) {
val formattedLabel = sdf.format(Date(hourly[i].time * 1000))
xAxisLabels.add(formattedLabel)
}
setChartAxisLabels(xAxisLabels)
Log.d(TAG, "label count = ${hourlyChart.xAxis.labelCount.toString()}")
})
return#async true
}
private fun setChartAxisLabels(labels: ArrayList<String>) {
// Populate the X axis with the hour labels
hourlyChart.xAxis.valueFormatter = IndexAxisValueFormatter(labels)
}
/**
* Sets up the chart with the appropriate
* customizations.
*/
private fun setUpChart() {
hourlyChart.apply {
description.isEnabled = false
setNoDataText("Data is loading...")
// enable touch gestures
setTouchEnabled(true)
dragDecelerationFrictionCoef = 0.9f
// enable dragging
isDragEnabled = true
isHighlightPerDragEnabled = true
setDrawGridBackground(false)
axisRight.setDrawLabels(false)
axisLeft.setDrawLabels(false)
axisLeft.setDrawGridLines(false)
// disable zoom functionality
setScaleEnabled(false)
setPinchZoom(false)
isDoubleTapToZoomEnabled = false
// disable the chart's legend
legend.isEnabled = false
// append extra offsets to the chart's auto-calculated ones
setExtraOffsets(0f, 0f, 0f, 10f)
data = LineData()
data.isHighlightEnabled = false
setVisibleXRangeMaximum(6f)
setBackgroundColor(resources.getColor(R.color.bright_White, null))
}
// X Axis setup
hourlyChart.xAxis.apply {
position = XAxis.XAxisPosition.BOTTOM
textSize = 14f
setDrawLabels(true)
setDrawAxisLine(false)
setDrawGridLines(false)
isEnabled = true
granularity = 1f // one hour
spaceMax = 0.2f // add padding start
spaceMin = 0.2f // add padding end
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
typeface = resources.getFont(R.font.work_sans)
}
textColor = resources.getColor(R.color.black, null)
}
// Left Y axis setup
hourlyChart.axisLeft.apply {
setDrawLabels(true)
setDrawGridLines(false)
setPosition(YAxis.YAxisLabelPosition.OUTSIDE_CHART)
// temperature values range (higher than probable temps in order to scale down the chart)
axisMinimum = 0f
axisMaximum = when (getUnitSystemValue()) {
UnitSystem.SI.name.toLowerCase(Locale.ROOT) -> 50f
UnitSystem.US.name.toLowerCase(Locale.ROOT) -> 150f
else -> 50f
}
}
// Right Y axis setup
hourlyChart.axisRight.apply {
setDrawGridLines(false)
isEnabled = false
}
}
}

CameraX: How to add pinch to zoom AND tap to focus? onClickListener and onTouchListener

I merged some CameraX tutorials so that it has pinch to zoom and tap to focus.
By themselves, they work well, but together, the OnClickListener and OnTouchListeners are interfering with each other.
I thought merging them under a single OnClickListener method where the pinch to zoom is executed on the ACTION_DOWN, and the tap to focus on the ACTION_UP, but only the tap to focus is running. Even if it did work, this feels a bit clunky and I'd appreciate some more advanced guidance.
zoomAndFocus is triggered by: "viewFinder.setOnClickListener{zoomAndFocus()}" in onCreate.
private fun zoomAndFocus(){
Log.d("onclick","detecting clck?")
viewFinder.setOnTouchListener { _, event ->
return#setOnTouchListener when (event.action) {
MotionEvent.ACTION_DOWN -> {
val listener = object : ScaleGestureDetector.SimpleOnScaleGestureListener() {
override fun onScale(detector: ScaleGestureDetector): Boolean {
val zoomRatio = camera?.cameraInfo?.zoomState?.value?.zoomRatio ?: 0f
val scale = zoomRatio * detector.scaleFactor
camera!!.cameraControl.setZoomRatio(scale)
return true
}
}
val scaleGestureDetector = ScaleGestureDetector(this, listener)
scaleGestureDetector.onTouchEvent(event)
true
}
MotionEvent.ACTION_UP -> {
val factory: MeteringPointFactory = SurfaceOrientedMeteringPointFactory(
viewFinder.width.toFloat(), viewFinder.height.toFloat()
)
val autoFocusPoint = factory.createPoint(event.x, event.y)
try {
camera?.cameraControl?.startFocusAndMetering(
FocusMeteringAction.Builder(
autoFocusPoint,
FocusMeteringAction.FLAG_AF
).apply {
//focus only when the user tap the preview
disableAutoCancel()
}.build()
)
} catch (e: CameraInfoUnavailableException) {
Log.d("ERROR", "cannot access camera", e)
}
viewFinder.performClick()
true
}
else -> false // Unhandled event.
}
}

Use this to add pinch to zoom and tap to focus together:
private fun setupZoomAndTapToFocus() {
val listener = object : ScaleGestureDetector.SimpleOnScaleGestureListener() {
override fun onScale(detector: ScaleGestureDetector): Boolean {
val currentZoomRatio: Float = cameraInfo.zoomState.value?.zoomRatio ?: 1F
val delta = detector.scaleFactor
cameraControl.setZoomRatio(currentZoomRatio * delta)
return true
}
}
val scaleGestureDetector = ScaleGestureDetector(viewFinder.context, listener)
viewFinder.setOnTouchListener { _, event ->
scaleGestureDetector.onTouchEvent(event)
if (event.action == MotionEvent.ACTION_DOWN) {
val factory = viewFinder.createMeteringPointFactory(cameraSelector)
val point = factory.createPoint(event.x, event.y)
val action = FocusMeteringAction.Builder(point, FocusMeteringAction.FLAG_AF)
.setAutoCancelDuration(5, TimeUnit.SECONDS)
.build()
cameraControl.startFocusAndMetering(action)
}
true
}
}

Camera X captures images in different rotation states

Okay, I went through different posts and find out that depending on mobile manufacturers there can be a complications such as capture images get rotated, so you have to be aware of that. What I did was:
fun rotateBitmap(bitmap: Bitmap): Bitmap? {
val matrix = Matrix()
when (getImageOrientation(bitmap)) {
ExifInterface.ORIENTATION_NORMAL -> return bitmap
ExifInterface.ORIENTATION_FLIP_HORIZONTAL -> matrix.setScale(-1f, 1f)
ExifInterface.ORIENTATION_ROTATE_270 -> matrix.setRotate(-90f)
ExifInterface.ORIENTATION_ROTATE_180 -> matrix.setRotate(180f)
ExifInterface.ORIENTATION_ROTATE_90 -> matrix.setRotate(90f)
ExifInterface.ORIENTATION_FLIP_VERTICAL -> {
matrix.setRotate(180f)
matrix.postScale(-1f, 1f)
}
ExifInterface.ORIENTATION_TRANSPOSE -> {
matrix.setRotate(90f)
matrix.postScale(-1f, 1f)
}
ExifInterface.ORIENTATION_TRANSVERSE -> {
matrix.setRotate(-90f)
matrix.postScale(-1f, 1f)
}
else -> return bitmap
}
This worked. But then I noticed something really weird and that might be related with how I configured Camera X configuration.
With the same device I get differently rotated Bitmaps (well, this should not happen. If devices rotates image weirdly, it should rotate images in both modes - in ImageAnalysesUseCase and ImageCaptureUseCase).
So, why is this happening and how can I fix it?
Code implementation:
Binding camera X to life-cycle:
CameraX.bindToLifecycle(
this,
buildPreviewUseCase(),
buildImageAnalysisUseCase(),
buildImageCaptureUseCase()
)
Preview use case:
private fun buildPreviewUseCase(): Preview {
val previewConfig = PreviewConfig.Builder()
.setTargetAspectRatio(config.aspectRatio)
.setTargetResolution(config.resolution)
.setTargetRotation(Surface.ROTATION_0)
.setLensFacing(config.lensFacing)
.build()
return AutoFitPreviewBuilder.build(previewConfig, cameraTextureView)
}
Capture use case:
private fun buildImageCaptureUseCase(): ImageCapture {
val captureConfig = ImageCaptureConfig.Builder()
.setTargetAspectRatio(config.aspectRatio)
.setTargetRotation(Surface.ROTATION_0)
.setTargetResolution(config.resolution)
.setCaptureMode(config.captureMode)
.build()
val capture = ImageCapture(captureConfig)
manualModeTakePhotoButton.setOnClickListener {
capture.takePicture(object : ImageCapture.OnImageCapturedListener() {
override fun onCaptureSuccess(imageProxy: ImageProxy, rotationDegrees: Int) {
viewModel.onManualCameraModeAnalysis(imageProxy, rotationDegrees)
}
override fun onError(useCaseError: ImageCapture.UseCaseError?, message: String?, cause: Throwable?) {
//
}
})
}
return capture
}
Analysis use case:
private fun buildImageAnalysisUseCase(): ImageAnalysis {
val analysisConfig = ImageAnalysisConfig.Builder().apply {
val analyzerThread = HandlerThread("xAnalyzer").apply { start() }
analyzerHandler = Handler(analyzerThread.looper)
setCallbackHandler(analyzerHandler!!)
setTargetAspectRatio(config.aspectRatio)
setTargetRotation(Surface.ROTATION_0)
setTargetResolution(config.resolution)
setImageReaderMode(config.readerMode)
setImageQueueDepth(config.queueDepth)
}.build()
val analysis = ImageAnalysis(analysisConfig)
analysis.analyzer = ImageRecognitionAnalyzer(viewModel)
return analysis
}
AutoFitPreviewBuilder:
class AutoFitPreviewBuilder private constructor(config: PreviewConfig,
viewFinderRef: WeakReference<TextureView>) {
/** Public instance of preview use-case which can be used by consumers of this adapter */
val useCase: Preview
/** Internal variable used to keep track of the use-case's output rotation */
private var bufferRotation: Int = 0
/** Internal variable used to keep track of the view's rotation */
private var viewFinderRotation: Int? = null
/** Internal variable used to keep track of the use-case's output dimension */
private var bufferDimens: Size = Size(0, 0)
/** Internal variable used to keep track of the view's dimension */
private var viewFinderDimens: Size = Size(0, 0)
/** Internal variable used to keep track of the view's display */
private var viewFinderDisplay: Int = -1
/** Internal reference of the [DisplayManager] */
private lateinit var displayManager: DisplayManager
/**
* We need a display listener for orientation changes that do not trigger a configuration
* change, for example if we choose to override config change in manifest or for 180-degree
* orientation changes.
*/
private val displayListener = object : DisplayManager.DisplayListener {
override fun onDisplayAdded(displayId: Int) = Unit
override fun onDisplayRemoved(displayId: Int) = Unit
override fun onDisplayChanged(displayId: Int) {
val viewFinder = viewFinderRef.get() ?: return
if (displayId == viewFinderDisplay) {
val display = displayManager.getDisplay(displayId)
val rotation = getDisplaySurfaceRotation(display)
updateTransform(viewFinder, rotation, bufferDimens, viewFinderDimens)
}
}
}
init {
// Make sure that the view finder reference is valid
val viewFinder = viewFinderRef.get() ?:
throw IllegalArgumentException("Invalid reference to view finder used")
// Initialize the display and rotation from texture view information
viewFinderDisplay = viewFinder.display.displayId
viewFinderRotation = getDisplaySurfaceRotation(viewFinder.display) ?: 0
// Initialize public use-case with the given config
useCase = Preview(config)
// Every time the view finder is updated, recompute layout
useCase.onPreviewOutputUpdateListener = Preview.OnPreviewOutputUpdateListener {
val viewFinder =
viewFinderRef.get() ?: return#OnPreviewOutputUpdateListener
// To update the SurfaceTexture, we have to remove it and re-add it
val parent = viewFinder.parent as ViewGroup
parent.removeView(viewFinder)
parent.addView(viewFinder, 0)
viewFinder.surfaceTexture = it.surfaceTexture
bufferRotation = it.rotationDegrees
val rotation = getDisplaySurfaceRotation(viewFinder.display)
updateTransform(viewFinder, rotation, it.textureSize, viewFinderDimens)
}
// Every time the provided texture view changes, recompute layout
viewFinder.addOnLayoutChangeListener { view, left, top, right, bottom, _, _, _, _ ->
val viewFinder = view as TextureView
val newViewFinderDimens = Size(right - left, bottom - top)
val rotation = getDisplaySurfaceRotation(viewFinder.display)
updateTransform(viewFinder, rotation, bufferDimens, newViewFinderDimens)
}
// Every time the orientation of device changes, recompute layout
displayManager = viewFinder.context
.getSystemService(Context.DISPLAY_SERVICE) as DisplayManager
displayManager.registerDisplayListener(displayListener, null)
// Remove the display listeners when the view is detached to avoid
// holding a reference to the View outside of a Fragment.
// NOTE: Even though using a weak reference should take care of this,
// we still try to avoid unnecessary calls to the listener this way.
viewFinder.addOnAttachStateChangeListener(object : View.OnAttachStateChangeListener {
override fun onViewAttachedToWindow(view: View?) {
displayManager.registerDisplayListener(displayListener, null)
}
override fun onViewDetachedFromWindow(view: View?) {
displayManager.unregisterDisplayListener(displayListener)
}
})
}
/** Helper function that fits a camera preview into the given [TextureView] */
private fun updateTransform(textureView: TextureView?, rotation: Int?, newBufferDimens: Size,
newViewFinderDimens: Size) {
// This should not happen anyway, but now the linter knows
val textureView = textureView ?: return
if (rotation == viewFinderRotation &&
Objects.equals(newBufferDimens, bufferDimens) &&
Objects.equals(newViewFinderDimens, viewFinderDimens)) {
// Nothing has changed, no need to transform output again
return
}
if (rotation == null) {
// Invalid rotation - wait for valid inputs before setting matrix
return
} else {
// Update internal field with new inputs
viewFinderRotation = rotation
}
if (newBufferDimens.width == 0 || newBufferDimens.height == 0) {
// Invalid buffer dimens - wait for valid inputs before setting matrix
return
} else {
// Update internal field with new inputs
bufferDimens = newBufferDimens
}
if (newViewFinderDimens.width == 0 || newViewFinderDimens.height == 0) {
// Invalid view finder dimens - wait for valid inputs before setting matrix
return
} else {
// Update internal field with new inputs
viewFinderDimens = newViewFinderDimens
}
val matrix = Matrix()
// Compute the center of the view finder
val centerX = viewFinderDimens.width / 2f
val centerY = viewFinderDimens.height / 2f
// Correct preview output to account for display rotation
matrix.postRotate(-viewFinderRotation!!.toFloat(), centerX, centerY)
// Buffers are rotated relative to the device's 'natural' orientation: swap width and height
val bufferRatio = bufferDimens.height / bufferDimens.width.toFloat()
val scaledWidth: Int
val scaledHeight: Int
// Match longest sides together -- i.e. apply center-crop transformation
if (viewFinderDimens.width > viewFinderDimens.height) {
scaledHeight = viewFinderDimens.width
scaledWidth = Math.round(viewFinderDimens.width * bufferRatio)
} else {
scaledHeight = viewFinderDimens.height
scaledWidth = Math.round(viewFinderDimens.height * bufferRatio)
}
// Compute the relative scale value
val xScale = scaledWidth / viewFinderDimens.width.toFloat()
val yScale = scaledHeight / viewFinderDimens.height.toFloat()
// Scale input buffers to fill the view finder
matrix.preScale(xScale, yScale, centerX, centerY)
// Finally, apply transformations to our TextureView
textureView.setTransform(matrix)
}
companion object {
/** Helper function that gets the rotation of a [Display] in degrees */
fun getDisplaySurfaceRotation(display: Display?) = when(display?.rotation) {
Surface.ROTATION_0 -> 0
Surface.ROTATION_90 -> 90
Surface.ROTATION_180 -> 180
Surface.ROTATION_270 -> 270
else -> null
}
/**
* Main entrypoint for users of this class: instantiates the adapter and returns an instance
* of [Preview] which automatically adjusts in size and rotation to compensate for
* config changes.
*/
fun build(config: PreviewConfig, viewFinder: TextureView) =
AutoFitPreviewBuilder(config, WeakReference(viewFinder)).useCase
}
}
If configuration is correct (it looks okay to me), then next idea was that maybe converting captured images objects to bitmap might be faulty. Below you can see implementation.
Capture mode uses this function:
fun imageProxyToBitmap(image: ImageProxy): Bitmap {
val buffer: ByteBuffer = image.planes[0].buffer
val bytes = ByteArray(buffer.remaining())
buffer.get(bytes)
return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}
Analysis mode uses this function:
fun toBitmapFromImage(image: Image?): Bitmap? {
try {
if (image == null || image.planes[0] == null || image.planes[1] == null || image.planes[2] == null) {
return null
}
val yBuffer = image.planes[0].buffer
val uBuffer = image.planes[1].buffer
val vBuffer = image.planes[2].buffer
val ySize = yBuffer.remaining()
val uSize = uBuffer.remaining()
val vSize = vBuffer.remaining()
val nv21 = ByteArray(ySize + uSize + vSize)
/* U and V are swapped */
yBuffer.get(nv21, 0, ySize)
vBuffer.get(nv21, ySize, vSize)
uBuffer.get(nv21, ySize + vSize, uSize)
val yuvImage = YuvImage(nv21, ImageFormat.NV21, image.width, image.height, null)
val out = ByteArrayOutputStream()
yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 50, out)
val imageBytes = out.toByteArray()
return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
} catch (e: IllegalStateException) {
Log.e("IllegalStateException", "#ImageUtils.toBitmapFromImage(): Can't read the image file.")
return null
}
}
So, weirdly, on few devices toBitmapFromImage() sometimes comes up upwards, but at the same time (same device) imageProxyToBitmap() returns image in correct rotation - it has to be the image to bitmap functions fault, right?Why is this happening (because capture mode returns image normally) and how to fix this?

Inside onImageCaptureSuccess, get the rotationDegrees and rotate your bitmap by that degree to get the correct orientation.
override fun onImageCaptureSuccess(image: ImageProxy) {
val capturedImageBitmap = image.image?.toBitmap()?.rotate(image.imageInfo.rotationDegrees.toFloat())
mBinding.previewImage.setImageBitmap(capturedImageBitmap)
showPostClickViews()
mCurrentFlow = FLOW_CAMERA
}
toBitmap() and rotate() are extension functions.
fun Image.toBitmap(): Bitmap {
val buffer = planes[0].buffer
buffer.rewind()
val bytes = ByteArray(buffer.capacity())
buffer.get(bytes)
return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}
fun Bitmap.rotate(degrees: Float): Bitmap =
Bitmap.createBitmap(this, 0, 0, width, height, Matrix().apply { postRotate(degrees) }, true)

CameraX returns the captured image with a rotation value in the callback, which can be used to rotate the image.
https://developer.android.com/reference/androidx/camera/core/ImageCapture.OnImageCapturedListener.html#onCaptureSuccess(androidx.camera.core.ImageProxy,%20int)

For Analyzer UseCases, you have to get rotationDegree coming through analyze method of ImageAnalysis.Analyzer and work accordingly.
Hope it helps!