I'm using CameraX's Analyzer use case with the MLKit's BarcodeScanner. I would like to crop portion of the image received from the camera, before passing it to the scanner.
What I'm doing right now is I convert ImageProxy (that I recieve in the Analyzer) to a Bitmap, crop it and then pass it to the BarcodeScanner. The downside is that it's not a very fast and efficient process.
I've also noticed the warning I get in the Logcat when running this code:
ML Kit has detected that you seem to pass camera frames to the
detector as a Bitmap object. This is inefficient. Please use
YUV_420_888 format for camera2 API or NV21 format for (legacy) camera
API and directly pass down the byte array to ML Kit.
It would be nice to not to do ImageProxy conversion, but how do I crop the rectangle I want to analyze?
What I've already tried is to set a cropRect field of the Image (imageProxy.image.cropRect) class, but it doesn't seem to affect the end result.
Yes, it's true that if you use ViewPort and set viewport to yours UseCases(imageCapture or imageAnalysis as here https://developer.android.com/training/camerax/configuration) you can get only information about crop rectangle especially if you use ImageAnalysis(because if you use imageCapture, for on-disk the image is cropped before saving and it doesn't work for ImageAnalysis and if you use imageCapture without saving on disk) and here solution how I solved this problem:
First of all set view port for use cases as here: https://developer.android.com/training/camerax/configuration
Get cropped bitmap to analyze
override fun analyze(imageProxy: ImageProxy) {
val mediaImage = imageProxy.image
if (mediaImage != null && mediaImage.format == ImageFormat.YUV_420_888) {
croppedBitmap(mediaImage, imageProxy.cropRect).let { bitmap ->
requestDetectInImage(InputImage.fromBitmap(bitmap, rotation))
.addOnCompleteListener { imageProxy.close() }
}
} else {
imageProxy.close()
}
}
private fun croppedBitmap(mediaImage: Image, cropRect: Rect): Bitmap {
val yBuffer = mediaImage.planes[0].buffer // Y
val vuBuffer = mediaImage.planes[2].buffer // VU
val ySize = yBuffer.remaining()
val vuSize = vuBuffer.remaining()
val nv21 = ByteArray(ySize + vuSize)
yBuffer.get(nv21, 0, ySize)
vuBuffer.get(nv21, ySize, vuSize)
val yuvImage = YuvImage(nv21, ImageFormat.NV21, mediaImage.width, mediaImage.height, null)
val outputStream = ByteArrayOutputStream()
yuvImage.compressToJpeg(cropRect, 100, outputStream)
val imageBytes = outputStream.toByteArray()
return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
}
Possibly there is a loss in conversion speed, but on my devices I did not notice the difference. I set 100 quality in method compressToJpeg, but mb if set less quality it can improve speed, it need test.
upd: May 02 '21 :
I found another way without convert to jpeg and then to bitmap. This should be a faster way.
Set viewport as previous.
Convert YUV_420_888 to NV21, then crop and analyze.
override fun analyze(imageProxy: ImageProxy) {
val mediaImage = imageProxy.image
if (mediaImage != null && mediaImage.format == ImageFormat.YUV_420_888) {
croppedNV21(mediaImage, imageProxy.cropRect).let { byteArray ->
requestDetectInImage(
InputImage.fromByteArray(
byteArray,
imageProxy.cropRect.width(),
imageProxy.cropRect.height(),
rotation,
IMAGE_FORMAT_NV21,
)
)
.addOnCompleteListener { imageProxy.close() }
}
} else {
imageProxy.close()
}
}
private fun croppedNV21(mediaImage: Image, cropRect: Rect): ByteArray {
val yBuffer = mediaImage.planes[0].buffer // Y
val vuBuffer = mediaImage.planes[2].buffer // VU
val ySize = yBuffer.remaining()
val vuSize = vuBuffer.remaining()
val nv21 = ByteArray(ySize + vuSize)
yBuffer.get(nv21, 0, ySize)
vuBuffer.get(nv21, ySize, vuSize)
return cropByteArray(nv21, mediaImage.width, cropRect)
}
private fun cropByteArray(array: ByteArray, imageWidth: Int, cropRect: Rect): ByteArray {
val croppedArray = ByteArray(cropRect.width() * cropRect.height())
var i = 0
array.forEachIndexed { index, byte ->
val x = index % imageWidth
val y = index / imageWidth
if (cropRect.left <= x && x < cropRect.right && cropRect.top <= y && y < cropRect.bottom) {
croppedArray[i] = byte
i++
}
}
return croppedArray
}
First crop fun I took from here: Android: How to crop images using CameraX?
And I found also another crop fun, it seems that it is more complicated:
private fun cropByteArray(src: ByteArray, width: Int, height: Int, cropRect: Rect, ): ByteArray {
val x = cropRect.left * 2 / 2
val y = cropRect.top * 2 / 2
val w = cropRect.width() * 2 / 2
val h = cropRect.height() * 2 / 2
val yUnit = w * h
val uv = yUnit / 2
val nData = ByteArray(yUnit + uv)
val uvIndexDst = w * h - y / 2 * w
val uvIndexSrc = width * height + x
var srcPos0 = y * width
var destPos0 = 0
var uvSrcPos0 = uvIndexSrc
var uvDestPos0 = uvIndexDst
for (i in y until y + h) {
System.arraycopy(src, srcPos0 + x, nData, destPos0, w) //y memory block copy
srcPos0 += width
destPos0 += w
if (i and 1 == 0) {
System.arraycopy(src, uvSrcPos0, nData, uvDestPos0, w) //uv memory block copy
uvSrcPos0 += width
uvDestPos0 += w
}
}
return nData
}
Second crop fun I took from here:
https://www.programmersought.com/article/75461140907/
I would be glad if someone can help improve the code.
I'm still improving the way to do it. But this will work for me now
CameraX crop image before sending to analyze
<androidx.constraintlayout.widget.ConstraintLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:paddingBottom="#dimen/_40sdp">
<androidx.camera.view.PreviewView
android:id="#+id/previewView"
android:layout_width="match_parent"
android:layout_height="0dp"
app:layout_constraintDimensionRatio="1:1"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" /></androidx.constraintlayout.widget.ConstraintLayout>
Cropping an image into 1:1 before passing it to analyze
override fun onCaptureSuccess(image: ImageProxy) {
super.onCaptureSuccess(image)
var bitmap: Bitmap = imageProxyToBitmap(image)
val dimension: Int = min(bitmap.width, bitmap.height)
bitmap = ThumbnailUtils.extractThumbnail(bitmap, dimension, dimension)
imageView.setImageBitmap(bitmap) //Here you can pass the crop[from the center] image to analyze
image.close()
}
**Function for converting into bitmap **
private fun imageProxyToBitmap(image: ImageProxy): Bitmap {
val buffer: ByteBuffer = image.planes[0].buffer
val bytes = ByteArray(buffer.remaining())
buffer.get(bytes)
return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}
You would use ImageProxy.SetCroprect to get the rect and then use CropRect to set it.
For example if you had imageProxy, you would do : ImageProxy.setCropRect(Rect) and then you would do ImageProxy.CropRect.
I found that solution (https://itnext.io/converting-pytorch-float-tensor-to-android-rgba-bitmap-with-kotlin-ffd4602a16b6) but when I tried to convert that way I found that the size of inputTensor.dataAsFloatArray is more than bitmap.width*bitmap.height. How works converting tensor to float array or is there any other possible method to convert pytorch tensor to bitmap?
val inputTensor = TensorImageUtils.bitmapToFloat32Tensor(
bitmap,
TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, TensorImageUtils.TORCHVISION_NORM_STD_RGB
)
// Float array size is 196608 when width and height are 256x256 = 65536
val res = floatArrayToGrayscaleBitmap(inputTensor.dataAsFloatArray, bitmap.width, bitmap.height)
fun floatArrayToGrayscaleBitmap (
floatArray: FloatArray,
width: Int,
height: Int,
alpha :Byte = (255).toByte(),
reverseScale :Boolean = false
) : Bitmap {
// Create empty bitmap in RGBA format (even though it says ARGB but channels are RGBA)
val bmp = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
val byteBuffer = ByteBuffer.allocate(width*height*4)
Log.d("App", floatArray.size.toString() + " " + (width * height * 4).toString())
// mapping smallest value to 0 and largest value to 255
val maxValue = floatArray.max() ?: 1.0f
val minValue = floatArray.min() ?: 0.0f
val delta = maxValue-minValue
var tempValue :Byte
// Define if float min..max will be mapped to 0..255 or 255..0
val conversion = when(reverseScale) {
false -> { v: Float -> ((v-minValue)/delta*255).toByte() }
true -> { v: Float -> (255-(v-minValue)/delta*255).toByte() }
}
// copy each value from float array to RGB channels and set alpha channel
floatArray.forEachIndexed { i, value ->
tempValue = conversion(value)
byteBuffer.put(4*i, tempValue)
byteBuffer.put(4*i+1, tempValue)
byteBuffer.put(4*i+2, tempValue)
byteBuffer.put(4*i+3, alpha)
}
bmp.copyPixelsFromBuffer(byteBuffer)
return bmp
}
None of the answers were able to produce the output I wanted, so this is what I came up with - it is basically only reverse engineered version of what happenes in TensorImageUtils.bitmapToFloat32Tensor().
Please note that this function only works if you are using MemoryFormat.CONTIGUOUS (which is default) in TensorImageUtils.bitmapToFloat32Tensor().
fun tensor2Bitmap(input: FloatArray, width: Int, height: Int, normMeanRGB: FloatArray, normStdRGB: FloatArray): Bitmap? {
val pixelsCount = height * width
val pixels = IntArray(pixelsCount)
val output = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
val conversion = { v: Float -> ((v.coerceIn(0.0f, 1.0f))*255.0f).roundToInt()}
val offset_g = pixelsCount
val offset_b = 2 * pixelsCount
for (i in 0 until pixelsCount) {
val r = conversion(input[i] * normStdRGB[0] + normMeanRGB[0])
val g = conversion(input[i + offset_g] * normStdRGB[1] + normMeanRGB[1])
val b = conversion(input[i + offset_b] * normStdRGB[2] + normMeanRGB[2])
pixels[i] = 255 shl 24 or (r.toInt() and 0xff shl 16) or (g.toInt() and 0xff shl 8) or (b.toInt() and 0xff)
}
output.setPixels(pixels, 0, width, 0, 0, width, height)
return output
}
Example usage then could be as follows:
tensor2Bitmap(outputTensor.dataAsFloatArray, bitmap.width, bitmap.height, TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, TensorImageUtils.TORCHVISION_NORM_STD_RGB)
// I faced the same problem, and I found the function itself
TensorImageUtils.bitmapToFloat32Tensor()
tortures the RGB colorspace. You should try to convert yuv to a bitmap and use
TensorImageUtils.bitmapToFloat32Tensor
instead for NOW.
// I modified the code from phillies (up) to get the coloful bitmap. Note that the format of an output tensor is typically NCHW.
// Here's my function in Kotlin. Hopefully it works in your case:
private fun floatArrayToBitmap(floatArray: FloatArray, width: Int, height: Int) : Bitmap {
// Create empty bitmap in ARGB format
val bmp: Bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
val pixels = IntArray(width * height * 4)
// mapping smallest value to 0 and largest value to 255
val maxValue = floatArray.max() ?: 1.0f
val minValue = floatArray.min() ?: -1.0f
val delta = maxValue-minValue
// Define if float min..max will be mapped to 0..255 or 255..0
val conversion = { v: Float -> ((v-minValue)/delta*255.0f).roundToInt()}
// copy each value from float array to RGB channels
for (i in 0 until width * height) {
val r = conversion(floatArray[i])
val g = conversion(floatArray[i+width*height])
val b = conversion(floatArray[i+2*width*height])
pixels[i] = rgb(r, g, b) // you might need to import for rgb()
}
bmp.setPixels(pixels, 0, width, 0, 0, width, height)
return bmp
}
Hopefully future releases of PyTorch Mobile will fix this bug.
I'm using CameraX and then FirebaseVision to read some text from the image. when I'm analyzing the Image I want to select a portion of the image, not the entire Image, something like when you use a barcode scanner.
class Analyzer : ImageAnalysis.Analyzer {
override fun analyze(imageProxy: ImageProxy?, rotationDegrees: Int) {
// how to crop the image in here?
val image = imageProxy.image
val imageRotation = degreesToFirebaseRotation(degrees)
if (image != null) {
val visionImage = FirebaseVisionImage.fromMediaImage(image, imageRotation)
val textRecognizer = FirebaseVision.getInstance().onDeviceTextRecognizer
textRecognizer.processImage(visionImage)
}
}
I want to know, is there any way to crop the image?
Your problem is exactly what I have tackled 2 months ago...
object YuvNV21Util {
fun yuv420toNV21(image: Image): ByteArray {
val crop = image.cropRect
val format = image.format
val width = crop.width()
val height = crop.height()
val planes = image.planes
val data =
ByteArray(width * height * ImageFormat.getBitsPerPixel(format) / 8)
val rowData = ByteArray(planes[0].rowStride)
var channelOffset = 0
var outputStride = 1
for (i in planes.indices) {
when (i) {
0 -> {
channelOffset = 0
outputStride = 1
}
1 -> {
channelOffset = width * height + 1
outputStride = 2
}
2 -> {
channelOffset = width * height
outputStride = 2
}
}
val buffer = planes[i].buffer
val rowStride = planes[i].rowStride
val pixelStride = planes[i].pixelStride
val shift = if (i == 0) 0 else 1
val w = width shr shift
val h = height shr shift
buffer.position(rowStride * (crop.top shr shift) + pixelStride * (crop.left shr shift))
for (row in 0 until h) {
var length: Int
if (pixelStride == 1 && outputStride == 1) {
length = w
buffer[data, channelOffset, length]
channelOffset += length
} else {
length = (w - 1) * pixelStride + 1
buffer[rowData, 0, length]
for (col in 0 until w) {
data[channelOffset] = rowData[col * pixelStride]
channelOffset += outputStride
}
}
if (row < h - 1) {
buffer.position(buffer.position() + rowStride - length)
}
}
}
return data
}
}
then convert bytearray into bitmap
object BitmapUtil {
fun getBitmap(data: ByteArray, metadata: FrameMetadata): Bitmap {
val image = YuvImage(
data, ImageFormat.NV21, metadata.width, metadata.height, null
)
val stream = ByteArrayOutputStream()
image.compressToJpeg(
Rect(0, 0, metadata.width, metadata.height),
80,
stream
)
val bmp = BitmapFactory.decodeByteArray(stream.toByteArray(), 0, stream.size())
stream.close()
return rotateBitmap(bmp, metadata.rotation, false, false)
}
private fun rotateBitmap(
bitmap: Bitmap, rotationDegrees: Int, flipX: Boolean, flipY: Boolean
): Bitmap {
val matrix = Matrix()
// Rotate the image back to straight.
matrix.postRotate(rotationDegrees.toFloat())
// Mirror the image along the X or Y axis.
matrix.postScale(if (flipX) -1.0f else 1.0f, if (flipY) -1.0f else 1.0f)
val rotatedBitmap =
Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true)
// Recycle the old bitmap if it has changed.
if (rotatedBitmap != bitmap) {
bitmap.recycle()
}
return rotatedBitmap
}
}
Please have a look at my open source project https://github.com/minkiapps/Firebase-ML-Kit-Scanner-Demo, I build a demo app where portion of the image proxy is cropped before it is processed by ml kit.
With Android CameraX Analyzer ImageProxy uses ImageReader under the hood with a default YUV_420_888 image format.
I'd like to convert it in OpenCV Mat in order to use OpenCV inside my analyzer:
override fun analyze(imageProxy: ImageProxy, rotationDegrees: Int) {
try {
imageProxy.image?.let {
// ImageProxy uses an ImageReader under the hood:
// https://developer.android.com/reference/androidx/camera/core/ImageProxy.html
// That has a default format of YUV_420_888 if not changed that's the default
// Android camera format.
// https://developer.android.com/reference/android/graphics/ImageFormat.html#YUV_420_888
// https://developer.android.com/reference/android/media/ImageReader.html
// Sanity check
if (it.format == ImageFormat.YUV_420_888
&& it.planes.size == 3
) {
// TODO - convert ImageProxy.image to Mat
} else {
// Manage other image formats
// TODO - https://developer.android.com/reference/android/media/Image.html
}
}
} catch (ise: IllegalStateException) {
ise.printStackTrace()
}
}
How can I do that?
Looking at OpenCV JavaCamera2Frame class in its GitHub repo you can write an Image extension function like that:
(ported to Kotlin)
// Ported from opencv private class JavaCamera2Frame
fun Image.yuvToRgba(): Mat {
val rgbaMat = Mat()
if (format == ImageFormat.YUV_420_888
&& planes.size == 3) {
val chromaPixelStride = planes[1].pixelStride
if (chromaPixelStride == 2) { // Chroma channels are interleaved
assert(planes[0].pixelStride == 1)
assert(planes[2].pixelStride == 2)
val yPlane = planes[0].buffer
val uvPlane1 = planes[1].buffer
val uvPlane2 = planes[2].buffer
val yMat = Mat(height, width, CvType.CV_8UC1, yPlane)
val uvMat1 = Mat(height / 2, width / 2, CvType.CV_8UC2, uvPlane1)
val uvMat2 = Mat(height / 2, width / 2, CvType.CV_8UC2, uvPlane2)
val addrDiff = uvMat2.dataAddr() - uvMat1.dataAddr()
if (addrDiff > 0) {
assert(addrDiff == 1L)
Imgproc.cvtColorTwoPlane(yMat, uvMat1, rgbaMat, Imgproc.COLOR_YUV2RGBA_NV12)
} else {
assert(addrDiff == -1L)
Imgproc.cvtColorTwoPlane(yMat, uvMat2, rgbaMat, Imgproc.COLOR_YUV2RGBA_NV21)
}
} else { // Chroma channels are not interleaved
val yuvBytes = ByteArray(width * (height + height / 2))
val yPlane = planes[0].buffer
val uPlane = planes[1].buffer
val vPlane = planes[2].buffer
yPlane.get(yuvBytes, 0, width * height)
val chromaRowStride = planes[1].rowStride
val chromaRowPadding = chromaRowStride - width / 2
var offset = width * height
if (chromaRowPadding == 0) {
// When the row stride of the chroma channels equals their width, we can copy
// the entire channels in one go
uPlane.get(yuvBytes, offset, width * height / 4)
offset += width * height / 4
vPlane.get(yuvBytes, offset, width * height / 4)
} else {
// When not equal, we need to copy the channels row by row
for (i in 0 until height / 2) {
uPlane.get(yuvBytes, offset, width / 2)
offset += width / 2
if (i < height / 2 - 1) {
uPlane.position(uPlane.position() + chromaRowPadding)
}
}
for (i in 0 until height / 2) {
vPlane.get(yuvBytes, offset, width / 2)
offset += width / 2
if (i < height / 2 - 1) {
vPlane.position(vPlane.position() + chromaRowPadding)
}
}
}
val yuvMat = Mat(height + height / 2, width, CvType.CV_8UC1)
yuvMat.put(0, 0, yuvBytes)
Imgproc.cvtColor(yuvMat, rgbaMat, Imgproc.COLOR_YUV2RGBA_I420, 4)
}
}
return rgbaMat
}
And so you can write:
override fun analyze(imageProxy: ImageProxy, rotationDegrees: Int) {
try {
imageProxy.image?.let {
// ImageProxy uses an ImageReader under the hood:
// https://developer.android.com/reference/androidx/camera/core/ImageProxy.html
// That has a default format of YUV_420_888 if not changed that's the default
// Android camera format.
// https://developer.android.com/reference/android/graphics/ImageFormat.html#YUV_420_888
// https://developer.android.com/reference/android/media/ImageReader.html
// Sanity check
if (it.format == ImageFormat.YUV_420_888
&& it.planes.size == 3
) {
val rgbaMat = it.yuvToRgba()
} else {
// Manage other image formats
// TODO - https://developer.android.com/reference/android/media/Image.html
}
}
} catch (ise: IllegalStateException) {
ise.printStackTrace()
}
}
private Mat convertYUVtoMat(#NonNull Image img) {
byte[] nv21;
ByteBuffer yBuffer = img.getPlanes()[0].getBuffer();
ByteBuffer uBuffer = img.getPlanes()[1].getBuffer();
ByteBuffer vBuffer = img.getPlanes()[2].getBuffer();
int ySize = yBuffer.remaining();
int uSize = uBuffer.remaining();
int vSize = vBuffer.remaining();
nv21 = new byte[ySize + uSize + vSize];
yBuffer.get(nv21, 0, ySize);
vBuffer.get(nv21, ySize, vSize);
uBuffer.get(nv21, ySize + vSize, uSize);
Mat yuv = new Mat(img.getHeight() + img.getHeight()/2, img.getWidth(), CvType.CV_8UC1);
yuv.put(0, 0, nv21);
Mat rgb = new Mat();
Imgproc.cvtColor(yuv, rgb, Imgproc.COLOR_YUV2RGB_NV21, 3);
Core.rotate(rgb, rgb, Core.ROTATE_90_CLOCKWISE);
return rgb;
}
This method converts the Camerax API YUV_420_888 image to OpenCV's Mat (RGB) object.
(Working 2021)
#shadowsheep solution is just fine if you need to get OpenCV Mat.
but if you want to get Bitmap and don't want to add opencv library into you project you can take a look at RenderScript solution in android/camera-samples repo
Also I made single Java file library at github. It will be useful if you want to get correct ByteBuffer without any row or pixel strides for futher processing (for instance with neural network engine).
I also compared all these approaches. OpenCV is the fastest.
Background
We record a video of the user's face, and usually the face is located at the upper half of the video.
Later we wish to view the video, but the aspect ratio of the PlayerView might be different than the one of the video, so there needs to be some scaling and cropping.
The problem
The only way I've found to scale the PlayerView so that it will be shown in the entire space it has but keeping the aspect ratio (which will result in cropping when needed, of course) , is by using app:resize_mode="zoom" . Here's a sample of how it works with center-crop: http://s000.tinyupload.com/?file_id=00574047057406286563 . The more the Views that show the content have a similar aspect ratio, the less cropping is needed.
But this is only for the center, meaning it takes a point of 0.5x0.5 of the video, and scale-crops from that point. This causes many cases of losing the important content of the video.
For example, if we have a video that was taken in portrait, and we have a square PlayerView and want to show the top area, this is the part that will be visible:
Of course, if the content itself is square, and the views are also square, it should show the entire content, without cropping.
What I've tried
I've tried searching over the Internet, StackOverflow (here) and on Github, but I couldn't find how to do it. The only clue I've found is about AspectRatioFrameLayout and AspectRatioTextureView, but I didn't find how to use them for this task, if it's even possible.
I was told (here) that I should use a normal TextureView , and provide it directly to SimpleExoPlayer using SimpleExoPlayer.setVideoTextureView. And to set a special transformation to it using TextureView.setTransform.
After a lot of trying what is best to use (and looking at video-crop repository , SuperImageView repository , and JCropImageView repository which have examples of scale/crop of ImageView and video), I've published a working sample that seems to show the video correctly, but I'm still not sure about it, as I also use an ImageView that's shown on top of it before it starts playing (to have a nicer transition instead of black content).
Here's the current code:
class MainActivity : AppCompatActivity() {
private val imageResId = R.drawable.test
private val videoResId = R.raw.test
private val percentageY = 0.2f
private var player: SimpleExoPlayer? = null
override fun onCreate(savedInstanceState: Bundle?) {
window.setBackgroundDrawable(ColorDrawable(0xff000000.toInt()))
super.onCreate(savedInstanceState)
if (cache == null) {
cache = SimpleCache(File(cacheDir, "media"), LeastRecentlyUsedCacheEvictor(MAX_PREVIEW_CACHE_SIZE_IN_BYTES))
}
setContentView(R.layout.activity_main)
// imageView.visibility = View.INVISIBLE
imageView.setImageResource(imageResId)
imageView.doOnPreDraw {
imageView.imageMatrix = prepareMatrixForImageView(imageView, imageView.drawable.intrinsicWidth.toFloat(), imageView.drawable.intrinsicHeight.toFloat())
// imageView.imageMatrix = prepareMatrix(imageView, imageView.drawable.intrinsicWidth.toFloat(), imageView.drawable.intrinsicHeight.toFloat())
// imageView.visibility = View.VISIBLE
}
}
override fun onStart() {
super.onStart()
playVideo()
}
private fun prepareMatrix(view: View, contentWidth: Float, contentHeight: Float): Matrix {
var scaleX = 1.0f
var scaleY = 1.0f
val viewWidth = view.measuredWidth.toFloat()
val viewHeight = view.measuredHeight.toFloat()
Log.d("AppLog", "viewWidth $viewWidth viewHeight $viewHeight contentWidth:$contentWidth contentHeight:$contentHeight")
if (contentWidth > viewWidth && contentHeight > viewHeight) {
scaleX = contentWidth / viewWidth
scaleY = contentHeight / viewHeight
} else if (contentWidth < viewWidth && contentHeight < viewHeight) {
scaleY = viewWidth / contentWidth
scaleX = viewHeight / contentHeight
} else if (viewWidth > contentWidth)
scaleY = viewWidth / contentWidth / (viewHeight / contentHeight)
else if (viewHeight > contentHeight)
scaleX = viewHeight / contentHeight / (viewWidth / contentWidth)
val matrix = Matrix()
val pivotPercentageX = 0.5f
val pivotPercentageY = percentageY
matrix.setScale(scaleX, scaleY, viewWidth * pivotPercentageX, viewHeight * pivotPercentageY)
return matrix
}
private fun prepareMatrixForVideo(view: View, contentWidth: Float, contentHeight: Float): Matrix {
val msWidth = view.measuredWidth
val msHeight = view.measuredHeight
val matrix = Matrix()
matrix.setScale(1f, (contentHeight / contentWidth) * (msWidth.toFloat() / msHeight), msWidth / 2f, percentageY * msHeight) /*,msWidth/2f,msHeight/2f*/
return matrix
}
private fun prepareMatrixForImageView(view: View, contentWidth: Float, contentHeight: Float): Matrix {
val dw = contentWidth
val dh = contentHeight
val msWidth = view.measuredWidth
val msHeight = view.measuredHeight
// Log.d("AppLog", "viewWidth $msWidth viewHeight $msHeight contentWidth:$contentWidth contentHeight:$contentHeight")
val scalew = msWidth.toFloat() / dw
val theoryh = (dh * scalew).toInt()
val scaleh = msHeight.toFloat() / dh
val theoryw = (dw * scaleh).toInt()
val scale: Float
var dx = 0
var dy = 0
if (scalew > scaleh) { // fit width
scale = scalew
// dy = ((msHeight - theoryh) * 0.0f + 0.5f).toInt() // + 0.5f for rounding
} else {
scale = scaleh
dx = ((msWidth - theoryw) * 0.5f + 0.5f).toInt() // + 0.5f for rounding
}
dy = ((msHeight - theoryh) * percentageY + 0.5f).toInt() // + 0.5f for rounding
val matrix = Matrix()
// Log.d("AppLog", "scale:$scale dx:$dx dy:$dy")
matrix.setScale(scale, scale)
matrix.postTranslate(dx.toFloat(), dy.toFloat())
return matrix
}
private fun playVideo() {
player = ExoPlayerFactory.newSimpleInstance(this#MainActivity, DefaultTrackSelector())
player!!.setVideoTextureView(textureView)
player!!.addVideoListener(object : VideoListener {
override fun onVideoSizeChanged(width: Int, height: Int, unappliedRotationDegrees: Int, pixelWidthHeightRatio: Float) {
super.onVideoSizeChanged(width, height, unappliedRotationDegrees, pixelWidthHeightRatio)
Log.d("AppLog", "onVideoSizeChanged: $width $height")
val videoWidth = if (unappliedRotationDegrees % 180 == 0) width else height
val videoHeight = if (unappliedRotationDegrees % 180 == 0) height else width
val matrix = prepareMatrixForVideo(textureView, videoWidth.toFloat(), videoHeight.toFloat())
textureView.setTransform(matrix)
}
override fun onRenderedFirstFrame() {
Log.d("AppLog", "onRenderedFirstFrame")
player!!.removeVideoListener(this)
// imageView.animate().alpha(0f).setDuration(5000).start()
imageView.visibility = View.INVISIBLE
}
})
player!!.volume = 0f
player!!.repeatMode = Player.REPEAT_MODE_ALL
player!!.playRawVideo(this, videoResId)
player!!.playWhenReady = true
// player!!.playVideoFromUrl(this, "https://sample-videos.com/video123/mkv/240/big_buck_bunny_240p_20mb.mkv", cache!!)
// player!!.playVideoFromUrl(this, "https://sample-videos.com/video123/mkv/720/big_buck_bunny_720p_1mb.mkv", cache!!)
// player!!.playVideoFromUrl(this#MainActivity, "https://sample-videos.com/video123/mkv/720/big_buck_bunny_720p_1mb.mkv")
}
override fun onStop() {
super.onStop()
player!!.setVideoTextureView(null)
// playerView.player = null
player!!.release()
player = null
}
companion object {
const val MAX_PREVIEW_CACHE_SIZE_IN_BYTES = 20L * 1024L * 1024L
var cache: com.google.android.exoplayer2.upstream.cache.Cache? = null
#JvmStatic
fun getUserAgent(context: Context): String {
val packageManager = context.packageManager
val info = packageManager.getPackageInfo(context.packageName, 0)
val appName = info.applicationInfo.loadLabel(packageManager).toString()
return Util.getUserAgent(context, appName)
}
}
fun SimpleExoPlayer.playRawVideo(context: Context, #RawRes rawVideoRes: Int) {
val dataSpec = DataSpec(RawResourceDataSource.buildRawResourceUri(rawVideoRes))
val rawResourceDataSource = RawResourceDataSource(context)
rawResourceDataSource.open(dataSpec)
val factory: DataSource.Factory = DataSource.Factory { rawResourceDataSource }
prepare(LoopingMediaSource(ExtractorMediaSource.Factory(factory).createMediaSource(rawResourceDataSource.uri)))
}
fun SimpleExoPlayer.playVideoFromUrl(context: Context, url: String, cache: Cache? = null) = playVideoFromUri(context, Uri.parse(url), cache)
fun SimpleExoPlayer.playVideoFile(context: Context, file: File) = playVideoFromUri(context, Uri.fromFile(file))
fun SimpleExoPlayer.playVideoFromUri(context: Context, uri: Uri, cache: Cache? = null) {
val factory = if (cache != null)
CacheDataSourceFactory(cache, DefaultHttpDataSourceFactory(getUserAgent(context)))
else
DefaultDataSourceFactory(context, MainActivity.getUserAgent(context))
val mediaSource = ExtractorMediaSource.Factory(factory).createMediaSource(uri)
prepare(mediaSource)
}
}
I had various issues on trying this till I got to the current situation, and I've updated this question multiple times accordingly. Now it even works with the percentageY I talked about, so I could set it to be from 20% of the top of the video, if I wish. However, I still think that it has a big chance that something is wrong, because when I tried to set it to 50% , I've noticed that the content might not fit the entire View.
I even looked at the source code of ImageView (here), to see how center-crop is used. When applied to the ImageView, it still worked as center-crop, but when I used the same technique on the video, it gave me a very wrong result.
The questions
My goal here was to show both ImageView and the video so that it will smoothly transition from a static image to a video. All that while having both have the top-scale-crop of 20% from the top (for example). I've published a sample project here to try it out and share people of what I've found.
So now my questions are around why this doesn't seem to work well for the imageView and/or video :
As it turns out, none of the matrix creations that I've tried work well for either ImageView or the video. What's wrong with it exactly? How can I change it for them to look the same? To scale-crop from the top 20%, for example?
I tried to use the exact matrix for both, but it seems each need it differently, even though both have the exact same size and content size. Why would I need a different matrix for each?
EDIT: after this question was answered, I've decided to make a small sample of how to use it (Github repository available here) :
import android.content.Context
import android.graphics.Matrix
import android.graphics.PointF
import android.net.Uri
import android.os.Bundle
import android.view.TextureView
import android.view.View
import androidx.annotation.RawRes
import androidx.appcompat.app.AppCompatActivity
import androidx.core.view.doOnPreDraw
import com.google.android.exoplayer2.ExoPlayerFactory
import com.google.android.exoplayer2.Player
import com.google.android.exoplayer2.SimpleExoPlayer
import com.google.android.exoplayer2.source.ExtractorMediaSource
import com.google.android.exoplayer2.source.LoopingMediaSource
import com.google.android.exoplayer2.trackselection.DefaultTrackSelector
import com.google.android.exoplayer2.upstream.*
import com.google.android.exoplayer2.upstream.cache.Cache
import com.google.android.exoplayer2.upstream.cache.CacheDataSourceFactory
import com.google.android.exoplayer2.upstream.cache.LeastRecentlyUsedCacheEvictor
import com.google.android.exoplayer2.upstream.cache.SimpleCache
import com.google.android.exoplayer2.util.Util
import com.google.android.exoplayer2.video.VideoListener
import kotlinx.android.synthetic.main.activity_main.*
import java.io.File
// https://stackoverflow.com/questions/54216273/how-to-have-similar-mechanism-of-center-crop-on-exoplayers-playerview-but-not
class MainActivity : AppCompatActivity() {
companion object {
private val FOCAL_POINT = PointF(0.5f, 0.2f)
private const val IMAGE_RES_ID = R.drawable.test
private const val VIDEO_RES_ID = R.raw.test
private var cache: Cache? = null
private const val MAX_PREVIEW_CACHE_SIZE_IN_BYTES = 20L * 1024L * 1024L
#JvmStatic
fun getUserAgent(context: Context): String {
val packageManager = context.packageManager
val info = packageManager.getPackageInfo(context.packageName, 0)
val appName = info.applicationInfo.loadLabel(packageManager).toString()
return Util.getUserAgent(context, appName)
}
}
private var player: SimpleExoPlayer? = null
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
if (cache == null)
cache = SimpleCache(File(cacheDir, "media"), LeastRecentlyUsedCacheEvictor(MAX_PREVIEW_CACHE_SIZE_IN_BYTES))
// imageView.visibility = View.INVISIBLE
imageView.setImageResource(IMAGE_RES_ID)
}
private fun prepareMatrix(view: View, mediaWidth: Float, mediaHeight: Float, focalPoint: PointF): Matrix? {
if (view.visibility == View.GONE)
return null
val viewHeight = (view.height - view.paddingTop - view.paddingBottom).toFloat()
val viewWidth = (view.width - view.paddingStart - view.paddingEnd).toFloat()
if (viewWidth <= 0 || viewHeight <= 0)
return null
val matrix = Matrix()
if (view is TextureView)
// Restore true media size for further manipulation.
matrix.setScale(mediaWidth / viewWidth, mediaHeight / viewHeight)
val scaleFactorY = viewHeight / mediaHeight
val scaleFactor: Float
var px = 0f
var py = 0f
if (mediaWidth * scaleFactorY >= viewWidth) {
// Fit height
scaleFactor = scaleFactorY
px = -(mediaWidth * scaleFactor - viewWidth) * focalPoint.x / (1 - scaleFactor)
} else {
// Fit width
scaleFactor = viewWidth / mediaWidth
py = -(mediaHeight * scaleFactor - viewHeight) * focalPoint.y / (1 - scaleFactor)
}
matrix.postScale(scaleFactor, scaleFactor, px, py)
return matrix
}
private fun playVideo() {
player = ExoPlayerFactory.newSimpleInstance(this#MainActivity, DefaultTrackSelector())
player!!.setVideoTextureView(textureView)
player!!.addVideoListener(object : VideoListener {
override fun onVideoSizeChanged(videoWidth: Int, videoHeight: Int, unappliedRotationDegrees: Int, pixelWidthHeightRatio: Float) {
super.onVideoSizeChanged(videoWidth, videoHeight, unappliedRotationDegrees, pixelWidthHeightRatio)
textureView.setTransform(prepareMatrix(textureView, videoWidth.toFloat(), videoHeight.toFloat(), FOCAL_POINT))
}
override fun onRenderedFirstFrame() {
// Log.d("AppLog", "onRenderedFirstFrame")
player!!.removeVideoListener(this)
imageView.animate().alpha(0f).setDuration(2000).start()
// imageView.visibility = View.INVISIBLE
}
})
player!!.volume = 0f
player!!.repeatMode = Player.REPEAT_MODE_ALL
player!!.playRawVideo(this, VIDEO_RES_ID)
player!!.playWhenReady = true
// player!!.playVideoFromUrl(this, "https://sample-videos.com/video123/mkv/240/big_buck_bunny_240p_20mb.mkv", cache!!)
// player!!.playVideoFromUrl(this, "https://sample-videos.com/video123/mkv/720/big_buck_bunny_720p_1mb.mkv", cache!!)
// player!!.playVideoFromUrl(this#MainActivity, "https://sample-videos.com/video123/mkv/720/big_buck_bunny_720p_1mb.mkv")
}
override fun onStart() {
super.onStart()
imageView.doOnPreDraw {
val imageWidth: Float = imageView.drawable.intrinsicWidth.toFloat()
val imageHeight: Float = imageView.drawable.intrinsicHeight.toFloat()
imageView.imageMatrix = prepareMatrix(imageView, imageWidth, imageHeight, FOCAL_POINT)
}
playVideo()
}
override fun onStop() {
super.onStop()
if (player != null) {
player!!.setVideoTextureView(null)
// playerView.player = null
player!!.release()
player = null
}
}
override fun onDestroy() {
super.onDestroy()
if (!isChangingConfigurations)
cache?.release()
}
fun SimpleExoPlayer.playRawVideo(context: Context, #RawRes rawVideoRes: Int) {
val dataSpec = DataSpec(RawResourceDataSource.buildRawResourceUri(rawVideoRes))
val rawResourceDataSource = RawResourceDataSource(context)
rawResourceDataSource.open(dataSpec)
val factory: DataSource.Factory = DataSource.Factory { rawResourceDataSource }
prepare(LoopingMediaSource(ExtractorMediaSource.Factory(factory).createMediaSource(rawResourceDataSource.uri)))
}
fun SimpleExoPlayer.playVideoFromUrl(context: Context, url: String, cache: Cache? = null) = playVideoFromUri(context, Uri.parse(url), cache)
fun SimpleExoPlayer.playVideoFile(context: Context, file: File) = playVideoFromUri(context, Uri.fromFile(file))
fun SimpleExoPlayer.playVideoFromUri(context: Context, uri: Uri, cache: Cache? = null) {
val factory = if (cache != null)
CacheDataSourceFactory(cache, DefaultHttpDataSourceFactory(getUserAgent(context)))
else
DefaultDataSourceFactory(context, MainActivity.getUserAgent(context))
val mediaSource = ExtractorMediaSource.Factory(factory).createMediaSource(uri)
prepare(mediaSource)
}
}
Here's a solution for ImageView alone, if needed:
class ScaleCropImageView(context: Context, attrs: AttributeSet?) : AppCompatImageView(context, attrs) {
var focalPoint = PointF(0.5f, 0.5f)
set(value) {
field = value
updateMatrix()
}
private val viewWidth: Float
get() = (width - paddingLeft - paddingRight).toFloat()
private val viewHeight: Float
get() = (height - paddingTop - paddingBottom).toFloat()
init {
scaleType = ScaleType.MATRIX
}
override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
super.onSizeChanged(w, h, oldw, oldh)
updateMatrix()
}
override fun setImageDrawable(drawable: Drawable?) {
super.setImageDrawable(drawable)
updateMatrix()
}
#Suppress("MemberVisibilityCanBePrivate")
fun updateMatrix() {
if (scaleType != ImageView.ScaleType.MATRIX)
return
val dr = drawable ?: return
imageMatrix = prepareMatrix(
viewWidth, viewHeight,
dr.intrinsicWidth.toFloat(), dr.intrinsicHeight.toFloat(), focalPoint, Matrix()
)
}
private fun prepareMatrix(
viewWidth: Float, viewHeight: Float, mediaWidth: Float, mediaHeight: Float,
focalPoint: PointF, matrix: Matrix
): Matrix? {
if (viewWidth <= 0 || viewHeight <= 0)
return null
var scaleFactor = viewHeight / mediaHeight
if (mediaWidth * scaleFactor >= viewWidth) {
// Fit height
matrix.postScale(scaleFactor, scaleFactor, -(mediaWidth * scaleFactor - viewWidth) * focalPoint.x / (1 - scaleFactor), 0f)
} else {
// Fit width
scaleFactor = viewWidth / mediaWidth
matrix.postScale(scaleFactor, scaleFactor, 0f, -(mediaHeight * scaleFactor - viewHeight) * focalPoint.y / (1 - scaleFactor))
}
return matrix
}
}
The question is how to manipulate an image like ImageView.ScaleType.CENTER_CROP but to shift the focus from the center to another location that is 20% from the top of the image. First, let's look at what CENTER_CROP does:
From the documentation:
CENTER_CROP
Scale the image uniformly (maintain the image's aspect ratio) so that both dimensions (width and height) of the image will be equal to or larger than the corresponding dimension of the view (minus padding). The image is then centered in the view. From XML, use this syntax: android:scaleType="centerCrop".
In other words, scale the image without distortion such that either the width or height of the image (or both width and height) fit within the view so that the view is completely filled with the image (no gaps.)
Another way to think of this is that the center of the image is "pinned" to the center of the view. The image is then scaled to meet the criteria above.
In the following video, the white lines mark the center of the image; the red lines mark the center of the view. The scale type is CENTER_CROP. Notice how the center points of the image and the view coincide. As the view changes size, these two points continue to overlap and always appear at the center of the view regardless of the view size.
So, what does it mean to have center crop-like behavior at a different location such as 20% from the top? Like center crop, we can specify that the point that is 20% from the top of the image and the point that 20% from the top of the view will be "pinned" like the 50% point is "pinned" in center crop. The horizontal location of this point remains at 50% of the image and view. The image can now be scaled to satisfy the other conditions of center crop which specify that either the width and/or height of the image will fit the view with no gaps. (Size of view is understood to be the view size less padding.)
Here is a short video of this 20% crop behavior. In this video, the white lines show the middle of the image, the red lines show the pinned point in the view and the blue line that shows behind the horizontal red line identifies 20% from the top of the image. (Demo project is on GitHub.
Here is the result showing the full image that was supplied and the video in a square frame that transition from the still image. .
MainActivity.kt
prepareMatrix() is the method that does the work to determine how to scale/crop the image. There is some additional work to be done with the video since it appears that the video is made to fit the TextureViewas a scale type "FIT_XY" when it is assigned to the TextureView. Because of this scaling, the media size must be restored before prepareMatrix() is called for the video
class MainActivity : AppCompatActivity() {
private val imageResId = R.drawable.test
private val videoResId = R.raw.test
private var player: SimpleExoPlayer? = null
private val mFocalPoint = PointF(0.5f, 0.2f)
override fun onCreate(savedInstanceState: Bundle?) {
window.setBackgroundDrawable(ColorDrawable(0xff000000.toInt()))
super.onCreate(savedInstanceState)
if (cache == null) {
cache = SimpleCache(File(cacheDir, "media"), LeastRecentlyUsedCacheEvictor(MAX_PREVIEW_CACHE_SIZE_IN_BYTES))
}
setContentView(R.layout.activity_main)
// imageView.visibility = View.INVISIBLE
imageView.setImageResource(imageResId)
imageView.doOnPreDraw {
imageView.scaleType = ImageView.ScaleType.MATRIX
val imageWidth: Float = ContextCompat.getDrawable(this, imageResId)!!.intrinsicWidth.toFloat()
val imageHeight: Float = ContextCompat.getDrawable(this, imageResId)!!.intrinsicHeight.toFloat()
imageView.imageMatrix = prepareMatrix(imageView, imageWidth, imageHeight, mFocalPoint, Matrix())
val b = BitmapFactory.decodeResource(resources, imageResId)
val d = BitmapDrawable(resources, b.copy(Bitmap.Config.ARGB_8888, true))
val c = Canvas(d.bitmap)
val p = Paint()
p.color = resources.getColor(android.R.color.holo_red_dark)
p.style = Paint.Style.STROKE
val strokeWidth = 10
p.strokeWidth = strokeWidth.toFloat()
// Horizontal line
c.drawLine(0f, imageHeight * mFocalPoint.y, imageWidth, imageHeight * mFocalPoint.y, p)
// Vertical line
c.drawLine(imageWidth * mFocalPoint.x, 0f, imageWidth * mFocalPoint.x, imageHeight, p)
// Line in horizontal and vertical center
p.color = resources.getColor(android.R.color.white)
c.drawLine(imageWidth / 2, 0f, imageWidth / 2, imageHeight, p)
c.drawLine(0f, imageHeight / 2, imageWidth, imageHeight / 2, p)
imageView.setImageBitmap(d.bitmap)
imageViewFull.setImageBitmap(d.bitmap)
}
}
fun startPlay(view: View) {
playVideo()
}
private fun getViewWidth(view: View): Float {
return (view.width - view.paddingStart - view.paddingEnd).toFloat()
}
private fun getViewHeight(view: View): Float {
return (view.height - view.paddingTop - view.paddingBottom).toFloat()
}
private fun prepareMatrix(targetView: View, mediaWidth: Float, mediaHeight: Float,
focalPoint: PointF, matrix: Matrix): Matrix {
if (targetView.visibility != View.VISIBLE) {
return matrix
}
val viewHeight = getViewHeight(targetView)
val viewWidth = getViewWidth(targetView)
val scaleFactorY = viewHeight / mediaHeight
val scaleFactor: Float
val px: Float
val py: Float
if (mediaWidth * scaleFactorY >= viewWidth) {
// Fit height
scaleFactor = scaleFactorY
px = -(mediaWidth * scaleFactor - viewWidth) * focalPoint.x / (1 - scaleFactor)
py = 0f
} else {
// Fit width
scaleFactor = viewWidth / mediaWidth
px = 0f
py = -(mediaHeight * scaleFactor - viewHeight) * focalPoint.y / (1 - scaleFactor)
}
matrix.postScale(scaleFactor, scaleFactor, px, py)
return matrix
}
private fun playVideo() {
player = ExoPlayerFactory.newSimpleInstance(this#MainActivity, DefaultTrackSelector())
player!!.setVideoTextureView(textureView)
player!!.addVideoListener(object : VideoListener {
override fun onVideoSizeChanged(width: Int, height: Int, unappliedRotationDegrees: Int, pixelWidthHeightRatio: Float) {
super.onVideoSizeChanged(width, height, unappliedRotationDegrees, pixelWidthHeightRatio)
val matrix = Matrix()
// Restore true media size for further manipulation.
matrix.setScale(width / getViewWidth(textureView), height / getViewHeight(textureView))
textureView.setTransform(prepareMatrix(textureView, width.toFloat(), height.toFloat(), mFocalPoint, matrix))
}
override fun onRenderedFirstFrame() {
Log.d("AppLog", "onRenderedFirstFrame")
player!!.removeVideoListener(this)
imageView.animate().alpha(0f).setDuration(2000).start()
imageView.visibility = View.INVISIBLE
}
})
player!!.volume = 0f
player!!.repeatMode = Player.REPEAT_MODE_ALL
player!!.playRawVideo(this, videoResId)
player!!.playWhenReady = true
// player!!.playVideoFromUrl(this, "https://sample-videos.com/video123/mkv/240/big_buck_bunny_240p_20mb.mkv", cache!!)
// player!!.playVideoFromUrl(this, "https://sample-videos.com/video123/mkv/720/big_buck_bunny_720p_1mb.mkv", cache!!)
// player!!.playVideoFromUrl(this#MainActivity, "https://sample-videos.com/video123/mkv/720/big_buck_bunny_720p_1mb.mkv")
}
override fun onStop() {
super.onStop()
if (player != null) {
player!!.setVideoTextureView(null)
// playerView.player = null
player!!.release()
player = null
}
}
companion object {
const val MAX_PREVIEW_CACHE_SIZE_IN_BYTES = 20L * 1024L * 1024L
var cache: com.google.android.exoplayer2.upstream.cache.Cache? = null
#JvmStatic
fun getUserAgent(context: Context): String {
val packageManager = context.packageManager
val info = packageManager.getPackageInfo(context.packageName, 0)
val appName = info.applicationInfo.loadLabel(packageManager).toString()
return Util.getUserAgent(context, appName)
}
}
fun SimpleExoPlayer.playRawVideo(context: Context, #RawRes rawVideoRes: Int) {
val dataSpec = DataSpec(RawResourceDataSource.buildRawResourceUri(rawVideoRes))
val rawResourceDataSource = RawResourceDataSource(context)
rawResourceDataSource.open(dataSpec)
val factory: DataSource.Factory = DataSource.Factory { rawResourceDataSource }
prepare(LoopingMediaSource(ExtractorMediaSource.Factory(factory).createMediaSource(rawResourceDataSource.uri)))
}
fun SimpleExoPlayer.playVideoFromUrl(context: Context, url: String, cache: Cache? = null) = playVideoFromUri(context, Uri.parse(url), cache)
fun SimpleExoPlayer.playVideoFile(context: Context, file: File) = playVideoFromUri(context, Uri.fromFile(file))
fun SimpleExoPlayer.playVideoFromUri(context: Context, uri: Uri, cache: Cache? = null) {
val factory = if (cache != null)
CacheDataSourceFactory(cache, DefaultHttpDataSourceFactory(getUserAgent(context)))
else
DefaultDataSourceFactory(context, MainActivity.getUserAgent(context))
val mediaSource = ExtractorMediaSource.Factory(factory).createMediaSource(uri)
prepare(mediaSource)
}
}
you can use app:resize_mode="zoom" in com.google.android.exoplayer2.ui.PlayerView
I had a similar problem and solved it by applying transformations on the TextureView whose Surface is used by ExoPlayer:
player.addVideoListener(object : VideoListener {
override fun onVideoSizeChanged(
videoWidth: Int,
videoHeight: Int,
unappliedRotationDegrees: Int,
pixelWidthHeightRatio: Float,
) {
removeVideoListener(this)
val viewWidth: Int = textureView.width - textureView.paddingStart - textureView.paddingEnd
val viewHeight: Int = textureView.height - textureView.paddingTop - textureView.paddingBottom
if (videoWidth == viewWidth && videoHeight == viewHeight) {
return
}
val matrix = Matrix().apply {
// TextureView makes a best effort in fitting the video inside the View. The first transformation we apply is for reverting the fitting.
setScale(
videoWidth.toFloat() / viewWidth,
videoHeight.toFloat() / viewHeight,
)
}
// This algorithm is from ImageView's CENTER_CROP transformation
val offset = 0.5f // the center in CENTER_CROP but you probably want a different value here
val scale: Float
val dx: Float
val dy: Float
if (videoWidth * viewHeight > viewWidth * videoHeight) {
scale = viewHeight.toFloat() / videoHeight
dx = (viewWidth - videoWidth * scale) * offset
dy = 0f
} else {
scale = viewWidth.toFloat() / videoWidth
dx = 0f
dy = (viewHeight - videoHeight * scale) * offset
}
setTransform(matrix.apply {
postScale(scale, scale)
postTranslate(dx, dy)
})
}
})
player.setVideoTextureView(textureView)
player.prepare(createMediaSource())
Note that unless you're using DefaultRenderersFactory you need to make sure that your video Renderer actually calls onVideoSizeChanged by for instance creating the factory like so:
val renderersFactory = RenderersFactory { handler, videoListener, _, _, _, _ ->
// Allows other renderers to be removed by R8
arrayOf(
MediaCodecVideoRenderer(
context,
MediaCodecSelector.DEFAULT,
DefaultRenderersFactory.DEFAULT_ALLOWED_VIDEO_JOINING_TIME_MS,
handler,
videoListener,
-1,
),
MediaCodecAudioRenderer(context, MediaCodecSelector.DEFAULT),
)
}