Develop Reference

MediaCodec gives IllegalStateException at getInputbuffer(index) in callback of OnInputBufferAvailable

I am decoding an mp3 file, first I convert the mp3 file into a chunks of byteArray of size 1000 and put it in a circularArray and then pass it to mediaCodec callback for decoding (decode one byteArray at a time), I follow this link. It is working fine for Samsung devices, but if I use other than Samsung devices (Vivo, pixel 3a) it crashes at the mediaCodec.getInputBuffer(index) in the callback of onInputBufferAvailable by giving the exception IllegalStateException. My code is as follows:
var decoder: MediaCodec = MediaCodec.createDecoderByType("audio/mpeg")
decoder.configure(format, null, null, 0)
decoder.setCallback(object : MediaCodec.Callback() {
override fun onInputBufferAvailable(mediaCodec: MediaCodec, i: Int) {
while (true) {
if (circularArray!!.size() > 0) {
val data: ByteArray = circularArray.popFirst()
val info = MediaCodec.BufferInfo()
val buffer = mediaCodec.getInputBuffer(i)
buffer!!.put(data, 0, data.size)
mediaCodec.queueInputBuffer(i, 0, data.size, 0, 0)
break
}
}
}
override fun onOutputBufferAvailable(mediaCodec: MediaCodec, i: Int, info: MediaCodec.BufferInfo) {
//DECODING PACKET ENDED
val outBuffer = mediaCodec.getOutputBuffer(i)
val chunk = ByteArray(info.size)
outBuffer!![chunk] // Read the buffer all at once
outBuffer!!.clear()
Log.d(TAG, "onOutputBufferAvailable: ${info.size}")
audioTrack!!.write(chunk, info.offset, info.offset + info.size)
mediaCodec.releaseOutputBuffer(i, false)
}
override fun onError(mediaCodec: MediaCodec, e: MediaCodec.CodecException) {}
override fun onOutputFormatChanged(mediaCodec: MediaCodec, mediaFormat: MediaFormat) {}
})
decoder!!.start()
I converted my file like this
val tempBuf = ByteArray(1000)
var byteRead: Int
try {
val bufferedInputStream = BufferedInputStream(FileInputStream(mp3File))
while (bufferedInputStream.read(tempBuf).also { byteRead = it } != -1) {
circularArray.addLast(tempBuf.copyOf())
}
bufferedInputStream.close()
Thread(aacDecoderAndPlayRunnable).start()
} catch (e: java.lang.Exception) {
Log.d(TAG, "fileToInputStream: ${e.message}")
e.printStackTrace()
null
}
The exception where the app crashes is
Even if I try to get the format form mediaCodec in the callback, it gives an exception and crashes anyway. I also checked supportedTypes from the codec it supports audio/mpeg.

First of all, the MediaCodec works with a queue of input buffers. And you can read more about it in the docs.
The second parameter of the onInputBufferAvailable callback is the index of the buffer. When calling getInputBuffer() you must pass this index instead of 0:
val buffer = mediaCodec.getInputBuffer(i)
Second, consider using the MediaExtractor instead of reading the file yourself. It supplies you will presentation timestamps and flags to pass into queueInputBuffer().
Third, you need to remove the while (true) loop. You can only queue one buffer per callback.

Mediaplayer showing error(1,-2147483648) while playing byteArray

I am getting some audio streaming data as base64 String, I convert it in byteArray and then write a file locally as mp3 file to play in mediaplayer. But the problem is mediaplayer througing error(1,-2147483648). How to solve this, I tried with many SO posts but nothing works.
**what I am trying to do is fetch base64 string save locally and play**.
val file = requireContext().getExternalFilesDir(null)?.absolutePath + "/audioRecording1.mp3"
val mediaPlayer = MediaPlayer()
try {
val output = FileOutputStream(file)
output.write(mp3SoundByteArray)
output.close()
val fis = FileInputStream(file)
mediaPlayer.setDataSource(fis.fd)
fis.close()
mediaPlayer.setAudioAttributes(
AudioAttributes.Builder().
setContentType(AudioAttributes.CONTENT_TYPE_MUSIC).
setUsage(AudioAttributes.USAGE_MEDIA).
build())
mediaPlayer.prepareAsync()
mediaPlayer.setOnPreparedListener {
mediaPlayer.start()
}
mediaPlayer.setOnErrorListener { mediaPlayer, i, i2 ->
Log.v("","${i,i2}")
true
}
}catch (e:Exception){
toast(e.message!!)
}
could you please tell me how to overcome this?

I am not sure, but it seams that you have trouble with file saving
fun saveFile(responseBody: ResponseBody?, pathWhereYouWantToSaveFile: String) {
val body = responseBody ?: return
var input: InputStream? = null
try {
val uri = Uri.parse(pathWhereYouWantToSaveFile)
input = body.byteStream()
val parcelFileDescriptor =
context.getContentResolver().openFileDescriptor(uri, FileConst.WRITE_MODE)
val fileOutputStream = FileOutputStream(parcelFileDescriptor?.fileDescriptor)
fileOutputStream.use { output ->
val bufferSize = BUFFER_SIZE.toInt()
val buffer = ByteArray(bufferSize)
var read: Int
while (input.read(buffer).also { read = it } != END_OF_FILE) {
output.write(buffer, START_OFFSET, read)
}
output.flush()
}
} catch (exception: Exception) {
logErrorIfDebug(exception)
} finally {
input?.close()
}
}
const val READ_MODE = "r"
const val WRITE_MODE = "w"
const val START_OFFSET = 0
const val END_OF_FILE = -1
const val BUFFER_SIZE = 4 * BYTES_IN_KILOBYTE
Try this in your viewModel or data sourse layer, then send result to UI layer and use there
Have you checked that your file saved correct? You can go to directory and try to open file. If everything okey, you can get it by uri in your media player.
Also you should check - perhaps you are creating another path for save and retrieve
Better way to use player is https://exoplayer.dev/
But native library also can work with internal uri path.

If you just take a random part of a base64 encoded audio stream then your bytearray will (after decoding) contain a part of an audiofile.
Some audio stream bytes.
Not a complete valid mp3 file with headers and such.
If you had said: am getting a mp3 file in one base64 String then your approch would be ok.

I have solved the issue without writing any header. below way.
val clipData =android.util.Base64.decode(data,0)
val output = FileOutputStream(file,true)
output.write(clipData)
output.close()

Android download image with Workmanager and show it in gallery

I wrote code that downloads some source of data from the internet (in this example picture), shows downloadPercentages while the process of downloading is ongoing and writes this file on android storage. works well and looks very nice everything except saving on android storage.
Code is written in 3 classes, but I will show only one that I think is relevant (DownloadWorker). If anyone thinks other classes might help, let me now in comment and I will add them.
DownloadWorker:
class DownloadWorker(val context: Context, params: WorkerParameters) : Worker(context, params) {
companion object {
const val FILE_NAME = "image.jpg"
}
override fun doWork(): Result {
try {
if (DownloadHelper.url == null) {
DownloadHelper.downloadState.postValue(DownloadState.Failure)
return Result.failure()
}
DownloadHelper.url?.apply {
if(!startsWith("https")) {
DownloadHelper.url = replace("http", "https")
}
}
val url = URL(DownloadHelper.url)
val connection = url.openConnection()
connection.connect()
val fileSize = connection.contentLength
val inputStream = connection.getInputStream()
val buffer = ByteArray(1024)
val file = context.getExternalFilesDir(Environment.DIRECTORY_DOWNLOADS)
val outputFile = File(file, FILE_NAME)
var len = 0
var total = 0L
val fos = FileOutputStream(outputFile)
len = inputStream.read(buffer)
while (len != -1) {
fos.write(buffer, 0, len)
total += len
val percent = ((total * 100) / fileSize).toInt()
DownloadHelper.downloadState.postValue(DownloadState.InProgress(percent))
len = inputStream.read(buffer)
}
fos.close()
inputStream.close()
DownloadHelper.downloadState.postValue(DownloadState.Success(outputFile))
} catch (e: Exception) {
DownloadHelper.downloadState.postValue(DownloadState.Failure)
return Result.failure()
}
return Result.success()
}
}
After download success, my image is not shown in gallery, or in downloaded files folder. To see this image you need to enter android storage, find in android data app package by name and navigate all the way to the image. Pretty complicated.
Can anyone help, thanks.

Google Speech to Text does not return results

I want Google Speech to text API to recognize a short phrase after I press a button. So I came up with the following code. But it keeps returning no results. I'm quite confused, there are results in there (the buffer etc.), the mic is working well and is enabled in the emulator. Google console also doesn't show errors.
Here's my code.
Click listener that starts the recording:
val clicker: View.OnClickListener = View.OnClickListener {
Log.d(TAG, "Starting record thread")
mAudioRecorder.record(LISTEN_TIME_MILLIS)
}
mReadButton.setOnClickListener(clicker)
Here's a broadcast receiver that processes the results and tries to send them to Google:
private val broadCastReceiver = object : BroadcastReceiver() {
override fun onReceive(contxt: Context?, intent: Intent?) {
if (intent!!.getBooleanExtra(RECORDING_SUCCESS, false)) {
val byteArrayExtra = intent.getByteArrayExtra(RECORDING_AUDIO)
val audioResultByteString: ByteString = ByteString.copyFrom(byteArrayExtra)
if (audioResultByteString.size() > 0) {
val audio: RecognitionAudio = RecognitionAudio.newBuilder()
.setContent(audioResultByteString).build()
val resultsList = mSpeechClient.recognize(config, audio).resultsList
if (resultsList.size > 0) {
for (result in resultsList) {
val resultText = result.alternativesList[0].transcript
}
}
Log.d(TAG, "- Done recognition. Result Qty: ${resultsList.size}")
}
}
}
}
Here is the AudioRecorder class function, which does the recording:
fun record(listenTimeMillis: Long) {
val byteString: ByteString = ByteString.EMPTY
mAudioRecorder = initAudioRecorder()
val mBuffer = ByteArray(4 * AudioRecord.getMinBufferSize(SAMPLE_RATE_HZ, CHANNEL, ENCODING))
mAudioRecorder!!.startRecording()
Thread {
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND)
Thread.sleep(listenTimeMillis)
val read = mAudioRecorder!!.read(mBuffer, 0, mBuffer.size, AudioRecord.READ_NON_BLOCKING)
val intent = Intent(RECORDING_COMPLETED_INTENT)
try {
if (read > 0) {
intent.putExtra(RECORDING_AUDIO, mBuffer)
intent.putExtra(RECORDING_SUCCESS, true)
}
LocalBroadcastManager.getInstance(context).sendBroadcast(intent)
} catch (e: Exception) {
Log.e(TAG, e.stackTrace.toString())
}
releaseAudioRecorder()
}.start()
}

I solved this. The thing to blame was a too small buffer size. So the recognition server was actually getting half a second of audio record which it obviously couldn't recognize.
val mBuffer = ByteArray(4 * AudioRecord.getMinBufferSize(SAMPLE_RATE_HZ, CHANNEL, ENCODING))
instead of 4 I put 200 and instead of AudioRecord.READ_NON_BLOCKING I have put AudioRecord.READ_BLOCKING and I read the buffer in a loop and increase the offset in each iteration. Then it started working.
val startTime = System.currentTimeMillis()
var deltaTime = 0L
var offset = 0
val intent = Intent(RECORDING_COMPLETED_INTENT)
val readChunk = 512
while (deltaTime < listenTimeMillis && offset < mBuffer.size) {
val read = mAudioRecord!!.read(mBuffer, offset, readChunk, AudioRecord.READ_BLOCKING)
if (read < 0) {
intent.putExtra(RECORDING_SUCCESS, false)
break; //if read with error, end here
}
deltaTime = System.currentTimeMillis() - startTime //startTime is a while loop breaking condition so it lestens only for specified amount of time
offset += readChunk
}

How to get&modify metadata to supported audio files on Android?

Background
Android supports various audio files encoding and decoding.
I record audio into an audio file using android.media.MediaRecorder class, but I also wish to show information about the files I've recorded (not standard data, but still just text, maybe even configurable by user), and I think it's best to store this information within the files.
examples of possible data to store: when it was recorded, where it was recorded, notes by the user...
The problem
The MediaRecorder class doesn't have any function that I can find, to add or even read metadata of the recorded audio file.
I also can't find a similar class that does it.
What I've tried
I tried searching how to do it for specific files types, and also tried to find a library that does it.
I haven't find even a clue about this information.
The only thing I've found for MediaRecorder class, is a function called "setLocation" , which is used to indicate where the recording has started (geographically), and looking at its code, I can see it sets parameters:
public void setLocation(float latitude, float longitude) {
int latitudex10000 = (int) (latitude * 10000 + 0.5);
int longitudex10000 = (int) (longitude * 10000 + 0.5);
if (latitudex10000 > 900000 || latitudex10000 < -900000) {
String msg = "Latitude: " + latitude + " out of range.";
throw new IllegalArgumentException(msg);
}
if (longitudex10000 > 1800000 || longitudex10000 < -1800000) {
String msg = "Longitude: " + longitude + " out of range";
throw new IllegalArgumentException(msg);
}
setParameter("param-geotag-latitude=" + latitudex10000);
setParameter("param-geotag-longitude=" + longitudex10000);
}
But setParameter is private, and I'm not sure if it's ok to put anything I want into it, even if I had a way to access it (reflection, for example) :
private native void setParameter(String nameValuePair);
I also don't get, given an audio/video file, how to get/modify this kind of information. It's not available for SimpleExoPlayer, for example.
The questions
How can I read,write, and modify metadata inside supported audio files of Android?
Are there any limitations/restrictions for those actions?
Which file formats are available for this?
Is it possible to add the metadata while I record the audio?
Is it possible perhaps via MediaStore ? But then how do I do those operations? And which files are supported? And does the metadata stay within the file?
EDIT: ok I've looked at the solution offered to me (here, repo here, based on here) , and it seems to work well. However, it doesn't work on latest version of the library that it uses (org.mp4parser.isoparser:1.9.37 dependency of mp4parser) , so I leave this question to be answered : Why doesn't it work on latest version of this library?
Code:
object MediaMetaDataUtil {
interface PrepareBoxListener {
fun prepareBox(existingBox: Box?): Box
}
#WorkerThread
fun <T : Box> readMetadata(mediaFilePath: String, boxType: String): T? {
return try {
val isoFile = IsoFile(FileDataSourceImpl(FileInputStream(mediaFilePath).channel))
val nam = Path.getPath<T>(isoFile, "/moov[0]/udta[0]/meta[0]/ilst/$boxType")
isoFile.close()
nam
} catch (e: Exception) {
null
}
}
/**
* #param boxType the type of the box. Example is "©nam" (AppleNameBox.TYPE). More available here: https://kdenlive.org/en/project/adding-meta-data-to-mp4-video/
* #param listener used to prepare the existing or new box
* */
#WorkerThread
#Throws(IOException::class)
fun writeMetadata(mediaFilePath: String, boxType: String, listener: PrepareBoxListener) {
val videoFile = File(mediaFilePath)
if (!videoFile.exists()) {
throw FileNotFoundException("File $mediaFilePath not exists")
}
if (!videoFile.canWrite()) {
throw IllegalStateException("No write permissions to file $mediaFilePath")
}
val isoFile = IsoFile(mediaFilePath)
val moov = isoFile.getBoxes<MovieBox>(MovieBox::class.java)[0]
var freeBox = findFreeBox(moov)
val correctOffset = needsOffsetCorrection(isoFile)
val sizeBefore = moov.size
var offset: Long = 0
for (box in isoFile.boxes) {
if ("moov" == box.type) {
break
}
offset += box.size
}
// Create structure or just navigate to Apple List Box.
var userDataBox: UserDataBox? = Path.getPath(moov, "udta")
if (userDataBox == null) {
userDataBox = UserDataBox()
moov.addBox(userDataBox)
}
var metaBox: MetaBox? = Path.getPath(userDataBox, "meta")
if (metaBox == null) {
metaBox = MetaBox()
val hdlr = HandlerBox()
hdlr.handlerType = "mdir"
metaBox.addBox(hdlr)
userDataBox.addBox(metaBox)
}
var ilst: AppleItemListBox? = Path.getPath(metaBox, "ilst")
if (ilst == null) {
ilst = AppleItemListBox()
metaBox.addBox(ilst)
}
if (freeBox == null) {
freeBox = FreeBox(128 * 1024)
metaBox.addBox(freeBox)
}
// Got Apple List Box
var nam: Box? = Path.getPath(ilst, boxType)
nam = listener.prepareBox(nam)
ilst.addBox(nam)
var sizeAfter = moov.size
var diff = sizeAfter - sizeBefore
// This is the difference of before/after
// can we compensate by resizing a Free Box we have found?
if (freeBox.data.limit() > diff) {
// either shrink or grow!
freeBox.data = ByteBuffer.allocate((freeBox.data.limit() - diff).toInt())
sizeAfter = moov.size
diff = sizeAfter - sizeBefore
}
if (correctOffset && diff != 0L) {
correctChunkOffsets(moov, diff)
}
val baos = BetterByteArrayOutputStream()
moov.getBox(Channels.newChannel(baos))
isoFile.close()
val fc: FileChannel = if (diff != 0L) {
// this is not good: We have to insert bytes in the middle of the file
// and this costs time as it requires re-writing most of the file's data
splitFileAndInsert(videoFile, offset, sizeAfter - sizeBefore)
} else {
// simple overwrite of something with the file
RandomAccessFile(videoFile, "rw").channel
}
fc.position(offset)
fc.write(ByteBuffer.wrap(baos.buffer, 0, baos.size()))
fc.close()
}
#WorkerThread
#Throws(IOException::class)
fun splitFileAndInsert(f: File, pos: Long, length: Long): FileChannel {
val read = RandomAccessFile(f, "r").channel
val tmp = File.createTempFile("ChangeMetaData", "splitFileAndInsert")
val tmpWrite = RandomAccessFile(tmp, "rw").channel
read.position(pos)
tmpWrite.transferFrom(read, 0, read.size() - pos)
read.close()
val write = RandomAccessFile(f, "rw").channel
write.position(pos + length)
tmpWrite.position(0)
var transferred: Long = 0
while (true) {
transferred += tmpWrite.transferTo(0, tmpWrite.size() - transferred, write)
if (transferred == tmpWrite.size())
break
//System.out.println(transferred);
}
//System.out.println(transferred);
tmpWrite.close()
tmp.delete()
return write
}
#WorkerThread
private fun needsOffsetCorrection(isoFile: IsoFile): Boolean {
if (Path.getPath<Box>(isoFile, "moov[0]/mvex[0]") != null) {
// Fragmented files don't need a correction
return false
} else {
// no correction needed if mdat is before moov as insert into moov want change the offsets of mdat
for (box in isoFile.boxes) {
if ("moov" == box.type) {
return true
}
if ("mdat" == box.type) {
return false
}
}
throw RuntimeException("I need moov or mdat. Otherwise all this doesn't make sense")
}
}
#WorkerThread
private fun findFreeBox(c: Container): FreeBox? {
for (box in c.boxes) {
// System.err.println(box.type)
if (box is FreeBox)
return box
if (box is Container) {
val freeBox = findFreeBox(box as Container)
if (freeBox != null) {
return freeBox
}
}
}
return null
}
#WorkerThread
private fun correctChunkOffsets(movieBox: MovieBox, correction: Long) {
var chunkOffsetBoxes = Path.getPaths<ChunkOffsetBox>(movieBox as Box, "trak/mdia[0]/minf[0]/stbl[0]/stco[0]")
if (chunkOffsetBoxes.isEmpty())
chunkOffsetBoxes = Path.getPaths(movieBox as Box, "trak/mdia[0]/minf[0]/stbl[0]/st64[0]")
for (chunkOffsetBox in chunkOffsetBoxes) {
val cOffsets = chunkOffsetBox.chunkOffsets
for (i in cOffsets.indices)
cOffsets[i] += correction
}
}
private class BetterByteArrayOutputStream : ByteArrayOutputStream() {
val buffer: ByteArray
get() = buf
}
}
Sample usage for writing&reading title:
object MediaMetaData {
#JvmStatic
#Throws(IOException::class)
fun writeTitle(mediaFilePath: String, title: String) {
MediaMetaDataUtil.writeMetadata(mediaFilePath, AppleNameBox.TYPE, object : MediaMetaDataUtil.PrepareBoxListener {
override fun prepareBox(existingBox: Box?): Box {
var nam: AppleNameBox? = existingBox as AppleNameBox?
if (nam == null)
nam = AppleNameBox()
nam.dataCountry = 0
nam.dataLanguage = 0
nam.value = title
return nam
}
})
}
#JvmStatic
fun readTitle(mediaFilePath: String): String? {
return MediaMetaDataUtil.readMetadata<AppleNameBox>(mediaFilePath, AppleNameBox.TYPE)?.value
}
}

It seems there's no way to do it uniformly for all supported audio formats in Android. There are some limited options for particular formats though, so I suggest to stick with one format.
MP3 is the most popular one and there should be a lot of options like this one.
If you don't want to deal with encoding/decoding, there are some options for a WAV format.
There's also a way to add a metadata track to a MP4 container using MediaMuxer (you can have an audio-only MP4 file) or like this.
Regarding MediaStore: here's an example (at the end of page 318) on how to add metadata to it just after using MediaRecorder. Though as far as I know the data won't be recorded inside the file.
Update
I compiled an example app using this MP4 parser library and MediaRecorder example from SDK docs. It records an audio, puts it in MP4 container and adds String metadata like this:
MetaDataInsert cmd = new MetaDataInsert();
cmd.writeRandomMetadata(fileName, "lore ipsum tralalala");
Then on the next app launch this metadata is read and displayed:
MetaDataRead cmd = new MetaDataRead();
String text = cmd.read(fileName);
tv.setText(text);
Update #2
Regarding m4a file extension: m4a is just an alias for an mp4 file with AAC audio and has the same file format. So you can use my above example app and just change the file name from audiorecordtest.mp4 to audiorecordtest.m4a and change audio encoder from MediaRecorder.AudioEncoder.AMR_NB to MediaRecorder.AudioEncoder.AAC.