Develop Reference

How can I catch a stream from Webview and change it to VOICE_CALL Stream with AEC?

I am using Agora, and it has some issues. One of them is the speaker's voice comes out to the media sound.
On the browser, it can't control the media volume, So, I created an app to handle this. In the app, I dispatch the volume up/down button to control media volume.
However, this method created howling issue. So, I'd like to send the sound to STREAM_VOICE_CALL and use AEC(Acoustic Echo Cancellation) API on Android so that the sound comes out to the right stream and it can handle the echo problem.
what I wrote,
private fun enableVoiceCallMode() {
with(audioManager) {
volumeControlStream = AudioManager.STREAM_VOICE_CALL
setStreamVolume(
AudioManager.STREAM_VOICE_CALL,
audioManager.getStreamVolume(AudioManager.STREAM_VOICE_CALL),
0
)
}
}
But this didn't work.
And also, I tried to apply AEC like this:
private fun enableEchoCanceler() {
if (AcousticEchoCanceler.isAvailable() && aec == null) {
aec = AcousticEchoCanceler.create(audioManager.generateAudioSessionId())
aec?.enabled = true
} else {
aec!!.enabled = false
aec!!.release()
aec = null
}
}
private fun releaseEchoCanceler() {
aec!!.enabled = false
aec?.release()
aec = null
}
However, I don't know if AcousticEchoCanceler.create(audioManager.generateAudioSessionId()) is correct way or not.
please help me out.

About development using android camera

I'm trying to develop application that using the MediaRecorder API that runs on HMT-1.
Android Studio is used for development, and the operating environment is Android 10 or higher.
While shooting a video using the MediaRecoder API, we are verifying whether the same microphone can be used in another process such as the SpeechRecognizer API.
Recording processing alone with the MediaRecoder API and voice input alone with the SpeechRecognizer API can be performed without problems.
However, if you try to record and input voice at the same time, an error will occur.
If you want to use the input voice for multiple processes, please let me know if you have any reference documents or samples.
MediaRecorder settings.
path = getExternalFilesDir(null)!!.path
mMediaRecorder = MediaRecorder()
mMediaRecorder!!.setAudioSource(MediaRecorder.AudioSource.MIC)
mMediaRecorder!!.setVideoSource(MediaRecorder.VideoSource.SURFACE)
mMediaRecorder!!.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
mMediaRecorder!!.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB)
mMediaRecorder!!.setAudioEncodingBitRate(16)
mMediaRecorder!!.setAudioSamplingRate(44100)
mMediaRecorder!!.setVideoSize(1024, 768)
mMediaRecorder!!.setVideoEncoder(MediaRecorder.VideoEncoder.H264)
mMediaRecorder!!.setVideoEncodingBitRate(10000000)
mMediaRecorder!!.setOutputFile(path + "/" + DateFormat.format("yyyyMMdd'-'kkmmss", Calendar.getInstance()) + ".mp4")
mMediaRecorder!!.setOnInfoListener(this)
mMediaRecorder!!.setMaxDuration(VIDEO_DURATION)
mMediaRecorder!!.setMaxFileSize(VIDEO_FILESIZE)
mMediaRecorder!!.setPreviewDisplay(mSurfaceHolder!!.surface)
val rotation = (getSystemService(WINDOW_SERVICE) as WindowManager)
if(rotation.defaultDisplay.rotation == 2){
mMediaRecorder!!.setOrientationHint(180)
}
mMediaRecorder!!.prepare()
try {
mMediaRecorder!!.start()
} catch (ex: IOException) {
ex.printStackTrace()
mMediaRecorder!!.release()
}
SpeechRecognizer settings.
mSpeechRecognizer = SpeechRecognizer.createSpeechRecognizer(applicationContext)
mSpeechRecognizer?.setRecognitionListener(createRecognitionListenerStringStream { recognize_text_view.text = it })
public fun onStart(View: View){
mSpeechRecognizer?.startListening(Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH))
}
public fun onStop(View: View){
mSpeechRecognizer?.stopListening()
}

Android Call Recording Incoming voice not getting recorded

I'm working auto call recorder app, I'm able to record voice call on below android 6 using MediaRecorder.AudioSource.VOICE_CALL,
From android 6 not able to record voice call using VOICE_CALL. I managed to record using MediaRecorder.AudioSource.MIC but here incoming voice not getting recorded and I want to record voice call in normal mode not in speaker on mode. Please help me on this. (I had tried on Xiomi Redmi 4a(android 6),not working).
myRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
myRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
myRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
myRecorder.setMaxDuration(60 * 60 * 1000);
AudioManager audiomanager =
(AudioManager)getSystemService(AUDIO_SERVICE);
audiomanager.setMode(2);
Edit : There is no issue with permissions.
Update : Anyone knows how to forcing another stream to MIC audio source. This requires native android code. Please help me on this
Refer this question for more details on routing audio

You need to use ndk. Here are examples of the functions that need to be done.
Load libmedia.so and libutils.so
int load(JNIEnv *env, jobject thiz) {
void *handleLibMedia;
void *handleLibUtils;
int result = -1;
lspr func = NULL;
pthread_t newthread = (pthread_t) thiz;
handleLibMedia = dlopen("libmedia.so", RTLD_NOW | RTLD_GLOBAL);
if (handleLibMedia != NULL) {
func = dlsym(handleLibMedia, "_ZN7android11AudioSystem13setParametersEiRKNS_7String8E");
if (func != NULL) {
result = 0;
}
audioSetParameters = (lasp) func;
} else {
result = -1;
}
handleLibUtils = dlopen("libutils.so", RTLD_NOW | RTLD_GLOBAL);
if (handleLibUtils != NULL) {
fstr = dlsym(handleLibUtils, "_ZN7android7String8C2EPKc");
if (fstr == NULL) {
result = -1;
}
} else {
result = -1;
}
cmd = CM_D;
int resultTh = pthread_create(&newthread, NULL, taskAudioSetParam, NULL);
return result;}
Function setParameters
int setParam(jint i, jint as) {
pthread_mutex_lock(&mt);
audioSession = (int) (as + 1);
kvp = "input_source=4";
kvps = toString8(kvp);
cmd = (int) i;
pthread_cond_signal(&cnd);
pthread_mutex_unlock(&mt);
return 0;}
Task AudioSetParameters
void *taskAudioSetParam(void *threadid) {
while (1) {
pthread_mutex_lock(&mt);
if (cmd == CM_D) {
pthread_cond_wait(&cnd, &mt);
} else if (audioSetParameters != NULL) {
audioSetParameters(audioSession, kvps);
}
pthread_mutex_unlock(&mt);
}
}
There is a library and an example of use https://github.com/ViktorDegtyarev/CallRecLib

Xiaomi devices always have problems with permission request even run-time or install-time.
I have an Xiaomi Redmi 3 pro, and it always force to Deny some permission when I install apps, so I must manually Allow it.
If your problem is the same, I found some workaround solution and it worked for me: How to get MIUI Security app auto start permission programmatically?

First these 3 permissions are needed in Manifest as well as a runtime permission request if the device is above Marshmallow,
<uses-permission android:name="android.permission.READ_PHONE_STATE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.CAPTURE_AUDIO_OUTPUT" />
MediaRecorder.AudioSource.VOICE_CALL is not supported on all phones so you need to continue using MediaRecorder.AudioSource.MIC.
I use this and works fine on most of the devices,
recorder = new MediaRecorder();
recorder.setAudioSource(audioSource);
recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
recorder.setOutputFile(your_path);
You need to set this to record your calls properly,
audioManager.setMode(AudioManager.MODE_IN_CALL);
raise volume level when you start recording
audioManager.setStreamVolume(AudioManager.STREAM_VOICE_CALL,audioManager.getStreamMaxVolume(AudioManager.STREAM_VOICE_CALL), 0);
When you stop recording set the mode to normal,
audioManager.setMode(AudioManager.MODE_NORMAL); and also set the stream volume to back how it was.

This could be a Permission related issue.
With the introduction of Android 6.0 Marshmallow, the app will not be granted any permission at installation time. Instead, the application has to ask the user for a permission one-by-one at run-time.
I hope you have included the code which explicitly asks for permissions on devices with Marshmallow and above.

In automatic call recorder (callU) have a option "SoundFX" If Enable Record Calls Two Side
Link

try
MediaRecorder.AudioSource.VOICE_COMMUNICATION
and see
https://androidforums.com/threads/android-phone-with-call-recording-function.181663/

How to check whether microphone is used by any background app

I have been searching for couple of days now and havent been able to find a suitable solution.
I am trying to check if any app in the background is using the microphone, so my app can use it, otherwise i want just to show message "Microphone in use by another app".
I tried checking all the applications in the background and their permissions but that doesnt solve my problem, since there is package wearable.app which asks for the permissions but it doesnt affect the audio, or it is not using it.
I tried the other solutions that i was able to find here or on google, but none of that seems to be the proper way.
All i want to check if the microphone is not being used, so my app can use it.
Any suggestion i will appreciate.

After searching more i found the solution and i am adding it here for anyone that needs it to find it easier.
private boolean validateMicAvailability(){
Boolean available = true;
AudioRecord recorder =
new AudioRecord(MediaRecorder.AudioSource.MIC, 44100,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_DEFAULT, 44100);
try{
if(recorder.getRecordingState() != AudioRecord.RECORDSTATE_STOPPED ){
available = false;
}
recorder.startRecording();
if(recorder.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING){
recorder.stop();
available = false;
}
recorder.stop();
} finally{
recorder.release();
recorder = null;
}
return available;
}

You can do it the other way around.
Get the microphone in your app.
Get a list of the installed apps, who have a RECORD permission.
Then check if one of these apps is on the foreground and if there is one release the microphone so that the other app can use it (for example when a phone call occurs).
A bit dirty practice but I think it is what you are looking for.
Cheers!

This is how is done
AudioManager am = (AudioManager)context.getSystemService(Context.AUDIO_SERVICE);
if(am.getMode()==AudioManager.MODE_IN_COMMUNICATION){
//Mic is in use
}
MODE_NORMAL -> You good to go. Mic not in use
MODE_RINGTONE -> Incoming call. The phone is ringing
MODE_IN_CALL -> A phone call is in progress
MODE_IN_COMMUNICATION -> The Mic is being used by another application

AudioManager.AudioRecordingCallback()
am.registerAudioRecordingCallback(new AudioManager.AudioRecordingCallback() {
#Override
public void onRecordingConfigChanged(List<AudioRecordingConfiguration> configs) {
super.onRecordingConfigChanged(configs);
try {
isMicOn = configs.get(0) != null;
}catch (Exception e)
{
isMicOn = false;
}
if (isMicOn) {
//microphone is on
} else {
// microphone is off
}
Toast.makeText(context, isMicOn ? "Mic on" : "Mic off", Toast.LENGTH_SHORT).show();
}
}, null);

I know this may sound a bit tedious or the long way... But have you considered recording a logcat? Record a log for both Kernel and apps. Recreate the issue, then compare both logs to see what program is occupied when the kernel utilizes the mic.

Since sharing audio input behaviour varies depending on Android versions, this answer aims to provide a complete solution based on the docs.
Pre-Android 10
Before Android 10 the input audio stream could only be captured by one
app at a time. If some app was already recording or listening to
audio, your app could create an AudioRecord object, but an error would
be returned when you called AudioRecord.startRecording() and the
recording would not start.
So, you can use this function to check if the mic is used by another app for pre Android 10 versions.
private fun isAnotherAppUsingMic(): Boolean {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) return false
createRecorder().apply {
try {
startRecording()
if (recordingState != AudioRecord.RECORDSTATE_RECORDING) {
return true
}
stop()
return false
} catch (e: IllegalStateException) {
return true
} finally {
release()
}
}
}
private fun createRecorder(): AudioRecord {
return AudioRecord(
MediaRecorder.AudioSource.MIC,
SAMPLE_RATE_HZ,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
2 * AudioRecord.getMinBufferSize(
SAMPLE_RATE_HZ,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT
)
)
}
const val SAMPLE_RATE_HZ = 44100
Android 10 and above
Android 10 imposes a priority scheme that can switch the input audio
stream between apps while they are running. In most cases, if a new
app acquires the audio input, the previously capturing app continues
to run, but receives silence.
So, for Android versions 10 and higher, in most cases your app will take priority if there is another app like voice or screen recorder is already running and then you start using mic in your app. But you will need to check for Voice/Video call as it has higher priority and mic won't be available for your app (it will receive silence). You can use below code to check if there is an active call:
private fun isVoiceCallActive(): Boolean {
val audioManager = context.getSystemService(Context.AUDIO_SERVICE) as AudioManager
return audioManager.mode in listOf(
AudioManager.MODE_IN_CALL,
AudioManager.MODE_IN_COMMUNICATION
)
}
In summary you can merge above two function to check if mic is available before you want to use it.
fun isMicAvailable() = !isAnotherAppUsingMic() && !isVoiceCallActive()

OpenSLES SLPlaybackRateItf SetRate not working on Android 5.x, but works on Android 4.x

OpenSLES SLPlaybackRateItf SetRate works on Android 4.x, but not on Android 5.x.
It seems as if the SetRate interface has no implementation on Android 5.x.
Below are some code snippets, and the complete code can be found here: https://github.com/jimrange/cocos2d-x/blob/audioEngineUpdate/cocos/audio/android/AudioEngine-inl.cpp
SLPlaybackRateItf _fdPlayerPlaybackRate;
SLObjectItf _fdPlayerObject;
Other initialization code not shown for brevity.
// get the playback rate interface
result = (*_fdPlayerObject)->GetInterface(_fdPlayerObject, SL_IID_PLAYBACKRATE, &_fdPlayerPlaybackRate);
if(SL_RESULT_SUCCESS != result){ ERRORLOG("get the rate interface failed"); break; }
SLpermille stepSize;
SLuint32 capabilities;
auto rangeResult = (*_fdPlayerPlaybackRate)->GetRateRange(_fdPlayerPlaybackRate, 0, &_minRate, &_maxRate, &stepSize, &capabilities);
if(SL_RESULT_SUCCESS != rangeResult)
{
log("%s error:%u",__func__, rangeResult);
}
At this point
_minRate == 500
_maxRate == 2000
stepSize == 0
capabilities == SL_RATEPROP_NOPITCHCORAUDIO
Later when calling the following, the result is SL_RESULT_SUCCESS, but the playback rate does not change.
void AudioEngineImpl::setPitch(int audioID,float pitch)
{
auto& player = _audioPlayers[audioID];
SLpermille playbackRate = (SLpermille)1000 * pitch;
if (playbackRate < player._minRate)
{
log("Warning: AudioEngine attempting to set rate:%d which is lower than minRate=%d.", playbackRate, player._minRate);
playbackRate = player._minRate;
}
else if (playbackRate > player._maxRate)
{
log("Warning: AudioEngine attempting to set rate:%d which is higher than maxRate=%d.", playbackRate, player._maxRate);
playbackRate = player._maxRate;
}
// This works on Android 4.4.4 on Nexus 7, but not on Android 5.x on Nexus 7 or Nexus 9.
auto result = (*player._fdPlayerPlaybackRate)->SetRate(player._fdPlayerPlaybackRate, playbackRate);
if(SL_RESULT_SUCCESS != result)
{
log("%s error:%u",__func__, result);
}
}
Here is what I am experiencing:
Requesting the SL_IID_PLAYBACKRATE interface returns SL_RESULT_SUCCESS.
I can query the interface for capabilities and I get a range of 500 to 2000, a step size of 0 and a capabilities flag of SL_RATEPROP_NOPITCHCORAUDIO. This is the same on Android 4.x and 5.x.
So the interface is telling me all is setup good and ready to go.
Calling SetRate returns SL_RESULT_SUCCESS on Android 4.x and Android 5.x.
But on 5.x the SetRate does not have any effect on the audio.
There is another SO post that suggests using SL_RATEPROP_PITCHCORAUDIO, but I want to use SL_RATEPROP_NOPITCHCORAUDIO so that there is no pitch correction when changing the rate of the audio. And SL_RATEPROP_NOPITCHCORAUDIO is the default setting, so no need to change that.
From the OpenSL_ES_Specification_1.0.1.pdf:
SL_RATEPROP_NOPITCHCORAUDIO - Plays audio at the current rate, but without pitch correction.
SL_RATEPROP_PITCHCORAUDIO - Plays audio at the current rate, but with pitch correction.
I am working on code to expand the Cocos2d-x AudioEngine.
The Cocos2d-x version is 3.7rc0 and the NDK version is ndk-r10d.