Develop Reference

Save video in every 5 second interval while video recoding is ON (Android OS)

I want to save the video every 5 seconds while the video recording is ON.
I have tried many solutions but I am facing a Glitch that is, the Last Saved Frame remains in preview for around 300ms.
I think the reason is in MediaRecorder class "Once a recorder has been stopped, it will need to be completely reconfigured and prepared before being restarted."
Thanks

I think it's impossible to do that with MediaRecorder. The better approach could be encoding video by using MediaCodec and storing encoded content bt using MediaMuxer.
Grafika is a project on Google Github account which is a dumping ground for Android graphics & media hacks. In this project, you can find good examples of using both MediaCodec and MediaMuxer classes.
I forked the Grafika project and did some modifications to support sequential segmented recording. You can find it here. When you run the application, select Show + capture camera item from the list and then set Output Segment Duration for example to 5 and then press Start recording button.
Please look at VideoEncoderCore and CameraCaptureActivity classes source code to find how it works. You can find here how it segments live camera feed to different files.

"I think the reason is in MediaRecorder class, "Once a recorder has been stopped, it will need to be completely reconfigured and prepared before being restarted"."
You can use multiple mediaMuxer's to encode separate files.
The camera should send data to fill a MediaMuxer object (which itself produces an .mp4 file).
When needed, you can start writing the Camera data to a second (different) MediaMuxer thus automatically creating a second new .mp4 file (on begin usage of the muxer).
The first MediaMuxer can then close and save its file. Your first segment is ready...
If needed, try to study this code for a guide on using Camera with mediaMuxer:
https://bigflake.com/mediacodec/CameraToMpegTest.java.txt
So you have a function that handles things when the 5 second interval has passed? In that function, could cycle the recording between two muxers, giving one a chance to close its file, while the other records the next segment and then vice-versa).
Instead of something like below (using MediaRecorder.OutputFormat.MPEG_4):
this.mMediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
You will instead create a new muxer (with MUXER_OUTPUT_MPEG_4):
//# create a new File to ssave into
File outputFile = new File(OUTPUT_FILENAME_DIR, "/yourFolder/Segment" + "-" + mySegmentNum + ".mp4");
String outputPath = outputFile.toString();
int format = MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4;
try { mMuxer = new MediaMuxer(outputPath, format); }
catch (IOException e) { Log.e(TAG, e.getLocalizedMessage()); }
And you stop a muxer with:
mMuxer1.stop(); mMuxer1.release();
PS:
Another option is to use Threads to run multiple MediaRecorders. It might help your situation. See the Android Background Process guide.

An AAC audio stream is playable in VLC for Android, but not in Exoplayer

I have an RTMP stream I want to play in my app using the Exoplayer library. My setup for that is as follows:
TrackSelector trackSelector = new DefaultTrackSelector();
RtmpDataSourceFactory rtmpDataSourceFactory = new RtmpDataSourceFactory(bandwidthMeter);
ExtractorsFactory extractorsFactory = new DefaultExtractorsFactory();
factory = new ExtractorMediaSource.Factory(rtmpDataSourceFactory);
factory.setExtractorsFactory(extractorsFactory);
createSource();
mPlayer = ExoPlayerFactory.newSimpleInstance(mActivity, trackSelector, new DefaultLoadControl(
new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE),
1000, // min buffer
3000, // max buffer
1000, // playback
2000, //playback after rebuffer
DefaultLoadControl.DEFAULT_TARGET_BUFFER_BYTES,
true
));
vwExoPlayer.setPlayer(mPlayer);
mPlayer.addListener(mVideoStreamHandler);
mPlayer.addVideoListener(new VideoListener() {
#Override
public void onVideoSizeChanged(int width, int height, int unappliedRotationDegrees, float pixelWidthHeightRatio) {
Log.d("hasil", "onVideoSizeChanged: w:" + width + ", h:" + height);
String res = width + "x" + height;
resolution.setText(res);
}
#Override
public void onRenderedFirstFrame() {
}
});
Where createSource() is as follows:
private void createSource() {
mMediaSource180 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_180));
mMediaSource360 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_360));
mMediaSource720 = factory.createMediaSource(Uri.parse(API.GAME_VIDEO_STREAM_URL_720));
mMediaSourceAudio = factory.createMediaSource(Uri.parse(API.GAME_AUDIO_STREAM_URL));
}
My current problem is that only the first three ExtractorMediaSources work fine in Exoplayer. The mMediaSourceAudio refuses to play in Exoplayer, but works just fine in the VLC Media Player for Android.
Right now I have a suspicion that the format is AAC-LTP, or whatever AAC variant that requires a codec available in VLC but not in default Android. However, I do not have access to the encoding process so I don't know for sure.
If this isn't the case, what is it?
EDIT:
I've been debugging the BandwidthMeter and added a MediaSourceEventListener. When I use the normal Video sources, onDownstreamFormatChanged() gets called, but not when I use that Audio Stream source.
In addition, the BandwidthMeter works fine, with bytes always downloaded in all parts of the stream and more bytes when the video stream comes in, but only in the Audio only stream that, when I call mPlayer.getBufferedPosition(), the returned value is always 0. Also, when I use the Audio Stream source, no OMX code was called - no decoders were set up.
Am I seeing a malformed audio stream, or do I need to change my Exoplayer's settings?
EDIT 2:
Further debugging reveals that, in all the Video streams and Audio stream, the same FlvExtractor is used. Even though the Video streams have the avc video track encoding and mp4a-latm audio track encoding. Is this normal?

Turns out it's because the stream was recognized to have two tracks/sampleQueues. One Audio track, and one track with null format. That null track was supposed to be the video track, which was supposed to exist according to the stream's flvHeader flag.
For now, I get around this by creating a custom MediaSource using a custom MediaPeriod. Said custom MediaPeriod having code to separate the video and audio tracks of the SampleQueues, then using the audio-only SampleQueue[] instead of the source SampleQueue[] when I want to play the audio-only stream.
Though this gives me another point of concern: There's something one can do to alter the 'has audio track (flag & 0x04) and video track (flag & 0x01)' flag in the rtmp stream, right?

Thanks for the comments, I'm new to ExoPlayer. But your comments helped me in debugging and getting multiple workarounds to the issue.
I tried to use custom MediaSource and custom MediaPeriod to address this audio issue. I have observed video format data coming after audio data incase of video+audio wowza stream, so the function maybeFinishPrepare() will wait for getting both video and audio format tag data before invoking onPrepared, incase if video tagData is received first. Incase of audio data received first, it wont wait and will call onPrepare().
With the above changes, I was able to play audio alone and video_audio wowza streams, where rtmp tagHeader with tagTypes were coming in the order of video tagData and then followed by audio data.
I wasn't able to use the same patch with srs server to play both audio_only and video_audio streams with the same changes. srs server is giving tagData in the order of audio and then video tagData,
So, I debugged further in FlvExtractor. In readFlvHeader, I have overriden the hasAudio and hasVideo variables. These variables will be set based on the first few tagHeaders(5 or 6). I used peekFully on input for 6 times in a loop. In each loop after fetching tagType and tagDataSize, tagDataSize is used to input.advancePeekPosition(), and tagType is used to identify whether we have audio/video format data in tagData. After peeking for first 6 consecutive tagHeaders, I was able to get actual values of hasAudio and hasVideo, and ignored the flvHeaders.flags, which were used to set these variables.
Custom FlvExtractor workaround, looked cleaner than custom MediaSource/MediaPeriod, as we will create those many tracks as necessary, as we are setting proper hasVideo/hasAudio values.

How to correctly send audio files to Google Speech API?

I'm trying to implement Google Speech API in Android by following this demo: https://github.com/GoogleCloudPlatform/android-docs-samples
I was able to successfully reproduce the example in my app by using the given "audio.raw" file located in R.raw, and everything works perfectly. However, when I try to use my own audio files, it returns "API successful" without any transcription text. I'm not sure if it has to do with the files' path or the encoding, so I'll include information on both just in case.
Encoding
My audio files are obtained by recording a voice through MediaRecorder. These are the settings:
myAudioRecorder = new MediaRecorder();
myAudioRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
myAudioRecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
myAudioRecorder.setAudioEncoder(MediaRecorder.OutputFormat.AMR_WB);
myAudioRecorder.setAudioSamplingRate(16000);
myAudioRecorder.setAudioEncodingBitRate(16000);
myAudioRecorder.setAudioChannels(1);
myAudioRecorder.setOutputFile(outputFile);
SpeechService's recognizeInputStream() function in the API:
mApi.recognize(
RecognizeRequest.newBuilder()
.setConfig(RecognitionConfig.newBuilder()
.setEncoding(RecognitionConfig.AudioEncoding.AMR_WB) //originally it was LINEAR16
.setLanguageCode("en-US")
.setSampleRateHertz(16000)
.build())
.setAudio(RecognitionAudio.newBuilder()
.setContent(ByteString.readFrom(stream))
.build())
.build(),
mFileResponseObserver);
Encoding guidelines by Google: https://cloud.google.com/speech/docs/best-practices
From what I understand, I can use AMR_WB and 16kHz instead of the default LINEAR16, I'm just not sure if I'm doing it right.
Path
This is the example that is fully working (with the audio file from the repo):
mSpeechService.recognizeInputStream(getResources().openRawResource(R.raw.audio));
However, none of the following options work, even with the exact same file:
InputStream inputStream = new URL("[website]/test/audio.raw").openStream();
mSpeechService.recognizeInputStream(inputStream);
Neither:
Uri uri = Uri.parse("android.resource://[package]/raw/audio");
InputStream inputStream = getActivity().getContentResolver().openInputStream(uri); //"getActivity()" because this is in a Fragment
mSpeechService.recognizeInputStream(inputStream);
To be clear, the result on the above paths is the same as on my custom audio files: "API successful" with no transcription. One of the options I have tried for my custom audio files, with the same thing happening, is this:
FileInputStream fis = new FileInputStream(filePath);
mSpeechService.recognizeInputStream(fis);
The only reason I'm not 100% sure the problem is in the path is because if the API is returning with success, then the file was found in the specified path. The problem should be the encoding, but then it's weird that the same file ("audio.raw") sent in different ways produces different results.
Anyway, thank you in advance! :)
EDIT:
To be clear, it's not that it returns an empty string in the transcription. It just never enters the "onSpeechRecognized" function that also exists in the demo, so no transcription is given.

Playing mp3 data compressed by lame mp3 with JLayer and Audiotrack in android

I have followed this example to convert raw audio data coming from AudioRecord to mp3, and it happened successfully, if I store this data in a file the mp3 file and play with music player then it is audible.
Now my question is instead of storing mp3 data to a file i need to play it with AudioTrack, the data is coming from the Red5 media server as live stream, but the problem is AudioTrack can only play PCM data, so i can only hear noise from my data.
Now i am using JLayer to my require task.
My code is as follows.
int readresult = recorder.read(audioData, 0, recorderBufSize);
int encResult = SimpleLame.encode(audioData,audioData, readresult, mp3buffer);
and this mp3buffer data is sent to other user by Red5 stream.
data received at other user is in form of stream, so for playing it the code is
Bitstream bitstream = new Bitstream(data.read());
Decoder decoder = new Decoder();
Header frameHeader = bitstream.readFrame();
SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream);
short[] pcm = output.getBuffer();
player.write(pcm, 0, pcm.length);
But my code freezes at bitstream.readFrame after 2-3 seconds, also no sound is produced before that.
Any guess what will be the problem? Any suggestion is appreciated.
Note: I don't need to store the mp3 data, so i cant use MediaPlayer, as it requires a file or filedescriptor.

just a tip, but try to
output.close();
bitstream.closeFrame();
after yours write code. I'm processing MP3 same as you do, but I'm closing buffers after usage and I have no problem.
Second tip - do it in Thread or any other Background process. As you mentioned these deaf 2 seconds, media player may wait until you process whole stream because you are loading it in same thread.
Try both tips (and you should anyway). In first, problem could be in internal buffers; In second you probably fulfill Media's input buffer and you locked app (same thread, full buffer cannot receive your input and code to play it and release same buffer is not invoked because writing locks it...)
Also, if you don't doing it now, check for 'frameHeader == null' due to file end.
Good luck.

You need to loop through the frames like this:
While (frameHeader = bitstream.readFrame()){
SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream);
short[] pcm = output.getBuffer();
player.write(pcm, 0, pcm.length);
bitstream.close();
}
And make sure you are not running them on main thread.(This is probably the reason of freezing.)

Using MediaRecorder to write to a buffer or FIFO

I am developing a low data rate VoIP kind of project . I need to capture audio at low data rates and store it in an internal buffer or FIFO (NOT in a file).
I would like to use low data rate .AMR encoders, which means AudioRecord is out. MediaRecorder looks like it does exactly what I want except that it seems to write to a file.
MediaRecorder takes a FileDescriptor... is there any way I can write a class that implements the FileDescriptor interface... acting as a sync for bytes... but instead of sending them to a file they are stored in a buffer? The documentation on FileDescriptor specifically says that Applications shouldn't write their own but why not and is it possible anyway?
http://docs.oracle.com/javase/1.4.2/docs/api/java/io/FileDescriptor.html
In short, I'd like to develop my own stream, and trick MediaRecorder to send data to it. Perhaps doing something tricky with opening both ends of a socket within the same APK and giving MediaRecorder the socket to write to? Using the socket as my FIFO? I'm somewhat new to this so any help/suggestions greatly appreciated.
I have a related question on the RX side. I'd like to have a buffer/fifo that feeds MediaPlayer. Can I trick MediaPlayer to accept data from a buffer fed by my own proprietary stream?

I know its a bit late to answer this question now...
...But if it helps here's the solution.
Android MediaRecorder's method setOutputFile() accepts FileDescriptor as a parameter.
As for your need a unix data pipe could be created and its FD could be passed as an argument in the following manner...
mediaRecorder.setOutputFile(getPipeFD());
FileDescriptor getPipeFD()
{
final String FUNCTION = "getPipeFD";
FileDescriptor outputPipe = null;
try
{
ParcelFileDescriptor[] pipe = ParcelFileDescriptor.createPipe();
outputPipe = pipe[1].getFileDescriptor();
}
catch(Exception e)
{
Log.e(TAG, FUNCTION + " : " + e.getMessage());
}
return outputPipe;
}
The ParcelFileDescriptor.createPipe() creates a Unix Data Pipe and returns an array of ParcelFileDescriptors. The first object refers to the read channel (Source Channel) and the second one refers to the write channel (Sink Channel) of the pipe. Use MediaRecorder object to write the recorded data to the write channel...
As far as MediaPlayer is concerned the same technique could be used by passing the FileDescriptor object related to the created pipe's read channel to the setDataSource() method...