I have been trying for a long time to render a video on a "Surface" class using MediaPlayer class. It was playing audio, but not the video. Everywhere I search, people talk about SurfaceView and SurfaceHolder but I have only a Surface object. How to crack this blocker?
This is how I tried,
public class SampleVideoPlayer{
private Uri mUrl;
private Surface mSurface;
private MediaPlayer mMediaPlayer;
private Context mContext;
public SampleVideoPlayer(Context context, String url, Surface surface){
mUrl = Uri.parse(url);
mSurface = surface;
mMediaPlayer = new MediaPlayer();
mContext = context;
}
public void playVideo() throws IOException {
mMediaPlayer.setDataSource(mContext, mUrl);
mMediaPlayer.setSurface(mSurface);
mMediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener(){
#Override
public void onPrepared(MediaPlayer mediaPlayer) {
mediaPlayer.start();
}
});
mMediaPlayer.prepareAsync();
}
}
Adding the Session Object I am passing,
public class MyTvSession extends TvInputService.Session implements Handler.Callback {
Context mContext;
String vidUrl;
Surface mSurface;
SampleVideoPlayer player = null;
SampleMediaPlayer mediaPlayer;
public MyTvSession(Context context){
super(context);
ChannelXmlReader reader = new ChannelXmlReader(context);
ArrayList<Channel> channels = reader.ReadXml();
mContext = context;
vidUrl = channels.get(0).url;
}
#Override
public boolean handleMessage(Message message) {
Log.d("HANDLE MESSAGE", message.toString());
return true;
}
#Override
public void onRelease() {
}
#Override
public boolean onSetSurface(Surface surface) {
if(surface != null)
Log.d("NOT NULL from SESSION", "NOTNULL");
mSurface = surface;
return true;
}
#Override
public void onSurfaceChanged(int format, int width, int height) {
super.onSurfaceChanged(format, width, height);
if(mediaPlayer != null)
mediaPlayer.mMediaPlayer.setSurface(mSurface);
Log.d("ONSURFACECHANGED", "Event");
}
#Override
public void onSetStreamVolume(float v) {
}
#Override
public boolean onTune(Uri uri) {
Log.d("TUNING CHANNEL", uri.toString());
try {
mediaPlayer = new SampleMediaPlayer(mContext, vidUrl, mSurface);
mediaPlayer.playVideo();
}catch(Exception e){
Log.d("MPEXCEPTION", Log.getStackTraceString(e));
}
return true;
}
#Override
public void onSetCaptionEnabled(boolean b) {
}
}
The Surface class is a thin wrapper around a buffer list shared with the backing surfaceflinger process, which is responsible for rendering to the display.
You can get one of these using the SurfaceView and its SurfaceHolder, which are tied to the lifecycle of the view. So be sure to get it after being called back when the surface has been created.
Alternatively, you can use a SurfaceTexture which is created using your own custom OpenGL context. With this approach you can render using your own OpenGL code or even pass it off to the media engine for rendering. You can also get a SurfaceTexture tied to the view subsystem by using TextureView (but like SurfaceView you'll need to use it at the appropriate time in its lifecycle.)
I have exactly the same problem. But it only happens on Philips TV. The same code runs fine on every other Android TV devices. The surface I get in onSetSurface is valid, sound is playing, but picture is black. When I close the app, the video is visible for a second. It seems to be in the background.
Related
I want to add "Fullscreen" button to the standard SimpleExoPlayer implementation from Google. I added the ImageButton and everything works well so far.
Then I decided that fullscreen will work in next way: after pressing on button, I open new activity with fullscreen SimpleExoPlayerView, pass there video uri and current position, initialize player and seek for given position. It works, but the player re-initialization takes 1-3 seconds depending on a device what don't want to have.
Now I would like to have an instance of player inside intent service and just reattach existing player to any view that wants to show it (preview or fullscreen view), like this:
mPlayerView.setPlayer(mPlayer);
The problem is that service will have the player and activity will have the view. No one of them will have both to be able to attach player to view. As a workaround, I think about making that the service class may have a static link to the player, so activity will be able to get it via static reference. But this seems like some code smell and I don't know if there won't be problems with communications between threads.
So, how can I pass the player from Service (that is not Serializable or Parcelable) to Activity or how can I pass a player view to the Service?
I have faced a similar issue with Video Playback using MediaPlayer. I have bound to MediaBrowserService using IBinder and then fetched the MediaPlayer instance. In your service provide a method that returns a reference to MediaPlayer. Something like this:
private MediaPlayer mediaPlayer;
#Override
public IBinder onBind(Intent intent) {
if (intent.getAction().equals("YOUR_INTENT")) {
return new LocalBinder();
}
return super.onBind(intent);
}
public class LocalBinder extends Binder{
public AudioService getService(){
return AudioService.this;
}
}
public MediaPlayer getMediaPlayer() {
return mediaPlayer;
}
In your Activity/Fragment then bind to MediaBrowserService using IBinder. In my implementation, I have used MediaPlayer, but I think in similar ways it can be used for Exoplayer.
public class MainActivity extends AppCompatActivity implements SurfaceHolder.Callback{
private MediaPlayer mediaPlayer;
private SurfaceView surfaceView;
private SurfaceHolder surfaceHolder;
private boolean isServiceBounded;
private boolean isSurfaceReady;
private ServiceConnection serviceConnection = new ServiceConnection() {
#Override
public void onServiceConnected(ComponentName name, IBinder service) {
isServiceBounded = true;
mediaPlayer = ((AudioService)service).getMediaPlayer();
if (isSurfaceReady) {
mediaPlayer.setDisplay(surfaceHolder);
}
}
#Override
public void onServiceDisconnected(ComponentName name) {
isServiceBounded = false;
}
};
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
surfaceView = findViewById(R.id.surfaceView);
surfaceHolder = surfaceView.getHolder();
surfaceHolder.addCallback(this);
bindService(new Intent("YOUR_INTENT"), serviceConnection, BIND_AUTO_CREATE);
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
surfaceHolder = holder;
isSurfaceReady = true;
if (mediaPlayer != null) {
mediaPlayer.setDisplay(holder);
}
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
}
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
isSurfaceReady = false;
if (mediaPlayer != null) {
mediaPlayer.setDisplay(null);
}
surfaceHolder = null;
}
#Override
protected void onDestroy() {
super.onDestroy();
unbindService(serviceConnection);
}
}
exoPlayer handle fullscreen good enough - try to change your layout params of playerView to match the screen on rotation or pressing button.
#Override
public void onConfigurationChanged(Configuration newConfig) {
super.onConfigurationChanged(newConfig);
int width;
int height;
View decorView = getWindow().getDecorView();
int uiOptions;
if(newConfig.orientation == Configuration.ORIENTATION_PORTRAIT){
Resources r = getResources();
float px = TypedValue.applyDimension(TypedValue.COMPLEX_UNIT_DIP, 300, r.getDisplayMetrics());
width = CoordinatorLayout.LayoutParams.MATCH_PARENT;
height = (int)px;
uiOptions = View.SYSTEM_UI_FLAG_VISIBLE;
} else {
uiOptions = View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION
| View.SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN
| View.SYSTEM_UI_FLAG_HIDE_NAVIGATION
| View.SYSTEM_UI_FLAG_FULLSCREEN
| View.SYSTEM_UI_FLAG_IMMERSIVE_STICKY;
}
decorView.setSystemUiVisibility(uiOptions);
final CoordinatorLayout.LayoutParams lp = new CoordinatorLayout.LayoutParams(width, height);
mAppBar.setLayoutParams(lp);
}
Also don't forget to add this to your activity manifest to prevent activity reload:
android:configChanges="keyboard|keyboardHidden|orientation|screenSize|screenLayout|smallestScreenSize|uiMode"
I want to run the front camera of a device from a background service, getting the captured frames by the camera in the background service using the callback that's called whenever a new frame is captured by the camera (eg, onPreviewFrame), and apply some real time processing on the obtained frames.
After searching I understand that there are two ways to run the camera in the background:
1- setting a SurfaceView with size 1 X 1.
2- Using SurfaceTexture (This does not require any view which seems more efficient.)
I tried both ways, I could only run the camera from the background, but the callbacks for getting frames were never called, though I tried different tricks.
The code I used for method 1 is available here:
https://stackoverflow.com/questions/35160911/why-onpreviewframe-is-not-being-called-from-a-background-service
Note that I've also tried to call onPreviewFrame inside surfaceChanged, but I always get that the camera is null when trying to use it to set the callback! I even tried to re-instantiate it as camera = Camer.open(), but I got an error saying "Fail to connect to camera service."
Here is the code for method 2:
public class BackgroundService extends Service implements TextureView.SurfaceTextureListener {
private Camera camera = null;
private int NOTIFICATION_ID= 1;
private static final String TAG = "OCVSample::Activity";
// Binder given to clients
private final IBinder mBinder = new LocalBinder();
private WindowManager windowManager;
private SurfaceTexture mSurfaceTexture= new SurfaceTexture (10);
public class LocalBinder extends Binder {
BackgroundService getService() {
// Return this instance of this service so clients can call public methods
return BackgroundService.this;
}
}//end inner class that returns an instance of the service.
#Override
public IBinder onBind(Intent intent) {
return mBinder;
}//end onBind.
#Override
public void onCreate() {
// Start foreground service to avoid unexpected kill
startForeground(NOTIFICATION_ID, buildNotification());
}
private Notification buildNotification () {
NotificationCompat.Builder notificationBuilder=new NotificationCompat.Builder(this);
notificationBuilder.setOngoing(true); //this notification should be ongoing
notificationBuilder.setContentTitle(getString(R.string.notification_title))
.setContentText(getString (R.string.notification_text_and_ticker))
.setSmallIcon(R.drawable.vecsat_logo)
.setTicker(getString(R.string.notification_text_and_ticker));
return(notificationBuilder.build());
}
#Override
public void onDestroy() {
Log.i(TAG, "surfaceDestroyed method");
}
/***********Ok now all service related methods were implemented**************/
/************now implement surface texture related methods*****************/
#Override
public void onSurfaceTextureAvailable(SurfaceTexture surfaceTexture, int i, int i2) {
//Now the surfaceTexture available, so we can create the camera.
camera = Camera.open(1);
mSurfaceTexture.setOnFrameAvailableListener(new SurfaceTexture.OnFrameAvailableListener() {
#Override
public void onFrameAvailable(SurfaceTexture surfaceTexture) {
Log.e(TAG, "frame captured");
}
});
//now try to set the preview texture of the camera which is actually the surfaceTexture that has just been created.
try {
camera.setPreviewTexture(mSurfaceTexture);
}
catch (IOException e){
Log.e(TAG, "Error in setting the camera surface texture");
}
camera.setPreviewCallback(new Camera.PreviewCallback() {
#Override
public void onPreviewFrame(byte[] bytes, Camera camera) {
Log.e(TAG, "frame captured");
}
});
// the preview was set, now start it
camera.startPreview();
}
#Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surfaceTexture, int i, int i2) {
//super implementation is sufficient.
}
#Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surfaceTexture) {
//stop the preview.
camera.stopPreview();
//release camera resource:
camera.release();
return false;
}
#Override
public void onSurfaceTextureUpdated(SurfaceTexture surfaceTexture) {
mSurfaceTexture.setOnFrameAvailableListener(new SurfaceTexture.OnFrameAvailableListener() {
#Override
public void onFrameAvailable(SurfaceTexture surfaceTexture) {
Log.e(TAG, "frame captured from update");
}
});
camera.stopPreview(); //stop current preview.
try {
camera.setPreviewTexture(mSurfaceTexture);
}
catch (IOException e){
Log.e(TAG, "Error in setting the camera surface texture");
}
camera.setPreviewCallback(new Camera.PreviewCallback() {
#Override
public void onPreviewFrame(byte[] bytes, Camera camera) {
Log.e(TAG, "frame captured");
}
});
camera.startPreview(); // start the preview again.
}
Here is how I call the service from an activity:
#Override
protected void onCreate(Bundle savedInstanceState) {
//call the method in the super class..
super.onCreate(savedInstanceState);
//inflate the xml layout.
setContentView(R.layout.main);
//prepare buttons.
ImageButton startButton = (ImageButton) findViewById(R.id.start_btn);
ImageButton stopButton = (ImageButton) findViewById(R.id.stop_btn);
//prepare service intent:
final Intent serviceIntent = new Intent(getApplicationContext(), BackgroundService.class);
//set on click listeners for buttons (this should respond to normal touch events and simulated ones):
startButton.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
//startActivity(serviceIntent);
//start eye gazing mode as a service running in the background:
isBound= getApplicationContext().bindService(serviceIntent, mConnection, Context.BIND_AUTO_CREATE);
}
});
stopButton.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
//stop eye gazing mode as a service running in the background:
getApplicationContext().unbindService(mConnection);
}
});
}//end onCreate.
private final ServiceConnection mConnection = new ServiceConnection() {
#Override
public void onServiceConnected(ComponentName className, IBinder service) {
BackgroundService.LocalBinder binder = (BackgroundService.LocalBinder) service;
mService = binder.getService();
isBound = true;
}
#Override
public void onServiceDisconnected(ComponentName className) {
isBound = false;
}
};
Please help, how can I get the frames callback to work? I tried a lot but I couldn't figure it out.
Any help is highly appreciated whether it uses method 1 or 2.
Thanks.
I have a MediaPlayer object that uses a SurfaceHolder object as a surface. There is a button on top of the video that takes me out of the video to a website. When that happens, I pause the player with player.pause(). When I return from the website, I resume the player with player.start(). I know that the surface gets destroyed when the activity is not displayed anymore, and it gets recreated as soon as the activity is restarted. In my surfaceCreated(), I set the surface for the player again (since it no longer has a surface at that point), and then resume. However, the player simply restarts the video from the beginning.
I've tried commenting out the line that takes me to the website, just to see if pause/start works properly and resumes from last spot. It does. I'm not sure why this behaviour doesn't happen when I leave and re-enter the video activity though.
I also tried using the player.seekTo() call. There was no difference. In fact, when I disabled the button taking me to a site to just pausing the video, with the seekTo() call the video ALSO started from the beginning despite position being not 0.
The player object is the same all the way throughout.
Just because the surface is a new one on restart, it doesn't know or care of its contents, does it? The player should be managing that, right?
I'm out of ideas at this point. Can anyone please offer any tips?
UPDATE: So I threw together a quick app just to eliminate any other external factors. Here's the full code for the video class (other class is just an activity with a play button):
public class VideoPlayer extends Activity implements MediaPlayer.OnCompletionListener,
MediaPlayer.OnErrorListener, MediaPlayer.OnPreparedListener, MediaPlayer.OnSeekCompleteListener, MediaPlayer.OnVideoSizeChangedListener,
SurfaceHolder.Callback {
private MediaPlayer player;
private SurfaceHolder mSurfaceHolder;
private SurfaceView mSurfaceView;
private Button leaveVideoButton;
private boolean isPaused = false;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.video_layout);
leaveVideoButton = (Button) findViewById(R.id.go_to_web);
leaveVideoButton.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View v) {
Uri uri = Uri.parse("http://www.google.com");
Intent intent = new Intent(Intent.ACTION_VIEW, uri);
pauseSteps();
startActivity(intent);
}
});
createPlayer();
createSurface();
}
private void createSurface() {
mSurfaceView = (SurfaceView) findViewById(R.id.surface);
mSurfaceHolder = mSurfaceView.getHolder();
mSurfaceHolder.addCallback(this);
mSurfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
}
private void createPlayer() {
player = new MediaPlayer();
player.setOnCompletionListener(this);
player.setOnErrorListener(this);
player.setOnPreparedListener(this);
player.setOnVideoSizeChangedListener(this);
player.setOnSeekCompleteListener(this);
}
private void pauseSteps() {
if(player.isPlaying()) {
player.pause();
isPaused = true;
}
}
private void playSteps() {
if(isPaused) {
isPaused = false;
player.start();
}
}
#Override
protected void onDestroy() {
super.onDestroy();
if (player.isPlaying()) {
player.stop();
}
player.reset();
player.release();
player = null;
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
player.setDisplay(holder);
if (!isPaused) {
try {
// player.setDataSource(path);
AssetFileDescriptor afd = getResources().openRawResourceFd(R.raw.video);
if (afd == null) return;
player.setDataSource(afd.getFileDescriptor(), afd.getStartOffset(), afd.getLength());
afd.close();
player.prepare();
} catch (IOException e) {
e.printStackTrace();
}
} else {
playSteps();
}
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
}
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
}
#Override
public void onCompletion(MediaPlayer mp) {
}
#Override
public boolean onError(MediaPlayer mp, int what, int extra) {
return false;
}
#Override
public void onPrepared(MediaPlayer mp) {
player.start();
}
#Override
public void onSeekComplete(MediaPlayer mp) {
}
#Override
public void onVideoSizeChanged(MediaPlayer mp, int width, int height) {
}
}
UPDATE 2: So I tried a different video and it resumed just fine from the same spot. This must be some encoding issue.
I'm building an app that records and plays back video. I would like to do so without affecting background music playback, i.e. if I begin playing a video, I do not want to pause other apps' audio. However, on Lollipop, Android's VideoView class automatically requests audio focus when the private method VideoView.openVideo() is called:
AudioManager am = (AudioManager) super.getSystemService(name);
am.requestAudioFocus(null, AudioManager.STREAM_MUSIC, AudioManager.AUDIOFOCUS_GAIN);
Any suggestions on how to get around this?
Starting with Android SDK 26 you may want to use VideoView and
if(Build.VERSION.SDK_INT >= Build.Version_CODES.O){
//set this BEFORE start playback
videoView.setAudioFocusRequest(AudioManager.AUDIOFOCUS_NONE)
}
For older version, there's a workaround described here: https://stackoverflow.com/a/31569930/993439
Basically, copy source code of VideoView and uncomment following lines
AudioManager am = (AudioManager) getContext().getSystemService(Context.AUDIO_SERVICE);
am.requestAudioFocus(null, AudioManager.STREAM_MUSIC, AudioManager.AUDIOFOCUS_GAIN);
I got around this with a stupid solution by copying whole source code of android.widget.VideoView of Lollipop and removing line you mentioned.
Make your own VideoView class. don't use extends VideoView since you can't override openVideo() method.
I don't recommend this as I thinking it's a temporary solution. VideoView Changed a lot between 4.1-5.0 so this can make RuntimeException on Android version other than Lollipop
Edit
I made approach MediaPlayer + SurfaceView as pinxue told us;
It respects aspect ratio within viewWidth and viewHeight.
final String finalFilePath = filePath;
final SurfaceHolder surfaceHolder = sv.getHolder();
final MediaPlayer mediaPlayer = new MediaPlayer();
final LinearLayout.LayoutParams svLayoutParams = new LinearLayout.LayoutParams(viewWidth,viewHeight);
surfaceHolder.addCallback(new SurfaceHolder.Callback(){
#Override
public void surfaceCreated(SurfaceHolder holder) {
try {
if(isDebug) {
System.out.println("setting VideoPath to VideoView: "+finalFilePath);
}
mediaPlayer.setDataSource(finalFilePath);
}catch (IOException ioe){
if(isDebug){
ioe.printStackTrace();
}
//mediaPlayer = null;
}
mediaPlayer.setDisplay(surfaceHolder);
mediaPlayer.prepareAsync();
mediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
#Override
public void onPrepared(MediaPlayer mp) {
if(isDebug){
System.out.println("Video is starting...");
}
// for compatibility, we adjust size based on aspect ratio
if ( mp.getVideoWidth() * svLayoutParams.height < svLayoutParams.width * mp.getVideoHeight() ) {
//Log.i("###", "image too wide, correcting");
svLayoutParams.width = svLayoutParams.height * mp.getVideoWidth() / mp.getVideoHeight();
} else if ( mp.getVideoWidth() * svLayoutParams.height > svLayoutParams.width * mp.getVideoHeight() ) {
//Log.i("###", "image too tall, correcting");
svLayoutParams.height = svLayoutParams.width * mp.getVideoHeight() / mp.getVideoWidth();
}
sv.post(new Runnable(){
#Override
public void run() {
sv.setLayoutParams(svLayoutParams);
}
});
mp.start();
}
});
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
if(isDebug){
System.out.println("surfaceChanged(holder, "+format+", "+width+", "+height+")");
}
}
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
try {
mediaPlayer.setDataSource("");
}catch (IOException ioe){
if(isDebug){
ioe.printStackTrace();
}
}
}
});
if(sv.post(new Runnable() {
#Override
public void run() {
sv.setLayoutParams(svLayoutParams);///
sv.setVisibility(View.VISIBLE);
}})){
if(isDebug) {
System.out.println("post Succeded");
}
}else{
if(isDebug) {
System.out.println("post Failed");
}
}
The accepted solution does not guarantee compatibility across all Android versions and is a dirty hack more than a true solution. I've tried all forms of hacks to get this working, yet none have worked to my satisfaction.
I have come up with a much better solution though - switch from a VideoView to a TextureView and load it with a MediaPlayer. There is no difference from the user's perspective, just no more audio stoppage.
Here's my use case for playing an MP4 looping:
private TextureView _introVideoTextureView;
private MediaPlayer _introMediaPlayer;
...
#Override
public void onCreate(...) {
_introVideoTextureView.setSurfaceTextureListener(new TextureView.SurfaceTextureListener() {
#Override
public void onSurfaceTextureAvailable(SurfaceTexture surfaceTexture, int width, int height) {
try {
destoryIntroVideo();
_introMediaPlayer = MediaPlayer.create(SignInActivity.this, R.raw.intro_video);
_introMediaPlayer.setSurface(new Surface(surfaceTexture));
_introMediaPlayer.setLooping(true);
_introMediaPlayer.setVideoScalingMode(MediaPlayer.VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING);
_introMediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
#Override
public void onPrepared(MediaPlayer mediaPlayer) {
mediaPlayer.start();
}
});
} catch (Exception e) {
System.err.println("Error playing intro video: " + e.getMessage());
}
}
#Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surfaceTexture, int i, int i1) {}
#Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surfaceTexture) {
return false;
}
#Override
public void onSurfaceTextureUpdated(SurfaceTexture surfaceTexture) {}
});
}
#Override
public void onDestroy() {
super.onDestroy();
destoryIntroVideo();
}
private void destoryIntroVideo() {
if (_introMediaPlayer != null) {
_introMediaPlayer.stop();
_introMediaPlayer.release();
_introMediaPlayer = null;
}
}
You may use MediaPlayer + SurfaceView instead.
use audioManager.abandonAudioFocus(null)
If you look at the VideoView code you will notice it calls the method audioManager.requestAudioFocus with null for the OnAudioFocusChangeListener. When you register a listener with the AudioManager it uses this method to make an ID for the listener
private String getIdForAudioFocusListener(OnAudioFocusChangeListener l) {
if (l == null) {
return new String(this.toString());
} else {
return new String(this.toString() + l.toString());
}
}
which generates the same ID every time you use null. So if you call abandonAudioFocus with null it will remove any listener that was added with null as the parameter for the OnAudioFocusChangeListener
I have a MediaPlayer in a Fragment which retains its instance on configuration changes. The player is playing a video loaded from my assets directory. I have the scenario set up with the goal of reproducing the YouTube app playback where the audio keeps playing during the configuration changes and the display is detached and reattached to the media player.
When I start the playback and rotate the device, the position jumps forward about 6 seconds and (necessarily) the audio cuts out when this happens. Afterwards, the playback continues normally. I have no idea what could be causing this to happen.
As requested, here is the code:
public class MainFragment extends Fragment implements SurfaceHolder.Callback, MediaController.MediaPlayerControl {
private static final String TAG = MainFragment.class.getSimpleName();
AssetFileDescriptor mVideoFd;
SurfaceView mSurfaceView;
MediaPlayer mMediaPlayer;
MediaController mMediaController;
boolean mPrepared;
boolean mShouldResumePlayback;
int mBufferingPercent;
SurfaceHolder mSurfaceHolder;
#Override
public void onInflate(Activity activity, AttributeSet attrs, Bundle savedInstanceState) {
super.onInflate(activity, attrs, savedInstanceState);
final String assetFileName = "test-video.mp4";
try {
mVideoFd = activity.getAssets().openFd(assetFileName);
} catch (IOException ioe) {
Log.e(TAG, "Can't open file " + assetFileName + "!");
}
}
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setRetainInstance(true);
// initialize the media player
mMediaPlayer = new MediaPlayer();
try {
mMediaPlayer.setDataSource(mVideoFd.getFileDescriptor(), mVideoFd.getStartOffset(), mVideoFd.getLength());
} catch (IOException ioe) {
Log.e(TAG, "Unable to read video file when setting data source.");
throw new RuntimeException("Can't read assets file!");
}
mMediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
#Override
public void onPrepared(MediaPlayer mp) {
mPrepared = true;
}
});
mMediaPlayer.setOnBufferingUpdateListener(new MediaPlayer.OnBufferingUpdateListener() {
#Override
public void onBufferingUpdate(MediaPlayer mp, int percent) {
mBufferingPercent = percent;
}
});
mMediaPlayer.prepareAsync();
}
#Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
super.onCreateView(inflater, container, savedInstanceState);
View view = inflater.inflate(R.layout.fragment_main, container, false);
mSurfaceView = (SurfaceView) view.findViewById(R.id.surface);
mSurfaceView.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View v) {
mMediaController.show();
}
});
mSurfaceHolder = mSurfaceView.getHolder();
if (mSurfaceHolder == null) {
throw new RuntimeException("SufraceView's holder is null");
}
mSurfaceHolder.addCallback(this);
return view;
}
#Override
public void onViewCreated(View view, Bundle savedInstanceState) {
super.onViewCreated(view, savedInstanceState);
mMediaController = new MediaController(getActivity());
mMediaController.setEnabled(false);
mMediaController.setMediaPlayer(this);
mMediaController.setAnchorView(view);
}
#Override
public void onResume() {
super.onResume();
if (mShouldResumePlayback) {
start();
} else {
mSurfaceView.post(new Runnable() {
#Override
public void run() {
mMediaController.show();
}
});
}
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
mMediaPlayer.setDisplay(mSurfaceHolder);
mMediaController.setEnabled(true);
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
// nothing
}
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
mMediaPlayer.setDisplay(null);
}
#Override
public void onPause() {
if (mMediaPlayer.isPlaying() && !getActivity().isChangingConfigurations()) {
pause();
mShouldResumePlayback = true;
}
super.onPause();
}
#Override
public void onSaveInstanceState(Bundle outState) {
super.onSaveInstanceState(outState);
}
#Override
public void onDestroyView() {
mMediaController.setAnchorView(null);
mMediaController = null;
mMediaPlayer.setDisplay(null);
mSurfaceHolder.removeCallback(this);
mSurfaceHolder = null;
mSurfaceView = null;
super.onDestroyView();
}
#Override
public void onDestroy() {
mMediaPlayer.release();
mMediaPlayer = null;
try {
mVideoFd.close();
} catch (IOException ioe) {
Log.e(TAG, "Can't close asset file..", ioe);
}
mVideoFd = null;
super.onDestroy();
}
// MediaControler methods:
#Override
public void start() {
mMediaPlayer.start();
}
#Override
public void pause() {
mMediaPlayer.pause();
}
#Override
public int getDuration() {
return mMediaPlayer.getDuration();
}
#Override
public int getCurrentPosition() {
return mMediaPlayer.getCurrentPosition();
}
#Override
public void seekTo(int pos) {
mMediaPlayer.seekTo(pos);
}
#Override
public boolean isPlaying() {
return mMediaPlayer.isPlaying();
}
#Override
public int getBufferPercentage() {
return mBufferingPercent;
}
#Override
public boolean canPause() {
return true;
}
#Override
public boolean canSeekBackward() {
return true;
}
#Override
public boolean canSeekForward() {
return true;
}
#Override
public int getAudioSessionId() {
return mMediaPlayer.getAudioSessionId();
}
}
The if block in the onPause method is not being hit.
Update:
After doing a bit more debugging, removing the interaction with the SurfaceHolder causes the problem to go away. In other words, if I don't setDisplay on the MediaPlayer the audio will work fine during the configuration change: no pause, no skip. It would seem there is some timing issue with setting the display on the MediaPlayer that is confusing the player.
Additionally, I have found that you must hide() the MediaController before you remove it during the configuration change. This improves stability but does not fix the skipping issue.
Another update:
If you care, the Android media stack looks like this:
MediaPlayer.java
-> android_media_MediaPlayer.cpp
-> MediaPlayer.cpp
-> IMediaPlayer.cpp
-> MediaPlayerService.cpp
-> BnMediaPlayerService.cpp
-> IMediaPlayerService.cpp
-> *ConcreteMediaPlayer*
-> *BaseMediaPlayer* (Stagefright, NuPlayerDriver, Midi, etc)
-> *real MediaPlayerProxy* (AwesomePlayer, NuPlayer, etc)
-> *RealMediaPlayer* (AwesomePlayerSource, NuPlayerDecoder, etc)
-> Codec
-> HW/SW decoder
Upon examining AwesomePlayer, it appears this awesome player takes the liberty of pausing itself for you when you setSurface():
status_t AwesomePlayer::setNativeWindow_l(const sp<ANativeWindow> &native) {
mNativeWindow = native;
if (mVideoSource == NULL) {
return OK;
}
ALOGV("attempting to reconfigure to use new surface");
bool wasPlaying = (mFlags & PLAYING) != 0;
pause_l();
mVideoRenderer.clear();
shutdownVideoDecoder_l();
status_t err = initVideoDecoder();
if (err != OK) {
ALOGE("failed to reinstantiate video decoder after surface change.");
return err;
}
if (mLastVideoTimeUs >= 0) {
mSeeking = SEEK;
mSeekTimeUs = mLastVideoTimeUs;
modifyFlags((AT_EOS | AUDIO_AT_EOS | VIDEO_AT_EOS), CLEAR);
}
if (wasPlaying) {
play_l();
}
return OK;
}
This reveals that setting the surface will cause the player to destroy whatever surface was previously being used as well as the video decoder along with it. While setting a surface to null should not cause the audio to stop, setting it to a new surface requires the video decoder to be reinitialized and the player to seek to the current location in the video. By convention, seeking will never take you further than you request, that is, if you overshoot a keyframe when seeking, you should land on the frame you overshot (as opposed to the next one).
My hypothesis, then, is that the Android MediaPlayer does not honor this convention and jumps forward to the next keyframe when seeking. This, coupled with a video source that has sparse keyframes, could explain the jumping I am experiencing. I have not looked at AwesomePlayer's implementation of seek, though. It was mentioned to me that jumping to the next keyframe is something that needs to happen if your MediaPlayer is developed with streaming in mind since the stream can be discarded as soon as it has been consumed. Point being, it might not be that far fetch to think the MediaPlayer would choose to jump forward as opposed to backwards.
Final Update:
While I still don't know why the playback skips when attaching a new Surface as the display for a MediaPlayer, thanks to the accepted answer, I have gotten the playback to be seamless during rotation.
Thanks to natez0r's answer, I have managed to get the setup described working. However, I use a slightly different method. I'll detail it here for reference.
I have one Fragment which I flag to be retained on configuration changes. This fragment handles both the media playback (MediaPlayer), and the standard TextureView (which provides the SurfaceTexture where the video buffer gets dumped). I initialize the media playback only once my Activity has finished onResume() and once the SurfaceTexture is available. Instead of subclassing TextureView, I simply call setSurfaceTexture (since it's public) in my fragment once I receive a reference to the SurfaceTexture. The only two things retained when a configuration change happens are the MediaPlayer reference, and the SurfaceTexture reference.
I've uploaded the source of my sample project to Github. Feel free to take a look!
I know this question is a tad old now, but I was able to get this working in my app without the skipping. The issue is the surface getting destroyed (killing whatever buffer it had in it). This may not solve all your issues because it targets API 16, but you can manage your own SurfaceTexture inside your custom TextureView where the video is drawn:
private SurfaceTexture mTexture;
private TextureView.SurfaceTextureListener mSHCallback =
new TextureView.SurfaceTextureListener() {
#Override
public void onSurfaceTextureAvailable(SurfaceTexture surface, int width,
int height) {
mTexture = surface;
mPlayer.setSurface(new Surface(mTexture));
}
#Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width,
int height) {
mTexture = surface;
}
#Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
mTexture = surface;
return false;
}
#Override
public void onSurfaceTextureUpdated(SurfaceTexture surface) {
mTexture = surface;
}
};
the key is returning false in onSurfaceTextureDestroyed and holding onto mTexture. When the view gets re-attached to the window you can set the surfaceTexture:
#Override
protected void onAttachedToWindow() {
super.onAttachedToWindow();
if (mTexture != null) {
setSurfaceTexture(mTexture);
}
}
This allows my view to continue playing video from EXACTLY where it left off.