Develop Reference

Long running service with large amount of data transfer to activity

I am using USB to communicate audio and video from some external hardware.
Video is displayed on a Surface as long as the app is in foreground but audio should keep playing even when in background.
Using threads doesn't seem to work, when the app goes to background the audio starts to stutter and sound very bad. I believe i need to use a service, am i wrong?
But from all the online documentation i can't figure out, what is the best type of service (or any) to use for such long running application and the best way to communicate the large amount of data between its Activity & Surface?
I looked into plain services, WorkManagers, ForegroundService, ServiceSocket and many others
Here is the current code i am using to run the streamed audio from the USB. It is using a coroutine and as said, audio starts to stutter when app in background
class AudioTrackPlayer {
private val channel = Channel<Boolean>(capacity = 15)
private var dataList = mutableListOf<ByteArray>()
private var readIndex = 0
private var writeIndex = 0
private val scope = CoroutineScope(Job() + Dispatchers.IO)
private var audio: AudioTrack? = null
var lastSize = 0
var isPlaying = false
companion object {
private const val DATA_LIST_SIZE = 11
}
init {
scope.launch {
startThread()
}
}
fun config(channels: Int, sampleRate: Int):Boolean {
//......
}
private suspend fun startThread() = withContext(Dispatchers.IO) {
var data:ByteArray
var isPlayingLast = false
var finalByteArray:ByteArray
channel.consumeEach {
if (audio != null) {
if (isPlaying) {
if (readIndex != writeIndex) {
isPlayingLast = true
data = dataList[readIndex]
lastSize = data.size - 8
if (audio?.playState != AudioTrack.PLAYSTATE_PLAYING) {
audio?.play()
}
audio?.write(data, 8, lastSize)
readIndex++
if (readIndex == DATA_LIST_SIZE) {
readIndex = 0
}
}
} else {
if (isPlayingLast) {
isPlayingLast = false
finalByteArray = ByteArray(lastSize * 2)
if (audio?.playState != AudioTrack.PLAYSTATE_PLAYING) {
audio!!.play()
}
audio!!.write(finalByteArray, 0, finalByteArray.size)
audio!!.stop()
audio!!.flush()
writeIndex = readIndex
}
}
}
}
}
fun write(data: ByteArray) {
scope.launch {
if (isPlaying) {
dataList.add(writeIndex, data)
writeIndex++
if (writeIndex == DATA_LIST_SIZE) {
writeIndex = 0
}
channel.send(true)
}
}
}
fun play() {
//...
}
fun stop() {
//....
}
}

WebRTC Remote Stream Not Showing on Web

I have an app that implements the video calling feature using WebRTC in android.
From the App, I am creating an Offer for calls and receiving from another App/ Web.
So, when I am working on app to app, everything is working fine, I can see the remote peer, and the remote peer can also see me.
But when I am working on app (creating offer) to web (receiver). In the app, I can see both local and remote Stream, but from the web, I can't see the remote stream only seeing the remote stream.
This is my android code while creating an offer
val pcConstraints = object : MediaConstraints() {
init {
optional.add(KeyValuePair("DtlsSrtpKeyAgreement", "true"))
optional.add(KeyValuePair("OfferToReceiveAudio", "true"))
optional.add(KeyValuePair("OfferToReceiveVideo", "true"))
}
}
val peerConnection = getOrCreatePeerConnection(mRemoteSocketId, "A")
peerConnection.createOffer(object : CustomSdpObserver() {
override fun onCreateSuccess(sessionDescription: SessionDescription?) {
Timber.d("onCreateSuccess: ")
peerConnection.setLocalDescription(CustomSdpObserver(), sessionDescription)
if (sessionDescription != null) {
// sending sessionDescription from here
}
// starting the call from here
}
}, pcConstraints)
Here Creating the peerConnection
private fun getOrCreatePeerConnection(socketId: String, role: String): PeerConnection {
Timber.tag("live").d("getOrCreatePeerConnection socketId $socketId role $role")
var peerConnection = peerConnectionMap[socketId]
if (peerConnection != null) {
return peerConnection
}
val rtcConfig = PeerConnection.RTCConfiguration(iceServers)
rtcConfig.bundlePolicy = PeerConnection.BundlePolicy.MAXBUNDLE
rtcConfig.rtcpMuxPolicy = PeerConnection.RtcpMuxPolicy.REQUIRE
rtcConfig.keyType = PeerConnection.KeyType.ECDSA
rtcConfig.iceTransportsType = PeerConnection.IceTransportsType.ALL
rtcConfig.enableCpuOveruseDetection = true
peerConnection =
peerConnectionFactory?.createPeerConnection(
rtcConfig,
pcConstraints,
object : CustomPCObserver() {
override fun onIceCandidate(p0: IceCandidate?) {
super.onIceCandidate(p0)
Timber.tag("live").d("getOrCreatePeerConnection onIceCandidate${p0} ")
if (p0 != null) {
SignalingServer.get()?.sendIceCandidate(p0)
}
}
override fun onAddStream(p0: MediaStream?) {
super.onAddStream(p0)
Timber.tag("live")
.d("onAddStream Remote MediaStream ${p0?.videoTracks?.size} ")
gotRemoteStream(p0!!)
}
override fun onRenegotiationNeeded() {
super.onRenegotiationNeeded()
Timber.tag("live").d("onRenegotiationNeeded")
}
})
peerConnection!!.addStream(localMediaStream)
peerConnectionMap[socketId] = peerConnection
Timber.tag("live")
.d("getOrCreatePeerConnection size $socketId ${peerConnectionMap.size} , ${peerConnectionMap.values} ")
return peerConnection
}
So, what am I missing, I believe in Web end somehow my local Stream is not received. Your help would be highly appreciated.

Delay in Connecting with XMPP Server using Smack

I have a connection delay that I cannot find explanations. Basically the connection to the local machine takes almost 1min in the Emulator and 20s+ on the physical device. I have tried everything I know and ended up with the code below. The time on device is now around 15s, but it is still a lot for just connecting.
Am not doing anything yet, just connecting.
Anyone with idea what I can do to optimize the delay?
class XMPPConnectionManager {
companion object {
const val JABBER_DOMAIN = "localhost"
const val JABBER_URL = "192.168.1.101"
const val JABBER_RESOURCE = "Resource"
const val JABBER_PORT = 5222
const val CONNECTION_TIMEOUT = 300
const val REPLY_TIMEOUT = 30000L
}
private val conn: XMPPTCPConnection by lazy {
val builder = XMPPTCPConnectionConfiguration.builder()
.setXmppDomain(JidCreate.domainBareFrom(JABBER_DOMAIN))
.setHostAddress(InetSocketAddress(JABBER_URL, JABBER_PORT).address)
.setResource(JABBER_RESOURCE)
.setCompressionEnabled(true)
.setSendPresence(false)
.setSecurityMode(ConnectionConfiguration.SecurityMode.disabled) //TODO: Enable it
//.enableDefaultDebugger()
.setConnectTimeout(CONNECTION_TIMEOUT)
val connection = XMPPTCPConnection(builder.build())
connection.replyTimeout = REPLY_TIMEOUT
//auto reconnection
val reconnectionManager = ReconnectionManager.getInstanceFor(connection)
reconnectionManager.enableAutomaticReconnection()
reconnectionManager.setReconnectionPolicy(ReconnectionManager.ReconnectionPolicy.RANDOM_INCREASING_DELAY)
connection.connect()
return#lazy connection
}
fun login(username: String?, password: String?): Boolean {
try {
if (!conn.isConnected) {
conn.connect()
}
if(!conn.isAuthenticated){
conn.login(username, password)
//Setup OX-IM and other listeners
}
} catch (e: Exception) {
return false
}
return true
}
}

How to cast web url from android app via miracast?

I try to cast youtube video from my android app to chromecast or smart tv through miracast.
But I can cast only source url for video like https://media.w3.org/2010/05/sintel/trailer.mp4
How can I cast web page with youtube or vimeo url video?
I know that YouTube app can cast video to some smart tv without chromecast and it looks like YouTube TV page. For example here https://www.youtube.com/watch?v=x5ImUYDjocY
I try to use Presentation API to set WebView in it:
#TargetApi(Build.VERSION_CODES.JELLY_BEAN_MR1)
class CastPresentation constructor(outerContext: Context?, display: Display?) : Presentation(outerContext, display) {
override fun onCreate(savedInstanceState: Bundle?) {
val wv = WebView(context)
wv.settings.javaScriptEnabled = true
wv.webChromeClient = WebChromeClient()
wv.loadUrl("https://www.youtube.com/watch?v=DxGLn_Cu5l0")
setContentView(wv)
super.onCreate(savedInstanceState)
}
}
But it doesn't affect. I don't undertand how to use it.
This is how I use it:
#TargetApi(Build.VERSION_CODES.JELLY_BEAN_MR1)
class CastDelegate constructor(private val activity: AppCompatActivity) {
private var mediaRouter: MediaRouter? = null
private var mediaRouteSelector: MediaRouteSelector? = null
// Variables to hold the currently selected route and its playback client
private var route: MediaRouter.RouteInfo? = null
private var remotePlaybackClient: RemotePlaybackClient? = null
private var presentation: Presentation? = null
// Define the Callback object and its methods, save the object in a class variable
private val mediaRouterCallback = object : MediaRouter.Callback() {
override fun onRouteSelected(router: MediaRouter, route: MediaRouter.RouteInfo) {
Timber.d("CastDelegate --> onRouteSelected: route=$route")
if (route.supportsControlCategory(MediaControlIntent.CATEGORY_REMOTE_PLAYBACK)) {
// Stop local playback (if necessary)
// ...
// Save the new route
this#CastDelegate.route = route
// Attach a new playback client
remotePlaybackClient = RemotePlaybackClient(activity, this#CastDelegate.route)
// Start remote playback (if necessary)
// ...
updatePresentation()
val uri = Uri.parse("https://media.w3.org/2010/05/sintel/trailer.mp4")
remotePlaybackClient?.play(uri, null, null, 0, null, object: RemotePlaybackClient.ItemActionCallback() {
override fun onResult(data: Bundle?, sessionId: String?, sessionStatus: MediaSessionStatus?, itemId: String?, itemStatus: MediaItemStatus?) {
super.onResult(data, sessionId, sessionStatus, itemId, itemStatus)
}
})
}
}
override fun onRouteUnselected(router: MediaRouter, route: MediaRouter.RouteInfo, reason: Int) {
Timber.d("CastDelegate --> onRouteUnselected: route=$route")
if (route.supportsControlCategory(MediaControlIntent.CATEGORY_REMOTE_PLAYBACK)) {
// Changed route: tear down previous client
this#CastDelegate.route?.also {
remotePlaybackClient?.release()
remotePlaybackClient = null
}
// Save the new route
this#CastDelegate.route = route
updatePresentation()
when (reason) {
MediaRouter.UNSELECT_REASON_ROUTE_CHANGED -> {
// Resume local playback (if necessary)
// ...
}
}
}
}
override fun onRoutePresentationDisplayChanged(router: MediaRouter?, route: MediaRouter.RouteInfo?) {
updatePresentation()
}
}
fun onCreate() {
// Get the media router service.
mediaRouter = MediaRouter.getInstance(activity)
// Create a route selector for the type of routes your app supports.
mediaRouteSelector = MediaRouteSelector.Builder()
// These are the framework-supported intents
.addControlCategory(MediaControlIntent.CATEGORY_REMOTE_PLAYBACK)
.build()
// val selectedRoute = mediaRouter?.selectedRoute ?: return
// val presentationDisplay = selectedRoute.presentationDisplay ?: return
// presentation = CastPresentation(activity, presentationDisplay)
// presentation?.show()
}
fun onStart() {
mediaRouteSelector?.also { selector ->
mediaRouter?.addCallback(selector, mediaRouterCallback,
MediaRouter.CALLBACK_FLAG_REQUEST_DISCOVERY)
}
updatePresentation()
}
fun onStop() {
mediaRouter?.removeCallback(mediaRouterCallback)
presentation?.dismiss()
presentation = null
}
fun onCreateOptionsMenu(menu: Menu?, inflater: MenuInflater?) {
// Attach the MediaRouteSelector to the menu item
val mediaRouteMenuItem = menu?.findItem(R.id.media_route_menu_item)
val mediaRouteActionProvider = MenuItemCompat.getActionProvider(mediaRouteMenuItem) as MediaRouteActionProvider
// Attach the MediaRouteSelector that you built in onCreate()
mediaRouteSelector?.also(mediaRouteActionProvider::setRouteSelector)
}
private fun updatePresentation() {
// Get the current route and its presentation display.
val selectedRoute = mediaRouter?.selectedRoute
val presentationDisplay = selectedRoute?.presentationDisplay
// Dismiss the current presentation if the display has changed.
if (presentation?.display != presentationDisplay) {
Timber.d("CastDelegate --> Dismissing presentation because the current route no longer " + "has a presentation display.")
presentation?.dismiss()
presentation = null
}
// Show a new presentation if needed.
if (presentation == null && presentationDisplay != null) {
Timber.d("CastDelegate --> Showing presentation on display: $presentationDisplay")
presentation = CastPresentation(activity, presentationDisplay)
try {
presentation?.show()
} catch (ex: WindowManager.InvalidDisplayException) {
Timber.d("CastDelegate --> Couldn't show presentation! Display was removed in the meantime.", ex)
presentation = null
}
}
}
}
As a result now playing video https://media.w3.org/2010/05/sintel/trailer.mp4 from
remotePlaybackClient?.play(...)

ExoPlayer not working with Ads

I have implemented the ExoPlayer in my application using the example from the Codelab : https://codelabs.developers.google.com/codelabs/exoplayer-intro/#3, algo with the example from https://medium.com/google-exoplayer/playing-ads-with-exoplayer-and-ima-868dfd767ea, the only difference is that I use AdsMediaSource instead of the deprecated ImaAdsMediaSource.
My Implementation is this:
class HostVideoFullFragment : Fragment(), AdsMediaSource.MediaSourceFactory {
override fun getSupportedTypes() = intArrayOf(C.TYPE_DASH, C.TYPE_HLS, C.TYPE_OTHER)
override fun createMediaSource(uri: Uri?, handler: Handler?, listener: MediaSourceEventListener?): MediaSource {
#C.ContentType val type = Util.inferContentType(uri)
return when (type) {
C.TYPE_DASH -> {
DashMediaSource.Factory(
DefaultDashChunkSource.Factory(mediaDataSourceFactory),
manifestDataSourceFactory)
.createMediaSource(uri, handler, listener)
}
C.TYPE_HLS -> {
HlsMediaSource.Factory(mediaDataSourceFactory)
.createMediaSource(uri, handler, listener)
}
C.TYPE_OTHER -> {
ExtractorMediaSource.Factory(mediaDataSourceFactory)
.createMediaSource(uri, handler, listener)
}
else -> throw IllegalStateException("Unsupported type for createMediaSource: $type")
}
}
private var player: SimpleExoPlayer? = null
private lateinit var playerView: SimpleExoPlayerView
private lateinit var binding: FragmentHostVideoFullBinding
private var playbackPosition: Long = 0
private var currentWindow: Int = 0
private var playWhenReady = true
private var inErrorState: Boolean = false
private lateinit var adsLoader: ImaAdsLoader
private lateinit var manifestDataSourceFactory: DataSource.Factory
private lateinit var mediaDataSourceFactory: DataSource.Factory
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
//Initialize the adsLoader
adsLoader = ImaAdsLoader(activity as Context, Uri.parse("https://pubads.g.doubleclick.net/gampad/ads?sz=640x480&iu=/124319096/external/ad_rule_samples&ciu_szs=300x250&ad_rule=1&impl=s&gdfp_req=1&env=vp&output=vmap&unviewed_position_start=1&cust_params=deployment%3Ddevsite%26sample_ar%3Dpremidpost&cmsid=496&vid=short_onecue&correlator="))
manifestDataSourceFactory = DefaultDataSourceFactory(
context, Util.getUserAgent(context, "BUO-APP"))//TODO change the applicationName with the right application name
//
mediaDataSourceFactory = DefaultDataSourceFactory(
context,
Util.getUserAgent(context, "BUO-APP"),//TODO change the applicationName with the right application name
DefaultBandwidthMeter())
}
private fun initializePlayer() {
/*
* Since the player can change from null (when we release resources) to not null we have to check if it's null.
* If it is then reset again
* */
if (player == null) {
//Initialize the Exo Player
player = ExoPlayerFactory.newSimpleInstance(DefaultRenderersFactory(activity as Context),
DefaultTrackSelector())
}
val uri = Uri.parse(videoURl)
val mediaSourceWithAds = buildMediaSourceWithAds(uri)
//Bind the view from the xml to the SimpleExoPlayer instance
playerView.player = player
//Add the listener that listens for errors
player?.addListener(PlayerEventListener())
player?.seekTo(currentWindow, playbackPosition)
player?.prepare(mediaSourceWithAds, true, false)
//In case we could not set the exo player
player?.playWhenReady = playWhenReady
//We got here without an error, therefore set the inErrorState as false
inErrorState = false
//Re update the retry button since, this method could have come from a retry click
updateRetryButton()
}
private inner class PlayerEventListener : Player.DefaultEventListener() {
fun updateResumePosition() {
player?.let {
currentWindow = player!!.currentWindowIndex
playbackPosition = Math.max(0, player!!.contentPosition)
}
}
override fun onPlayerStateChanged(playWhenReady: Boolean, playbackState: Int) {
//The player state has ended
//TODO check if there is going to be a UI change here
// if (playbackState == Player.STATE_ENDED) {
// showControls()
// }
// updateButtonVisibilities()
}
override fun onPositionDiscontinuity(#Player.DiscontinuityReason reason: Int) {
if (inErrorState) {
// This will only occur if the user has performed a seek whilst in the error state. Update
// the resume position so that if the user then retries, playback will resume from the
// position to which they seek.
updateResumePosition()
}
}
override fun onPlayerError(e: ExoPlaybackException?) {
var errorString: String? = null
//Check what was the error so that we can show the user what was the correspond problem
if (e?.type == ExoPlaybackException.TYPE_RENDERER) {
val cause = e.rendererException
if (cause is MediaCodecRenderer.DecoderInitializationException) {
// Special case for decoder initialization failures.
errorString = if (cause.decoderName == null) {
when {
cause.cause is MediaCodecUtil.DecoderQueryException -> getString(R.string.error_querying_decoders)
cause.secureDecoderRequired -> getString(R.string.error_no_secure_decoder,
cause.mimeType)
else -> getString(R.string.error_no_decoder,
cause.mimeType)
}
} else {
getString(R.string.error_instantiating_decoder,
cause.decoderName)
}
}
}
if (errorString != null) {
//Show the toast with the proper error
Toast.makeText(activity as Context, errorString, Toast.LENGTH_LONG).show()
}
inErrorState = true
if (isBehindLiveWindow(e)) {
clearResumePosition()
initializePlayer()
} else {
updateResumePosition()
updateRetryButton()
}
}
}
private fun clearResumePosition() {
//Clear the current resume position, since there was an error
currentWindow = C.INDEX_UNSET
playbackPosition = C.TIME_UNSET
}
/*
* This is for the multi window support
* */
private fun isBehindLiveWindow(e: ExoPlaybackException?): Boolean {
if (e?.type != ExoPlaybackException.TYPE_SOURCE) {
return false
}
var cause: Throwable? = e.sourceException
while (cause != null) {
if (cause is BehindLiveWindowException) {
return true
}
cause = cause.cause
}
return false
}
private fun buildMediaSourceWithAds(uri: Uri): MediaSource {
/*
* This content media source is the video itself without the ads
* */
val contentMediaSource = ExtractorMediaSource.Factory(
DefaultHttpDataSourceFactory("BUO-APP")).createMediaSource(uri) //TODO change the user agent
/*
* The method constructs and returns a ExtractorMediaSource for the given uri.
* We simply use a new DefaultHttpDataSourceFactory which only needs a user agent string.
* By default the factory will use a DefaultExtractorFactory for the media source.
* This supports almost all non-adaptive audio and video formats supported on Android. It will recognize our mp3 file and play it nicely.
* */
return AdsMediaSource(
contentMediaSource,
/* adMediaSourceFactory= */ this,
adsLoader,
playerView.overlayFrameLayout,
/* eventListener= */ null, null)
}
override fun onStart() {
super.onStart()
if (Util.SDK_INT > 23) {
initializePlayer()
}
}
override fun onResume() {
super.onResume()
hideSystemUi()
/*
* Starting with API level 24 Android supports multiple windows.
* As our app can be visible but not active in split window mode, we need to initialize the player in onStart.
* Before API level 24 we wait as long as possible until we grab resources, so we wait until onResume before initializing the player.
* */
if ((Util.SDK_INT <= 23 || player == null)) {
initializePlayer()
}
}
}
The ad never shows and if it shows it shows a rendering error E/ExoPlayerImplInternal: Renderer error. which never allows the video to show. I've run the examples from the IMA ads https://developers.google.com/interactive-media-ads/docs/sdks/android/ example code and it doesn't work neither. Does anyone have implemented the Exo Player succesfully with the latest ExoPlayer library version?
Please Help. Thanks!

When on an emulator, be sure to enable gpu rendering on the virtual device

The problem is that the emulator can not render videos. Therefore it wasn't showing the ads or the video. Run the app on a phone and it will work