I am developing a karaoke app which consist of animation and playback song. The most important thing here is very precise synchronization between both.
I implemented it few months ago and on iOS 9/10 everything was working great. We had a problem with Android app (ExoPlayer) because there were too big delay between playback and animation. Finally we found out that the problem was caused by wrong song format (mp3) - which was explained here: https://github.com/google/ExoPlayer/issues/3233. We fixed it by packing song into mp4 format (on iOS there is still mp3 used).
I use AVPlayer with HLS
self.player = AVPlayer(url: url)
//
var currentTime_: Double {
if let player = self.player {
return player.currentTime().seconds
}
return 0
}
Everything was ok until iOS 11 release. On iOS 11 there is the same desynchronization (player.currentTime is not accurate) that was on Android.
What's interesting is that even app that was build on iOS 10 SDK and is available on AppStore doesn't work properly on iOS 11 - but it still works great on iOS 10.
Nothing on server side was changed. So Apple had to change something in decoding/buffering. Right now we are working on changing audio format/decoding but still I am curious - why that has happened? Anyone encountered something similar?
Regards
You need use this option when create AVURLAsset
let asset = AVURLAsset(url: url, options: [AVURLAssetPreferPreciseDurationAndTimingKey: true])
If you not use this option, AVPlayer measures mp3 audio duration by file length, so it has small error to get duration.
As woosiki pointed it out, you need to use the AVURLAsset. Just keep in mind that loading of large files will take longer due to the pre-calculation of the precise duration. You may want to put a loader indicator.
func prepareAudioFile(with track: Track) {
guard let url = URL(string: track.streamURL) else {
return
}
let asset = AVURLAsset(url: url,
options: [AVURLAssetPreferPreciseDurationAndTimingKey: true])
let playerItem = AVPlayerItem(asset: asset)
player = AVPlayer(playerItem: playerItem)
observePlayerProgress()
}
private func observePlayerProgress() {
let interval = CMTimeMake(value: 1, timescale: 5)
player.addPeriodicTimeObserver(forInterval: interval, queue: .main) { [weak self] time in
if let track = self?.player.currentItem, track.status == .readyToPlay {
// Once the loading is finished this callback will return
// Here you can calculate and set duration and elapsed time (track.duration, time.seconds)
}
}
}
Related
My website plays a background song and some audio effects at the same time. Although the song loads fine (I can see its duration via Audio.duration property), it doesn't play, while the audio effects play (I have already clicked on the screen to activate audio playing). On Firefox in my PC it plays without problem. The audio file is a 10 MB mp3 (but my Android is at home wi-fi). Below is part of my code.
var music,clicked=false,downloaded=false;
function play() {
music.play();
}
function canPlay() {
downloaded = true;
if (clicked) {
play();
}
}
function touchStart(e) {
if (!clicked) {
clicked = true;
if (downloaded) {
play();
}
}
}
function load() {
music = new Audio('mp3/music.mp3');
music.addEventListener('canplaythrough',canPlay,false);
window.addEventListener('touchstart',touchStart,false);
}
The load function is called in the onload of body. The clicked and downloaded variables are required because I never know what happens first: the user click or the audio finish downloading.
On Google mobile-friendly test, it says the audio file couldn’t be loaded (Status: Other error). I've read this test doesn't wait for more than 3 seconds. However, reading the time of the song in seconds doesn't prove it has downloaded?
Why won't the song play on mobile?
EDIT
I created a timer interval that tells me the position (music.currentTime) of the song once every minute. On desktop, it gives the right time. On mobile, it always gives 0, meaning that the song, although loaded (?), isn't really being played.
EDIT 2
I investigated the promise returned by music.play(), and noticed it was giving the error: "(NotAllowedError: play() failed because the user didn't interact with the document first)". It seems that touchStart doesn't count as a "user interaction" the way that mouseDown does...
In the end, I removed both touchStart and touchEnd (each with his own preventDefault), leaving the song to be played after the resulting mouseDown. I've kept the touchMove, however, because it's different from the mouseMove, and required for my site.
for awhile now, I am working on a media playing app. In this app, I also programmed a little 5 Band EQ using mainly this code:
try
{
AppPreferencesClass ap = new AppPreferencesClass(ctx);
if (Activity_Player.eq != null)
{
Activity_Player.eq.Dispose();
}
Activity_Player.eq = new Android.Media.Audiofx.Equalizer(0, Activity_Player.mediaPlayerSessionId);
Activity_Player.eq.SetEnabled(true);
await Task.Run(() =>
{
if (Activity_Player.EqActive)
{
if (ap.getAwesomeSound())
{
Activity_Player.eq.SetBandLevel(0, Convert.ToInt16(Activity_Equalizer.awesomesound0));
Activity_Player.eq.SetBandLevel(1, Convert.ToInt16(Activity_Equalizer.awesomesound1));
Activity_Player.eq.SetBandLevel(2, Convert.ToInt16(Activity_Equalizer.awesomesound2));
Activity_Player.eq.SetBandLevel(3, Convert.ToInt16(Activity_Equalizer.awesomesound3));
Activity_Player.eq.SetBandLevel(4, Convert.ToInt16(Activity_Equalizer.awesomesound4));
}
else
{
Activity_Player.eq.SetBandLevel(0, Convert.ToInt16(ap.getEQ0()));
Activity_Player.eq.SetBandLevel(1, Convert.ToInt16(ap.getEQ1()));
Activity_Player.eq.SetBandLevel(2, Convert.ToInt16(ap.getEQ2()));
Activity_Player.eq.SetBandLevel(3, Convert.ToInt16(ap.getEQ3()));
Activity_Player.eq.SetBandLevel(4, Convert.ToInt16(ap.getEQ4()));
}
}
});
}
catch
{
}
For many days, this worked just fine but out of NO WHERE, the catch block sometimes gets activated. But only occasionally.On other times, try works fine but there are just no more changes to the audio being played. This is odd enough, since I never changed anything on this code after it starting working.
I then tried another phone (Samsung S4) on my code and the eq worked just perfectly.
So this got me googleing and I think I might have heard that there can only be as many audiosession IDs after you just would run out. I tested and the audio session ID used here is somewhere at 74,000.
So this could be an issue I thought but this would easialy be tested because I already had this very app running in the google play store just an older version of it. I am 100 percent positive, that in this version the EQ worked on my phone. Otherwise I would have not uploaded that version.
Anyway, I downloaded my old app from the play store and here we go:
It doesnt work anymore. The EQ in the old version also has simply NO effect on the audio. While ofcourse on my other phones this old version works perfectly.
Before I am going to reset my own personal phone I wanted to ask you guys if this could be infact the case.
Another thing is, that I am using many static variables in order to get the EQ to work right. Actually, the variable EQ itself is static. Do maybe static variables sort of leave a "trace" behind and maybe I have set the eq up just "too" many times? Although I am disposing of the object before intialising it again (see in my code).
Summing up:
1.) Can there maybe be a maxmium number of EQ or AudioSessionIDs and I have passed those?
2.) Can creating static variables over and over again in my code cause a memory leak so big, even deinstalling the app doesnt do anything?
Thank you!
This is the error log:
11-20 12:16:43.736 E/AudioEffect(16990): set(): AudioFlinger could not create effect, status: -38
11-20 12:16:43.736 E/AudioEffects-JNI(16990): Error setting AudioEffect
11-20 12:16:43.737 E/AudioEffect-JAVA(16990): Error code -19 when initializing AudioEffect.
Thread started: #311-20 12:16:43.745 V/MediaPlayerNative(16990): unrecognized message: (6, 0, 0)
After 2 days of googeling and trying evetything out, here is the issue:
NOT CALLING RELEASE() will have you eventually have to REBOOT your phone. It wont allow too many instances of an EQ to be set.
Solution:
if (eq != null)
{
eq.Release();
}
I'm using react-native-sound to playback looped audio files (exactly mp3 format). The problem is that there is a gap between the loops.
Here is my code example:
loadSound = (s) => {
let sound = new Sound(s, Sound.MAIN_BUNDLE, (error) => {
if (error) {
console.log('failed to load the sound', error);
return;
} else {
sound.setNumberOfLoops(-1);
}
});
return sound;
}
let sound = loadSound('campfire.mp3')
sound.play();
Is there any workaround how to make the loops to sound smooth?
Actually, this issue is open on github, but there is no solution yet...
There are gaps between loops because of the bug in Android MediaPlayer (which is used under the hood).
To fix the problem I've created the react-native-audio-exoplayer module, which has pretty same functionality as the react-native-sound, but it is based on the Android ExoPlayer. The API is a bit different but more robust and modern (have a look at the docs).
Later if I have time I'll do a pull request to introduce ExoPlayer in the react-native-sound.
But for now feel free to use react-native-audio-exoplayer as a good workaround (for now it's implemented only for Android).
Using the below version of the react-native-sounds fixed the gap issue.
https://github.com/Menardi/react-native-sound-gapless.git
From the README of the git repo:
Simply setting a track to loop on Android does not make it gapless,
instead leaving a short silence before restarting the track. However,
Android does actually support gapless playback between different
tracks. So, this fork essentially loads the same track twice, and gets
Android to handle the gapless playback for us. The downside is that
you will use more memory than before, because the track is loaded
twice. For most cases, this shouldn't matter, but you can profile your
app in Android Studio if you are concerned.
I have same issue with looping the sound. What I did to make it work is to call the same play sound function once sound play completed. I hope this will helps you.
function playBackground(){
bgSound = new Sound(bgMusic, (error, sound) => {
if (error) {
alert('error' + error.message);
return;
}
bgSound.play((success) => {
if(success)
{
// console.log("Play completed");
playBackground();
}
bgSound.release();
});
});
}
Im building a react-native application.
Im trying to meter the current sound level (in decibel).
Libraries in use: react-native-audio and react-native-sound.
There is anybody familiar with this feature?
Thank you.
With the react-native-audio library, you can get the count for the only iOS.
For the currentMetering in Android, you need to customize in a native module. I have updated it add the following things in your package.json file instead of your code. You will get the currentMetering count for the Android as well as iOS.
"react-native-audio": "git+https://github.com/Harsh2402/react-native-audio.git",
You can use react-native-audio currentMetering value - in order to get the sound level in real-time.
First, you will have to initialise your recorder (which i will assume youve done). I use prepareRecordingAtPath in a similar way to below
AudioRecorder.prepareRecordingAtPath(audioPath, {
SampleRate: 22050,
Channels: 1,
AudioQuality: "Low",
AudioEncoding: "aac",
MeteringEnabled: true
});
then once you've called AudioRecorder.startRecording(); (Note you have access to .pause() and .stop() methods also
Now for handling the audio level, you are going to have to retrieve the data that is returned by the onProgress method. From what i remember, there should be some currentMetering value that you can access. Note that defining this behaviour will trigger the action every time a different decibel reading is retrieved. Like so
AudioRecorder.onProgress = data => {
let decibels = Math.floor(data.currentMetering);
//DO STUFF
};
Hope this helps,
Good day,
I'm currently building a website for myself (I'm a composer/producer), and I'm using Media Element for the main demo page. The page is under construction at the following link:
http://www.vincentrubinetti.com/listen.html
http://www.vincentrubinetti.com/listen.js
Below are what I think are the relevant functions and code:
function initPlayer()
{
player = new MediaElementPlayer('#listen_player',{
success: function (mediaElement, domObject)
{
mediaElement.addEventListener('play', resumeSong, false);
mediaElement.addEventListener('ended', playNextSong, false);
mediaElement.addEventListener('pause', pauseSong, false);
}
});
[omitted]
}
function setSong(element)
{
if (element != "")
{
unselectAllSongs();
document.getElementById(element).className = "listen_song_highlight";
[omitted]
var newSrc = document.querySelector("#"+element+" .listen_song_source").title;
player.pause();
player.setSrc(newSrc);
player.load();
}
}
function playSong(element)
{
document.querySelector("#"+element+" .listen_song_status").innerHTML = "playing";
player.play();
}
function playNextSong()
{
var newSong = document.querySelector(".listen_song_highlight + div.listen_song");
if (autoplay && newSong != null)
{
setSong(newSong.id);
playSong(newSong.id);
}
else
{
document.querySelector(".listen_song_highlight .listen_song_status").innerHTML = "stopped";
}
}
All the CSS end of this seems to be fine. It finds the next song on the list and sets the new source; I've verified it does this properly via alert and other debugging. However, on the Android default browser and the Dolphin browser, it seems to not be able to load the mp3 sometimes, and prematurely triggers the "ended" event to go to the next song. The result is that it appears to skip 2-3 songs after one is done playing. And it takes some finagling even to get it to play one, clicking on the song divs and the player play button. I can type in the same url that's in the html, and the browser will play/download it just fine, no problems accessing or loading it.
Here are the 3 mp3 files that repeat down the playlist, for reference. They are by me, but are placeholders for the real music.
NEW/music/creation.mp3
NEW/music/startup.mp3
NEW/music/win.mp3
Note that all of this works properly on Chrome, Firefox, IE8+, and Android Chrome (I haven't tested iPhone or iPad yet).
Is my diagnosis correct? Can anyone point me in the right direction? Is there any experience where MediaElement doesn't work properly on the Android default and Dolphin browsers?
I have the same problem in android (but with video...)
I was told that this is an issue with the file not being downloaded in time -- i.e. the browser starts the download, but since the file isn't downloaded it is 0:00 long, so the ended event is triggered.
If there is someway to make it so that the video / audio can be downloaded faster it solves the problem.
Strangely enough, if the device has good WiFi then this problem basically disappears.
See what John Dyer said about it here:
https://github.com/johndyer/mediaelement/issues/543