Develop Reference

How to disable surface detection in ARCore Android

I am working on a project and facing an issue with ARCore. I used ARCore Location in my project, I set the location of object using latitude and longitude. but when I see it in the device, object location varies in AR.
CompletableFuture<ViewRenderable> exampleLayout = ViewRenderable.builder()
.setView(this, R.layout.example_layout)
.build();
// When you build a Renderable, Sceneform loads its resources in the background while returning
// a CompletableFuture. Call thenAccept(), handle(), or check isDone() before calling get().
CompletableFuture<ModelRenderable> andy = ModelRenderable.builder()
.setSource(this, R.raw.andy)
.build();
CompletableFuture.allOf(
exampleLayout,
andy)
.handle(
(notUsed, throwable) -> {
// When you build a Renderable, Sceneform loads its resources in the background while
// returning a CompletableFuture. Call handle(), thenAccept(), or check isDone()
// before calling get().
if (throwable != null) {
DemoUtils.displayError(this, "Unable to load renderables", throwable);
return null;
}
try {
exampleLayoutRenderable = exampleLayout.get();
andyRenderable = andy.get();
hasFinishedLoading = true;
} catch (InterruptedException | ExecutionException ex) {
DemoUtils.displayError(this, "Unable to load renderables", ex);
}
return null;
});
// Set an update listener on the Scene that will hide the loading message once a Plane is
// detected.
arSceneView
.getScene()
.setOnUpdateListener(
frameTime -> {
if (!hasFinishedLoading) {
return;
}
if (locationScene == null) {
// If our locationScene object hasn't been setup yet, this is a good time to do it
// We know that here, the AR components have been initiated.
locationScene = new LocationScene(this, this, arSceneView);
// Now lets create our location markers.
// First, a layout
LocationMarker layoutLocationMarker = new LocationMarker(
77.398151,
28.540926,
getExampleView()
);
// An example "onRender" event, called every frame
// Updates the layout with the markers distance
layoutLocationMarker.setRenderEvent(new LocationNodeRender() {
#SuppressLint("SetTextI18n")
#Override
public void render(LocationNode node) {
View eView = exampleLayoutRenderable.getView();
TextView distanceTextView = eView.findViewById(R.id.textView2);
distanceTextView.setText(node.getDistance() + "M");
}
});
// Adding the marker
locationScene.mLocationMarkers.add(layoutLocationMarker);
// Adding a simple location marker of a 3D model
locationScene.mLocationMarkers.add(
new LocationMarker(
77.398151,
28.540926,
getAndy()));
}
Frame frame = arSceneView.getArFrame();
if (frame == null) {
return;
}
if (frame.getCamera().getTrackingState() != TrackingState.TRACKING) {
return;
}
if (locationScene != null) {
locationScene.processFrame(frame);
}
if (loadingMessageSnackbar != null) {
for (Plane plane : frame.getUpdatedTrackables(Plane.class)) {
if (plane.getTrackingState() == TrackingState.TRACKING) {
hideLoadingMessage();
}
}
}
});
// Lastly request CAMERA & fine location permission which is required by ARCore-Location.
ARLocationPermissionHelper.requestPermission(this);
The major problem in this is that it detects surface and place image according to that, If there is any possibility to disable surface detection in this then it works perfectly.

Modify the session configuration with EnablePlaneFinding = false and then disable and reenable the ARCoreSession. That would disable plane finding but would keep existing planes as they were at the moment.
If you don't want to disable the session you could force an OnEnable() call on the session without disabling it:
var session = GameObject.Find("ARCore Device").GetComponent<ARCoreSession>();
session.SessionConfig.EnablePlaneFinding = false; session.OnEnable();

You can use hide() method to hide the plane discovery in android. Also, by setting setEnabled() method as false to disable the plane renderer.
Try like this,
arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment);
arFragment.getPlaneDiscoveryController().hide();
arFragment.getPlaneDiscoveryController().setInstructionView(null);
arFragment.getArSceneView().getPlaneRenderer().setEnabled(false);

How do you solve high latency with UDP on mobile devices

I am having an issue getting the network latency to be low for my multiplayer shooter game made with Unity. I am using UDP to send player positions from the game client to my amazon server and back to another game client. My game clients are sending 60 byte UDP packets to the amazon server at a rate of 8 packets per second.
When I play the game on two separate iOS devices (iPhone 7 and iPad mini) the network latency is very low and the players are able to see each other move instantly. However, if I run the game on my iPhone 7 and play against another player who is using a samsung galaxy s4 running android, which is a lower powered device, I experience 5 second latency on the android device when receiving player positions from the iOS device. The odd thing is that the iOS device can receive player positions from the android device instantly. The same issue happens when playing on the iPhone 7 against a client on mac, except the iPhone 7 experiences the 5 second latency when receiving player positions from the mac client. The issue also happens when playing on the Samsung Galaxy S4 against a client on mac, where the galaxy s4 experiences the 5 second latency.
I have tried to increase the udp receive buffer size to 16 MB but it didn't change anything.
Here is a sample of my game client code where I send player positions to the amazon server:
void Update(){
// manually constrain rotation because rigidbody constraints don't work
this.transform.rotation = Quaternion.Euler(new Vector3(0, this.transform.rotation.eulerAngles.y, 0));
this.transform.position = new Vector3(this.transform.position.x, boatHeightConstant, this.transform.position.z);
// Speed Boost Handling
if(isSpeedBoosting == true){
tiltFactor = tiltModifier * (VelocityRatio() + 1.0f);
speedBoostTimer += Time.deltaTime;
}
else{
tiltFactor = VelocityRatio() + 1.0f;
}
if(speedBoostTimer >= speedBoostDuration){
isSpeedBoosting = false;
speedBoostTimer = 0f;
endSpeedBoost = true;
}
if(endSpeedBoost == true){
GetComponentInChildren<MainWeapon>().EndFireRateBoost();
endSpeedBoost = false;
}
// Attack Boost Handling
if(isAttackBoosting == true){
attackBoostTimer += Time.deltaTime;
}
if(attackBoostTimer >= attackBoostDuration){
isAttackBoosting = false;
attackBoostTimer = 0f;
endAttackBoost = true;
}
if(endAttackBoost == true){
GetComponentInChildren<MainWeapon>().ResetDamage();
GetComponentInChildren<MainWeapon>().ResetHeatUpRate();
endAttackBoost = false;
}
if (GetComponent<InputManager>().GetInputType() == 0 || GetComponent<InputManager>().GetInputType() == 1 )
{
if (isSpeedBoosting == true)
{
Move(speedModifier);
}
else
{
Move(1f);
}
if (syncTimer <= 0f) {
syncTimer = networkRefreshRate;
SyncTransform ();
} else {
syncTimer -= Time.deltaTime;
}
}
else
{
NetworkMove();
}
}
/// <summary>
/// This function is constantly called to upload the local player's position to the server so the server is
/// aware of this player's movement and can share this local players current position with other players in the game.
/// </summary>
public void SyncTransform() {
if (isLocalPlayer == true && client != null && client.IsConnected()) {
client.SendPlayerTransform(SharedData.storage["userId"], transform.position, transform.rotation, currentLife);
}
}
Here is a sample of the UDP sender class in my game client:
public void SendPlayerTransform(string playerId, Vector3 position, Quaternion rotation, int currentLife) {
Message.Writer writer = new Message.Writer(Message.MessageType.PlayerTransform, udpClient, remoteEndPoint);
// Write the timestamp of this message
writer.WriteLong(DateTime.UtcNow.Ticks);
// write the player id
writer.WriteString(SharedData.storage["userId"]);
// write the position vector
writer.WriteFloatArray(CommonGameFunctions.ConvertVectorToFloatArray(position));
// write the rotation vector
writer.WriteFloatArray(CommonGameFunctions.ConvertQuaternionToFloatArray(rotation));
writer.WriteInt (currentLife);
// Now send the message
writer.Send();
}
Here is a sample of where I receive the UDP messages on the game client:
public int HandleUdpMessages() {
if (udpTimerStarted == false) {
lastUdpMessageReceivedTime = DateTime.Now;
udpTimerStarted = true;
} else if (udpTimerStarted == true && udpClient.Available == 0){
TimeSpan t = DateTime.Now - lastUdpMessageReceivedTime;
if (t.Seconds >= 10f) {
// no message received for last 10 seconds then throw IO exception
//throw new SocketException();
}
}
if (udpClient.Available > 0) {
var messageReader = new Message.Reader (udpClient);
messageReader.BlockingRead (ref localEndPoint, UdpReceiveTimeout);
var messageType = messageReader.ReadMessageTypeUdp ();
lastUdpMessageReceivedTime = DateTime.Now;
Debug.Log ("Received udp message: " + messageType);
switch (messageType) {
// Player position update message
case Message.MessageType.PlayerTransform:
HandlePlayerTransform (messageReader);
break;
case Message.MessageType.PlayerScore:
HandlePlayerScore (messageReader);
break;
case Message.MessageType.RockHealth:
HandleRockHealth (messageReader);
break;
case Message.MessageType.PlayerHealth:
HandlePlayerHealth (messageReader);
break;
case Message.MessageType.ShieldHealth:
HandleShieldHealth (messageReader);
break;
default:
Debug.LogError ("Unhandled message " + messageType);
break;
}
}
return 0;
}
public void HandlePlayerTransform(Message.Reader reader)
{
long timeStamp = reader.ReadLong ();
string playerId = reader.ReadString();
if (playerMessageTimeStamps [playerId].latestPlayerTransform > timeStamp)
return;
Vector3 position = new Vector3();
Quaternion rotation = new Quaternion();
// read position
position = CommonGameFunctions.ConvertFloatArrayToVector3(reader.ReadFloatArray(3));
rotation = CommonGameFunctions.ConvertFloatArrayToQuaternion(reader.ReadFloatArray(4));
// Now update the transform of the right player
Player player = Player.playerTable[playerId];
if (player == null) {
return;
}
player.SetNetworkPostion(position);
player.SetNetworkRotation(rotation);
}
On my server this is the main game loop which runs on its own dedicated thread.
// Now all the players are connected to the server
// We can start the main game loop
while (gameRunning == true) {
HandlePlayersWhoDroppedOut ();
if (PlayersLeftGame () == true) {
DisconnectAllPlayers ();
Debug.LogError ("Player's left match, returning from thread");
return;
} else {
foreach (NetworkPlayer p in participants) {
try {
p.HandleTcpMessages ();
p.HandleUdpMessages ();
} catch (IOException e) {
droppedPlayers.Add (p);
}
}
try {
RespawnRocksIfDestroyed ();
} catch (System.IO.IOException e) {
DisconnectAllPlayers ();
return;
Debug.LogError ("Failed to spawn rocks");
}
}
}

The problem with my code was that I was reading exactly one UDP message in each iteration of the UDP message handler function. I changed my function to read ALL the available UDP messages in the buffer and the lag reduced by 80%. UDP messages queue up in the buffer quicker than the message handler function repeats so this is why the problem was happening.

Games.RealtimeMultiplayer.getWaitingRoomIntent null pointer exception

I create a room and it gets successfully made. And my onRoomCreated method gets called...
#Override
public void onRoomCreated(int statusCode, Room room) {
mRoomId = room.getRoomId();
Intent i = Games.RealTimeMultiplayer.getWaitingRoomIntent(gApiClient, room, 2);
startActivityForResult(i, RC_WAITING_ROOM);
}
Then in my onActivityResult...
Room r = data.getExtras().getParcelable(Multiplayer.EXTRA_ROOM);
ArrayList<String> invitees = new ArrayList<String>();
for (Participant p : r.getParticipants()) {
invitees.add(p.getPlayer().getPlayerId()); //<---NULL POINTER!
}
I get that null pointer. Why?
EDIT: The android docs say this about the getPlayer() method...
Returns the Player that this participant represents. Note that this may be null if the identity of the player is unknown. This occurs in automatching scenarios where some players are not permitted to see the real identity of others.
That is why I am getting null, because my room is through auto-matching.
Now the question is. How can I create a turnbasedgame using only participant IDs? Not Player IDs

Now that I see what you are asking more clearly (my fault, not yours), here is how I do it:
(for clarification I use LibGDX, so may be some interface stuff you don't need, and I am still using GamesClient not the new API methods, but is for all intents the same)
First, the final call I look to start my game is onRoomConnected
#Override
public void onRoomConnected(int statusCode, Room room) {
//dLog("onRoomConnected");
mRoomCurrent = room;
mParticipants = room.getParticipants();
mMyID = room.getParticipantId(aHelper.getGamesClient().getCurrentPlayerId());
//dLog("The id is " + mMyID);
try {
bWaitRoomDismissedFromCode = true;
finishActivity(RC_WAITING_ROOM);
} catch (Exception e) {
//dLog("would have errored out in waiting room");
}
//tell the Game the room is connected
if (statusCode == GamesClient.STATUS_OK) {
theGameInterface.onRoomConnected(room.getParticipantIds(), mMyID, room.getCreationTimestamp() );
} else {
leaveRoom();
}
}
So, now have all the participantIDs.. now in my Game code (where I sent that List of Ids), I sort the list of IDs so that in determining Player order, it is the same methodology for all Players. First I build my opponents.
private void buildOpponents() {
// this creates a new opponent with a View on the Stage()
//sort the participants the same for all players
sortParticipantIDs();
for (String s : mParticipantIds) {
if(s.contains(mMyID) || mMyID.contains(s)) continue;
newOpponentWindow ow = new newOpponentWindow(s, MyAssetManager.getMySkin(), getStage());
Opponent o = new Opponent(this, s);
mapOpponents.put(s, o);
o.setWindow(ow);
getStage().addActor(ow);
}
setOpponentWindowPositions();
}
Then after some more setup I start Play and my first Time through, I have chosen that whoever is the top ID gets the honor of starting (I find this randomizes play enough, without having to do another method.. .but you can let the top ID do another method, and send that out to the other Players) Note this checks over my Opponents to determine Starting Player if someone leaves the room later in the game as well.
private boolean determineIfStartingBidder() {
Collections.sort(mParticipantIds);
// now look thru list
// if the number is mine then return true
// if the number is not mine.. and opponent is not Out of Game or Disconnected.. then return false
for (String s : mParticipantIds) {
if(s.contains(mMyID) || mMyID.contains(s)){
return true;
}
if(mapOpponents.get(s).getCurrentState() == currentState.DISCONNECTED || mapOpponents.get(s).getCurrentState() == currentState.OUTOFGAME ||
mapOpponents.get(s).getCurrentState() == currentState.LOSTGAME) {
continue;
}
return false;
}
return false;
}
Then in your game logic, just go through your ParticipantID list in whatever manner makes sense to pass the baton around! This works well, since all the calls for passing messages require the ParticipantID, and are there for easy grab n go!
Prior Answer Below ------------------------------------------------
try
data.getParcelableExtra(Multiplayer.EXTRA_ROOM);
no need for the getExtras

Sound recognition in Android

I want my Android app to recognize sound. For example I want to know if the sound from microphone is a clapping or knocking or something else.
Do I need to use math, or can I just use some library for that?
If there are any libraries for sound analysis please let me know. Thanks.

Musicg library is useful for whistle detection. Concerning claps, I wouldn't recommend use it, cause it reacts to every loud sound (even speech).
For clap and other percussive sounds detection I recommend TarsosDSP. It has a simple API with a rich functionality (pitch detection and so on). For clap detection you can use something like (if you use TarsosDSPAndroid-v3):
MicrophoneAudioDispatcher mDispatcher = new MicrophoneAudioDispatcher((int) SAMPLE_RATE, BUFFER_SIZE, BUFFER_OVERLAP);
double threshold = 8;
double sensitivity = 20;
mPercussionDetector = new PercussionOnsetDetector(22050, 1024,
new OnsetHandler() {
#Override
public void handleOnset(double time, double salience) {
Log.d(TAG, "Clap detected!");
}
}, sensitivity, threshold);
mDispatcher.addAudioProcessor(mPercussionDetector);
new Thread(mDispatcher).start();
You can tune your detector by adjusting sensitivity (0-100) and threshold (0-20).
Good luck!

There is an Api that works very well for your needs in my opinion.
http://code.google.com/p/musicg/
Good Luck!!!

You don't need math and you don't need AudioRecord. Just check MediaRecorder.getMaxAmplitude() every 1000 milliseconds.
this code and this code might be helpful.
Here is some code you will need.
public class Clapper
{
private static final String TAG = "Clapper";
private static final long DEFAULT_CLIP_TIME = 1000;
private long clipTime = DEFAULT_CLIP_TIME;
private AmplitudeClipListener clipListener;
private boolean continueRecording;
/**
* how much louder is required to hear a clap 10000, 18000, 25000 are good
* values
*/
private int amplitudeThreshold;
/**
* requires a little of noise by the user to trigger, background noise may
* trigger it
*/
public static final int AMPLITUDE_DIFF_LOW = 10000;
public static final int AMPLITUDE_DIFF_MED = 18000;
/**
* requires a lot of noise by the user to trigger. background noise isn't
* likely to be this loud
*/
public static final int AMPLITUDE_DIFF_HIGH = 25000;
private static final int DEFAULT_AMPLITUDE_DIFF = AMPLITUDE_DIFF_MED;
private MediaRecorder recorder;
private String tmpAudioFile;
public Clapper() throws IOException
{
this(DEFAULT_CLIP_TIME, "/tmp.3gp", DEFAULT_AMPLITUDE_DIFF, null, null);
}
public Clapper(long snipTime, String tmpAudioFile,
int amplitudeDifference, Context context, AmplitudeClipListener clipListener)
throws IOException
{
this.clipTime = snipTime;
this.clipListener = clipListener;
this.amplitudeThreshold = amplitudeDifference;
this.tmpAudioFile = tmpAudioFile;
}
public boolean recordClap()
{
Log.d(TAG, "record clap");
boolean clapDetected = false;
try
{
recorder = AudioUtil.prepareRecorder(tmpAudioFile);
}
catch (IOException io)
{
Log.d(TAG, "failed to prepare recorder ", io);
throw new RecordingFailedException("failed to create recorder", io);
}
recorder.start();
int startAmplitude = recorder.getMaxAmplitude();
Log.d(TAG, "starting amplitude: " + startAmplitude);
do
{
Log.d(TAG, "waiting while recording...");
waitSome();
int finishAmplitude = recorder.getMaxAmplitude();
if (clipListener != null)
{
clipListener.heard(finishAmplitude);
}
int ampDifference = finishAmplitude - startAmplitude;
if (ampDifference >= amplitudeThreshold)
{
Log.d(TAG, "heard a clap!");
clapDetected = true;
}
Log.d(TAG, "finishing amplitude: " + finishAmplitude + " diff: "
+ ampDifference);
} while (continueRecording || !clapDetected);
Log.d(TAG, "stopped recording");
done();
return clapDetected;
}
private void waitSome()
{
try
{
// wait a while
Thread.sleep(clipTime);
} catch (InterruptedException e)
{
Log.d(TAG, "interrupted");
}
}
/**
* need to call this when completely done with recording
*/
public void done()
{
Log.d(TAG, "stop recording");
if (recorder != null)
{
if (isRecording())
{
stopRecording();
}
//now stop the media player
recorder.stop();
recorder.release();
}
}
public boolean isRecording()
{
return continueRecording;
}
public void stopRecording()
{
continueRecording = false;
}
}

I realize this is a year old, but I stumbled across it. I'm pretty sure that general, open domain sound recognition is not a solved problem. So, no, you're not going to find any kind of library to do what you want on Android, because such code doesn't exist anywhere yet. If you pick some restricted domain, you could train a classifier to recognize the kinds of sounds your interested in, but that would require lots of math, and lots of examples of each of the potential sounds. It would be pretty cool if the library you wanted existed, but as far as I know, the technology just isn't there yet.

Programmatically pressing a imageButton doesn't execute immediately

playTune is called whenever a user selects a button. The problem is the line myButton.setPressed(true); is only called after the tune is played. This defies logic, so I'm wondering whether Android lines up events in an queue? I would like the line myButton.setPressed(true); to be called as it appears in the code. i.e. before a note is played.
private void playTune() {
isTunePlaying = true;
//Get no of notes user selected
String selectedValue = noteCountValues[noteCountIdx];
Log.v(LOG_TAG, "selectedValue:"+selectedValue+" noteCountIdx:"+noteCountIdx);
int noOfNotes;
if ("ALL".equals(selectedValue)){
noOfNotes = 50;
}
else{
noOfNotes = Integer.parseInt(selectedValue);
}
TuneManager tuneManager = TuneManager.getInstance(this);
Log.v(LOG_TAG, "tuneNamesIdx:"+tuneNamesIdx);
Tune tune = tuneManager.getTune(tuneNamesIdx+1);
Log.v(LOG_TAG, " tuneTitle:"+tune.getTitle()+" tuneNoOfNotes:"+tune.getNotes().size());
//Initialise expectedNotes
expectedNotes = new StringBuffer();
//Get notes and play
List<Note> notes = tune.getNotes();
//for (Note note:tune.getNotes()){
for (int i=1; i<=notes.size() && i<=noOfNotes; i++){
Log.v(LOG_TAG, "i:"+i+" notesSize:"+notes.size()+" noOfNotes:"+noOfNotes);
//Highlight note
if (isHighlightNotesOn){
final View myButton = (ImageButton) findViewById(R.id.middle_c);
myButton.setPressed(true);
}
Note note = notes.get(i-1);
int notePos = soundManager.getNotePosition(note);
Log.v(LOG_TAG, "current note:"+note.getName()+" playing notePos:"+notePos+" setting duration:"+note.getDurationMS());
soundManager.playSound(notePos);
//Add to expectedNotes
expectedNotes.append(" "+note.getName()+",");
//Sleep for the duration
try{
Thread.currentThread().sleep(note.getDurationMS());//sleep for 1000 ms
}
catch(InterruptedException ie){
}
}
isTunePlaying = false;
//Initialise actualNotesPlayed i.e. start from after the tine has finished
actualNotesPlayed = new StringBuffer();
}

Not sure if this is what you are running into, but if you cause an event from something event driven (happening on the UI thread) it of course cannot be processed on the UI thread until after your current event handler returns.