I am trying to automate one part of an android application I wrote. I this activity I have a edittext which is used to enter a product number.
Once the product number is entered the user clicks on the software keyboards send button to initiate a search for that product, however I am not able to simulate the softkeyboard's send/enter key using java-appium.
The following code works well in android 4.4.4 and below but not in android 6.0+
element.sendKeys(productNumber + "\n");
The code below does not work either
element.sendKeys(productNumber + Keys.ENTER);
I have found that only this solution is the most reliable one - mocking manual submit button tap.
static void submit() {
Dimension screen = mobileDriver.manage().window().getSize();
mobileDriver.tap(1, screen.getWidth() - 20, screen.getHeight() - 20);
}
Related
I am trying to receive the scanned barcode result from a device paired via (Bluetooth/USB) to an android device.
so many topics said :
most plug-in barcode scanners (that I've seen) are made as HID profile devices so whatever they are plugged into should see them as a Keyboard basically.
source
So I am using this code to receive the result of the scan:
#Override
public boolean dispatchKeyEvent(KeyEvent event) {
if (viewModel.onTriggerScan()) {
//1
char pressedKey = (char) event.getUnicodeChar();
viewModel.addCharToCode(pressedKey);
//2
String fullCode = event.getCharacters();
viewModel.fullCode(fullCode);
//check if the scan is done, received all the chars
if (event.getAction() == EditorInfo.IME_ACTION_DONE) {
//does this work ?
viewModel.gotAllChars();
//3
String fullCode2 = event.getCharacters();
viewModel.fullCode(fullCode2);
}
return true;
} else
return super.dispatchKeyEvent(event);
}
Note: I don't have a barcode scanner device for the test.
which code will receive the result ?? (1 or 2 or 3 ?)
You won't ever see an IME_ACTION_DONE, that's something that's Android only and an external keyboard would never generate.
After that, it's really up to how the scanner works. You may get a full key up and key down for each character. You may not, and may receive multiple characters per event. You may see it finish with a terminator (like \n) you may not- depends on how the scanner is configured. Unless you can configure it yourself or tell the user how to configure it, you need to be prepared for either (which means treating the data as done either after seeing the terminator, or after a second or two once new data stops coming in.
Really you need to buy a configurable scanner model and try it in multiple modes and make ever mode works. Expect it to take a few days in your schedule.
Workaround solution but it works 100%.
the solution is based on clone edittext (hidden from the UI), this edit text just receives the result on it, adds a listener, and when the result arrives gets it and clears the edittext field. An important step, when you try to receive the result(trigger scan) make sure that edittext has the focus otherwise you wil not get the result.
Quick steps:
1- create editText (any text field that receives inputs) in your layout
2- set its visibility to "gone" and clear it.
3- add onValueChangeListener to your edittext.
4- focus your edittext when you start trigger the scan
5- each time you the listener call, get the result and clear edittext
Note: never miss to focus your edittext whenever you start trigger scan.
Note: this method work(99%) for all external scan device and any barcode type.
I'm trying to incorporate Pepper's built in Android tablet more in DialogFlow interactions. Particularly, my goal is to open applications installed on the tablet itself for people to use while they're talking with Pepper. I'm aware there is a 'j-tablet-browser' app installed on Pepper's end that can let a person browse the tablet like an ordinary Android device, but I would like to take it one step further and directly launch an Android app, like Amazon's Alexa.
The best solution I can up with is:
User says specific utterance (e.g. "Pepper, open Alexa please")
DialogFlow launches the j-tablet-browser behavior
{
"speak": "Sure, just a second",
"action": "startApp",
"action_parameters": {
"appId": "j-tablet-browser/."
}
}
User navigates the Android menu manually to tap the Alexa icon
My ideal goal is to make the process seamless:
User says specific utterance (e.g. "Pepper, open Alexa please")
DialogFlow launches the Alexa app installed on the Android tablet
Does anyone have an idea how this could be done?
This is quite a broad question so I'll try and focus on the specifics for launching an app with a Dialogflow chatbot. If you don't already have a QiSDK Dialogflow chatbot running on Pepper, there is a good tutorial here which details the full process. If you already have a chatbot implemented I hope the below steps are general enough for you to apply to your project.
This chatbot only returns text results for Pepper to say, so you'll need to make some modifications to allow particular actions to be launched.
Modifying DialogflowDataSource
Step 2 on this page of the tutorial details how to send a text query to Dialogflow and get a text response. You'll want to modify it to return the full reponse object (including actions), not just the text. Define a new function called detectIntentFullResponse for example.
// Change this
return response.queryResult.fulfillmentText
// to this
return response.queryResult
Modifying DialogflowChatbot
Step 2 on this page shows how to implement a QiSDK Chatbot. Add some logic to check for actions in the replyTo function.
var response: DetectIntentResponse? = null
// ...
response = dataSource.detectIntentFullResponse(input, dialogflowSessionId, language)
// ...
return if (reponse.action != null) {
StandardReplyReaction(
ActionReaction(qiContext, response), ReplyPriority.NORMAL
)
} else if (reponse.answer != null) {
StandardReplyReaction(
SimpleSayReaction(qiContext, reponse.answer), ReplyPriority.NORMAL
)
} else {
StandardReplyReaction(
EmptyChatbotReaction(qiContext), ReplyPriority.FALLBACK
)
}
Now make a new Class, ActionReaction. Note that the below is incomplete, but should serve as an example of how you can determine which action to run (if you want others). Look at SimpleSayReaction for more implementation details.
class ActionReaction internal constructor(context: QiContext, private val response: DetectIntentResponse) :
BaseChatbotReaction(context) {
override fun runWith(speechEngine: SpeechEngine) {
if (response.action == "launch-app") {
var appID = response.parameters.app.toString()
// launch app at appID
}
}
}
As for launching the app, various approaches are detailed in other questions, such as here. It is possible to extend this approach to do other actions, such as running or retrieving online data.
Hi i have written testCases using espresso for the Android app.
Below is my code. my requirement is i need to manally Login to my app by entering credentials before i test the title bar Text.
So i am going into sleep for 2 min. when i enter credentials and click Login button below error is coming.
"D/InputEventConsistencyVerifier: TouchEvent: Touch event stream contains events from multiple sources: previous device id 0, previous source 2, new device id 0, new source 1002 "
Please let me know how to achieve this....
#Test
public void checkTitleBarText() throws InterruptedException {
sleep(120000);
onView(withId(R.id.drawer_layout)).perform(actionOpenDrawer());
Assert.assertEquals("SomeText", (String) textView.getText());
}
Why can't you just login with espresso? I'd be much easier...
However, If it's really your requirement, You can always launch tests with debuger attached and put a breakpoint in test code (which is stopping test thread). Login manually then and resume execution.
Intro: I want to create a POC on Android Security which requires to identify if there is any KeyLogger running on Android device or not. And if it is running or installed on device then, disable it throughout my Android application.
Queries:
1.) Is this possible to create Android keyloggers which intercepts keyboard events and running in background as services?
2.) Is this possible to identify if any of the background process handelling keyboard events?
3.) Can I stop any other background service (not owned by me) by my application code?
Please help me with suitable links if you have.
I know this question already has an accepted answer but I disagree with the accepted answer!
This can be done in android via the accessibility APIs.
Take a look at the below program :
Type Machine
It uses the accessibility APIs, and correctly stores every key stroke you type in any application which is essentially what a key logger does!
EDIT : I am not exactly sure what TypeMachine uses but you should be able to get the text from accessibility service using this method.
For Example, the following code can be used to get a new text:
void onViewTextChanged(AccessibilityEvent accessibilityEvent, AccessibilityNodeInfo accessibilityNodeInfo) {
List text = accessibilityEvent.getText();
CharSequence latestText = (CharSequence) text.get(0);
Log.i(MY_TAG, latestText.toString());
}
As a follow up on #ananth's answer, here is a complete code example on how to implement a keylogger using Accessibility Service.
But, this requires permissions to bind your Service with the system and also the user has to explicitly turn on the Service you create by navigating to Settings>Accessibility>{Your app name} on Android devices. So, if you have evil intensions, good luck with that.
Step 1: Paste this in your manifest file.
<application>
...
<service android:name=".MyAccessibilityService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
</service>
</application>
Step 2: Create a class for your Accessibility Service.
public class MyAccessibilityService extends AccessibilityService {
#Override
public void onAccessibilityEvent(AccessibilityEvent event) {
final int eventType = event.getEventType();
String eventText = null;
switch(eventType) {
/*
You can use catch other events like touch and focus
case AccessibilityEvent.TYPE_VIEW_CLICKED:
eventText = "Clicked: ";
break;
case AccessibilityEvent.TYPE_VIEW_FOCUSED:
eventText = "Focused: ";
break;
*/
case AccessibilityEvent.TYPE_VIEW_TEXT_CHANGED:
eventText = "Typed: ";
break;
}
eventText = eventText + event.getText();
//print the typed text in the console. Or do anything you want here.
System.out.println("ACCESSIBILITY SERVICE : "+eventText);
}
#Override
public void onInterrupt() {
//whatever
}
#Override
public void onServiceConnected() {
//configure our Accessibility service
AccessibilityServiceInfo info=getServiceInfo();
info.eventTypes = AccessibilityEvent.TYPE_VIEW_TEXT_CHANGED;
info.feedbackType = AccessibilityServiceInfo.FEEDBACK_SPOKEN;
info.notificationTimeout = 100;
this.setServiceInfo(info);
}
}
That's it. Now, everything the user types in any app on their phone will be logged to the console. You can configure the above class to listen to the keystrokes from certain specified apps only if you want to. The above class is independent of your MainActivity. The service will get registered as soon as your app is installed. But, again, in order to work, this requires manual toggling of settings as mentioned previously.
Read the docs for detailed explanation on what Accessibility is and details on each of the classes and methods used above.
After research for 1 whole day I reached at below conclusion.
Android does not allow you to intercepts default soft keyboard inputs from background services. The only way to intercepts these events are custom keyboards.
I have summarized it as follows:
In Android Key logging for keyboard events is not supported in background services. Some of the links are as follows:
Point 1: Google Android Developer
As soft input methods can use multiple and inventive ways of inputting text, there is no guarantee that any key press on a soft keyboard will generate a key event: this is left to the IME's discretion, and in fact sending such events is discouraged. You should never rely on receiving KeyEvents for any key on a soft input method. In particular, the default software keyboard will never send any key event to any application targetting Jelly Bean or later, and will only send events for some presses of the delete and return keys to applications targetting Ice Cream Sandwich or earlier.
http://developer.android.com/reference/android/view/KeyEvent.html
Android Jelly Bean is: 4.1 to 4.3.1
Android IceCream Sandwich: 4.0
Key presses on soft input methods are not required to trigger the methods in this listener, and are in fact discouraged to do so. The default android keyboard will not trigger these for any key to any application targetting Jelly Bean or later, and will only deliver it for some key presses to applications targetting Ice Cream Sandwich or earlier.
http://developer.android.com/reference/android/text/method/KeyListener.html
Point 2: Stack Overflow
KeyEvents can only be handled by Activities as they are the interface to the user pressing the keys and only when they are in the foreground. Even Services that run in the background are not intended to react on user input.
Android - Listener for hard keys press at background
Is it possible to create an Android Service that listens for hardware key presses?
Point 3: Romain Guy
Romain Guy (https://stackoverflow.com/users/298575/romain-guy) who works for Google also confirms it
onKeyDown in a service? (Global Hot Keys)
Point 4: Some Other reference:
Google android-developers Group : https://groups.google.com/forum/#!topic/android-developers/o--GUWmqXdI
It can be done only by using Custom KeyBoard: get pressed key and throw another key in android
Please add your comments if you think that I have missed anything.
I'm creating an Android app for a chinese client and they need map integration, so Google maps is not an option since all Google services are blocked in China. I'm trying to use Baidu maps, which is called Baidu LBS (location-based services) cloud.
Getting a basic map with no overlays to work was relatively easy. The process is described here (in Chinese, but the code speaks for itself if you don't understand the language). Downloading the latest Baidu Android SDK (v3.2.0 at time of writing) and integrating it into my Eclipse project as a library was no problem, but don't trust the documentation in that link too much even though it is the official one. Their examples often contain code that wouldn't even compile. The name of the .jar file for example was completely different from what you see in their screenshot.
Oh and also their .jar library is obfuscated which is super annoying to work with :-(
I needed to register a Baidu account and go to their control center to generate a key. To create an access key ("ak") for mobile you need to enter the SHA1 fingerprint of the keystore which signs your app, followed by the package name specified in your manifest.
Then I added the generated key to my manifest under the tag
<meta-data android:name="com.baidu.lbsapi.API_KEY" android:value="xxx...xxx" />
I then copied code from their sample project's CloudSearchActivity because I have specific coordinates that I would like to display. I implemented the CloudListener interface as shown:
#Override
public void onGetSearchResult(final CloudSearchResult result, final int error)
{
Log.w("onGetSearchResult", "status=" + result.status + ". size=" + result.size + ". total=" + result.total + ". error=" + error);
if(null != result && null != result.poiList && 0 < result.poiList.size())
{
mBaiduMap.clear();
final BitmapDescriptor bitmapDescriptor=BitmapDescriptorFactory.fromResource(R.drawable.icon_address_grey);
LatLng latitudeLongitude;
LatLngBounds.Builder builder=new Builder();
for(final CloudPoiInfo info : result.poiList)
{
latitudeLongitude=new LatLng(info.latitude, info.longitude);
final OverlayOptions overlayOptions=new MarkerOptions().icon(bitmapDescriptor).position(latitudeLongitude);
mBaiduMap.addOverlay(overlayOptions);
builder.include(latitudeLongitude);
}
final LatLngBounds bounds=builder.build();
MapStatusUpdate mapStatusUpdate=MapStatusUpdateFactory.newLatLngBounds(bounds);
mBaiduMap.animateMapStatus(mapStatusUpdate);
}
}
And I added code to launch a query (also copied from their sample project):
#Override
public View onCreateView(final LayoutInflater layoutInflater, final ViewGroup viewGroup,
final Bundle savedInstanceState)
{
// initialize needs to be called
SDKInitializer.initialize(getApplication());
CloudManager.getInstance().init(MyFragment.this);
view=(ViewGroup)layoutInflater.inflate(R.layout.fragment_map, viewGroup, false);
mMapView=(MapView)view.findViewById(R.id.baiduMapView);
mBaiduMap=mMapView.getMap();
NearbySearchInfo info=new NearbySearchInfo();
info.ak="xxx...xxx";
info.geoTableId=12345;
info.tags="";
info.radius=30000;
info.location="116.403689,39.914957";
CloudManager.getInstance().nearbySearch(info);
return view;
}
Unfortunately I keep getting a status value of 102 from the server (according to this API page that means STATUS_CODE_SECURITY_CODE_ERROR. Now I don't know what to do.
Things that I don't understand:
Why do I need to repeat my access key ("ak") when building the query? Is it not enough to have it in the manifest once?
What is this "geoTableId" value in the query supposed to be?
Any ideas?
After many hours of research I have made some progress on the open questions.
The reason for the "ak" field in a cloud search query is not duplication, it is in fact a different access key. Somewhere in a hidden place Baidu says that access keys "for mobile" will not work for these cloud searches, you need an ak "for server". So the solution is to go back to the Baidu control center and create another key "for server". This key needs to be used in the query, while the "for mobile" key needs to remain in the manifest.
geoTableId is an identifier of your account, not unsimilar to the access keys. It is a (currently) 5 digit number that you need to obtain in the Baidu control center. The other keys were generated in the tab titled "API控制台" (API control desk), but for the geoTableId you need to switch to the tab called "数据管理" (data management). There I think I needed to press the "创建" (~create) button on top left, then enter a name, select "是" (yes) where they ask if this is for release (not sure about that translation) and then click "保存" (save). After this, your freshly generated number is displayed in the top field in parentheses behind the name you chose just now.
These steps have allowed me to send "successful" queries where the server answers with status 0 (STATUS_CODE_SUCCEED). However, so far all the answers I get are empty, I have yet to find a query which produces a non-empty answer. If anyone manages to do that, please let me know!