So i've recently started using Xamarin.UITest in my mobile application. Its great for testing the UI, navigation etc. However im slightly confused as to how the best way is to test all the other parts of the app i.e the database, web connection, model and so on.
The project is cross-platform using a shared project.
Should these other tests be in along with the UITests or do they need their own separate project?
Furthermore in Xamarin Studio on OSX is there a way to test particularly the .Net only code without having to build and launch the whole iOS app?
Thanks
This question is really broad.
UI tests should only test the UI itself
Does navigation work correctly?
Is everything displayed like it should?
Are UI components visible, enabled, read-only etc.
Do components display correct data?
Your business logic should be tested in a unit test project
Do your classes process incoming data correctly?
Do they trigger the correct methods with the correct parameters in all cases?
Do they prepare the correct calls to webservices, databases or third party libraries? (You don't test the correct behaviour of other pieces of software here)
Do they throw exceptions as expected?
Unit tests are helping you the most to fix and avoid bugs. They are in general much more important for high quality code than all the other tests.
Integration tests are used to test the rest
Do databases, webservices, file system, operating system API work like expected?
Does it all work correctly together?
Does it all work fast enough?
There are many good books and links about good testing. It's way too much then to do it here.
Just a few examples:
http://blog.stevensanderson.com/2009/08/24/writing-great-unit-tests-best-and-worst-practises/
https://developer.xamarin.com/guides/ios/deployment,_testing,_and_metrics/touch.unit/
http://blog.falafel.com/hold-right-learning-xamarin-unit-tests/
How to do integration testing in .NET with real files?
With these links, StackOverflow and Google you will find out that your unit tests should stay in a separate project. Mock your business logic as good as possible in your UI tests. So that they don't test all your code. Otherwise you will result in UI tests which often fail in order to changes inside your business logic. You would have to change the UI tests everytime you change the unit tests. That becomes annoying and frustrating quickly.
The main scenario that you're asking about is an integration test.
Since your tests drive the app from the UI layer, it's of course invoking all the code in the app and touching all the backend systems. But you're hoping to validate that the app and its supporting set of backend services all work correctly.
First of all, you'll need a way to expose data about your backend service to the test. You already did this, since the app probably uses a REST API and a database. For tests, you'd probably only want to do this for a staging or pre-production endpoint, somewhere that you won't be at risk for exposing real data by accident. You could configure a simple RESTful API that gives information about your backend services, and then invoke that from the test. You could also use backdoor methods which, in turn, call the API and prevent you from needing to make extra API calls from the test. Let's take a basic example, creating a new user.
Your test code might look something like this pseudocode:
var uniqueUser = Guid.NewGuid().ToString();
app.EnterText(firstName, "Test 1");
app.EnterText(lastName, uniqueUser);
app.Tap(createUserButton);
app.WaitForNoElement(e => e.Id("progress"));
app.WaitForElement(e => e.Text("User successfully created"));
// At this point, call your API using RestSharp or whatever makes sense for you
string endpoint = "https://my.server.net/user/getByLastName/" + uniqueUser;
var client = new RestClient(endpoint);
var result = ExecuteRequestHereAndItShouldReturnUserJSON();
Assert.IsTrue(result != null && result.FirstName == "Test 1");
That type of thing.
On teardown, you'd want to invoke a method to delete the test user by its id to avoid clogging up your database with test users created by the thousands of devices in Test Cloud. You could do this via backdoor or REST, whichever is safer depending on your current setup.
As for testing the code itself, just create a separate unit test project and use NUnit. There are a lot of great resources on testing listed in the other answer that can help you understand how to write good unit tests, when/how to use mocking, etc.
Related
I am looking for ways to basically manipulate data that is shown in my app by intercepting the API calls it makes and then basically manipulate the data ( positive and negative scenarios) and then observe how my App behaves. I have been manually intercepting these calls and doing it in Charles Proxy tool but I want to get rid of it and looking for a way to do this through Automation. Set up my test data intercept the API calls through a proxy and then run my functional tests on the App. I am looking to do this for both Android and IPhone Apps.
Note: I have been using Appium for Automation.
Appium is a black-box test automation tool, it has no API to manipulate with application on server side. On Android, it allows you to start Activity/Fragment with pre-defined state, but it is not equal to what you are doing.
I think the right way here is to mock your server, there are multiple libraries, one example.
Then you can make a build for Android/iOS apps to use your mocked server instead of real server. In this case you will pre-define response instead of overriding it with proxy.
You can ask your developer to log the things you want to intercept or if they are already being intercepted in logs.You just need to collect the logs from ADB and use that in your appium code.
You can get the adb logs using :
List<LogEntry> logEntries = driver.manage().logs().get("logcat").getAll();
for(LogEntry entries:logEntries){
if(entries.getMessage().contains("*Things you need from logs*")){
entry= entries.getMessage();
}
}
return entry;
I am trying to create the bare minimum of a chatbot with dialogflow in an Android app. Initially I do not need the bot to hook into things or have context and entities. I just want it to match questions with answers as good as it can.
I do not come from the web world so this https://dialogflow.com/docs/getting-started/basic-fulfillment-conversation confuses me. The final part of the tutorial is not finished and it leaves me hanging a bit.
As an Android dev I want the app to:
call a rest api with the user text
or possibly:
invoke an SDK and have a local pretrained agent and not involve the internet an all
You can download the Agent but that is just some json files. I found an Dialogflow.SDK but it seems to be related to speech. In the dialog flow console you can 'copy curl':
curl 'https://api.dialogflow.com/v1/query?v=20170712&query=Hi&lang=en&sessionId=xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx' -H 'Authorization:Bearer xxxxxxxxxxxxxxxxxxxxxxxxx'
I want to access the agent like that from multiple instances of an app.
Do you always need a 'fulfillment'?
The guide mentions creating folders on your local machine(in conjunction with fulfillment) are these only used to do a 'deploy' from your local machine to configure the web hook in the cloud? Or are they also used to deploy the entire Agent to the cloud project? (which i can't seem to find any instructions for, unless it's the same as for webhooks?)
gcloud beta functions deploy weatherWebhook --stage-bucket [BUCKET_NAME] --trigger-http
I have read questions with both tags DialogFlow and Android but most of them seem to involve Google Android Assistent which I have no interest in.
You do not need fulfillment. This is primarily for Dialogflow to take advanced actions or to use more complicated logic to handle the response.
You also cannot run Dialogflow processing on your device - all operations take place in the cloud. So you don't need to download the zip file unless you want to make a backup copy of your agent.
The REST interface you're looking for is the one you found. It handles more than speech - the /query endpoint can take a query parameter which contains "Natural language text to be processed."
I need some advices and guidance related to TDD of android applications
I'm developing a complex android application (an automated robo player for a web game). Now i have a pilot for it, and before start continue with it's development, i want to make full TDD for it.
The main purpose of this application is run as a background service that communicates with a http server - it will send http requests with json content
It has to 2 modes of running:
- service mode - it will run in background- read some data configuration from database and communicate with the server
- GUI mode - communication with server on demand + configuration for running in service mode.
GUI layer is designed as MVP, so all business done in a module (a MVP unit) is separated from view, therefore, all app logic is "NON android dependent".
I have several big layers that i need to make unit tests for:
- domain data access :
database storage (ormlite)
android specific storage - shared prefs..,
files
- server communication - http client (spring android rest template)
http content conversion (gson)
- background services - android services
- GUI - activities
- app business - android independent - algorithms, computations, ...
In conclusion will need TDD - for every of these layers individually, but also integration tests for a complete flow.
tests included :
- database DAO tests
- http client requests
- GSON conversion TDDs
- business logic - simple tests for methods
- unit testsfor running and scheduled background services
- activity unit testing
- test suites (service + DAO + json conversion + http requests)
My first question is what TDD frameworks would be best for my needs? Should i use mocking or not ?
Please give me some directions to go for. Thanks
At first, for automatic testing : junit.
For fluent assertions, you could look at fest-assertions. They provides a lot of examples, here. But enjoy this one on the String :
assertThat("Frodo").startsWith("Fro").endsWith("do").hasSize(5);
It isn't just harmonious, it increase also your productivity.
Since you want to write unit tests, you will have to write mock objects. Of course, you can do it without any framework, but it is very tedious.
Personnaly, i am big fan of Mockito. It's api is great and very fluent. Besides, you could the use the BDDMockito static methods to write test in a BDD manner. Look :
//given
given(seller.askForBread()).willReturn(new Bread());
//when
Goods goods = shop.buyBread();
//then
assertThat(goods, containBread());
Sometimes you will depend on class that you can't mock, for example a final class like BluetoothAdapter. In these cases (and only in these), you could use PowerMock.
Look also at this stuff :
https://stackoverflow.com/questions/2336488/resources-for-tdd-on-android
[Why Android isn’t ready for TDD, and how I tried anyway](
http://daverog.wordpress.com/2009/12/14/why-android-isnt-ready-for-tdd-and-how-i-tried-anyway/)
Introduction to Android Testin
There is also a book : Android Application Testing Guide
Since you mentioned integration tests, you might also be interested in RoboGherk as an automated acceptance testing framework for Android that was recently announced by my friend at LeanDog.
The comment for the class BillingService recommends that:
You should modify and obfuscate this code before using it.
OK, but what must be modified?
The class name? The TAG used for logging? Method names and data members? The logic and program flow itself? Other?
In other words, I can understand the need for obfuscation, but how can I get away with implementing the recommendation without rewriting everything from scratch (potentially with bugs that are worse than not modifying anything)?
I'm working on this at this moment and my approach, so far, is as follows:
I'm using the BillingReceiver, Billing Service, PurchaseObserver and ResponseHandler.
I've moved all the Constants into my own Constants class and all the above classes are included in my own package.
I've done away with the PurchaseDatabase class and integrated parts of it into my own SQLite database, DBAdapter and data access classes.
I've changed the CatalogEntry into my own model object and my UI will be quite different to the example e.g. RadioButton group rather than Spinner for product items (I only have 4).
It says in the Security class 'For a secure implementation, all of this code should be implemented on a server that communicates with the application'. I'm 'fortunate' that my app has to contact my server anyway so I'll be implementing these security measures on the server and I'll be doing my own validation of the purchase info passed to the server. I'm looking to secure this part of the comms using SSL and I already require a prior username/password (hashed and salted) which is stored on my server.
I'm cutting out any other superfluous code which I'm not using e.g. payload editing.
Some of the methods have 5 or 6 parameters in their signature e.g. onPurchasestateChanged() - I was thinking about combining these into a single wrapper object (but haven't done so yet).
I'm testing it slowly and thoroughly, so that I understand what's going on, and following the recommendations. I used the complete sample at first to make sure it worked and tested the static responses. Then I started making my own changes while still doing static testing. I'm still testing with static responses and I will follow the flow of messages to understand the interchanges going on. Once I'm happy with this I'll test with my own product Id's and try and satisfy myself on the data and its security.
I've thought that the developerPayload string could also be signed and encypted and when returned to my server, decrypted and checked for integrity.
Finally, I'll obfuscate the code using ProGuard and follow some of the tips for doing so which are available on StackOverflow.
Hope this helps.
No good news here: You need to change anything you can, in addition to using Proguard. This includes merging classes, splitting them, moving certain methods from one module to another, and especially encrypting the purchase information stored into the database, as the description for the PurchaseDatabase class suggests:
You should use an obfuscator before storing any information to
persistent storage. The obfuscator should use a key that is specific
to the device and/or user. Otherwise an attacker could copy a database
full of valid purchases and distribute it to others.
The reason is that with tools like AntiLVL it is very easy to compare your decompiled (obfuscated!) code to the original sample and deduct from it whatever needed to compromise it. It is impossible to completely prevent cracking, but you should try to make it as difficult as possible.
They explain it as follows:
The in-app billing sample application is publicly distributed and can
be downloaded by anyone, which means it is relatively easy for an
attacker to reverse engineer your application if you use the sample
code exactly as it is published. The sample application is intended to
be used only as an example. If you use any part of the sample
application, you must modify it before you publish it or release it as
part of a production application.
In particular, attackers look for known entry points and exit points
in an application, so it is important that you modify these parts of
your code that are identical to the sample application.
It means don't use the code as provided, change some part of it so that hackers won't be able to know what code you use.
Basically, I don't think they meant the billingService itself, but the way you use it in your application.
I'm writing an Android app which sometimes needs to request data through HTTP from a REST API. I'm using the Apache DefaultHttpClient for performing requests. Is there a way to write tests for this app and "replace" DefaultHttpClient's response when running the tests so that test results are always consistent?
As an example of the things I'd like to test, one of the web services I'm accessing takes a string and performs a text search, returning a paged list of objects. I need to test the cases where the list is empty, the list fits in the first page, or the list is larger than a page and the app needs to make several requests to get the complete list.
I'm not the developer of this web API nor can modify its responses, so I can't change what it returns. For the above example, if I want to test the case where the list returned is empty, I could just search for a string which I'm sure won't return any results, but the other two cases are harder because what the service can return is always changing.
I think ideally I would have a way to get a modified DefaultHttpClient when running tests, that returns a hardcoded result for requests to a given URL instead of actually doing the network request. This way I would always get consistent results independently of the real web service's response.
I'm currently using Robotium for testing but I'm open to using other tools too.
Yes, you can definitely "fake" responses when using the HttpClient framework. It's quite convoluted, and I will have to leave most of the details up to you, but I will give you a quick overview:
Implement ClientHttpRequestFactory, mainly so you can override the createRequest() method so you can...
Return your custom implementation of ClientHttpRequest, in which you can override the execute() method so you can ...
Return your custom implementation of ClientHttpResponse in which you will finally be able to return your fake response data, e.g. getBody() can return the content of a file, you can hardcode the headers in getHeaders(), etc.
The rest is figuring out how to best tie all these shenanigans to your service layer.
You might give Charles a try for something like this. Sorta a non-code solution.
http://www.charlesproxy.com/
I use the Charles' reverse proxies and the map local tool for things like this.
What you do is point your request at your local box on the reverse proxy port. Charles in turn can be configured to provide a static hard-coded flat file but to your app it looks like a 100% genuine web service response.
There are lots of other cool things you can do with Charles - watch traffic from your android app to and from your server and breakpoints (which allows you to tweak requests and responses before they are sent and received). Definitely worth checking out.
Another option is to use Dependency Injection so that you can change the HttpClient when running the tests. Check out Guice if you are interested.
I'm guessing you are interested in writing functional tests using the standard Android Junit testing framework. So you could just implement the parts of the API you are using on your own webserver and point at that server when running your tests.
If you'd prefer your tests to be self-contained, you could implement an Http server that runs on the device. Examples of using the Http server available in the Android class library are here and here.