I am trying to import some data from server to my android app. Server gets data from sql server and returns it throw a HttpRequest. I parse the data from HttpResponse and store it in sqlite database upon the JSONObject is parsed.
However the number of objects to be inserted is high and the operation may be canceled intentional(by pressing pause) or unintentional(internet problem). So I have below choices:
1- Ignore inserting into database after parsing JSONObject and wait for complete successful response from server: This solution is highly bad because if a problem occurs the user should start importing data again.
2- Make a feedback to server when I insert a row to database. So if I resume importing data just new records are imported from server: It is good but imposes extra network communication and also may affect performance.
3- Get one data packed file and try to parse it: So I am sure pausing the operation will not cause to data lost. But I prefer another solution rather than working with file.
What is the best way to handle this issue?
Thanks
I went through this choice two weeks ago.
I found the better way to manage these data is to build a web service in the server (for example, in PHP; i was using a wordpress blog), send to the web service an offset and limit request and then download the data in JSON format.
In this way, if you have 100 rows to query, you can tell the web service to give you the first 10 results, then you can parse and store them, and then, you can go back with another query asking the results from the row 11 to the row 20, and so on.
In this way, the app will manage many sequential requests (and the related low-weight JSON answers), instead of an extremely big JSON file.
You can also manage all these operation in background or with AsyncTask so, in the meantime, the user can work on the foreground activity.
Related
In my android application im using web-service to get information about food the problem that if i write appl instead of apple,row,skin it dosent fill automatically and an error message will be shown,, what i did i stored 2000 name from web service in sqlite and search through them is it wrong to do this ? and how can i search with misspelling in web-service without the error message ?
The api url http://api.nal.usda.gov/ndb/search/?format=json&q=apple&sort=n&max=25&offset=0&api_key=DEMO_KEY
Salam Enas,
If the data do change frequently (like a movies list) then it's not ok to store it in the sqlite and it's better to be stored in the server database.
If the data does not change frequently, then it's ok to store them on the sqlite database although the app size will increase by 2 Mb which is a drawback.
You may also want to consider other APIs if you are not committed to use this one.
When storing data on the server, HTTP requests burden can be alleviated by caching data on the mobile.
good luck :)
I am building an android application in which I need to parse the data from the Database through JSON using Volley. I have 17,000+ rows in the table, I need to fetch all the row from the table via JSON and I need to store it in SQLite. Once downloaded the user can modify the fields of all row. Once the fields have been modified. The whole data need to be sent back to the server(Like synchronize).
What my problem is, If I attempt to download this huge data to my app. It's getting crashed. Is there any way that I can do to make this app work. Also tell me how to parse huge data from the server using volley.
I think it crashes because of timeOutError. When volley has to parse a large amount of data, you should set the timeout for your volley request. Default timeout is 2500ms, set it to 10000ms or higher it'll work for ex.
strReq.setRetryPolicy(new DefaultRetryPolicy(10000,
DefaultRetryPolicy.DEFAULT_MAX_RETRIES, DefaultRetryPolicy.DEFAULT_BACKOFF_MULT));
It will not consume all the time that you allotted, but it will be disconnected once all the data is fetched
timeoutError will not give you crash, but onErrorResponse would be called, please make sure that tables that you are going to insert into were created before making insert operation.
1- Use Gson for parsing data and store in ormlite database.
2-After the modification to database tables you should be synchronize table row bye row using sync adapter to server.
3-Make Service for synchronize to server
I think you are getting OutOfMemoryException, am i right ?
Make paging for huge data, you can pull 100 by 100 or 1000 by 1000 from server,then you can store them in sqlite directly
Enable largheap in application manifest.
Use this link to know how to enable it
I am using retrofit to download large file.
Try to parse using jsonreader for large data.
I am not sure what type of server technology you are using. Downloading 17K + data and uploading the same amount overtime is I guess not the optimum use of technology.
Try some technology which reduces your burden about data sync and parsing. I am sure you are aware about Firebase by google. that one of the solution in this case.
Let me know if I misunderstood your question in this case.
I need some help, my app created a database on creation at the minute.
What I want is to get the information in a database on a server or PC and load it into my app.
It just needs to be from server to app, not back again.
I have no idea how I would do this, does anyone have any ideas?
first, I should say that your question is so wide. It contains alot of technologies.
So, I'll give you a breif walk through here. you should figure out the rest on your own. and then, you can come back and ask more specific questions.
That said, here is the path you should take:
On the server, implement a REST API that respond to the caller with the data in the server database.
example: http://your.server.com/api/somedata
calling this url with GET method should return the data you want in JSON format.
from your android application, call this url and save the response in a String.
parse the JSON String ( you can use gson ).
save the parsed data to your local database.
If you are the one developing the server api, then this book is a recommended reading:
RESTful Web Services
Update
Say you have a table in your server database called (TABLE) with the columns COL_1, COL_2, COL_3.
You could implement a php page called 'TABLE.php' that return the following:
{
"items": [
{"COL_1": "value_1", "COL_2": "value_2", "COL_3": "value_3"},
{"COL_1": "value_2", "COL_2": "value_3", "COL_3": "value_4"},
{"COL_1": "value_4", "COL_2": "value_5", "COL_3": "value_6"},
...
]
}
In your android application call http://your.server.com/api/TABLE.php page with GET method, parse the JSON returned (above) and save it to your local database.
You can do this for each database table on your server, until you have all your data saved locally in your android application.
This probably wont be the best design decision, but considering your knowledge about the topics, this could be a fair starting point.
Using HttpClient you can set things up to execute a POST to some servlet or similar app on the server. That app needs to then send data back to your Android app which is waiting for a response from the server. Then your app can parse the data and stuff it's own database.
Sync data from the server using json parser
I have developed android application for offline concept.When internet connected more than 4000 records synchronized from the server. If i add only one data in the server. when i do second time synchronizing, that particular data to be synchronize from the server.My problem is when i do second time synchronizing data it took same time as first time synchronizing. how to avoid this,please advice me.
Synchronizing working fine. But if i press the sync button second time. whole data can be synchronizing again. Please explain in detail. I have little bit idea i don't know whether it is correct or not.
My suggestion:
Flag to be set in the server side and client side. If data synchronized flag to be set as 0. if data newly synchronized it should be set as 1.
if data loss. unfortunatly data cache cleared in application. how to do this?. please explain me.
Check this video
Here a googler explain how to build REST applications - there is a lot of usefull thing about sync.
If you build server and client by youw own - you can store timestamp of the last connection and each time you synchronize you pass this time to the server and depend on it you can pass only modified columns (you have to store timeStamp of modification for data or list of modified columns from key timestamps)
First of all, the strategy that i have at the moment was achieved by searching here in stackoverflow, but by some reason, the "next step" was never mentioned here..
So i'm currently developing an android app that needs to fetch some data
from a remote database. I've searched in the web and at the moment i'm
able to get the data i want through a http request (rest webservice +
json response). My problem is, i can only fetch data of one database
query to my app in the same database session..
I wanted some strategy where i could send the http request, and them
perform 2 or 3 database querys and return them back to my app, without
the need to open 2 or 3 times a database connection, or sending one
http request for every query.
I know i can do some tricky manipulations with the json response, like
putting all the database query's response separated by some marks i
choose, but i was looking for some clean solution.
Can you help me achieve this pls?
Thanks
Unless you create some combined data structure which is the result of all the queries and then convert it to a JSON string, then your suggestion is the only other way to go, I feel. The combined data structure can then be recreated from the JSON received at the client end.