I have an application in android, a database on the server and I wrote api in django. I want for my application to work online and offline. So far I did local database in android and api returns query results in json. (database is very simply, just one table with 7 fields). But I don't know nothing about databases performance. I wonder if a good strategy is to getting all rows from table (from server), delete all rows from local database, and save all new rows to local database? Or update all fields in existing rows? Or maybe online/offline app is not a good idea? How long can take downloading 1000 rows from database on server?
Maybe this is silly question, but really, I'm looking for a solution ;)
my model in Django looks like that:
class Quote(models.Model):
user = models.ForeignKey(User)
quote = models.TextField(verbose_name="Quotation")
accepted = models.BooleanField(default=False, verbose_name="Accepted")
deleted = models.BooleanField(default=False, verbose_name="Deleted")
date = models.DateTimeField(verbose_name="Date", auto_now=True)
good = models.PositiveIntegerField(default=0, verbose_name="Rate good")
bad = models.PositiveIntegerField(default=0, verbose_name="Rate bad")
class Favourite(models.Model):
user = models.ForeignKey(User)
quote = models.ForeignKey(Quote)
And I send by json only table Quote (username, quote, accepted, deleted, date, good, bad).
An api function:
#csrf_exempt
def api(request):
if request.method == 'POST':
operation = request.POST['operation']
print 'operation', operation
response_data = {}
response_data['result'] = 'FAILED'
if operation == 'getFive':
itemsList = []
ID = int(request.POST['id'])
if ID == -1:
querySet = Quote.objects.filter(deleted=False).order_by('-id')[:5]
else:
querySet = Quote.objects.filter(id__lt=ID, deleted=False).order_by('-id')[:5]
maxID = Quote.objects.filter(deleted=False).order_by('-id')[0].id
minID = Quote.objects.filter(deleted=False).order_by('id')[0].id
for item in querySet:
tmp = {}
tmp['id'] = item.id
tmp['username'] = item.user.username
tmp['quote'] = item.quote
tmp['accepted'] = item.accepted
tmp['deleted'] = item.deleted
tmp['date'] = dateformat.format(item.date, "Y-m-d H:i:s")
tmp['good'] = item.good
tmp['bad'] = item.bad
itemsList.append(tmp)
print item.id, item.quote
response_data['result'] = 'SUCCESS'
response_data['items'] = itemsList
response_data['maxID'] = maxID
response_data['minID'] = minID
response_data['items'] = itemsList
return HttpResponse(json.dumps(response_data), content_type="application/json")
return redirect("/")
My problem is that I don't want to user wait very long time to run the application and I don't how long data can be downloaded by smartphone with HSDPA/3G/LTE etc. where are ~1000 rows in table. Let's say there is ~1500 characters by one row. If I counted good it's 1500 B * 1000 rows = 1500000 Bytes = 1464 KB = ~1,42 MB... So I think it's much, but I'm not an expert.
I forgot add, that when row have 'deleted'=True, I don't send it to smartphone.
And my question again: Sending all rows from server to android app, every time when app starts, is a good idea?
Thanks, and sorry for my english ;).
I can explain a solution for ya, but I cannot really show you how to do it.
You only want to get all rows if its the first time the user is opening your app. So youll do an update if not exist to your local DB.
When you are doing this, you will want it to be in the background so the user can go ahead and start poking around your application. Maybe do a toast to display that the background thread is downloading the records and it might take a while.
Break up the download with a cursor to only download x records at a time. Download 50 and return, download 50 and return. That way the user can see the records being downloaded and they do not have to wait until ALL records are returned.
Related
I just start using Firebase in Android and I wrote query like this.
messageSearchQuery = FirebaseDatabase.getInstance().getReference().child("Message")
.child(conversationID).limitToLast(INITIAL_MESSAGE_COUNT)
.orderByChild("timestamp").startAt(participant.joiningDate).endAt(endAt);
I can only give limitToFirst, limitToLast, startAt, endAt. (I think there is no center)
I need to give timestamp (e.g 10 am sharp) and i wanna query 10 message before 10 am and 10 message after 10 am. (but i don't know when it start, when it end).
Edit
What I want to do is something like this.
messageSearchQuery = FirebaseDatabase.getInstance().getReference().child("Message").
child(conversationID).
orderByChild("timestamp").equalTo(endAt).limitToFirst(INITIAL_MESSAGE_COUNT).limitToLast(INITIAL_MESSAGE_COUNT);
But if I do, it crash like this.
java.lang.IllegalArgumentException: Can't call limitToLast on query with previously set limit!
How shall I do?
My data in firebase is something like this.
You can select a range by combining startAt and endAt, both of which operate on the child you order on. For example if you want messages between 10am and 10pm on a certain date, you'd do:
long timestampOf10am = ...;
long timestampOf10pm = ...;
messageSearchQuery = FirebaseDatabase.getInstance().getReference().child("Message")
.child(conversationID)
.orderByChild("timestamp").startAt(timestampOf10am).endAt(timestampOf10pm)
.limitToLast(INITIAL_MESSAGE_COUNT);
Update: If you need 10 items before 10am and 10 items after 10am, you'll need to fire two queries and merge the results client-side.
long timestampOf10am = ...;
beforeMessageSearchQuery = FirebaseDatabase.getInstance().getReference().child("Message")
.child(conversationID)
.orderByChild("timestamp").startAt(timestampOf10am)
.limitToFirst(INITIAL_MESSAGE_COUNT);
afterMessageSearchQuery = FirebaseDatabase.getInstance().getReference().child("Message")
.child(conversationID)
.orderByChild("timestamp").endAt(timestampOf10am)
.limitToLast(INITIAL_MESSAGE_COUNT);
I'm pretty new to Android app development so please forgive my naivety.
I'm currently trying to develop an app that can pull data from a google spreadsheet and write data to it.
I've completed the quickstart tutorial so my code is the same as that right now. It all works correctly.
My issue is I need to be able to read from my own spreadsheet and I don't really understand the code used so I'm struggling to know where to start.
I've looked at this to try and implement the authorisation in fewer steps which I thought might make the code easier to understand - but again my lack of knowledge means I don't know how this fits into all the methods I currently have.
I've looked at the developer documentation and tried to replace the code from the quickstart which retrieves data from the app with this:
ValueRange result = service.spreadsheets().values().get(spreadsheetId, range).execute();
int numRows = result.getValues() != null ? result.getValues().size() : 0;
System.out.printf("%d rows retrieved.", numRows);
But again this is different to the code I already have so doesn't fit in as the getDataFromApi() method requires a return statement. I've tried just changing the spreadsheetId to that of my own spreadsheet and changing the range value to the cells I need,
/**
* Fetch a list of names and majors of students in a sample spreadsheet:
* https://docs.google.com/spreadsheets/d/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/edit
* #return List of names and majors
* #throws IOException
*/
private List<String> getDataFromApi() throws IOException {
String spreadsheetId = "1-hL78Jm9-HijPx9UHthFxcXatkIhA2FR-AQ1lrCUbEg";
String range = "Go Mix 12 0506!B6:D";
List<String> results = new ArrayList<String>();
ValueRange response = this.mService.spreadsheets().values()
.get(spreadsheetId, range)
.execute();
List<List<Object>> values = response.getValues();
if (values != null) {
results.add("Name, Major");
for (List row : values) {
results.add(row.get(1) + ", " + row.get(3));
}
}
return results;
}
but I'm clearly going about it wrong because I get this error:
Invalid index 3, size is 3
I've also followed up to video 15 on this tutorial but it bypasses the need for any authorisation so didn't help.
Basically I've hit a brick wall and need someone to explain to me in simple terms how I can either work with the code from the quickstart tutorial to work with my own spreadsheet and fix that error! OR (more preferable to myself) explain how to do it myself from scratch. NB the spreadsheet cannot be public so I will need authorisation.
I hope that all makes sense!
You are asking to retrieve three columns of data (B through D), but then attempting to read a fourth column (rows.get(3)). In Java (and in most programming languages), lists and arrays are 0-based. So if you want the first item, you call list.get(0), and if you want the third you call list.get(2). This is evident based on the fact that the error message tells you your index of 3 is invalid since the list is size 3 -- you're attempting to read something beyond the end of the list.
I have a simple firebase database: /rides is a list of simple objects like this
{
car: "Toyota"
minutes: 15
}
and I need to display sum of minutes of all the rides. The obvious solution is to load all the rides and calculate the sum. But if I have several hundreds of rides this is very slow, up to several seconds.
So it seems I have to maintain a separate field /totalMinutesin the database for this. But thus I will have to manually update /totalMinutes every time I add/remove/change a ride. Anyway this is not a big deal of work.
But what if I need to calculate total minutes only for a subset of rides? For instance only for "Toyota" cars or "Ford" cars? Manual maintaining /totalMinutesFord, /totalMinutesToyota now doesn't seem so easy.
So what is the correct way to maintain such dynamic values in firebase?
Firebase has no way to get automatically calculate values based on the data in your database.
So your two options are:
calculate the value whenever you update the data
retrieve all the data and calculate the value on the client
You already (wisely) decided that retrieving all data is not a good idea. Your users will be grateful for that.
So that leaves calculating the derived values whenever you update the data of a ride. I'm not sure why doing that for multiple values would be more difficult than doing it for a single value. It may be more code, but it's pretty much the same code:
var ride = { car: "Toyota", minutes: 15 };
ref = new Firebase('https://yours.firebaseio.com/');
ref.child('rides').push(ride);
ref.child('totalMinutes').transaction(function(current_value) {
return (current_value || 0) + ride.minutes;
});
ref.child('totalMinutes'+ride.car).transaction(function(current_value) {
return (current_value || 0) + ride.minutes;
})
I am developing the following functionality:
an user picks a date and gets ListView populated by SimpleCoursorLoader (queries are executed in the background).
User frequently choices adjacent dates and there might be a lot of duplicate queries.
I tested the application and discovered that in case of high frequency requests - it runs very slow.
In order to speedup my application I decided to implement cache where results of queries will be stored. Key - date and value-?
Is it worth doing and what techniques could you advice?
1) Yes, it's really worth doing since DB access is relatively slow (even with such a great thing like SQLite)
2) Considering what I've got from your post I'd suggest using LongSparseArray: key will be date from database (long), stored value - your cached data object (Bundle etc). The reasons are it's:
naturally sorted
sort order is maintained on changes
fast
memory efficient
3) When you need to load overlapping/adjacent interval you have to check bounds and load only absent part
4) If a situation is possible when you cache non-adjacent intervals - you need to manage loaded intervals bounds as well. But if you do it only for list scroll purposes you may omit this (if you don't stop loading data on fling gesture)
About my experience: I've got about 3 times payoff using caching. But actual results depends on database scheme etc. You may get even more
I found MatrixCursor useful for the purpose of caching. I keep HashMap.
Logic: if no request has been done - issue it, get Cursor, convert it to MatrixCursor and write to cache.
Here is the snippet for convertion:
private MatrixCursor cursorToMatrixCursor(Cursor c) {
MatrixCursor result = new MatrixCursor(c.getColumnNames());
if (c.moveToFirst()) {
do {
ArrayList<String> columnValues = new ArrayList<>();
final int nOfColumns = c.getColumnCount();
for(int col = 0; col < nOfColumns; ++col)
columnValues.add(c.getString(col));
result.addRow(columnValues);
} while (c.moveToNext());
}
return result;
}
I am sure there is an easy solution to this one but I figured I would check with all the folks here first.
I am working on a database creation and management application for android where the user creates and manages data along the line of what PHPMyAdmin does for regular computers.
I have the section where the user creates a database and can insert tables with the appropriate styled data into the system.
The next priority is selecting which DB to enter and modify its contents. Is there a way to display the available databases, along with its table contents, in the form of a list-view for the user to enter and edit the desired data??
I know that this is a rather dull question, but this is basically the last piece of the puzzle for me to fit into this app before it is operational in a raw format. If you need any further information, or any code to examine, I will be happy to provide.
Thanks again for everyone's assistance.
Here's a good tutorial on SQLite databases and displaying contents in a ListView:
http://www.vogella.com/articles/AndroidSQLite/article.html#databasetutorial
It doesn't go over editing that much, but it's easy to see where he puts the values into the database.
thenewboston on YouTube is a good resource for Android tutorials and he goes over SQLite databases:
http://www.youtube.com/watch?v=gEg9OdufXmM
It's pretty comprehensive and slow if you already kinda know what you're doing so here is where he goes over inserting data/editing the database if you just wanna jump to that:
http://www.youtube.com/watch?v=S3Z4e7KgNdU
I know this below can be optimized but for now just create such a method to do it automatically. The method...
Creates an empty database of a random name for a second,
Saves the new database's location - getDatabasePath,
Quikly deletes the empty database,
Removes filename form the saved path to get the directory path olny,
List all files in the database path excluding '-journal' files.
And it goes like this:
ArrayList<String> arr_list_of_db_files = getDBFILES();
pivate ArrayList<String> getDBFILES()
{
ArrayList<String> arr = new ArrayList<String>;
String db_path, rand_name, str_tmp;
//ad.1-2. random file name for db
rand_name = new Random().nextInt((4000000-2000+1)+2000).toString()+".db";
db_path = openOrCreateDatabase(rand_name, MODE_PRIVATE, null).getPath();
//ad.3.
deleteDatabase(rand_name);
//ad.4.
db_path = db_path.replace("/" + rand_name, "");
//ad.5.
File [] files = new File(db_path).listFiles();
if (files == null) { return null; }
//so now we get the filenames one by one
for (int i = 0; i < files.length; i++)
{
str_tmp = files[i].getName();
if (!str_tmp.endsWith("-journal"))
{ arr.add(str_tmp); }
}
return arr;
}
Btw, I cant test the code, but I hope it is fine and some of you find it useful.
Edited: I have optimized the above code.