Ive tried a thousand things. As of right now the only way for me to query anything is to get the entire list and look through it that way! which takes way to much time. How can I query something in google app engine, for example pull only the entities that have > 100 votes for example.
Tried to user cursor but not sure how it works. I know it can use a cursor but how do I set it up with google app engine since my database isnt in my app per say??
Ive tried... but this dose not work at all..
Cursor cursor = ("select * from Votes WHERE Votes >" + 250 , null);
quotes endpoint.listquotes().setCursor(cursor).execute();
and
String query = ("select * from Votes WHERE Votes >= 40");
quotes endpoint.listquotes().setCursor(query).execute();
Im following the tic-tac-toe example https://github.com/GoogleCloudPlatform/appengine-endpoints-tictactoe-java and https://developers.google.com/eclipse/docs/endpoints-addentities In the example I just switched notes for quotes.
Heres my current code for example on how im getting the entities.
protected CollectionResponseQuotes doInBackground(Context... contexts) {
Quotesendpoint.Builder endpointBuilder = new Quotesendpoint.Builder(
AndroidHttp.newCompatibleTransport(),
new JacksonFactory(),
new HttpRequestInitializer() {
public void initialize(HttpRequest httpRequest) { }
});
Quotesendpoint endpoint = CloudEndpointUtils.updateBuilder(
endpointBuilder).build();
try {
quotes = endpoint.listquotes().execute();
for (Quotes quote : quotes.getItems()) {
if (quote.getVotes() > 3) {
quoteList.add(quote);
}
}
Here is the code that Google generated in the app engine for me when I created the endpoint. It looks like it will query somehow but I cant figure it out. They are two different projects.
#Api(name = "quotesendpoint", namespace = #ApiNamespace(ownerDomain = "projectquotes.com" ownerName = "projectquotes.com", packagePath = ""))
public class quotesEndpoint {
/**
* This method lists all the entities inserted in datastore.
* It uses HTTP GET method and paging support.
*
* #return A CollectionResponse class containing the list of all entities
* persisted and a cursor to the next page.
*/
#SuppressWarnings({ "unchecked", "unused" })
#ApiMethod(name = "listquotes")
public CollectionResponse<quotes> listquotes(
#Nullable #Named("cursor") String cursorString,
#Nullable #Named("limit") Integer limit) {
EntityManager mgr = null;
Cursor cursor = null;
List<quotes> execute = null;
try {
mgr = getEntityManager();
Query query = mgr.createQuery("select from quotes as quotes");
if (cursorString != null && cursorString != "") {
cursor = Cursor.fromWebSafeString(cursorString);
query.setHint(JPACursorHelper.CURSOR_HINT, cursor);
}
if (limit != null) {
query.setFirstResult(0);
query.setMaxResults(limit);
}
execute = (List<quotes>) query.getResultList();
cursor = JPACursorHelper.getCursor(execute);
if (cursor != null)
cursorString = cursor.toWebSafeString();
// Tight loop for fetching all entities from datastore and accomodate
// for lazy fetch.
for (quotes obj : execute)
;
} finally {
mgr.close();
}
return CollectionResponse.<quotes> builder().setItems(execute)
.setNextPageToken(cursorString).build();
In Google App Engine you need to set up a servlet to query the database for you and then return the results in JSON, see here for more information:
https://developers.google.com/appengine/docs/java/datastore/queries
https://github.com/octo-online/robospice
https://developers.google.com/appengine/docs/java/#Requests_and_Servlets
https://code.google.com/p/google-gson/
You would end up querying using http:// your-url/query? + query string
EDIT:
Preview!
This is a Preview release of Google Cloud Endpoints. As a result, the
API is subject to change and the service itself is currently not
covered by any SLA or deprecation policy. These characteristics will
be evaluated as the API and service moves towards General
Availability, but developers should take this into consideration when
using the Preview release of Google Cloud Endpoints.
Most likely the cursor function is still in development. But I'm also unsure why you would want to use Cursors, as Collections are so much easier to work with... Wouldn't you prefer to do what's below then the awful code above? :)
ScoreCollection scores = service.scores().list().execute();
Update your list method to take in a filter attribute
#SuppressWarnings({ "unchecked", "unused" })
#ApiMethod(name = "listZeppaUserInfo")
public CollectionResponse<ZeppaUserInfo> listZeppaUserInfo(
#Nullable #Named("filter") String filterString,
#Nullable #Named("cursor") String cursorString,
#Nullable #Named("limit") Integer limit) {
PersistenceManager mgr = null;
Cursor cursor = null;
List<ZeppaUserInfo> execute = null;
try {
mgr = getPersistenceManager();
Query query = mgr.newQuery(ZeppaUserInfo.class);
if (isWebSafe(cursorString)) {
cursor = Cursor.fromWebSafeString(cursorString);
HashMap<String, Object> extensionMap = new HashMap<String, Object>();
extensionMap.put(JDOCursorHelper.CURSOR_EXTENSION, cursor);
query.setExtensions(extensionMap);
} else if (isWebSafe(filterString)){
// query has a filter
query.setFilter(filterString);
}
if (limit != null) {
query.setRange(0, limit);
}
execute = (List<ZeppaUserInfo>) query.execute();
cursor = JDOCursorHelper.getCursor(execute);
if (cursor != null)
cursorString = cursor.toWebSafeString();
// Tight loop for fetching all entities from datastore and
// accomodate
// for lazy fetch.
for (ZeppaUserInfo obj : execute)
;
} finally {
mgr.close();
}
return CollectionResponse.<ZeppaUserInfo> builder().setItems(execute)
.setNextPageToken(cursorString).build();
}
Related
I want to load several contacts via Xamarin.Contacts.AddressBook, at the moment I have something like:
var loookupIDs = /* load 10 saved contact IDs */
var addressBook = new AddressBook(context) { PreferContactAggregation = true };
foreach(var id in loookupIDs)
{
var contact = addressBook.Load(id);
names.Add(contact.DisplayName);
}
However, this is really slow (tested on Android device) - even just loading 10 contacts. Is there a way to batch up the loading so it's faster? Or is the only option to use platform specific APIs instead of the Xamarin wrapper.
Yes, Xamarin.Mobile is kind of slow. It combines all possible contacts (phones, mails, etc) and all possible fields, which is not recommended by Android reference manual.
I recommend you to use native way to query your contacts with Cursor and filter it for your needs. Sadly, Xamarin dev mixed up all constants, so it is not trivial task.
Here is complete example
public class PhoneContactInfo
{
public string PhoneContactID { get; set; }
public string ContactName { get; set; }
public string ContactNumber { get; set; }
}
public IEnumerable<PhoneContactInfo> GetAllPhoneContacts(IEnumerable<int> filterIds = null)
{
Log.Debug("GetAllPhoneContacts", "Getting all Contacts");
var arrContacts = new System.Collections.Generic.List<PhoneContactInfo>();
PhoneContactInfo phoneContactInfo = null;
var uri = ContactsContract.CommonDataKinds.Phone.ContentUri;
string[] projection = { ContactsContract.Contacts.InterfaceConsts.Id,
ContactsContract.Contacts.InterfaceConsts.DisplayName,
ContactsContract.CommonDataKinds.Phone.Number
};
//String[] strings = filterIds.Select(k => Convert.ToString(k)).ToArray();
//string whereClause = ContactsContract.Contacts.InterfaceConsts.Id + " = ? ";
var cursor = MainActivity.ContextHolder.ContentResolver.Query(uri, projection,
null,
null,
null);
cursor.MoveToFirst();
while (cursor.IsAfterLast == false)
{
int phoneContactID = cursor.GetInt(cursor.GetColumnIndex(ContactsContract.Contacts.InterfaceConsts.Id));
if (filterIds.Contains(phoneContactID))
{
String contactNumber = cursor.GetString(cursor.GetColumnIndex(ContactsContract.CommonDataKinds.Phone.Number));
String contactName = cursor.GetString(cursor.GetColumnIndex(ContactsContract.Contacts.InterfaceConsts.DisplayName));
phoneContactInfo = new PhoneContactInfo()
{
PhoneContactID = Convert.ToString(phoneContactID),
ContactName = contactName,
ContactNumber = contactNumber
};
arrContacts.Add(phoneContactInfo);
}
cursor.MoveToNext();
}
cursor.Close();
cursor = null;
Log.Debug("GetAllPhoneContacts", "Got all Contacts");
return arrContacts;
}
If you wish to add some fancy async
public Task<IEnumerable<PhoneContactInfo>> GetAllPhoneContactsAsync(IEnumerable<int> filterIds)
{
return Task.FromResult(GetAllPhoneContacts(filterIds));
}
Also take a look at commented whereClause. You possibly can construct 'SQL like' where clause to make this query even more faster. Just build a string with several '=' and 'or'
P.S.
I didn't measure performance differences, if anyone has decent statistics i will be grateful
It looks like you access AdressBook for each loookupID, this might cause your speed issue.
Try:
1) Fetch all contacts, or only those you might be interested in. (Use Linq)
2) Do further work with found contacts
Example from Xamarin docs:
http://blog.xamarin.com/introducing-xamarin-contacts/
var book = new AddressBook (this) {
PreferContactAggregation = true
};
foreach (Contact c in book.Where (c => c.LastName == "Smith")) {
print (c.DisplayName);
foreach (Phone p in c.Phones)
print ("Phone: " + p.Number);
foreach (Email e in c.Emails)
print ("Email: " + e.Address);
}
I am querying CouchBaseLite view but I am getting this exception every time I query it. It is not returning the result.
detailMessage: last sequence < 0(-1)
View:
com.couchbase.lite.View viewItemsByDate = database.getView(String.format("%s/%s", designDocName, byDateViewName));
viewItemsByDate.setMap(new Mapper() {
#Override
public void map(Map<String, Object> document, Emitter emitter) {
Object createdAt = document.get("text");
if (createdAt != null) {
emitter.emit(createdAt.toString(), null);
}
}
}, "1.0");
and My query to this view is:
com.couchbase.lite.View view = database.getView(byDateViewName);
Query query = view.createQuery();//database.createAllDocumentsQuery();
List<Object> keyArray = new ArrayList<Object>();
keyArray.add("A");
keyArray.add("B");
query.setKeys(keyArray);
QueryEnumerator rowEnum = query.run();
for (Iterator<QueryRow> it = rowEnum; it.hasNext();) {
QueryRow row = it.next();
Log.d("Document ID:", row.getDocumentId());
}
And I have documents in the database with key "text": "A" and "text": "B" but still it is showing me exception. last sequence < 0(-1)
Why it could not find the rows that I queried for?
The problem is that you are querying a different view name than the view name you defined.
Change
database.getView(String.format("%s/%s", designDocName, byDateViewName));
to
database.getView(byDateViewName);
and it fixes the problem.
See Couchbase Lite Android issue #66
I'm trying to work on the sample GAE / Android app. There is Place Entity.
In generated PlaceEndpoint class there is a method:
#ApiMethod(name = "listGame")
public CollectionResponse<Place> listPlace(
#Nullable #Named("cursor") String cursorString,
#Nullable #Named("limit") Integer limit) {
EntityManager mgr = null;
Cursor cursor = null;
List<Game> execute = null;
try {
mgr = getEntityManager();
Query query = mgr.createQuery("select from Place as Place");
if (cursorString != null && cursorString != "") {
cursor = Cursor.fromWebSafeString(cursorString);
query.setHint(JPACursorHelper.CURSOR_HINT, cursor);
}
if (limit != null) {
query.setFirstResult(0);
query.setMaxResults(limit);
}
execute = (List<Game>) query.getResultList();
cursor = JPACursorHelper.getCursor(execute);
if (cursor != null)
cursorString = cursor.toWebSafeString();
// Tight loop for fetching all entities from datastore and accomodate
// for lazy fetch.
for (Game obj : execute)
;
} finally {
mgr.close();
}
return CollectionResponse.<Game> builder().setItems(execute)
.setNextPageToken(cursorString).build();
}
As I understand cursor and limit all optional params.
However I can't figure out how to pass them using Placeednpoint class on the client side:
Placeendpoint.Builder builder = new Placeendpoint.Builder(AndroidHttp.newCompatibleTransport(), new JacksonFactory(), null);
builder = CloudEndpointUtils.updateBuilder(builder);
Placeendpoint endpoint = builder.build();
try {
CollectionResponsePlace placesResponse = endpoint.listPlace().execute();
} catch (Exception e) {
e.printStackTrace();
Normally, when params are not nullable I would pass them in endpoint.listPlace() method. But when params are nullable, client side app doesn't see alternative constructor, that would accept params.
How am I supposed to pass them then?
For passing parameters from client side while sending a query request through cloud endpoints, you need to add provision for setting parameters. To send the required parameter from android , the class where you would define the REST Path and method type, should include an option to the set cursor and limit. For example for the String cursorstring :
#com.google.api.client.util.Key
private String cursorstring;
public String getCursorstring() {
return cursorstring;
}
public ListPlace setCursorstring(String cursorstring) {
this.cursorstring = cursorstring;
return this;
}
Finally while calling the endpoint method from your android code, you should pass a value using the setCursorstring, which will be something like:
CollectionResponsePlace placesResponse = endpoint.listPlace().setCursorstring("yourcursorstring").execute();
There is an additional way. You can simply set your nullable value 'cursor' it like this:
CollectionResponsePlace placesResponse =
endpoint.listPlace().set("cursor", "Your Cursorstring").execute();
Where you can
Faced the same issue but backend developed using Java.
The solution provided by Tony m worked like a charm.
Simply had to adapt how the method is called on Android and it worked.
I have read several posts here on speed issues when looping through a cursor and tried the answers given in these posts such as e.g. do not use getcolumnindex in the loop call this once etc.
However with a database having around 2400 records it takes around 3 to 5 minutes to finish.
The loop is running in an async task method so that it does not hang up the device and the database is handled via a database adapter.
The loop code is as follows :
while (!exportrec.isAfterLast()) {
if ( exportrec.moveToNext() ) {
fulldate = exportnumberformatter(exportrec.getInt(daye))
+"/"+exportnumberformatter(exportrec.getInt(monthe))+"/"
+String.valueOf(exportrec.getInt(yeare));
fulltime = exportnumberformatter(exportrec.getInt(houre))+":"
+exportnumberformatter(exportrec.getInt(mine))+":"
+exportnumberformatter(exportrec.getInt(sece));
noiseid = exportrec.getInt(typee);
exportedinfo += exporttypes[id] +","+exportrec.getString(notee)+","+
fulldate+","+fulltime+" \n" ;
}
}
The exportnumberformatter does the following :
public String exportnumberformatter(int i) {
String result = Integer.toString(i);
if (result.length() >1 ) {
return Integer.toString(i);
}
String zeroprefix = "";
zeroprefix = "0"+result;
return zeroprefix ;
}
The cursor is called as follows before the loop to get the data :
exportrec = MD.GetAllLogs(2, "date_sort");
exportrec.moveToFirst();
The MD is the database adapter and the GetAllLogs Method (this has been played with to try and speed things up and so the date_sort that is used is really ignored here):
public Cursor GetAllLogs(Integer i,String sortfield)
{
String sorted = "";
if (i == 1 ) {
sorted = "DESC";
} else if (i == 2) {
sorted = "ASC";
}
return mDB.query(DB_TABLE, new String[] {COL_ID, COL_TYPE,COL_IMAGE, COL_INFO,COL_IMAGE,COL_HOUR,COL_SEC,COL_MIN,COL_DAY,COL_MON,COL_YEAR,COL_SORT_DATE},
null, null, null, null, COL_ID+" "+sorted);
}
When I created the table in the database it had no indexes so I created these via the upgrade method. However they did not error or appear to fail when I did this but what I do not know is A) does the database/table need rebuilding after an index is created and B) how to tell if they have been created ? the two indexes were based on the ID as the first and a field that holds the year month day hour minute second all in on Long Integer.
I am concerned that the loop appears to be taking this long to read through that many records.
Update:
rtsai2000's and the suggestion from CL answer has improved the speed from minutes to seconds
Your exportedInfo String is growing and growing. Save the results in an array and Stringify later (such as with StringBuilder).
You are not closing your cursor after reading the records.
List<String> exportedInfo = new ArrayList<String>();
Cursor exportrec = GetAllLogs();
try {
while (exportrec.moveToNext()) {
String info = String.format("%s, %s, %02d/%02d/%02d, %02d:%02d:%02d",
exporttypes[id],
exportrec.getString(notee),
exportrec.getInt(daye),
exportrec.getInt(monthe),
exportrec.getInt(yeare),
exportrec.getInt(houre),
exportrec.getInt(mine),
exportrec.getInt(sece));
exportedInfo.add(info);
}
} finally {
exportrec.close();
}
return exportedInfo;
I have a SQLite table (on Android) that has numerous fields, but certain fields are repeated/denormalized. I would like to select a distinct set of this data and use them as actual objects.
Example
books table
title summary author
Little Johnny A funny kid Johnny Himself
Big Johnny A funny adult Johnny Himself
I would like to extract one author from this list ("Johnny Himself") and would expect I should be able to do this with ORMLite instead of manually with Java.
I would like to select a distinct set of this data and use them as actual objects.
ORMLite supports a distinct() method on the QueryBuilder that should do what you want. So your code would look something like:
List<Book> results = booksDao.queryBuilder()
.distinct().selectColumns("author").query();
In this case, the resulting Book objects would only have the author field set and not the id field or anything else. If you just wanted the author names instead of objects then you could do:
GenericRawResults<String[]> rawResults =
booksDao.queryRaw("SELECT DISTINCT author FROM books");
for (String[] resultColumns : rawResults) {
String author = resultColumns[0];
...
}
This is my application code
public class DbHelper<T> {
private Class<T> c;
private DatabaseHelper db;
public DbHelper(Class<T> c) {
this.c = c;
db = DatabaseHelper.getInstance();
}
This is a good idea
public List<T> queryForBuilderDistinct(int offset, int limit, String ColumnsName,
String orderName, boolean isAsc) {
try {
Dao<T, Integer> dao = db.getDao(c);
QueryBuilder<T, Integer> queryBuilder = dao.queryBuilder();
if (offset != 0) {
queryBuilder.offset((long) offset);
}
if (limit != 0) {
queryBuilder.limit((long) limit);
}
if (orderName != null) {
queryBuilder.orderBy(orderName, isAsc);
}
queryBuilder.distinct().selectColumns(ColumnsName);
return dao.query(queryBuilder.prepare());
} catch (SQLException e) {
LogUtil.e(TAG, "queryForBuilderDistinct", e);
}
return new ArrayList<T>();
}