I've been trying to build some functionality into my app too allow user-generated data (EEG recordings) to be sent to a central BigQuery database.
I've never done any networking code in Java before, so I shied away from doing the POST or REST-based strategies recommended here. The BigQuery Java client library seemed to be exactly what I needed, though I was completely confused why it wouldn't officially support Android.
Still, I came across this example Android app (from Google no less) that promised to do exactly what I wanted with the BigQuery Client library. I incorporated it into my app as follows:
// .... in an AsyncTask
#Override
protected String doInBackground(String... params) {
String CSV_CONTENT = params[0];
try {
AssetManager am = MainApplication.getInstance().getAssets();
InputStream isCredentialsFile = am.open(CREDENTIALS_FILE);
BigQuery bigquery = BigQueryOptions.builder()
.authCredentials(AuthCredentials.createForJson(isCredentialsFile))
.projectId( PROJECT_ID )
.build().service();
TableId tableId = TableId.of(DATASET,TABLE);
Table table = bigquery.getTable(tableId);
int num = 0;
Log.d("Main", "Sending CSV: ");
WriteChannelConfiguration configuration = WriteChannelConfiguration.builder(tableId)
.formatOptions(FormatOptions.csv())
.build();
try (WriteChannel channel = bigquery.writer(configuration)) {
num = channel.write(ByteBuffer.wrap(CSV_CONTENT.getBytes(StandardCharsets.UTF_8)));
} catch (Exception e) {
Log.d("Main", e.toString());
}
Log.d("Main", "Loading " + Integer.toString(num) + " bytes into table " + tableId);
} catch (Exception e) {
Log.d("Main", "Exception: " + e.toString());
}
return "Done";
}
This runs without any errors and fires off an API call that is detected by Google Cloud Storage. However, it returns error 200 (job was cancelled) every time. I don't understand how this could be since I'm not doing anything in the code to cancel the request and I don't see how the async task I put the call in could be cancelled.
Was this just a bad example app I copied and a bad usage of the BigQuery Client? If so, what's the best way to send data to BigQuery from Android?
Related
My server sends the list of videoID to Android. Now, I want to show Title, Thumbnail and Number of Comments on these videos in List View. I have done this in web using GET request to https://www.googleapis.com/youtube/v3/videos?part=snippet&id={VIDEO_ID}&key={YOUR_API_KEY} but how to do this in Android? Is there any YouTube SDK to initialize YouTube object? How do I retrieve this information from YouTube using VideoID?
EDIT: I have found a way to this using YouTube Data API Client Library for Java but It is giving runtime error without any explanation.
Here is the code I used
/**
* Define a global instance of a Youtube object, which will be used
* to make YouTube Data API requests.
*/
private static YouTube youtube;
youtube = new YouTube.Builder(new NetHttpTransport(), new JacksonFactory(), new HttpRequestInitializer(){
public void initialize(com.google.api.client.http.HttpRequest request) throws IOException {
}
}).setApplicationName("youtube-cmdline-search-sample").build();
// Call the YouTube Data API's videos.list method to retrieve videos.
VideoListResponse videoListResponse = youtube.videos().
list("snippet").setId(videoId).execute();
// Since the API request specified a unique video ID, the API
// response should return exactly one video. If the response does
// not contain a video, then the specified video ID was not found.
List<Video> videoList = videoListResponse.getItems();
if (videoList.isEmpty()) {
System.out.println("Can't find a video with ID: " + videoId);
return;
}
Video video = videoList.get(0)
// Print information from the API response.
}
YouTube provides (at least) two official libraries relevant to your question:
YouTube Android Player API
YouTube Data API Client Library for Java
As the name already suggests, the first library is specifically developed for the Android platform. Its focus is on enabling you to incorporate video playback functionality into an app by providing a player framework. If your goal is to enable users to simply play YouTube videos, then is probably easiest to implement. Do note that this library requires the official YouTube app to be installed on the device.
The second library is more generic (although there are separate instructions for using it on Android) and provides a wrapper around YouTube's Data API to make interfacing with it a little easier. Hence, it allows you to do basically everything you can also do with the web API. As such, it solves a different problem than the Android Player API and is more likely the way to go if you want full control over how you display video data in your own UI.
Your third option would be to do exactly what you did for your web-based solution: make the API call yourself, parse the response and bind up the relevant data to your UI components. Various networking libraries (i.e. Retrofit) can greatly simplify this process.
Refer my post here. I just tried this method for my project and it works very nicely. You don't need the above code or any google api jar imports. Just replace the HTTP request with your HTTP request.
Output is returned in JSON, for which you can use a JSON parser jar to retrieve the title,thumbnails and other details you may require, as I have described in my answer there.
Try this:
protected void requestYoutubeVideos(String text) {
try {
youtube = new YouTube.Builder(new NetHttpTransport(), new JacksonFactory(), new HttpRequestInitializer() {
public void initialize(HttpRequest request) throws IOException {
}
}).setApplicationName("My app name").build();
// Define the API request for retrieving search results.
YouTube.Search.List query = youtube.search().list("id");
// Set your developer key from the Google Cloud Console for
// non-authenticated requests. See:
// https://cloud.google.com/console
query.setKey(YOUTUBE_API_KEY);
query.setQ(text);
query.setMaxResults(NUMBER_OF_VIDEOS_RETURNED);
// To increase efficiency, only retrieve the fields that the
// application uses.
query.setFields("items(id)");
query.setOrder("viewCount");
// Restrict the search results to only include videos. See:
// https://developers.google.com/youtube/v3/docs/search/list#type
query.setType("video");
SearchListResponse searchResponse = query.execute();
List<SearchResult> list = searchResponse.getItems();
Log.e("Youtube search", "list ===> " + list);
//Get Info for each video id
for (SearchResult video: list) {
youtubeList.add(video);
YouTube.Videos.List query2 = youtube.videos().list("id,contentDetails,snippet,statistics").setId(video.getId().getVideoId());
query2.setKey(YOUTUBE_API_KEY);
query2.setMaxResults((long) 1);
query2.setFields("items(id,contentDetails,snippet,statistics)");
VideoListResponse searchResponse2 = query2.execute();
List<Video> listEachVideo = searchResponse2.getItems();
Video eachVideo = listEachVideo.get(0);
}
} catch (GoogleJsonResponseException e) {
Log.e("Youtube search", "There was a service error: " + e.getDetails().getCode() + " : "
+ e.getDetails().getMessage());
} catch (IOException e) {
Log.e("Youtube search", "There was an IO error: " + e.getCause() + " : " + e.getMessage());
} catch (Throwable t) {
t.printStackTrace();
}
}
and do not forget to call it from another thread:
new Thread(new Runnable() {
#Override
public void run() {
try {
requestYoutubeVideos("Harry el Sucio Potter");
} catch (Exception ex) {
ex.printStackTrace();
}
}
}).start();
I want to design an Android file viewer for Google Drive.
At first, I implemented the app by using of the Google Android API, as follows,
private void retrieveNextPage(){
if(mHasMore == false)
return;
Query query = new Query.Builder().setPageToken(mNextPageToken).build();
com.google.android.gms.drive.Drive.DriveApi.query(getGoogleApiClient(), query).setResultCallback(metadataBufferResultResultCallback);
}
However, the Android Drive API only allows the app to view and fetch the files that created by itself. I cannot access other files on the drive through the app.
Therefore, I turned to another option, directly manipulate the Java Drive API.
According to the example on developer guide for Java,
https://developers.google.com/drive/web/quickstart/quickstart-java
The users have to manually copy and paste the "Authorization Code" between the browser and app, which is not a practical way to acquire the Access Token in Android.
To come out a new way, I used the GoogleAuthUtil in Android API to acquire the Access Token, coincided with the GoogleCredential and Drive in Java API to fetch the file list, as follows,
private static List<File> retrieveFiles(Drive service) throws IOException{
List<File> result = new ArrayList<File>();
Files.List request = service.files().list();
do {
try{
FileList fileList = request.execute();
result.addAll(fileList.getItems());
request.setPageToken(fileList.getNextPageToken());
}catch (IOException e){
Log.d(dbgT + "JavaRetrieveFiles", "Retrieved Failed");
request.setPageToken(null);
}
}while (request.getPageToken() != null && request.getPageToken().length() > 0);
return result;
}
private class RetrieveTokenTask extends AsyncTask<String, Void, String>{
#Override
protected String doInBackground(String... params){
String accountName = params[0];
String scopes = "oauth2:" + "https://www.googleapis.com/auth/drive";
String token = null;
try{
token = GoogleAuthUtil.getToken(getApplicationContext(), accountName, scopes);
}
catch (IOException e){
Log.e(excpTAG, "IO Exception: " + e.getMessage());
}
catch (UserRecoverableAuthException e){
startActivityForResult(e.getIntent(), REQ_SIGN_IN_REQUIRED);
}
catch (GoogleAuthException e)
{
Log.e(excpTAG, "GoogleAuthException: " + e.getMessage());
}
return token;
}
#Override
protected void onPostExecute(String s){
super.onPostExecute(s);
//Get Access Token
Log.d( dbgT + "Token", s);
EditText tokenText = (EditText) findViewById(R.id.tokenText);
tokenText.setText(s);
EditText fileNameText = (EditText) findViewById(R.id.editTextMeta);
GoogleCredential credential = new GoogleCredential().setAccessToken(s);
HttpTransport httpTransport = new NetHttpTransport();
JsonFactory jsonFactory = new JacksonFactory();
Drive service = new Drive.Builder(httpTransport, jsonFactory, null).setHttpRequestInitializer(credential).build();
List<File> fileList;
try{
fileList = retrieveFiles(service);
for(int i=0; i< fileList.size(); i++)
fileNameText.append(fileList.get(i).getTitle());
}catch(IOException e){
Log.d(dbgT + "RetrieveFileList", "IO Exception" );
}
}
}
Unfortunately, the app always crashes by the causing of NetworkOnMainThreadException when request.execute() in retrieveFiles is invoked.
I checked my access token s, it is usually in form of ya29.xxx...etc., and it can also be passed to my other .NET program for retrieving files from Google Drive. Therefore I can certain the access token is correct.
So my question is, how to create a correct GoogleCredential by using of access token, instead of applying authorization code in setFromTokenResponse ?
Thanks in advance.
Many thanks for Andy's tips, this problem is simply caused by the network operations occurs on the main thread, which is a very basic newbie error.
The Drive in Google Drive SDK for Java, using network libraries without any background/thread worker, and now it is functional after I put the retrieveFiles() into background.
Applying the GoogleAuthUtil in Google Play Android SDK to acquire the access token, and followed by GoogleCredential+Drive in Java SDK that use the token to do the file operation in Google Drive.
This is a right way to avoid the scope restriction in Android SDK for Google Drive, allowing the developers to acquire the full permissive of accessing Google Drive.
Burcu Dogan wrote some example code showing how to sync a local preferences file to the user's Google Drive appfolder, found here: https://github.com/googledrive/appdatapreferences-android
I've converted this example to use the current Drive SDK, now shipping with Google Play Services.
If I update the cloud Drive file with device 1, and then run the following code on device 2, I'm getting a stale "modified" timestamp from the metadata. I'm assuming this is because the results are from a local cache of the Drive file:
Step 1. Look up the preferences file by name, with a query:
/**
* Retrieves the preferences file from the appdata folder.
* #return Retrieved preferences file or {#code null}.
* #throws IOException
*/
public DriveFile getPreferencesFile() throws IOException
{
if (mDriveFile != null)
return mDriveFile;
GoogleApiClient googleApiClient = getGoogleApiClient();
if (!googleApiClient.isConnected())
LOGW(TAG, "getPreferencesFile -- Google API not connected");
else
LOGD(TAG, "getPreferencesFile -- Google API CONNECTED");
Query query = new Query.Builder()
.addFilter(Filters.contains(SearchableField.TITLE, FILE_NAME))
.build();
DriveApi.MetadataBufferResult metadataBufferResult =
Drive.DriveApi.query(getGoogleApiClient(), query).await();
if (!metadataBufferResult.getStatus().isSuccess()) {
LOGE(TAG, "Problem while retrieving files");
return null;
}
MetadataBuffer buffer = metadataBufferResult.getMetadataBuffer();
LOGD(TAG, "Preference files found on Drive: " +
buffer.getCount());
if (buffer.getCount() == 0)
{
// return null to indicate the preference file doesn't exist
mDriveFile = null;
// create a new preferences file
// mDriveFile = insertPreferencesFile("{}");
}
else
mDriveFile = Drive.DriveApi.getFile(
getGoogleApiClient(),
buffer.get(0).getDriveId());
// Release the metadata buffer
buffer.release();
return mDriveFile;
}
Step 2. Get the metadata for the file:
// Get the metadata
DriveFile file;
DriveResource.MetadataResult result = file.getMetadata(getGoogleApiClient()).await();
Metadata metadata = result.getMetadata();
// Get the modified dates
metadata.getModifiedDate();
More curiously, after running the code below (which just lists the appdatafolder files and their content) the metadata modified date, fetched above, becomes correct!! Why???
/**
*
* Simple debug activity that lists all files currently in Drive AppFolder and their contents
*
*/
public class ActivityViewFilesInAppFolder extends BaseActivity {
private static final String TAG = "ActivityViewFilesInAppFolder";
private TextView mLogArea;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Add a text view to the window
ScrollView layout = new ScrollView(this);
setContentView(layout);
mLogArea = new TextView(this);
layout.addView(mLogArea);
ApiClientAsyncTask<Void, Void, String> task = new ApiClientAsyncTask<Void, Void, String>(this) {
#Override
protected String doInBackgroundConnected(Void[] params) {
StringBuffer result = new StringBuffer();
MetadataBuffer buffer = Drive.DriveApi.getAppFolder(getGoogleApiClient())
.listChildren(getGoogleApiClient()).await().getMetadataBuffer();
result.append("found " + buffer.getCount() + " files:\n");
for (Metadata m: buffer) {
DriveId id = m.getDriveId();
DriveFile file = Drive.DriveApi.getFile(getGoogleApiClient(), id);
DriveContents contents = file.open( getGoogleApiClient(),
DriveFile.MODE_READ_ONLY, null).await().getDriveContents();
FileInputStream is = new FileInputStream(contents.getParcelFileDescriptor()
.getFileDescriptor());
try {
BufferedReader bf = new BufferedReader(new InputStreamReader(is, Charsets.UTF_8));
String line=null; StringBuffer sb=new StringBuffer();
while ((line=bf.readLine()) != null ) {
sb.append(line);
}
contents.discard(getGoogleApiClient());
result.append("*** " + m.getTitle() + "/" + id + "/"
+ m.getFileSize() + "B:\n [" + sb.toString() + "]\n\n");
} catch (IOException e) {
throw new RuntimeException(e);
}
}
buffer.release();
return result.toString();
}
#Override
protected void onPostExecute(String s) {
if (mLogArea != null) {
mLogArea.append(s);
Map<String, ?> values = PreferenceManager
.getDefaultSharedPreferences(ActivityViewFilesInAppFolder.this).getAll();
String localJson = new GsonBuilder().create().toJson(values);
LOGD(TAG, "Local: " + localJson);
LOGD(TAG, "File: " + s);
}
}
};
task.execute();
}
}
Is the metadata reading from a cached local copy, unless something kicks it?
Does anyone know how to force these APIs to always pull the results from the remote Drive file?
I have an answer to your question. Well 'kind of answer', and I'm sure you will not be happy with it.
I had used RESTful API in my app before switching to GDAA. And after I did, I realized that GDAA, another layer with timing delays I have no control over, is causing issues in an app that attempts to keep multiple Android devices synchronized. See SO 22980497 22382099, 22515028, 23073474 and just grep for 'requestSync'.
I was hoping that GDAA implemented some kind of GCM logic to synchronize 'behind-the-scenes'. Especially when there is the 'addChangeListener()' method that seems to be designed for that. It does not look to be the case (at least not around Sept 2014). So, I backed off to a true-and-tested scheme of using RESTful API to talk to Google Drive with DataProvider and SyncAdapter logic behind it (much like shown in the UDACITY Class here).
What I'm not happy about, is somewhat ambiguous documentation of GDAA using terms like 'synchronize' not telling us if it is 'local' of 'network' synchronization. And not answering questions like the SO 23073474 mentioned above.
It appears (and I am not a Google insider) that GDAA has been designed for apps that do not immediately synchronize between devices. Unfortunately this has not been mentioned here or here - see 1:59, costing me a lot of time and frustration.
Now the question is: Should I (we) wait until we get 'real time' synchronization from GDAA, or should we go ahead and work on home-grown GCM based sync on top of RESTful-DataProvider-SyncAdapter?
Well, I personally will start working on GCM sync and will maintain an easy-to-use miniapp that will test GDAA behavior as new versions of Google Play Services come out. I will update this answer as soon as I have the 'test miniapp' ready and up in GitHub. Sorry I did not help much with the problem itself.
Well, I just found the secret sauce to trick the new Drive API into reading metadata from the remote file instead of the local cache.
Even though reading metadata doesn't require the file to be opened, it turns out that the file needs to be opened!
So the working code to read the latest metadata from the cloud is as follows:
DriveFile file;
// Trick Google Drive into fetching the remote file
// which has the latest metadata
file.open( getGoogleApiClient(), DriveFile.MODE_READ_ONLY, null).await();
DriveResource.MetadataResult result = file.getMetadata(getGoogleApiClient()).await();
Metadata metadata = result.getMetadata();
// Get the modified date
metadata.getModifiedDate();
Question for Google -- is this working as intended? The metadata is cached unless you first open the file read_only?
I'm migrating my app from C2DM to GCM but the server on which i'll be sending the registration id will still be entertaining C2DM ids. So my question is what can I do from my app so that server can identify that this is GCM id. Google recommends to send a new bit along with registration id. How can I do that. Please give an exmaple.
Here's what you need to do if you're not using AppEngine.
In your server, you should already have a table that stores all the registration id (reg_id). You need to add another column to this table, say 'is_gcm_reg_id'. You would default all existing rows to '0' for this column since they are currently all C2DM reg_ids. You could also just create a new table to store the GCM reg_ids, if you like.
Then in your GCM enabled application, you just need to let your server know the reg_id is a GCM reg_id. You didn't mention how you upload the reg_id to your server (Web Service call? Simple POST request?). If using Web Service call, just create a new method that will be used exclusively by your new GCM-enabled app. If using POST request, just add another key value pair like 'gcm=true' and your server should look for this kvp.
Finally, you should have all the pieces needed to know the reg_id is of GCM, and update the 'is_gcm_reg_id' column field (or a new GCM table) accordingly.
Assuming you're using App Engine to implement GCM/C2DM.
In the deviceInfo entity for your App Engine there is already a "type" field. This is the field you will modify to either say "ac2dm" or "gcm".
An easy way to do this is to create a new registration request that simply passes "gcm" instead of "ac2dm". Take a look in your AppEngine project and locate the RegistrationInfo class. Look at the register() method. Simply replace "ac2dm" with "gcm".
EX:
// original
public void register() {
log.info("register " + this);
try {
doRegister(getDeviceRegistrationId(), "ac2dm", getDeviceId(), getAccountName());
} catch (Exception e) {
log.info("Got exception in registration: " + e + " - " + e.getMessage());
for (StackTraceElement ste : e.getStackTrace()) {
log.info(ste.toString());
}
}
log.info("Successfully registered");
}
// new version
public void register() {
log.info("register " + this);
try {
doRegister(getDeviceRegistrationId(), "gcm", getDeviceId(), getAccountName());
} catch (Exception e) {
log.info("Got exception in registration: " + e + " - " + e.getMessage());
for (StackTraceElement ste : e.getStackTrace()) {
log.info(ste.toString());
}
}
log.info("Successfully registered");
}
You will probably have to recompile the RPC service to get this to work.
Now, it's up to you to make sure your server checks for the "gcm" tag of the deviceInfo entity and acts accordingly. Also, read this if you haven't seen it https://developer.android.com/guide/google/gcm/c2dm.html
I'm trying to indicate the authentication / sync status of an account using the AccountAuthenticator and SyncAdapter. I've been through the samples, and can get it working alright.
How can I set the indicator to red just like the GMail account?
I'd also like to add additional status indicators on the sync adapter page. See picture below:
Answering my own question for future team knowledge...
Getting the indicator to change color was fairly easy after some experimentation. Start by creating a project based on thecode supplied in the SDK sample projects, modify as follows:
1) Fake the initial login from the server during the AuthenticationActivity. Once past the initial check, the system will start it's periodic sync attempts.
/**
* Called when the authentication process completes (see attemptLogin()).
*/
public void onAuthenticationResult(boolean result) {
Log.i(TAG, "onAuthenticationResult(" + result + ")");
// Hide the progress dialog
hideProgress();
// Override the result, we don't care right now....
result = true;
if (result) {
if (!mConfirmCredentials) {
finishLogin();
} else {
finishConfirmCredentials(true);
}
} else {
Log.e(TAG, "onAuthenticationResult: failed to authenticate");
if (mRequestNewAccount) {
// "Please enter a valid username/password.
mMessage.setText(getText(R.string.login_activity_loginfail_text_both));
} else {
// "Please enter a valid password." (Used when the
// account is already in the database but the password
// doesn't work.)
mMessage.setText(getText(R.string.login_activity_loginfail_text_pwonly));
}
}
}
2) Modify the "onPerformSync()" method within the SyncAdapter. The key here are the "syncResult.stats" fields. While modifying them, I found that inserting multiple errors didn't get the effect I wanted. Also noting that the counts didn't seem to be recorded across sync attempts (i.e. the fails always come in as zero). The "lifetimeSyncs" is a static variable that keeps count across sync attempts. This modified code will continue to alternate between green and red...
#Override
public void onPerformSync(Account account, Bundle extras, String authority, ContentProviderClient provider, SyncResult syncResult) {
List<User> users;
List<Status> statuses;
String authtoken = null;
try {
// use the account manager to request the credentials
authtoken = mAccountManager.blockingGetAuthToken(account, Constants.AUTHTOKEN_TYPE, true );
// fetch updates from the sample service over the cloud
//users = NetworkUtilities.fetchFriendUpdates(account, authtoken, mLastUpdated);
// update the last synced date.
mLastUpdated = new Date();
// update platform contacts.
Log.d(TAG, "Calling contactManager's sync contacts");
//ContactManager.syncContacts(mContext, account.name, users);
// fetch and update status messages for all the synced users.
//statuses = NetworkUtilities.fetchFriendStatuses(account, authtoken);
//ContactManager.insertStatuses(mContext, account.name, statuses);
if (SyncAdapter.lifetimeSyncs-- <= 0 ){
//mAccountManager.invalidateAuthToken(Constants.ACCOUNT_TYPE, authtoken);
syncResult.stats.numAuthExceptions++;
//syncResult.delayUntil = 60;
lifetimeSyncs = 5;
}
} catch (final AuthenticatorException e) {
syncResult.stats.numParseExceptions++;
Log.e(TAG, "AuthenticatorException", e);
} catch (final OperationCanceledException e) {
Log.e(TAG, "OperationCanceledExcetpion", e);
} catch (final IOException e) {
Log.e(TAG, "IOException", e);
Log.d(TAG, extras.toString());
syncResult.stats.numAuthExceptions++;
syncResult.delayUntil = 60;
//extras.putString(AccountManager.KEY_AUTH_FAILED_MESSAGE, "You're not registered");
} catch (final ParseException e) {
syncResult.stats.numParseExceptions++;
Log.e(TAG, "ParseException", e);
}
}
That's it, enjoy playing with the delays and other variables too...