Develop Reference

getKeysByPrefix does not return defaults, only remote values

Firebase remote config has a method to return a set of keys. But it only returns keys that exists on the server-side.
(The keys i've defined in remoteconfig_defaults.xml do not.)
Used code:
FirebaseRemoteConfig instance = FirebaseRemoteConfig.getInstance();
instance.setDefaults(R.xml.remoteconfig_defaults);
Set<String> keysSet = instance.getKeysByPrefix("");
And xml:
<defaultsMap xmlns:android="http://schemas.android.com/apk/res/android">
<entry>
<key>loading_phrase</key>
<value>Fetching config…</value>
</entry>
...
</defaultsMap>
Any ideas on how to get the keys of the defaults as well?

As the the defaults are in XML, an XML-parser can be used to get the keys.
public static Set<String> getDefaultKeys(Context context, #XmlRes int xmlRes) {
Set<String> keys = new HashSet<>();
XmlResourceParser xmlResourceParser = context.getResources().getXml(xmlRes);
try {
boolean isInKeyTag = false;
while (xmlResourceParser.getEventType() != XmlResourceParser.END_DOCUMENT) {
if (xmlResourceParser.getEventType() == XmlResourceParser.START_TAG) {
String tagName = xmlResourceParser.getName();
if (tagName.equals("key")) {
isInKeyTag = true;
}
}
if (xmlResourceParser.getEventType() == XmlResourceParser.END_TAG){
String tagName = xmlResourceParser.getName();
if (tagName.equals("key")){
isInKeyTag = false;
}
}
if (xmlResourceParser.getEventType() == XmlResourceParser.TEXT){
if (isInKeyTag){
String key = xmlResourceParser.getText();
keys.add(key);
}
}
xmlResourceParser.next();
}
return keys;
} catch (Exception e) {
return null;
}
}

Android : java.lang.OutOfMemoryError: Failed to allocate with free bytes and 70MB until OOM when using gson.toJson()

In my Android application I am getting below exception when I try to sync my data to the server which is large in size. I get this exception when data size is more than 20 MB I think. I am saving bitmap images as string using base64 encoding after sacle down the image size which makes such huge data.
04-18 13:51:51.957 16199-16816/com.example.myproject.app E/art﹕ Throwing OutOfMemoryError "Failed to allocate a 128887990 byte allocation with 16777216 free bytes and 70MB until OOM"
04-18 13:51:52.037 16199-16816/com.example.myproject.app E/AndroidRuntime﹕ FATAL EXCEPTION: Thread-4482
Process: com.example.myproject.app, PID: 16199
java.lang.OutOfMemoryError: Failed to allocate a 128887990 byte allocation with 16777216 free bytes and 70MB until OOM
at java.lang.AbstractStringBuilder.enlargeBuffer(AbstractStringBuilder.java:95)
at java.lang.AbstractStringBuilder.append0(AbstractStringBuilder.java:146)
at java.lang.StringBuffer.append(StringBuffer.java:219)
at java.io.StringWriter.write(StringWriter.java:167)
at com.google.gson.stream.JsonWriter.string(JsonWriter.java:570)
at com.google.gson.stream.JsonWriter.value(JsonWriter.java:419)
at com.google.gson.internal.bind.TypeAdapters$16.write(TypeAdapters.java:426)
at com.google.gson.internal.bind.TypeAdapters$16.write(TypeAdapters.java:410)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:112)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:239)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:97)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:61)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
How to solve this problem? I know that this comes when I convert the data from class to json using Gson. Below is my code:
SimpleDateFormat dtf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss",Locale.ENGLISH);
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(Date.class, new JsonDeserializer<Date>() {
#Override
public Date deserialize(JsonElement json, Type type, JsonDeserializationContext deserializationContext) throws JsonParseException {
String frStr = json.getAsJsonPrimitive().getAsString();
Date retDate =null;
try {
retDate = dtf.parse(frStr);
} catch (ParseException e) {
e.printStackTrace();
}
return retDate;
}
});
builder.registerTypeAdapter(Date.class, new JsonSerializer<Date>() {
#Override
public JsonElement serialize(Date src, Type typeOfSrc, JsonSerializationContext context) {
String jsDate = dtf.format(src);
return new JsonPrimitive(jsDate);
}
});
builder.registerTypeAdapter(byte[].class, new JsonDeserializer<byte[]>() {
#Override
public byte[] deserialize(JsonElement json, Type type, JsonDeserializationContext deserializationContext) throws JsonParseException {
return Base64.decode(json.getAsString(), Base64.NO_WRAP);
}
});
gson = builder.create();
attDataAcc.setAttList(attList);
String jsonAttAccts = gson.toJson(attDataAcc, AttachmentDataList.class);
HttpEntity<String> entityAtt = new HttpEntity<String>(jsonAttAccts,headers);
ResponseEntity<String> restResA = restTemplate.exchange(strUrl+"/saveAttToServer", HttpMethod.POST, entityAtt, String.class);
public class Attachment implements Serializable {
#DatabaseField(columnName = "id",id = true)
private String id;
#DatabaseField(columnName = "user_id")
private Integer userId;
#DatabaseField(columnName = "attachment_id")
private String attachmentId;
#DatabaseField(columnName = "file_name")
private String fileName;
#DatabaseField(columnName = "file_data")
private String fileData;
#DatabaseField(columnName = "date",dataType=DataType.DATE)
private Date date;
public Attachment() {
super();
// TODO Auto-generated constructor stub
}
public Attachment(String id, Integer userId, String attachmentId, String fileName, String fileData, Date date) {
this.id = id;
this.userId = userId;
this.attachmentId = attachmentId;
this.fileName = fileName;
this.fileData = fileData;
this.date = date;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public Integer getUserId() {
return userId;
}
public void setUserId(Integer userId) {
this.userId = userId;
}
public String getAttachmentId() {
return attachmentId;
}
public void setAttachmentId(String attachmentId) {
this.attachmentId = attachmentId;
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public String getFileData() {
return fileData;
}
public void setFileData(String fileData) {
this.fileData = fileData;
}
public Date getDate() {
return date;
}
public void setDate(Date date) {
this.date = date;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Attachment that = (Attachment) o;
if (id != null ? !id.equals(that.id) : that.id != null) return false;
if (userId != null ? !userId.equals(that.userId) : that.userId != null) return false;
if (attachmentId != null ? !attachmentId.equals(that.attachmentId) : that.attachmentId != null) return false;
if (fileName != null ? !fileName.equals(that.fileName) : that.fileName != null) return false;
if (fileData != null ? !fileData.equals(that.fileData) : that.fileData != null) return false;
if (date != null ? !date.equals(that.date) : that.date != null) return false;
}
#Override
public int hashCode() {
int result = id != null ? id.hashCode() : 0;
result = 31 * result + (userId != null ? userId.hashCode() : 0);
result = 31 * result + (attachmentId != null ? attachmentId.hashCode() : 0);
result = 31 * result + (fileName != null ? fileName.hashCode() : 0);
result = 31 * result + (fileData != null ? fileData.hashCode() : 0);
result = 31 * result + (date != null ? date.hashCode() : 0);
return result;
}
#Override
public String toString() {
return userFileName;
}
}
public class AttachmentDataList implements Serializable {
private ArrayList<Attachment> attList;
public ArrayList<Attachment> getAttList() {
return attList;
}
public void setAttList(ArrayList<Attachment> attList) {
this.attList = attList;
}
}

You're running into OutOfMemoryError because you're using inefficient and very memory-consuming Base64 transformations. Another hit here is Gson: it does not provide any raw-write methods for JsonWriter and JsonReader classes: the most you can do here is writing/reading a single string value. Collecting a huge input to a single string is another very memory-consuming operation: check your stacktrace to make sure that a string builder instance is used under the very hood -- and this is just to write a single value to the output stream. In short, it all looks like this (if I'm not mistaken about your code, because it seems to lack really important parts, so I'm just trying to reconstruct your scenario):
Obtaining a byte array (that would be a new object, probably a clone of another byte array);
Converting the byte array to a Base64 encoded string (it also hits performance because the would clone the byte array to create a defensive copy);
Converting ALL to a String gson.toJson(attDataAcc, AttachmentDataList.class); -- another huge hit.
All of this is extremely memory consuming. And it would be nice if Gson could support raw write to the output stream, but currently it lacks any of it.
In theory, you might overcome this issue by just writing to the underlying streams (probably directly from your byte array source without any massive transformations because Base64 can be streamed as well thus consuming memory minimum). You mentioned Gson 2.6.2, but I'm working with Gson 2.8.0, so the below solution can 100% work with Gson 2.8.0 only, and may not work even for any other minor Gson versions because it uses reflection in order to "hack" the JsonWriter class.
final class ByteArrayTypeAdapter
extends TypeAdapter<byte[]> {
// These two methods and one field from the super class privates are necessary to make it all work
private static final Method writeDeferredNameMethod;
private static final Method beforeValueMethod;
private static final Field writerField;
static {
try {
writeDeferredNameMethod = JsonWriter.class.getDeclaredMethod("writeDeferredName");
writeDeferredNameMethod.setAccessible(true);
beforeValueMethod = JsonWriter.class.getDeclaredMethod("beforeValue");
beforeValueMethod.setAccessible(true);
writerField = JsonWriter.class.getDeclaredField("out");
writerField.setAccessible(true);
} catch ( final NoSuchMethodException | NoSuchFieldException ex ) {
throw new RuntimeException(ex);
}
}
// This type adapter is effectively a singleton having no any internal state
private static final TypeAdapter<byte[]> byteArrayTypeAdapter = new ByteArrayTypeAdapter();
private ByteArrayTypeAdapter() {
}
// But making the constructor private and providing access to the instance via the method, we make sure that the only instance exists and it's safe
static TypeAdapter<byte[]> getByteArrayTypeAdapter() {
return byteArrayTypeAdapter;
}
#Override
public void write(final JsonWriter out, final byte[] bytes)
throws IOException {
try {
// Since we're writing a byte[] array, that's probably a field value, make sure that the corresponding property name has been written to the output stream
writeDeferredNameAndFlush(out);
// Now simulate JsonWriter.value(byte[]) if such a method could exist
writeRawBase64ValueAndFlush(bytes, (Writer) writerField.get(out));
} catch ( IllegalAccessException | InvocationTargetException ex ) {
throw new IOException(ex);
}
}
#Override
public byte[] read(final JsonReader in) {
// If necessary, requires more hacks...
// And this is crucial for the server-side:
// In theory, the client can generate HUGE Base64 strings,
// So the server could crash with OutOfMemoryError too
throw new UnsupportedOperationException();
}
private static void writeDeferredNameAndFlush(final Flushable out)
throws IOException, IllegalAccessException, InvocationTargetException {
writeDeferredNameMethod.invoke(out);
beforeValueMethod.invoke(out);
// Flush is necessary: the JsonWriter does not know that we're using its private field intruding to its privates and may not flush
out.flush();
}
private static void writeRawBase64ValueAndFlush(final byte[] bytes, final Writer writer)
throws IOException {
// Writing leading "
writer.write('\"');
// This comes from Google Guava
final BaseEncoding baseEncoding = BaseEncoding.base64();
final OutputStream outputStream = baseEncoding.encodingStream(writer);
// This too
// Note that we just r_e_d_i_r_e_c_t streams on fly not making heavy transformations
ByteStreams.copy(new ByteArrayInputStream(bytes), outputStream);
// This is necessary too
outputStream.close();
// Writing trailing "
writer.write('\"');
// Flush again to keep it all in sync
writer.flush();
}
}
I know it's a hack, but it's better than just getting OutOfMemoryError constantly.
Now, just make it work with Spring RestTemplates:
// Gson is thread-safe and can be re-used
private static final Gson gson = new GsonBuilder()
// SimpleDateFormat may be NOT thread-safe so you should not share the single SimpleDateFormat between threads
// However Gson supports date/time formats out of box
.setDateFormat("yyyy-MM-dd HH:mm:ss")
// Registering byte[] to the type adapter
.registerTypeAdapter(byte[].class, getByteArrayTypeAdapter())
.create();
private static final RestTemplate restTemplate = new RestTemplate();
private static final String URL = "http://localhost";
public static void main(final String... args) {
sendPostRequest("hello world".getBytes(), byte[].class);
}
private static void sendPostRequest(final Object object, final Type type) {
// This is where we're binding the output stream I was asking in the question comments
final RequestCallback requestCallback = request -> gson.toJson(object, type, new OutputStreamWriter(request.getBody()));
// Spring RestTemplates stuff here...
final SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
final ResponseExtractor<String> responseExtractor = new HttpMessageConverterExtractor<>(String.class, restTemplate.getMessageConverters());
restTemplate.setRequestFactory(requestFactory);
// Let it fly
restTemplate.execute(URL, POST, requestCallback, responseExtractor);
}
Note that you might write a specialized type adapter for special types that could write directly to output stream so you could not get rid of byte[] at all. You could also vote up for this issue at the official Gson issue tracker: https://github.com/google/gson/issues/971 and probably there could no any need in using any Java Reflection API hacks in a future version of Gson.

Need help on simplify a list of strings

Can someone help me on simplfy this lists of strings:
public class MyList {
// New customers
public static final String mNew1 = "email1";
public static final String mNew2 = "email2";
public static final String mNew3 = "email3";
public static final String mNew4 = "email4";
// Old customers
public static final String mOld1 = "email1";
public static final String mOld2 = "email2";
public static final String mOld3 = "email3";
}
public class App extends Application {
public static boolean mIsNew = false;
public static boolean mIsOld = false;
Pattern emailPattern = Patterns.EMAIL_ADDRESS;
Account[] accounts = AccountManager.get(context).getAccounts();
for (Account account : accounts) {
if (emailPattern.matcher(account.name).matches()) {
String possibleEmail = account.name;
if (MyList.mNew1.matches(possibleEmail) || MyList.mNew2.matches(possibleEmail) ||
MyList.mNew3.matches(possibleEmail) || MyList.mNew4.matches(possibleEmail)) {
mIsNew = true;
}
if (MyList.mOld1.matches(possibleEmail) || MyList.mOld2.matches(possibleEmail) || MyList.mOld3.matches(possibleEmail)) {
mIsOld = true;
}
}
}
Since the old customers email will be over 10.000 in less then a week can you suggest me an easy way to pick the strings from the MyList class and enable the right boolean? I.E if oneOfTheStringInThisList.matches(possibleEmail) mIsOld = true.
I'm not really familiar with lists of strings, sorry for my noob question! Thankyou!

You can get the values by reflection:
private void Something()
{
MyList list = new MyList();
HashSet<String> oldMails = new HashSet<String>();
HashSet<String> newMails = new HashSet<String>();
try {
GetAllMails(list, oldMails, newMails);
} catch (IllegalAccessException e) {
// TODO
}
boolean mIsOld = oldMails.contains("email4");
boolean mIsNew = newMails.contains("email4");
}
private void GetAllMails(MyList list, HashSet<String> oldMails, HashSet<String> newMails) throws IllegalAccessException
{
Field[] allFields = MyList.class.getDeclaredFields();
for(Field f : allFields)
{
if (f.getName().startsWith("mNew"))
{
newMails.add(f.get(list).toString());
}
else if (f.getName().startsWith("mOld"))
{
oldMails.add(f.get(list).toString());
}
}
}
You should store the HashSets in memory because reflection is not very performant.

How to track Omniture Custom Traffic Variable in Android?

In Site Catalyst we have defined Custom Traffic Variable "Item Bought". Now how do i send request to update this custom variable from Android library? i don't find much help here
Please find my code below,
public class TrackingHelper {
private static final String TRACKING_RSID = "MY_TRACKING_RSID";
private static final String TRACKING_SERVER = "MY_TRACKING_SERVER";
static ADMS_Measurement measurement = null;
public static void startActivity(Activity activity) {
measurement = ADMS_Measurement.sharedInstance(activity);
measurement.startActivity(activity);
}
public static void stopActivity() {
measurement = ADMS_Measurement.sharedInstance();
measurement.stopActivity();
}
public static void configureAppMeasurement(Activity activity) {
if (measurement == null) {
measurement = ADMS_Measurement.sharedInstance(activity);
measurement.configureMeasurement(TRACKING_RSID, TRACKING_SERVER);
measurement.setOfflineTrackingEnabled(true);
measurement.setDebugLogging(true);
}
}
public static void trackItemBought() {
if (measurement != null) {
measurement = ADMS_Measurement.sharedInstance();
Hashtable<String, Object> contextData = new Hashtable<String, Object>();
contextData.put("Item Bought", "Item Bought");
measurement.setEvents("Item Bought");
measurement.track(contextData);
}
}
}

I can't vouch for the android code, but here are some general thoughts:
When you send a context data variable, you send a name-value pair, so I suspect this line:
contextData.put("Item Bought", "Item Bought");
Should be something more like this (I'm going to say the item being bought is a basketball):
contextData.put("Item Bought", "Basketball");
Then someone with access to processing rules in SiteCatalyst will need to configure a rule to say that that variable ("Item Bought") belongs to its assigned custom traffic variable (prop14, for instance). http://www.jasonegan.net/2011/04/07/omniture-sitecatalyst-15-context-variables-processing-rules/ talks about how to do that a bit, but the article generally assumes you have a certain level of knowledge of SiteCatalyst.

Just make the following change in your code.
public class TrackingHelper {
private static final String TRACKING_RSID = "MY_TRACKING_RSID";
private static final String TRACKING_SERVER = "MY_TRACKING_SERVER";
static ADMS_Measurement measurement = null;
public static void startActivity(Activity activity) {
measurement = ADMS_Measurement.sharedInstance(activity);
measurement.startActivity(activity);
}
public static void stopActivity() {
measurement = ADMS_Measurement.sharedInstance();
measurement.stopActivity();
}
public static void configureAppMeasurement(Activity activity) {
if (measurement == null) {
measurement = ADMS_Measurement.sharedInstance(activity);
measurement.configureMeasurement(TRACKING_RSID, TRACKING_SERVER);
measurement.setOfflineTrackingEnabled(true);
measurement.setDebugLogging(true);
}
}
public static void trackItemBought() {
if (measurement != null) {
measurement = ADMS_Measurement.sharedInstance();
Hashtable<String, Object> contextData = new Hashtable<String, Object>();
contextData.put("Item Bought", "Item Bought");
measurement.setEvents("Item Bought");
measurement.contextData(contextData);
measurement.track();
}
}

Parsing large text file efficiency

Due to simplicity i have a text file with entries separated by ; and parses every line into an object. The problem is that the text file contains almost 10 000 rows.
I also need to create keys for each object im parsing so i can filter the results in a search interface.
It takes almost 16 seconds in emulator to parse the text and add the keys. I'm i doing something wrong here? Or is there a more efficient way?
Here is my database singleton:
public class Database {
private static Database instance = null; private final Map<String, List<Stop>> mDict = new ConcurrentHashMap<String, List<Stop>>();
public static Database getInstance() { if (instance == null) { instance = new Database(); } return instance; } public List<Stop> getMatches(String query) {
List<Stop> list = mDict.get(query);
return list == null ? Collections.EMPTY_LIST : list;
}
private boolean mLoaded = false;
/**
* Loads the words and definitions if they haven't been loaded already.
*
* #param resources Used to load the file containing the words and definitions.
*/
public synchronized void ensureLoaded(final Resources resources) {
if (mLoaded) return;
new Thread(new Runnable() {
public void run() {
try {
loadStops(resources);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}).start();
}
private synchronized void loadStops(Resources resources) throws IOException
{
if (mLoaded) return;
Log.d("database", "loading stops");
InputStream inputStream = resources.openRawResource(R.raw.allstops);
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
try {
String line;
while((line = reader.readLine()) != null) {
String[] strings = TextUtils.split(line, ";");
addStop(strings[0], strings[1], strings[2]);
}
} finally {
reader.close();
}
Log.d("database", "loading stops completed");
mLoaded = true;
}
private void addStop(String name, String district, String id) {
Stop stop = new Stop(id, name, district);
int len = name.length();
for (int i = 0; i < len; i++) {
String prefix = name.substring(0, len - i).toLowerCase();
addMatch(prefix, stop);
}
}
private void addMatch(String query, Stop stop) {
List<Stop> matches = mDict.get(query);
if (matches == null) {
matches = new ArrayList<Stop>();
mDict.put(query, matches);
}
matches.add(stop);
}
}
Here is some sample data:
Mosseporten Senter;Norge;9021014089003000;59.445422;10.701055;273
Oslo Bussterminal;Norge;9021014089004000;59.911369;10.759665;273
Långegärde;Strömstad;9021014026420000;58.891462;11.007767;68
Västra bryggan;Strömstad;9021014026421000;58.893080;11.009997;7
Vettnet;Strömstad;9021014026422000;58.903184;11.020739;7
Ekenäs;Strömstad;9021014026410000;58.893610;11.048821;7
Kilesand;Strömstad;9021014026411000;58.878472;11.052983;7
Ramsö;Strömstad;9021014026430000;58.831531;11.067402;7
Sarpsborg;Norge;9021014089002000;59.280937;11.111763;273
Styrsö;Strömstad;9021014026180000;58.908110;11.115818;7
Capri/Källviken;Strömstad;9021014026440000;58.965200;11.124384;63
Lindholmens vändplan;Strömstad;9021014026156000;58.890212;11.128393;64
Öddö;Strömstad;9021014026190000;58.923490;11.130767;7
Källviksdalen;Strömstad;9021014026439000;58.962414;11.131962;64
Husevägen;Strömstad;9021014026505000;58.960094;11.133535;274
Caprivägen;Strömstad;9021014026284000;58.958404;11.134281;64
Stensviks korsväg;Strömstad;9021014026341000;59.001499;11.137203;63
Kungbäck;Strömstad;9021014026340000;59.006056;11.140313;63
Kase;Strömstad;9021014026173000;58.957649;11.141904;274

You should add the information into a SQLite database and ship the app with the database in res/raw.
Additionally, the db file can often be effectively compressed into a zip file.
See this for more information: Ship an application with a database

The fastest way to load that data into memory is to place it right into .java file. E.g. stopNames={"Stop1", "Stop2", ...}; latitudes={...};
I do this in my public transit app, loading similar amounts of the data this way takes under a second, filtering is instant.