I have an upstream server that accepts image submissions using rest. The submitted image is part of a JSON payload similar to this one
{
"name": "Blah.jpg",
"uploader": "user1",
"image": "<base64.....>"
}
Using this strategy works for small images but generates Out of Memory errors on larger images.
Is it possible to stream the base64 component of the image? Pass in something like an iterator that will be used to read chunks of the image, base64 them and send them directly to the network?
Not with Gson or Moshi. Both of these libraries require strings to be in memory before emitting them to a stream.
I solved this with the following, in a class that extends okhttp3.RequestBody:
private void writeFile(File file, BufferedSink sink) throws IOException {
byte buf[] = new byte[3000];
try (FileInputStream fin = new FileInputStream(file)) {
while (fin.read(buf) >= 0) {
byte encoded[] = Base64.encodeBase64(buf);
sink.write(encoded);
}
}
}
It uses Android's android.util.Base64 Apache Commons' org.apache.commons.codec.binary.Base64 to encode a buffered chunk of data.
I ended up writing the other json fields separately, with enough granularity that I could insert the file record exactly where I needed to.
EDIT:
As you can see in my edits above, I ended up switching to Apache commons-codec, via compile 'commons-codec:commons-codec:1.5' in my build.gradle file.
I didn't have time to investigate why the Android SDK solution didn't work. I tried their Base64.encode(buf, Base64.NO_WRAP) as suggested elsewhere - supposedly equivalent to Apache Commons' encodeBase64(byte[]) - but this did not work, hence the switch.
The problem could have been on our backend, so don't rule out Android SDK's solution based on my post alone - I just wanted to add this note so readers can see the code snippet that actually worked for me in the end.
Related
This question can be a possible duplicate of the question here, but I need a little bit more info on the subject. My partner(android) and me(ios) are trying to exchange some data, which is an identity key. This key is generated through the curve25519 wrapper of the Signal Protocol. The key consists of a public as well as a private key. The key is of the type defined here as ECKeyPair. Now I want to transfer the publicKey as NSData with each other, read iOS to Android and vice versa. The approaches taken and resulting issues are listed below.
From iOS, tried to make the data(publicKey) as a base64 encoded string and used json serialisation to transfer to Android, But android is not able to decode the base64 string correctly. I feel its because while converting the key to base64 the data size is changed somehow, for eg when I print the data the size is always more than the 32 bytes its supposed to be. The algorithm which check for the data validation rejects the public key from within the Android implementation saying not a valid data for the key.
Tried to transfer the public key from Android to iOS by casting as a ByteArray. The byte array created from the Android consists of Signed Integers and I am not able to invoke Data(bytes: <Array<UInt8>>) because its applicable to Unsigned Integers. When I try to convert this to signed Integers then it results in a similar validity check failure like in point one mentioned above.
An almost similar scenario is described in this issue here. So I would like to know the following.
Why is it that some data, which is base64 encoded in a particular plateform, like iOS, differes when decoded in another platform, say Android.
Why cant iOS succesfully read in a ByteArray created by an Android OS. Or am I mistaken? If so please guide me.
How can I transfer data between iOS and Android successfully like in this particular case? I am familiar with normal data transfers like images or files, but its just not working in my case of cryptographic data. I know about ProtoBuff and for the sake of feasibility I would prefer json or likewise.
Adding some details that might be helpfull.
The publickey generated at iOS
The public key generated at Android.
[12,-55,99,72,15,-101,99,-13,99,-56,-47,19,-21,90,-17,-39,-119,-33,44,-87,-18,-24,-53,-29,-100,34,-60,69,-61,24,8,92].
Do note the difference in signs between iOS and Android.
Any help is appreciated.
Thank You
EDIT: Adding the exception screen shot at android.
As you can see the data is discarded as BadKey Type. I transferred the data from iOS to Android by Base64Encoding to string.
Its long time but for that particular scenario I used Hex String. Please be known hex string conversion for large data like videos is impractical as it freezes the device.
Also, sometime later for another project, I have implemented a simple encryption between Android and iOS. And these are what I used. Maybe this will help. These use base64 encoding.
Android Encryption and Decryption
val cryptor = AES256JNCryptor();
val password = "6dpC295ei9"
val text = yourString.toByteArray(charset("UTF-8"))
val encrypt = cryptor.encryptData(text, password.toCharArray()) // your encryption
val encode = Base64.encodeToString(encrypt, Base64.NO_WRAP) // encryptedtext
val decode = Base64.decode(encode, Base64.NO_WRAP)
val text = String(cryptor.decryptData(decode, password.toCharArray())) // your decryption
iOS Encryption and Decryption
let password = "6dpC295ei9"
extension String {
var encrypted : String? {
guard let plainData = data(using: .utf8) else {
return nil
}
return RNCryptor.encrypt(data: plainData, withPassword: password).base64EncodedString(options: .init(rawValue: 0))
}
var decrypted : String? {
guard let cipherData = Data.init(base64Encoded: self, options: .init(rawValue: 0)), let plainData = try? RNCryptor.decrypt(data: cipherData, withPassword: password), let plainText = String(bytes: plainData, encoding: .utf8) else {
return "~ DECRYPTION FAILED ~"
}
return plainText
}
}
I'm curious to know that without commercial product for obfuscation, is there any way where I can store API url and parameters safely which cannot be compiled in reverse engineering? I have tried all my apps and their API url and code is easy to read. I'm concerned about security.
Hide Url in Environmental variables,BuildConfig and Android Studio
One simple way to avoid this bad practice is to store your values
inside an environmental variable, so only your machine knows it, then
read this values in some way and inject them in your code at build
time. Let’s see how to do that using Android Studio, Gradle, and
BuildConfig.
First, we need to create these environmental vars. In Linux and Mac,
create or edit the file ~/.gradle/gradle.properties (pay attention to
the actual Gradle User Home directory position) and add some values:
WEBServiceBaseURL="http://192.168.2.102:2323/"
WEBServiceBaseSMSURL="https://www.example.com/"
Second, in your module’s build.gradle file, add these lines
//Add these lines
def Base_URL = '"' + WEBServiceBaseURL + '"' ?: '"Define BASE URL"';
def SMS_Base_URL = '"' + WEBServiceBaseSMSURL + '"' ?: '"Define SMS BASE URL"';
android.buildTypes.each { type ->
type.buildConfigField 'String', 'Base_URL', WEBServiceBaseURL
type.buildConfigField 'String', 'SMS_Base_URL', WEBServiceBaseSMSURL
}
Use in Java File Like
BuildConfig.Base_URL it will return URL String
public static Retrofit getClient() {
if (retrofit==null) {
retrofit =new Retrofit.Builder()
.baseUrl(BuildConfig.Base_URL)
.addConverterFactory(GsonConverterFactory.create())
.build();
}
return retrofit;
}
I found a solution to hide base url to keep api secured with NDK. Keep base64 encoded string inside cpp file and call that from java class and decode base64.
Include c++ (NDK) support to your project. You can include this to your new or old project.
Your cpp file name can be like (native-lib.cpp)
Search online base64 encoder and encode your base url. Now keep encoded string inside cpp file
Inside cpp file sample code is like:
#include <jni.h>
#include <string>
extern "C" JNIEXPORT jstring JNICALL
Java_com_touhidapps_MyProject_utils_MyConstants_baseUrlFromJNI(JNIEnv *env, jobject) {
std::string mUrl = "aHR0cDovL2FwaS5leGFtcGxlLmNvbS8="; //"http://api.example.com/";
return env->NewStringUTF(mUrl.c_str());
}
Inside MyConstants.java class: (where I kept all api urls.)
// load c++ library
static {
System.loadLibrary("native-lib");
}
public static native String baseUrlFromJNI();
// decode base64 to a string and get normal url
public static String getSecureBaseUrl() {
String mUrl = baseUrlFromJNI();
try {
String text = new String(Base64.decode(mUrl, Base64.DEFAULT), "UTF-8");
return text;
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
mUrl = "http://demo.example.com/"; // don't change this link. This will not execute normally, if exception happens then it will return a demo url.
return mUrl;
}
Now you can get your original url like below:
public static final String API_BASE = "" + getSecureBaseUrl();
Your question is not ideal for the StackOverflow as the topic is too broad and primarily opinion based. However, I thought I can share some of my thoughts as an answer here.
Hiding API urls with code obfuscation is definitely a good idea and it may work in some cases as well if you want to hide those. You might consider encrypting the API url in your code as well and store the encrypted url in your SharedPreferences or in local storage which needs to be decrypted again each time when you're using your API url to call a web service.
But none of these can't ensure that your API urls are uncrackable. If someone really wants to get your API urls s/he can easily get those by tracking the network that you're using to call the web services.
So encrypting API urls and obfuscating the variable names to hide the API urls will not work in most of the cases as you expected. And yes, I don't see any security breach in getting your API urls either. Because, the API server should be designed in a way that it can block unwanted service calls an attacker is making through an API. You might consider thinking of setting up a firewall in your host machines or can setup a basic authentication protocol which will protect your data. There are a lot of ways to prevent these security breach activities. You might also consider reading this article which I found useful to get a heads-up on how you can protect your APIs to be abused.
Hope that helps.
I am decoding an image(ARGB) that is 8000x8000 pixels, so in uncompressed form it reaches
(2 + 2 + 2 + 2) * (8000 * 8000) = 488 MB
and crashes the android. I don't want to sample the image because I am converting the bitmap to byte array and sending it in a PUT request. I have tried "decodeRegion" but I don't know how to stitch the data(i.e. byte arrays) back together , since they have the head info at start and just concatenating them isn't helping.
Use an HTTP client library that allows you to upload from a file or stream, so that you do not need to decode the image and try to hold it in memory. OkHttp has options for this; see this recipe for streaming a POST request, this recipe for POSTing a file, or this recipe for multipart POSTs. Those techniques should be adaptable to a PUT request.
Why are you reading in a large image, decoding it, then posting the byte array? That's the wrong way to do it.
If your API actually requires the decoded bytes, fix it. More likely it wants the file's raw data. In which case you just need to use any networking API that gives you an OutputStream, and read in the file's data 1 MB at a time, reading it from the File's InputStream and writing it to the socket's OutputStream
Hello I am currently developing an android app in biometric fingerprint reader , the app captures and sends the png format a SOAP service, but need to be sent in the WSQ format and then send it as an arrangement of bits in the service , thank you.
I guess after three years you already solved it somehow, but in case anyone else is looking for a solution, you can try the WSQ for Android library. (Disclaimer: I wrote the library.) Add the following dependency to your build.gradle file:
implementation 'com.gemalto.wsq:wsq-android:1.0'
and use the following code to produce the WSQ:
byte[] pngData = ...; //obtain the PNG data from somewhere
Bitmap png = BitmapFactory.decodeByteArray(pngData, 0, pngData.length);
byte[] wsqData = new WSQEncoder(png).encode();
Im struggling to find a solution or library with good documentation describing how to submit a multipart image to a server. I had a look at loopJ, which I think is very promising however I am still unsure what the best solution is. It would be great if someone could give me advice/strategy or code example I have used in my iOS, (AFNetworking) shown below:
AFHTTPRequestOperationManager *manager = [AFHTTPRequestOperationManager manager];
NSDictionary *parameters = #{#"userID": #"1234567890"};
// add parameters first (user id) and then multipart image
[manager POST:MAIN_URL parameters:parameters constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
// add multipart image
[formData appendPartWithFileData:imageData name:#"uploadFile" fileName:#"uploadFile" mimeType:#"image/jpg"];
} success:^(AFHTTPRequestOperation *operation,
id responseObject) {
NSLog(#"Success: %#", responseObject);
} failure:^(AFHTTPRequestOperation *operation,
NSError *error) {
NSLog(#"Error: %#", error);
}];
Where imageData is the Image that needs to be uploaded.
I used the Apache's Multipart Entity Builder. Then just convert your image into a byte array or input stream.
http://hc.apache.org/httpcomponents-client-ga/httpmime/apidocs/org/apache/http/entity/mime/MultipartEntityBuilder.html
http://loopj.com/android-async-http/ seems to be pretty easy to use for android networking including multipart (just like AFNetworking for IOS).
The only thing I'm still struggling with, how to update a view (progress bar) while it's uploading. I also need to upload bunch of files synchronously (one at a time) which also seems to be not as easy as I thought (as least I have not found a solution for it yet). If anyone has any hints on using loopj asynchronously -- would be much appreciated.