Develop Reference

How to decode json string as UTF-8?

I've been working with json for some time and the issue is the strings I decode are encoded as Latin-1 and I cannot get it to work as UTF-8. Because of that, some characters are shown incorrectly (ex. ' shown as ').
I've read a few questions here on stackoverflow, but they doesn't seem to work.
The json structure I'm working with look like this (it is from YouTube API):
...
"items": [
{
...
"snippet": {
...
"title": "Powerbeats Pro “Totally Wireless” Except when you need a wire",
...
}
}
]
I encode it with:
response = await http.get(link, headers: {HttpHeaders.contentTypeHeader: "application/json; charset=utf-8"});
extractedData = json.decode(response.body);
dataTech = extractedData["items"];
And then what I tried was changing the second line to:
extractedData = json.decode(utf8.decode(response.body));
But this gave me an error about wrong format. So I changed it to:
extractedData = json.decode(utf8.decode(response.bodyBytes));
And this doesn't throw the error, but neither does it fix the problem. Playing around with headers does neither.
I would like the data to be stored in dataTech as they are now, but encoded as UTF-8. What am I doing wrong?

Just an aside first: UTF-8 is typically an external format, and typically represented by an array of bytes. It's what you might send over the network as part of an HTTP response. Internally, Dart stores strings as UTF-16 code points. The utf8 encoder/decoder converts between internal format strings and external format arrays of bytes.
This is why you are using utf8.decode(response.bodyBytes); taking the raw body bytes and converting them to an internal string. (response.body basically does this too, but it chooses the bytes->string decoder based on the response header charset. When this charset header is missing (as it often is) the http package picks Latin-1, which obviously doesn't work if you know that the response is in a different charset.) By using utf8.decode yourself, you are overriding the (potentially wrong) choice being made by http because you know that this particular server always sends UTF-8. (It may not, of course!)
Another aside: setting a content type header on a request is rarely useful. You typically aren't sending any content - so it doesn't have a type! And that doesn't influence the content type or content type charset that the server will send back to you. The accept header might be what you are looking for. That's a hint to the server of what type of content you'd like back - but not all servers respect it.
So why are your special characters still incorrect? Try printing utf8.decode(response.bodyBytes) before decoding it. Does it look right in the console? (It very useful to create a simple Dart command line application for this type of issue; I find it easier to set breakpoints and inspect variables in a simple ten line Dart app.) Try using something like Wireshark to capture the bytes on the wire (again, useful to have the simple Dart app for this). Or try using Postman to send the same request and inspect the response.
How are you trying to show the characters. If may simply be that the font you are using doesn't have them.

just add the header : 'Accept': 'application/json; charset=UTF-8';
it worked for me

My header looks like :
final response = await http.get(url, headers: {
'Content-Type': 'application/json; charset=UTF-8',
'Accept': 'application/json',
'Authorization': 'Bearer $token',
});
And the response is handled like:
Map<String, dynamic> data = json.decode(utf8.decode(response.bodyBytes));

Send large data OkHttp3 outofmemory

I want to send a PDF image to the server in Android.
The server spec needs I should use Base64 encoding.
So I should convert the PDF Image file to Base64 String.
Below is the HTTP POST request:
<?xml version="1.0" encoding="UTF-8"?>
<request>
<auth>
<!-- auth info -->
</auth>
<imagefile>
<filename>Test Attachment</filename>
<type>pdf</type>
<data>"HERE IS BASE64 String"</data>
</imagefile>
</request>
And I used below code, it is just Http post request code.
fun post(url: String, xml: String) = Observable.defer {
val request = Request.Builder()
.url(url)
.post(RequestBody.create(contentType, xml))
.build()
okHttpClient.newCall(request).execute()
}
In case of the small pdf file, it's ok. (Works fine)
But when I used the large pdf file(over 20MB), out of memory occurs.
So I tried another way like below.
fun post(url: String, xmlFile: File) = Observable.defer {
val request = Request.Builder()
.url(url)
.post(RequestBody.create(contentType, xmlFile))
.build()
okHttpClient.newCall(request).execute()
}
The difference is that I don't use String, but File.
I created a file using the Base64 encoded text with XML syntax.
But it is failed too.
How can I send the large data in Android without OOM?

You're going to need to find a streaming base64 encoder and use that to write a RequestBody that streams to the server. There is no such encoder built-in to OkHttp or Okio but you can use the one in Android itself. Use wrap on the OutputStream that you'll get in the writeTo method of your custom RequestBody.
https://developer.android.com/reference/java/util/Base64.Encoder

How to send special character like å in header from okhttp library

Is it possible to send special characters in headers from okhttp library? Right now my app crashes showing the following error:
java.lang.IllegalArgumentException: Unexpected char 0xe5 at 1 in username value: påfyll
at okhttp3.Headers$Builder.checkNameAndValue(Headers.java:320)
This is how i am sending the request.
okhttp3.Request request = new okhttp3.Request.Builder()
.url(AppConfig.CONCERT_LIST)
.addHeader("Content-Type", "application/json; charset=UTF-8")
.addHeader("username", "påfulo")
.addHeader("accessToken", "12345ASDFGsf98")
.build();

It is not that simple, you will have to encode it and have your server to decode it.
You can find more information about this problem here :
https://github.com/square/okhttp/issues/2016

I think the best way to do it ASCII char rather then string.On the server you can get original value from it.
You can find your ASCII character down here-:
http://ee.hawaii.edu/~tep/EE160/Book/chap4/subsection2.1.1.1.html

How to decode a very large image on android?

I am decoding an image(ARGB) that is 8000x8000 pixels, so in uncompressed form it reaches
(2 + 2 + 2 + 2) * (8000 * 8000) = 488 MB
and crashes the android. I don't want to sample the image because I am converting the bitmap to byte array and sending it in a PUT request. I have tried "decodeRegion" but I don't know how to stitch the data(i.e. byte arrays) back together , since they have the head info at start and just concatenating them isn't helping.

Use an HTTP client library that allows you to upload from a file or stream, so that you do not need to decode the image and try to hold it in memory. OkHttp has options for this; see this recipe for streaming a POST request, this recipe for POSTing a file, or this recipe for multipart POSTs. Those techniques should be adaptable to a PUT request.

Why are you reading in a large image, decoding it, then posting the byte array? That's the wrong way to do it.
If your API actually requires the decoded bytes, fix it. More likely it wants the file's raw data. In which case you just need to use any networking API that gives you an OutputStream, and read in the file's data 1 MB at a time, reading it from the File's InputStream and writing it to the socket's OutputStream

File uploading Not getting regular callback response?

I am trying to upload a 20mb video file to cloud using http put method. File is uploading as byte array format. To show the progress of upload, I add progress callback with entity. But now I'm getting the call back response after uploading the full file and response show 100.00.
How do I get the callback after in every 10kb uploading?
Here is my code:
byte[] videoBytes = ous.toByteArray();
httput.setEntity(new ByteArrayEntity(videoBytes));
httput.setEntity(new ProgressHttpEntityWrapper(httput.getEntity(), progressCallback));