As per this link below:
http://java.sun.com/developer/technicalArticles/Programming/PerfTuning/
You can speed up loading of bitmaps (or any files) if you do the buffering yourself (i.e., instead of using BufferedInputStream, you handle the buffering yourself).
In particular, Approach 4 looks promising (slurp whole file at a time). However, I have no idea how to implement that in android. Here's the Java code:
import java.io.*;
public class readfile {
public static void main(String args[]) {
if (args.length != 1) {
System.err.println("missing filename");
System.exit(1);
}
try {
int len = (int)(new File(args[0]).length());
FileInputStream fis =
new FileInputStream(args[0]);
byte buf[] = new byte[len];
fis.read(buf);
fis.close();
int cnt = 0;
for (int i = 0; i < len; i++) {
if (buf[i] == '\n')
cnt++;
}
System.out.println(cnt);
}
catch (IOException e) {
System.err.println(e);
}
}
}
This technique is not optimized for Android and will likely run poorly. The convention is to use AndroidHttpClient:
Subclass of the Apache DefaultHttpClient that is configured with reasonable default settings and registered schemes for Android, and also lets the user add HttpRequestInterceptor classes.
If you really want to use Sun's code above, you should be careful because you will likely exceed the VM heap budget when the size of the file exceeds the amount of heap space available to the application.
It would be wise to first check if there is sufficient heap space left using ActivityManager. See also the elaborate answer to this question.
Edit:
I've found an example of sending an InputStream via POST. Here a file is being read from a resource (res/data.xml), but you could replace the InputStream with the FileInputStream from your snippet. Converting the InputStream to a byte array does essentially the same as your code: read the entire file into memory and push it into the request. This is a notorious cause of OutOfMemoryErrors, so take care that you don't read files that are too large (I would suggest less than 1 MB).
public void executeMultipartPost() throws Exception {
try {
InputStream is = this.getAssets().open("data.xml");
HttpClient httpClient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost("http://w3mentor.com/Upload.aspx");
byte[] data = IOUtils.toByteArray(is);
InputStreamBody isb = new InputStreamBody(new ByteArrayInputStream(data),"uploadedFile");
StringBody sb1 = new StringBody("someTextGoesHere");
StringBody sb2 = new StringBody("someTextGoesHere too");
MultipartEntity multipartContent = new MultipartEntity();
multipartContent.addPart("uploadedFile", isb);
multipartContent.addPart("one", sb1);
multipartContent.addPart("two", sb2);
postRequest.setEntity(multipartContent);
HttpResponse res = httpClient.execute(postRequest);
res.getEntity().getContent().close();
} catch (Throwable e) {
// handle exception here
}
}
Related
This question already has answers here:
Java multiple file transfer over socket
(3 answers)
Closed 5 years ago.
I have been spending forever on this and can not seem to work it out. I am self taught and not very familiar with this so forgive me if it is a remedial question. I am sending data from Android to .Net server. Data is getting corrupt on encoding, I know this I am just not sure how to fix. I am using the .Net Async server sample code found here: Microsoft Async Sample
My Android client code is:
try {
final Socket sock = new Socket();
final int timeOut = (int) TimeUnit.SECONDS.toMillis(5); // 5 sec wait period
sock.connect(new InetSocketAddress("localhost", 11000), timeOut);
if (sock.isConnected()==true){
BufferedReader in = new BufferedReader(new InputStreamReader(sock.getInputStream()));
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(sock.getOutputStream()));
String FileName = "myfile.jpg";
StringBuilder hd = new StringBuilder();
try {
String FilePath= Environment.getExternalStorageDirectory() + "/mydir/" + FileName;
File file = new File(FilePath);
FileInputStream is = new FileInputStream(file);
byte[] chunk = new byte[40960];
int chunkLen = 0;
while ((chunkLen = is.read(chunk)) != -1) {
//String str = new String(Base64.encodeToString(chunk, Base64.NO_WRAP));
//String str = new String(chunk, "ASCII");
String str = new String(chunk, "UTF-8");
out.write(str);
}
//out.write(hd.toString());
} catch (FileNotFoundException fnfE) {
// file not found, handle case
} catch (IOException ioE) {
// problem reading, handle case
}
out.write("<EOF>");
out.flush();
StringBuilder returnString = new StringBuilder();
String line;
while ((line = in.readLine()) != null) {
returnString.append(line).append('\n');
}
out.close();
in.close();
sock.close();
}else{
sock.close();
}
} catch (IOException e) {
e.printStackTrace();
}
As you can see in my comments, I have tried base64 and UTF-8. I get all kinds of errors on the server when I do that. If I use Base64 I get not part of Base64 error (extra padding etc.). UTF8 writes the file but it is corrupt. When I send it all as one Base64 string it works fine as I use 'Dim data As Byte() = Convert.FromBase64String(FileData)' but as expected it throws memory errors in Android for large files hence the chunking. I am sending some plain ASCII text along with it so I parse out the non-ASCII stuff to write the file. I am super stuck, any help would be much appreciated. Thanks in advance.
You don't have to encode it at all. Just write it directly as bytes using an OutputStream. Simpler, quicker, better.
I found the answer. It was so weird but makes sense now.
byte[] chunk = new byte[30000];
int chunkLen = 0;
while ((chunkLen = is.read(chunk)) != -1) {
String str = new String(Base64.encodeToString(chunk, Base64.NO_WRAP));
out.write(str);
}
I had to change the chunk size to a multiple of 3 then my base64 encoding worked great. Found it here and gave an up vote. Thank you 'mjv'. Link to the answer
I have a big JSON input (download the file) API and I don´t know how to parse this data. I need:
Save this data (entire JSON input) to text file or database. What is the best way for this?
Load this data from text file or database and create JSONArray from JSON tag "list" (first tag)
The solution should be fast and support Android 2.3. What you have recomend for this? Any ideas?
My code:
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(urls[0]);
HttpResponse httpResponse;
httpResponse = httpClient.execute(httpGet);
HttpEntity httpEntity = httpResponse.getEntity();
... and what next ?...
FYI:
EntityUtils throws OutOfMemoryException
EDIT:
I try to save data to file like this:
InputStream inputStream = httpEntity.getContent();
FileOutputStream output = new FileOutputStream(Globals.fileNews);
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, len);
}
And it´s OK. I load data:
FileInputStream fis = null;
StringBuffer fileContent = new StringBuffer("");
fis = new FileInputStream(Globals.fileNews);
byte[] buffer = new byte[1024];
while (fis.read(buffer) != -1) {
fileContent.append(new String(buffer));
}
But how convert StringBuffer to JSONObject? fileContent.ToString() is not ideal, sometimes I get OutOfMemoryException.
First of all: Dispose the HttpClient. Google discourages it:
Unfortunately, Apache HTTP Client does not, which is one of the many
reasons we discourage its use.
Source: developer.android.com
A good replacement is Google Volley. You have to build the JAR yourself but it just works like charm. I use for my setups Google Volley with OkHttp-Stack and GSON requests.
In your case you would write another Request which just writes the response out to the SD-card chunk by chunk. You don't buffer the string before! And some logic to open an input-stream from the file you wrote and give it to your JSON Deserializer. Jackson and GSON are able to handle streams out of the box.
Of course everything works with Android 2.3.
Don't, I repeat, don't try to dump the whole serialized stuff into a string or something. That's almost a OutOfMemoryException guarantee.
I need to upload a file which is vary from 10-100mb. When I try to upload the file, it gives me outofmemory error in line
sb.append(Base64.encode(data));
How can I solve this issue? My web service is SOAP webservice. Here is the complete code of the method.
public static String fileToBase64(String path) throws IOException {
File imagefile = new File(path);
byte[] data = new byte[3000];
FileInputStream fin = null;
StringBuffer sb = new StringBuffer();
try {
fin = new FileInputStream(imagefile);
while(fin.read(data) >= 0) {
sb.append(Base64.encode(data));
}
return sb.toString();
} catch (Exception e) {
// TODO: handle exception
} finally{
fin.close();
}
return null;
}
I see two methods:
Use streamed XML writer and chunked content to avoid excessive memory consumption. Not sure if it is applicable here.
Use MTOM to send SOAP messages with attachments.
This method is preferable, but you must add required functionality to your webservice.
I have created a HTTP file server with the objective of transferring media files (mp3, ogg etc) to an Android device. When the server is accessed from the android browser like
10.0.2.2:portNumber/path/to/file
The server initiates the file download process. Of course the customer would not do such a thing, Its fine for testing the file server.
I m new to Android development and have learned that httpclient package can manage get/post requests. Here is the sample code I have been using for reading the response
DefaultHttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
try {
HttpResponse execute = client.execute(httpGet);
InputStream content = execute.getEntity().getContent();
BufferedReader buffer = new BufferedReader(new InputStreamReader(content));
String s = "";
while ((s = buffer.readLine()) != null) {
response += s;
}
} catch (Exception e) {
e.printStackTrace();
}
return response;
The above code works fine when the server sends a list of file in JSON. Since the server part of sending file has been coded, the point where I m stuck is in retrieving the media file on android.
I m confused about how to receive the mp3 files send by the server. Should they be read in a stream ? Thanks
Yes, you want to read the file onto disk via an inputstream.
Here's an example. If you don't want a file download progress bar then remove the progress related code.
try {
File f = new File("yourfilename.mp3");
if (f.exists()) {
publishProgress(100,100);
} else {
int count;
URL url = new URL("http://site:port/your/mp3file/here.mp3");
URLConnection connection = url.openConnection();
connection.connect();
int lengthOfFile = connection.getContentLength();
long total = 0;
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream(f);
byte data[] = new byte[1024];
while ((count = input.read(data)) != -1) {
total += count;
publishProgress((int)(total/1024),lengthOfFile/1024);
output.write(data, 0, count);
}
output.flush();
output.close();
input.close();
}
} catch (Exception e) {
Log.e("Download Error: ", e.toString());
}
I have an application where i need to download a large amount of data via a SOAP call to a webservice into the application when it is first run. The response is then sent to a function which converts the XML and stores the data in a db file.
The data is more than 16MB in size and i have a java.lang.OutOfMemoryError everytime.
Modifying the webservice to give out smaller amounts of data is not an option.
Is there a way to be able to download the large data? Something like an InputStream perhaps?
This is my code
public Protocol[] getProtocols() {
String METHOD_NAME = "GetProtocols";
String SOAP_ACTION = "urn:protocolpedia#GetProtocols";
Log.d("service", "getProtocols");
SoapObject response = invokeMethod(METHOD_NAME, SOAP_ACTION);
return retrieveProtocolsFromSoap(response);
}
private SoapObject invokeMethod(String methodName, String soapAction) {
Log.d(TAG, "invokeMethod");
SoapObject request = GetSoapObject(methodName);
SoapSerializationEnvelope envelope = getEnvelope(request);
return makeCall(envelope, methodName, soapAction);
}
Can anyone suggest what should be done in this case?
Thanks and regards
Mukul
Just an update, I found that the "call" method in AndroidHttpTransport was running out of memory at this line -
if (debug) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[256];
while (true) {
int rd = is.read(buf, 0, 256);
if (rd == -1)
break;
bos.write(buf, 0, rd);
}
bos.flush();
buf = bos.toByteArray(); //Goes out of memory here
responseDump = new String(buf);
is.close();
is = new ByteArrayInputStream(buf);
the call to toByteArray takes a lot of memory, so to overcome this, instead of converting the response to a byte array, i now directly write it to an XML file, and this is saved at a location of my choice. Here -
if (debug) {
FileOutputStream bos = new FileOutputStream("/data/data/com.mypackage.myapp/response.xml");
byte[] buf = new byte[1048576];
int current = 0; int i=0; int newCurrent = 0;
while ((current = inputStream.read(buf)) != -1) {
newCurrent = newCurrent + current;
Log.d("current", "Current = " + current + " total = "+newCurrent+" i = "+i++);
bos.write(buf, 0, current);
}
bos.flush();
}
The device no longer runs out of memory, and i have a custom parse method that takes this XML and writes it to the DB.
Two strategies to help you solve this problem:
Save your SOAP XML stream directly to disk as you download it. Don't store it in memory.
Parse it using a SAX-style parser, where you don't load the whole DOM in memory, but rather parse it in chunks.
Depending on the kind of XML you are handling, using SAX parsers is usually harder in code; you will have to keep track of many things yourself, and you won't be able to "jump" from section to section of your DOM tree. But the memory consumption will be way lower.
Take note, however, that many "high-level" network communication libraries usually load the whole XML DOM in memory, which might be the case here. You will probably have to create and manage the HTTP connection yourself, and then manually parse the result.
Fixed!
I downloaded/copied HttpTransportSE java class from here (after copied, some code errors can occur, but they are all quick fixable) and added to my package:
https://github.com/mosabua/ksoap2-android/blob/master/ksoap2-j2se/src/main/java/org/ksoap2/transport/HttpTransportSE.java
removed from my Connection class this row:
import org.ksoap2.transport.HttpsTransportSE;
and substituted this code in my new HttpTransportSE.java file:
if (debug) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[256];
while (true) {
int rd = is.read(buf, 0, 256);
if (rd == -1)
break;
bos.write(buf, 0, rd);
}
bos.flush();
buf = bos.toByteArray(); //Goes out of memory here
responseDump = new String(buf);
is.close();
is = new ByteArrayInputStream(buf);
}
with this
if (debug) {
FileOutputStream bos = new FileOutputStream(file);
byte[] buf = new byte[256];
while (true) {
int rd = is.read(buf, 0, 256);
if (rd == -1) {
break;
}
bos.write(buf, 0, rd);
}
bos.flush();
}
where "file" is a simple file object like new File("/sdcard/","myFile.xml") for example