This question already has answers here:
Java multiple file transfer over socket
(3 answers)
Closed 5 years ago.
I have been spending forever on this and can not seem to work it out. I am self taught and not very familiar with this so forgive me if it is a remedial question. I am sending data from Android to .Net server. Data is getting corrupt on encoding, I know this I am just not sure how to fix. I am using the .Net Async server sample code found here: Microsoft Async Sample
My Android client code is:
try {
final Socket sock = new Socket();
final int timeOut = (int) TimeUnit.SECONDS.toMillis(5); // 5 sec wait period
sock.connect(new InetSocketAddress("localhost", 11000), timeOut);
if (sock.isConnected()==true){
BufferedReader in = new BufferedReader(new InputStreamReader(sock.getInputStream()));
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(sock.getOutputStream()));
String FileName = "myfile.jpg";
StringBuilder hd = new StringBuilder();
try {
String FilePath= Environment.getExternalStorageDirectory() + "/mydir/" + FileName;
File file = new File(FilePath);
FileInputStream is = new FileInputStream(file);
byte[] chunk = new byte[40960];
int chunkLen = 0;
while ((chunkLen = is.read(chunk)) != -1) {
//String str = new String(Base64.encodeToString(chunk, Base64.NO_WRAP));
//String str = new String(chunk, "ASCII");
String str = new String(chunk, "UTF-8");
out.write(str);
}
//out.write(hd.toString());
} catch (FileNotFoundException fnfE) {
// file not found, handle case
} catch (IOException ioE) {
// problem reading, handle case
}
out.write("<EOF>");
out.flush();
StringBuilder returnString = new StringBuilder();
String line;
while ((line = in.readLine()) != null) {
returnString.append(line).append('\n');
}
out.close();
in.close();
sock.close();
}else{
sock.close();
}
} catch (IOException e) {
e.printStackTrace();
}
As you can see in my comments, I have tried base64 and UTF-8. I get all kinds of errors on the server when I do that. If I use Base64 I get not part of Base64 error (extra padding etc.). UTF8 writes the file but it is corrupt. When I send it all as one Base64 string it works fine as I use 'Dim data As Byte() = Convert.FromBase64String(FileData)' but as expected it throws memory errors in Android for large files hence the chunking. I am sending some plain ASCII text along with it so I parse out the non-ASCII stuff to write the file. I am super stuck, any help would be much appreciated. Thanks in advance.
You don't have to encode it at all. Just write it directly as bytes using an OutputStream. Simpler, quicker, better.
I found the answer. It was so weird but makes sense now.
byte[] chunk = new byte[30000];
int chunkLen = 0;
while ((chunkLen = is.read(chunk)) != -1) {
String str = new String(Base64.encodeToString(chunk, Base64.NO_WRAP));
out.write(str);
}
I had to change the chunk size to a multiple of 3 then my base64 encoding worked great. Found it here and gave an up vote. Thank you 'mjv'. Link to the answer
Related
This is related to a situation I find myself in working with saving text files in Unity on Android, then reading them in native Android.
One of the files we read is a HMACMD5 signature, created with the code,
byte[] bData = System.Text.Encoding.UTF8.GetBytes (data);
byte[] bKey = System.Text.Encoding.UTF8.GetBytes (key);
using (HMACMD5 hmac = new HMACMD5(bKey)) {
byte[] signature = hmac.ComputeHash (bData);
return System.Convert.ToBase64String (signature);
}
And then written to the phone with,
public static void SaveText (string path, string data) {
using (FileStream fs = new FileStream(path, FileMode.Create)) {
using (StreamWriter sw = new StreamWriter(fs)) {
sw.Write (data);
}
}
}
The other string we're saving is a JSON string dump. The signature has a newline character at the end of the string, but the JSON string doesn't. I know I can manually add one, but this question is about reading the accurate file contents.
On Android, based on previous SO answers, I read the file with,
String readFile(File file) {
StringBuilder text = new StringBuilder();
try {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
text.append(line);
text.append("\n");
}
br.close();
}
catch (IOException e) {
MyLogger.e(LOG_TAG, "Error opening file " + file.getPath(), e);
}
return text.toString();
}
I'm manually adding the newline character after every line, but if I do this, I don't accurately read the JSON file, which doesn't have a newline character at the end. If I don't add the newline, I don't accurately read the signature file, which does.
You better then do not use readLine() but read().
I am new to ADK. Recently I wrote a program to get strings from
other devices. It's OK except for Chinese string. Can anyone
help to fix this problem?
my codes look like this
mFileDescriptor = mUsbManager.openAccessory(mAccessory);
FileDescriptor fd = mFileDescriptor.getFileDescriptor();
mInputStream = new FileInputStream(fd);
byte[] b = new byte[512];
int n;
String str;
while ((n = mInputStream.read(b)) != -1)
{
......
str = (new String(b, 0, n, Charset.forName("UTF-8")));
......
}
Chinese language is represented in Unicode as multi-byte characters. So, to read it properly, you need to use specific reader objects which can understand and interpret the multi-byte character encoding concept.
Essentially, you need to do something like this:
try {
BufferedReader bufReader = new BufferedReader(
new InputStreamReader(inputstream,Charset.forName("UTF-8")));
String line = null;
while ((line= bufReader.readLine())!= null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
I am calling a HTML page via a web servise . I need to get hole source code of HTML page.
My problem is that, when I convert the http response to string I am getting only some part of HTML page. How do I can get hole HTML page .Please help me.
//paramString1 = url,paramString = header, paramList = paramiters
public String a(String paramString1, String paramString2, List paramList)
{
String str1 = null;
HttpPost localHttpPost = new HttpPost(paramString1);
localHttpPost.addHeader("Accept-Encoding", "gzip");
InputStream localInputStream = null;
try
{
localHttpPost.setEntity(new UrlEncodedFormEntity(paramList));
localHttpPost.setHeader("Referer", paramString2);
HttpResponse localHttpResponse = this.c.execute(localHttpPost);
int i = localHttpResponse.getStatusLine().getStatusCode();
localInputStream = localHttpResponse.getEntity().getContent();
Header localHeader = localHttpResponse.getFirstHeader("Content-Encoding");
if ((localHeader != null) && (localHeader.getValue().equalsIgnoreCase("gzip")))
{
GZIPInputStream localObject = null;
localObject = new GZIPInputStream(localInputStream);
Log.d("API", "GZIP Response decoded!");
BufferedReader localBufferedReader = new BufferedReader(new InputStreamReader((InputStream)localObject, "UTF-8"));
StringBuilder localStringBuilder = new StringBuilder();
while(true){
String str2 = localBufferedReader.readLine();
if (str2 == null)
break;
localHttpResponse.getEntity().consumeContent();
str1 = localStringBuilder.toString();
localStringBuilder.append(str2);
continue;
}
}
}
catch (IOException localIOException)
{
localHttpPost.abort();
}
catch (Exception localException)
{
localHttpPost.abort();
}
Object localObject = localInputStream;
return (String)str1;
Are you receiving the HTML in the variable paramString1?, in that case, are you encoding the String somehow or its just plane HTML?
Maybe the HTML special characters are breaking your response. Try encoding the String with urlSafe Base64 in your server side, and decoding it in the client side:
You can use the function Base64 of Apache Commons.
Server Side:
Base64 encoder = new Base64(true);
encoder.encode(yourBytes);
Client side:
Base64 decoder = new Base64(true);
byte[] decodedBytes = decoder.decode(paramString1);
HttpPost localHttpPost = new HttpPost(new String(decodedBytes));
You may not get the complete source code in your stringBuilder as it must be exceeding the max size of stringBuilder as StringBuilder is set of arrays. If u want to store that particular sourcecode. You may try this: The inputStream (which contains html source code) data, store directly into a File. Then you will have complete source code in that file and then perform file operation to whatever you require. See if this may help you.
So I'm unexpectedly receiving a FileNotFoundException. As you can see, shortly before I call the FileReader, I call FileInputStream which works fine. I've tried putting the FileReader in its own Try/Catch clause, but receive the same result. I've gutted most of the lines unnecessary to my question from this block. (Ultimately I call LineNumberReader as well, though I removed it from the block because I'm not even getting that far.)
String FILENAME = "file.txt";
try {
byte[] buffer = new byte[128];
String toStr = new String();
TextView view = (TextView)findViewById(R.id.textview);
FileInputStream fis = openFileInput(FILENAME); /////File is found successfully here/////
fis.read(buffer);
fis.close();
toStr = new String(buffer);
view.append(toStr);
FileReader fr = new FileReader(FILENAME); /////FileNotFoundExceptionThrownHere/////
/////do stuff here/////
fr.close();
}
catch (FileNotFoundException e) {
TextView view = (TextView)findViewById(R.id.textview);
view.append("file not found!");
}
catch (IOException e) {
TextView view = (TextView)findViewById(R.id.textview);
view.append("IO error!");
}
Also, please keep in mind when answering that I'm still somewhat of a novice when it comes to java. I have experience in a couple other languages but java is a bit of a different breed of monster to me. Any help would be tremendously appreciated!
openFileInput() and new FileReader() do not take the same parameter.
openFileInput("file.txt") is equivalent to new FileReader(new File(getFilesDir(), "file.txt")).
I have an application where i need to download a large amount of data via a SOAP call to a webservice into the application when it is first run. The response is then sent to a function which converts the XML and stores the data in a db file.
The data is more than 16MB in size and i have a java.lang.OutOfMemoryError everytime.
Modifying the webservice to give out smaller amounts of data is not an option.
Is there a way to be able to download the large data? Something like an InputStream perhaps?
This is my code
public Protocol[] getProtocols() {
String METHOD_NAME = "GetProtocols";
String SOAP_ACTION = "urn:protocolpedia#GetProtocols";
Log.d("service", "getProtocols");
SoapObject response = invokeMethod(METHOD_NAME, SOAP_ACTION);
return retrieveProtocolsFromSoap(response);
}
private SoapObject invokeMethod(String methodName, String soapAction) {
Log.d(TAG, "invokeMethod");
SoapObject request = GetSoapObject(methodName);
SoapSerializationEnvelope envelope = getEnvelope(request);
return makeCall(envelope, methodName, soapAction);
}
Can anyone suggest what should be done in this case?
Thanks and regards
Mukul
Just an update, I found that the "call" method in AndroidHttpTransport was running out of memory at this line -
if (debug) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[256];
while (true) {
int rd = is.read(buf, 0, 256);
if (rd == -1)
break;
bos.write(buf, 0, rd);
}
bos.flush();
buf = bos.toByteArray(); //Goes out of memory here
responseDump = new String(buf);
is.close();
is = new ByteArrayInputStream(buf);
the call to toByteArray takes a lot of memory, so to overcome this, instead of converting the response to a byte array, i now directly write it to an XML file, and this is saved at a location of my choice. Here -
if (debug) {
FileOutputStream bos = new FileOutputStream("/data/data/com.mypackage.myapp/response.xml");
byte[] buf = new byte[1048576];
int current = 0; int i=0; int newCurrent = 0;
while ((current = inputStream.read(buf)) != -1) {
newCurrent = newCurrent + current;
Log.d("current", "Current = " + current + " total = "+newCurrent+" i = "+i++);
bos.write(buf, 0, current);
}
bos.flush();
}
The device no longer runs out of memory, and i have a custom parse method that takes this XML and writes it to the DB.
Two strategies to help you solve this problem:
Save your SOAP XML stream directly to disk as you download it. Don't store it in memory.
Parse it using a SAX-style parser, where you don't load the whole DOM in memory, but rather parse it in chunks.
Depending on the kind of XML you are handling, using SAX parsers is usually harder in code; you will have to keep track of many things yourself, and you won't be able to "jump" from section to section of your DOM tree. But the memory consumption will be way lower.
Take note, however, that many "high-level" network communication libraries usually load the whole XML DOM in memory, which might be the case here. You will probably have to create and manage the HTTP connection yourself, and then manually parse the result.
Fixed!
I downloaded/copied HttpTransportSE java class from here (after copied, some code errors can occur, but they are all quick fixable) and added to my package:
https://github.com/mosabua/ksoap2-android/blob/master/ksoap2-j2se/src/main/java/org/ksoap2/transport/HttpTransportSE.java
removed from my Connection class this row:
import org.ksoap2.transport.HttpsTransportSE;
and substituted this code in my new HttpTransportSE.java file:
if (debug) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[256];
while (true) {
int rd = is.read(buf, 0, 256);
if (rd == -1)
break;
bos.write(buf, 0, rd);
}
bos.flush();
buf = bos.toByteArray(); //Goes out of memory here
responseDump = new String(buf);
is.close();
is = new ByteArrayInputStream(buf);
}
with this
if (debug) {
FileOutputStream bos = new FileOutputStream(file);
byte[] buf = new byte[256];
while (true) {
int rd = is.read(buf, 0, 256);
if (rd == -1) {
break;
}
bos.write(buf, 0, rd);
}
bos.flush();
}
where "file" is a simple file object like new File("/sdcard/","myFile.xml") for example