I have a curious problem, respectively a weird effect of using my self-programmed android App.
My app reads out the HTML-source-code of a website and parse it for my desired information. And it work... oh well, not really consistent.
Scenario 1: I use my WLan at home and run my app -> All is working fine. All desired items can be seen in my ListView
Scenario 2: I use my mobile Internet, like Edge or HSDPA -> My ListView is only presenting 1 Item. All of the others are vanished...
I don' t know why. Could there be any time-out, that detain the app to read out the whole HTML-site? But all of the other items would directly follow in the next line of the HTML-source-code...
I have no idea how could I fix it. On google I didn' t find anyone else with the same problem.
Regards, Julian
Here is some code
// With this I get the HTML-source-code
URL url = new URL("http://www.area4.de);
URLConnection conn = url.openConnection();
DataInputStream dataIn = new DataInputStream(conn.getInputStream());
BufferedReader reader = new BufferedReader(new InputStreamReader(dataIn, "UTF-8"));
String line;
// Then I parse the code with
while ((line=reader.readLine()) != null)
{
if (line.contains(searchPattern))
al.add(line); //al is an ArrayList
}
That was all I do in my app till now (besides presenting the arrayList in a ListView).
The source code of the site you can see in your browser (Ctrl + u). I search for these lines
THIRTY SECONDS TO MARS //
DROPKICK MURPHYS //
With 3G I only get thirty-seconds-to-mars...
Ah, I solved it. I searched, as it can be seen above, with this code-snippet
while ((line=reader.readLine()) != null)
{
if (line.contains(searchPattern))
al.add(line); //al is an ArrayList
}
With WLan (and my emulator) I really have a new line for each band e. g.:
line1
line2
line3
....
But with Edge or HDSPA all lines I get with Wlan are written in one line.
line1line2line3.... And with my regex i delte all before and after the line when I find a desired target. Hope you understand, it' s difficult to explain it in a foreign language.
A simple
while (line.contains(searchPattern))
fixed it.
You can always try reading whole http response before sending it for parsing. This way you get to see whole document is loaded properly.
Related
My code used to work, it does not work anymore, I tried troubleshooting and can't figure out why.
I have this piece of code in my PHP:
$android_id_01 = $_GET['pmysql_room_id'];
$android_id_02 = "";
$f = fopen("00_android_id_01.txt", "w");
fwrite($f, print_r($android_id_01, true));
fclose($f);
$f = fopen("00_android_id_02.txt", "w");
fwrite($f, print_r($android_id_02, true));
fclose($f);
For troubleshooting I created two android IDs ($android_id_01 and $android_id_02) which are both empty (The first one is From Android and the second one I created directly from PHP).
Now when I launch my Android device, the PHP file is executed from server side and both the text files are created empty and identical. Now my code only works when I use $android_id_02 and not $android_id_01 from the code below:
if ($android_id == '')
{
//my code
}
(Yes when I use either one of the $android_id_01 OR $android_id_02 I rename it to $android_id and comment out the other one)
My question is, although this was working yesterday, why does it work with $android_id_02 = ""; and not $android_id_01 = $_GET['pmysql_room_id']; even though they are both empty????
I don't know what changed from yesterday to today.
Ok after a bit of troubleshooting I found a solution, strange though.
On the server side "display_errors" under PHP settings must be turned off. Somehow having this on interferes with the json_encode sent back to android client. (even though my code is not generating any errors)
I am facing a strange issue, and I am not able to debug it out. I have implemented a logic for uploading stream of data and am using Volley for the same, I have customized a logic little bit in HurlStack, addBodyIfExists api,so that body of type "application/octet-stream" can be handled.
My logic is to post progress to user, so that UI can be updated indicating user progress in upload, below my logic for same.
int toRead = length; // File length
byte[] data = new byte[4096];
connection.setDoOutput(true);
if(length != -1) {
connection.setFixedLengthStreamingMode(length);
} else {
connection.setChunkedStreamingMode(4096);
}
OutputStream os;
int i;
int count;
os = connection.getOutputStream();
int progress= 0;
try {
for(i = 0; (count= is.read(data)) > 0; ++i) { // is, is not null and contains a valid input stream
os.write(data, 0, count); // at this line am getting unexpected end of stream
progress+= count;
if(i % 20 == 0) {
rs.deliverProgress(progress, 0L);
progress= 0;
}
}
os.flush();
} finally {
if(is != null) {
is.close();
}
if(os != null) {
os.close();
}
}
on executing above code am getting this, although I have verified, output stream is not null, neither do input stream, it fails in first iteration of read loop itself, am seeing it has read 4096 bytes and then trying to write the same.
java.net.ProtocolException: unexpected end of stream
at com.android.okhttp.internal.http.HttpConnection$FixedLengthSink.close(HttpConnection.java:326)
at com.android.okio.RealBufferedSink.close(RealBufferedSink.java:174)
at com.android.okio.RealBufferedSink$1.close(RealBufferedSink.java:142)
any help in debugging above will he highly appreciated.
This may help you :
That exception is thrown by FixedLengthInputStream when the expected number of bytes (usually set in the content-length header of the response) is larger than the actual data in the response.
Check that the content-length header is correct. (If you're supplying your own value for the content length, make sure it is correct.)
It would help to see your code that sets up the input stream.
Already Fixed it, please add "Accept-Encoding", "identity" in header, then the server-side will get command that it will not modify the response, then send back to Clients.
If you have checked everywhere in your code and tried every solution in stackoverflow and github but the issue still occurs, and you have only tested your code on emulator, then, you should try to run your code on your real device instead. Maybe it will work, or maybe it won't, but if you feel desperate, just have a try, seriously. I was astonished when I happened to find that my code ran with bugs on emulator everytime but successfully on my mobile phone. Besides, the code also ran sucessfully on others' android emulators. So I guess there is something wrong in my android studio configuration that I can't find out. I have no idea why this happen, just like we don't know why "Clean Project/Invalidate caches" sometimes works better than any solution.
It is a little strange that your data length might be unknown.
Is it a media file? Or a live stream?
Anyway, I tried to upload my live stream data. And it happened in the same error.
I added this setting to the Connection and solved my problem.
Transfer-Encoding : chunked
("setChunkedStreamingMode" didn't work. I still don't know why.)
This happens for me on android emulator and doesn't happen on my physical android device.
I was doing GET request to flask server running on 0.0.0.0 on my laptop from the android app.
To fix it on the emulator, add the servers ip address in the emulators proxy.
see How to set up Android emulator proxy settings
The exact problem i had was unexpected end of stream retrofit
I am opening a connection setting up a custom protocol like this:
WebSocketSubProtocol d = new WebSocketSubProtocol("MyCustomProto",WebSocketEncoding.TEXT);
mJWC.addSubProtocol(d);
mJWC.open(mURL);
But... Server side, I receive tis in the protocol string
"org.jwebsocket.json MyCustomProto"
How can I remove from the string the "org.jwebsocket.json" ?
I don't wanna do it server side...
Thanks!
I will answer to my own question.
By calling the "addSubProtocol" doesn't seem to be the right solution for couple of reasons:
if you call those 3 lines of code multiple time (if the first time the connection failed for example..) well the the protocol string would be something like
"org.jwebsocket.json MyCustomProto MyCustomProto"
It just keep adding the protocol..
So I found a turn around. Now I don't use that "addSubProtocol" but instead I defined the protocol directly when I create the socket
mJWC = new BaseTokenClient("client||"+code+"||"+name,WebSocketEncoding.TEXT);
Voila.. Now no more "org.jwebsocket.json" anymore
I'm using JDOM with my Android project, and every time I get a certain set of characters in my server response, I end up with these error messages:
05-04 10:08:46.277: E/PARSE: org.jdom.input.JDOMParseException: Error on line 95 of document UTF-8: At line 95, column 5263: unclosed token
05-04 10:08:46.277: E/Error Handler: Handler failed: org.jdom.input.JDOMParseException: Error on line 1: At line 1, column 0: syntax error
When I make the same query through google chrome, I can see that all of the XML came through fine, and that there are in fact no areas where a token is not closed. I have run into this problem several times throughout the development of the application, and the solution has always been to remove odd ascii characters (copyright logos, or trademark characters, etc. that got copied/pasted into those data fields). How can I get it to either a remove those characters, or b strip them and continue the function. Here's an example of one of my parse functions.
public static boolean parseUserData(BufferedReader br) {
SAXBuilder builder = new SAXBuilder();
Document document = null;
try {
document = builder.build(br);
/* XML Output to Logcat */
if (document != null) {
XMLOutputter outputter = new XMLOutputter(
Format.getPrettyFormat());
String xmlString = outputter.outputString(document);
Log.e("XML", xmlString);
}
Element rootNode = document.getRootElement();
if (!rootNode.getChildren().isEmpty()) {
// Do stuff
return true;
}
} catch (Exception e) {
GlobalsUtil.errorUtil
.setErrorMessage("Error Parsing XML: User Data");
Log.e(DEBUG_TAG, e.toString());
return false;
}
}
It distinctly sounds like a character encoding issue. I think duffymo is correct in his assessment. I have two comments though ....
If you are getting your data through a URL you should be using the URLConnection.getContentType() to get the charset (if it is set and the charset is not null) to set up the InputStreamReader on the URL's InputStream...
Have you tried JDOM 2.0.1? It is the first JDOM version that is fully tested on Android... (and the only 'supported' JDOM version on Android). JDOM 2.0.1 also has a number of performance tweaks, and memory optimizations that should make your processing faster. It also fixes a number of bugs.... though from what I see you should not run in to any bug problems.....
Check out https://github.com/hunterhacker/jdom/wiki/JDOM2-Migration-Issues and https://github.com/hunterhacker/jdom/wiki/JDOM2-and-Android
Is the BufferedReader constructed to take the encoding argument? Perhaps you need to tell the Reader or InputStream that you pass to use UTF-8.
I've been trying all day to make this thing works but it's still not right yet. I've checked so many posts around here and tested so many different implementations that I'dont know where to look now...
Here is my situation, I have a small php test file (gz.php) on my server wich looks like this :
header("Content-Encoding: gzip");
print("\x1f\x8b\x08\x00\x00\x00\x00\x00");
$contents = gzcompress("Is it working?", 9);
print($contents);
This is the simplest I could do and it works fine with any web browser.
Now I have an Android activity using Jsoup that has this code :
URL url = new URL("http://myServerAdress.com/gz.php");
doc = Jsoup.parse(url, 1000);
Which cause an empty EOFException on the "Jsoup.parse" line.
I've read everywhere that Jsoup is supposed to parse gzipped content without having to do anything special, but obviously, there's something missing.
I've tried many other ways like using Jsoup.connect().get() or InpuStream, GZipInputStream and DataInpuStream. I did try the gzDeflate() and gzencode() methods from PHP as well but no luck either. I even tried not to declare the header-encoding in PHP and try to deflate the content later...but it was as clever as effective...
It has to be something "stupid" I'm missing but I just can't tell what... anybody has an idea?
(ps : I'm using Jsoup 1.7.0, so the latest one as of now)
The asker indicated in a comment that gzcompress was writing a CRC that was both incorrect and incomplete, according to information from here, the operative code being:
// Display the header of the gzip file
// Thanks ck#medienkombinat.de!
// Only display this once
echo "\x1f\x8b\x08\x00\x00\x00\x00\x00";
// Figure out the size and CRC of the original for later
$Size = strlen($contents);
$Crc = crc32($contents);
// Compress the data
$contents = gzcompress($contents, 9);
// We can't just output it here, since the CRC is messed up.
// If I try to "echo $contents" at this point, the compressed
// data is sent, but not completely. There are four bytes at
// the end that are a CRC. Three are sent. The last one is
// left in limbo. Also, if we "echo $contents", then the next
// byte we echo will not be sent to the client. I am not sure
// if this is a bug in 4.0.2 or not, but the best way to avoid
// this is to put the correct CRC at the end of the compressed
// data. (The one generated by gzcompress looks WAY wrong.)
// This will stop Opera from crashing, gunzip will work, and
// other browsers won't keep loading indefinately.
//
// Strip off the old CRC (it's there, but it won't be displayed
// all the way -- very odd)
$contents = substr($contents, 0, strlen($contents) - 4);
// Show only the compressed data
echo $contents;
// Output the CRC, then the size of the original
gzip_PrintFourChars($Crc);
gzip_PrintFourChars($Size);
Jonathan Hedley commented, "jsoup just uses a normal Java GZIPInputStream to parse the gzip, so you'd hit that issue with any Java program." The EOFException is presumably due to the incomplete CRC.