I've been trying all day to make this thing works but it's still not right yet. I've checked so many posts around here and tested so many different implementations that I'dont know where to look now...
Here is my situation, I have a small php test file (gz.php) on my server wich looks like this :
header("Content-Encoding: gzip");
print("\x1f\x8b\x08\x00\x00\x00\x00\x00");
$contents = gzcompress("Is it working?", 9);
print($contents);
This is the simplest I could do and it works fine with any web browser.
Now I have an Android activity using Jsoup that has this code :
URL url = new URL("http://myServerAdress.com/gz.php");
doc = Jsoup.parse(url, 1000);
Which cause an empty EOFException on the "Jsoup.parse" line.
I've read everywhere that Jsoup is supposed to parse gzipped content without having to do anything special, but obviously, there's something missing.
I've tried many other ways like using Jsoup.connect().get() or InpuStream, GZipInputStream and DataInpuStream. I did try the gzDeflate() and gzencode() methods from PHP as well but no luck either. I even tried not to declare the header-encoding in PHP and try to deflate the content later...but it was as clever as effective...
It has to be something "stupid" I'm missing but I just can't tell what... anybody has an idea?
(ps : I'm using Jsoup 1.7.0, so the latest one as of now)
The asker indicated in a comment that gzcompress was writing a CRC that was both incorrect and incomplete, according to information from here, the operative code being:
// Display the header of the gzip file
// Thanks ck#medienkombinat.de!
// Only display this once
echo "\x1f\x8b\x08\x00\x00\x00\x00\x00";
// Figure out the size and CRC of the original for later
$Size = strlen($contents);
$Crc = crc32($contents);
// Compress the data
$contents = gzcompress($contents, 9);
// We can't just output it here, since the CRC is messed up.
// If I try to "echo $contents" at this point, the compressed
// data is sent, but not completely. There are four bytes at
// the end that are a CRC. Three are sent. The last one is
// left in limbo. Also, if we "echo $contents", then the next
// byte we echo will not be sent to the client. I am not sure
// if this is a bug in 4.0.2 or not, but the best way to avoid
// this is to put the correct CRC at the end of the compressed
// data. (The one generated by gzcompress looks WAY wrong.)
// This will stop Opera from crashing, gunzip will work, and
// other browsers won't keep loading indefinately.
//
// Strip off the old CRC (it's there, but it won't be displayed
// all the way -- very odd)
$contents = substr($contents, 0, strlen($contents) - 4);
// Show only the compressed data
echo $contents;
// Output the CRC, then the size of the original
gzip_PrintFourChars($Crc);
gzip_PrintFourChars($Size);
Jonathan Hedley commented, "jsoup just uses a normal Java GZIPInputStream to parse the gzip, so you'd hit that issue with any Java program." The EOFException is presumably due to the incomplete CRC.
Related
My code used to work, it does not work anymore, I tried troubleshooting and can't figure out why.
I have this piece of code in my PHP:
$android_id_01 = $_GET['pmysql_room_id'];
$android_id_02 = "";
$f = fopen("00_android_id_01.txt", "w");
fwrite($f, print_r($android_id_01, true));
fclose($f);
$f = fopen("00_android_id_02.txt", "w");
fwrite($f, print_r($android_id_02, true));
fclose($f);
For troubleshooting I created two android IDs ($android_id_01 and $android_id_02) which are both empty (The first one is From Android and the second one I created directly from PHP).
Now when I launch my Android device, the PHP file is executed from server side and both the text files are created empty and identical. Now my code only works when I use $android_id_02 and not $android_id_01 from the code below:
if ($android_id == '')
{
//my code
}
(Yes when I use either one of the $android_id_01 OR $android_id_02 I rename it to $android_id and comment out the other one)
My question is, although this was working yesterday, why does it work with $android_id_02 = ""; and not $android_id_01 = $_GET['pmysql_room_id']; even though they are both empty????
I don't know what changed from yesterday to today.
Ok after a bit of troubleshooting I found a solution, strange though.
On the server side "display_errors" under PHP settings must be turned off. Somehow having this on interferes with the json_encode sent back to android client. (even though my code is not generating any errors)
My first question here, & a newbie to everything including coding.
I am using django-rest-framework to call a rest api from an android app I am building.
I am calling another view's method from the api_view method of django-rest-framework.
This works great with the GET method, but fails while using the POST method.
Below is the code:
View 1:
#api_view(['GET', 'POST'])
def get_dos(request):
return View2.get_data(request)
View 2:
def get_data(request):
#if I print request.body, it goes through without problems
#print request.body
#but commenting it fails when I read from request.POST.get below:
var1 = int(request.POST.get('test', 0))
#fails in the above statement.
I have tried commenting all middleware classes in the settings file, but no luck :(. I keep getting "You cannot access body after reading from request's data stream" error.
Am I doing something silly? Any assistance appreciated. Thanks!
I am facing a strange issue, and I am not able to debug it out. I have implemented a logic for uploading stream of data and am using Volley for the same, I have customized a logic little bit in HurlStack, addBodyIfExists api,so that body of type "application/octet-stream" can be handled.
My logic is to post progress to user, so that UI can be updated indicating user progress in upload, below my logic for same.
int toRead = length; // File length
byte[] data = new byte[4096];
connection.setDoOutput(true);
if(length != -1) {
connection.setFixedLengthStreamingMode(length);
} else {
connection.setChunkedStreamingMode(4096);
}
OutputStream os;
int i;
int count;
os = connection.getOutputStream();
int progress= 0;
try {
for(i = 0; (count= is.read(data)) > 0; ++i) { // is, is not null and contains a valid input stream
os.write(data, 0, count); // at this line am getting unexpected end of stream
progress+= count;
if(i % 20 == 0) {
rs.deliverProgress(progress, 0L);
progress= 0;
}
}
os.flush();
} finally {
if(is != null) {
is.close();
}
if(os != null) {
os.close();
}
}
on executing above code am getting this, although I have verified, output stream is not null, neither do input stream, it fails in first iteration of read loop itself, am seeing it has read 4096 bytes and then trying to write the same.
java.net.ProtocolException: unexpected end of stream
at com.android.okhttp.internal.http.HttpConnection$FixedLengthSink.close(HttpConnection.java:326)
at com.android.okio.RealBufferedSink.close(RealBufferedSink.java:174)
at com.android.okio.RealBufferedSink$1.close(RealBufferedSink.java:142)
any help in debugging above will he highly appreciated.
This may help you :
That exception is thrown by FixedLengthInputStream when the expected number of bytes (usually set in the content-length header of the response) is larger than the actual data in the response.
Check that the content-length header is correct. (If you're supplying your own value for the content length, make sure it is correct.)
It would help to see your code that sets up the input stream.
Already Fixed it, please add "Accept-Encoding", "identity" in header, then the server-side will get command that it will not modify the response, then send back to Clients.
If you have checked everywhere in your code and tried every solution in stackoverflow and github but the issue still occurs, and you have only tested your code on emulator, then, you should try to run your code on your real device instead. Maybe it will work, or maybe it won't, but if you feel desperate, just have a try, seriously. I was astonished when I happened to find that my code ran with bugs on emulator everytime but successfully on my mobile phone. Besides, the code also ran sucessfully on others' android emulators. So I guess there is something wrong in my android studio configuration that I can't find out. I have no idea why this happen, just like we don't know why "Clean Project/Invalidate caches" sometimes works better than any solution.
It is a little strange that your data length might be unknown.
Is it a media file? Or a live stream?
Anyway, I tried to upload my live stream data. And it happened in the same error.
I added this setting to the Connection and solved my problem.
Transfer-Encoding : chunked
("setChunkedStreamingMode" didn't work. I still don't know why.)
This happens for me on android emulator and doesn't happen on my physical android device.
I was doing GET request to flask server running on 0.0.0.0 on my laptop from the android app.
To fix it on the emulator, add the servers ip address in the emulators proxy.
see How to set up Android emulator proxy settings
The exact problem i had was unexpected end of stream retrofit
I want to access a android device from python to download some photos.
libmtp works from the CLI.
Than pymtp. It's been around for a while but it's designed for python 2 and i'm using python 3. Meanwhile fixed several minor issues but i'm stuck at an error from function get_filelisting
specially this section:
ret = []
next = files
while next:
ret.append(next.contents)
if (next(next.contents) is None):
break
next = next(next.contents)
The error is related to the "next".
That section looks strange to me, i've been coding in python for a while but i'm new to ctypes. Tried a lot of variants, they all failed. The "next" could be confusing with python buildin function so i renamed it to nextpointer and came to this code:
ret = []
nextpointer = files
while nextpointer:
ret.append(nextpointer.contents)
nextpointer = nextpointer.contents.next
It seems to work but did it work by accident ? does it have any design flaws ? Could anyone with experience on python ctypes confirm this a solution ? Any suggestion welcome.
From python2.7 documentation
next(iterator[, default])
Retrieve the next item from the iterator by calling its next() method. If default is given, it is returned if the iterator is
exhausted, otherwise StopIteration is raised.
from python3 documentation
next(iterator[, default])
Retrieve the next item from the iterator by calling its __next__() method. If default is given, it is returned if the iterator is
exhausted, otherwise StopIteration is raised.
Notice that next() method was removed from python3 but the function still exists.
This is all I can say about the next function and .next()/__next__() methods.
I downloaded the pymtp module and get_filelisting() is slightly different from what you posted in your ported code, here it is:
ret = []
next = files
while next:
ret.append(next.contents)
if (next.contents.next == None):
break
next = next.contents.next
If none of this helped you (which probably didn't :D), the version of pymtp library that I am using is 0.0.6 download using pip.
I have a form in which user can add information and add images. The images are base64 encoded so everything is stored in a json object. This object is sent to the server (with $resource) when the user submits it.
If a user adds for example 3 Images with about 2MB per Image, has a shitty connection like Edge, and wants to upload it it seems like it's taking forever. The user just sees the $ionicLoading overlay without information how long it will take or how much % are already uploaded.
The UX is bad because the user could assume, that the app froze or is in an endless loop and that it's just a bad app.
I have the following ideas but no idea if they are possible or
Is there a way in angular, cordova or ionic to get the browserinformation how much % are already uploaded?
Is there a way to get the current uploadspeed? I could get the size of my object by getting the length of my stringified JSON Object, divide it 1024 to get the kB. Then i would check the current uploadSpeed every second and add the current uploadspeed to a variable. With this information i could calculate the uploaded % approximately
JQuery ajaxForm Plugin?
Sounds like what you are looking for is progress events from XHR2.
Assuming your server is setup to handle XHR2 and return content-length, In plain JavaScript:
function upload(blobOrFile) {
var xhr = new XMLHttpRequest();
xhr.open('POST', '/server', true);
xhr.onload = function(e) { ... };
// Listen to the upload progress.
var progressBar = document.querySelector('progress');
xhr.upload.onprogress = function(e) {
if (e.lengthComputable) {
progressBar.value = (e.loaded / e.total) * 100;
progressBar.textContent = progressBar.value; // Fallback for unsupported browsers.
}
};
xhr.send(blobOrFile);
}
upload(new Blob(['hello world'], {type: 'text/plain'}));
Upload speed is also calculable using the information returned in the progress event, as you described.
As for this implementation in AngularJS/Ionic, it seems like this is a longstanding issue within the framework that $http doesn't really support progress events.
I have seen implementations that utilize a special angular directive written for this, or even utilize a jQuery file upload implementation.