I have an API on rails 4 that accepts HTTP requests in the form of a file upload. Everything works fine on Localhost but on Heroku the POST request doesn't seem to do anything.
This is what my Android POST request looks like:
public static byte[] postData(String operation, byte[] binaryData) {
String urlString = baseUrl + "/" + operation;
String boundary = "uahbkjqtjgecuaoehuaebkjahj";
byte[] postData = null;
URLConnection urlConnection;
DataInputStream responseDataInputStream;
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
StringBuffer startBuffer = new StringBuffer("--").append(boundary).append("\r\n");
startBuffer.append("Content-Disposition: form-data; name=\"data\"; ").append("filename=\"data.dat\"\r\n");
startBuffer.append("Content-Type: application/octet-stream\r\n\r\n");
StringBuffer endBuffer = new StringBuffer("\r\n--").append(boundary).append("--\r\n");
String startRequestData = startBuffer.toString();
String endRequestData = endBuffer.toString();
try {
URL url = new URL(urlString);
urlConnection = url.openConnection();
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
urlConnection.setUseCaches(false);
urlConnection.setConnectTimeout(5000); //5 seconds
urlConnection.setReadTimeout(5000);//5 seconds
urlConnection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
urlConnection.connect();
DataOutputStream _request = new DataOutputStream(urlConnection.getOutputStream());
// Write the start portion of the request
byteArrayOutputStream.write(startRequestData.getBytes());
postData = byteArrayOutputStream.toByteArray();
_request.write(postData);
// Write the Binary Packet
_request.write(binaryData);
// Write the end portion of the request
byteArrayOutputStream.reset();
byteArrayOutputStream.write(endRequestData.getBytes());
postData = byteArrayOutputStream.toByteArray();
_request.write(postData);
_request.flush();
_request.close();
// Read in the response bytes
InputStream is = urlConnection.getInputStream();
responseDataInputStream = new DataInputStream(is);
byteArrayOutputStream.reset();
byte[] buffer = new byte[responseDataInputStream.available()];
while (responseDataInputStream.read(buffer) != -1) {
byteArrayOutputStream.write(buffer);
buffer = new byte[responseDataInputStream.available()];
}
byte[] responseData = byteArrayOutputStream.toByteArray();
return responseData;
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (SocketTimeoutException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return new byte[0];
}
My rails controller
skip_before_filter :verify_authenticity_token
respond_to :raw
before_filter :read_file
def read_file
data = params[:data].tempfile
data_compressed = ''
File.open(data, 'r') do |file|
file.each do |line|
data_compressed.concat(line)
end
end
#json_data = Zlib::Inflate.inflate(data_compressed)
end
def an_action
#processing stuff
response = Zlib::Deflate.deflate(j_response)
send_data #json_response.to_s
end
Heroku logs shows that the controller action is hit but nothing more from the logs
2015-05-05T22:39:29.860161+00:00 heroku[router]: at=info method=POST path="/api/login" host=[my app].herokuapp.com request_id=9d9c91b1-2ce2-4db5-9b54-3dea0322e211 fwd="197.237.24.179" dyno=web.1 connect=4ms service=12ms status=500 bytes=1683
2015-05-05T22:40:40.952219+00:00 heroku[router]: at=info method=POST path="/api/signup" host=[my app].herokuapp.com request_id=4adac44c-66e6-4001-a568-8eb913176091 fwd="197.237.24.179" dyno=web.1 connect=4ms service=8ms status=500 bytes=1683
After shifting my focus from Heroku to the Android code I figured out that there was an endless loop at this section of the code:
while (responseDataInputStream.read(buffer) != -1) {
byteArrayOutputStream.write(buffer);
buffer = new byte[responseDataInputStream.available()];
}
When theres nothing to read the return value is 0 instead of -1. So I updated it to:
while (responseDataInputStream.read(buffer) > 0) {
byteArrayOutputStream.write(buffer);
buffer = new byte[responseDataInputStream.available()];
}
Something strange is that on localhost the first piece of code works and it should work on production too. The documentation states that -1 is returned if the end of stream is reached DataInputStream.read(). Maybe thats a discussion for another day, for now I'm using the second piece of code.
EDIT
This issue has haunted me for weeks and after alot of googling and tweaking of the code I would like to point out that this approach was the WRONG route. The code worked on a WIFI connection but always failed on 3G. So i'll list the code changes that finally worked.
use HttpURLConnection instead of URLConnection. Reason here
Increased the size of Connect and Read timeouts from 5000 to 30000 and 120000 respectively.
Revert the while condition to while (responseDataInputStream.read(buffer) != -1)
Related
I am running a flask server at the backend. I want to read the image from mobile app and send it to the server to detect faces.
This is the java code(client side) for sending image as bytes -
public class client {
public static void main(String [] args) throws Exception{
String url = "http://127.0.0.1:8080/facial";
// 2. create obj for the URL class
URL obj = new URL(url);
// 3. open connection on the url
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type","image/jpeg");
con.setDoInput(true);
con.setDoOutput(true);
try {
System.out.println("Reading image from disk. ");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
baos.flush();
File file = new File("jpeg.jpg");
BufferedImage image1 = ImageIO.read(file);
ImageIO.write(image1, "jpg", baos);
baos.flush();
System.out.println(baos.size());
byte[] bytes = baos.toByteArray();
baos.close();
System.out.println("Sending image to server. ");
OutputStream out = con.getOutputStream();
DataOutputStream image = new DataOutputStream(out);
image.writeInt(bytes.length);
image.write(bytes, 0, bytes.length);
System.out.println("Image sent to server. ");
image.close();
// close the output stream
out.close();
}catch (Exception e) {
System.out.println("Exception: " + e.getMessage());
}
// define object for the reply from the server
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
//Get response from server
int responseCode = con.getResponseCode();
System.out.println("Response Code : " + responseCode);
// read in the response from the server
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
System.out.println(inputLine);
}
// close the input stream
in.close();
}
}
This is my server code -
def get_facial(data):
face_api_url = 'https://southeastasia.api.cognitive.microsoft.com/face/v1.0/detect'
# Set image_url to the URL of an image that you want to analyze.
headers = {'Ocp-Apim-Subscription-Key': subscription_key,
"Content-Type":"application/octet-stream"
}
params = {
'returnFaceId': 'true',
'returnFaceLandmarks': 'false',
'returnFaceAttributes': 'age,gender,headPose,smile,facialHair,glasses,' +
'emotion,hair,makeup,occlusion,accessories,blur,exposure,noise'
}
response = requests.post(face_api_url, params=params, headers=headers, data=data)
faces = response.json()
res={}
import pdb; pdb.set_trace()
res["status"] = '200'
res["num"] = str(len(faces))
return res
#app.route('/facial',methods=['POST'])
def facial():
import pdb; pdb.set_trace()
data=bytes(request.get_data())
res={}
try:
res = get_facial(data)
except:
res['status'] = '404'
print(res)
return json.dumps(res)
After examining - I sent the same image from another python file and checked the size of the data. It was 102564 bytes and it works but
the same image read and sent from java code is 106208 bytes. I don't know where exactly the mistake is.
Any help is appreciated !!:-)
I found a quick fix to this problem -
Path path = Paths.get("jpeg.jpg");
byte[] fileContents = Files.readAllBytes(path);
image.write(fileContents, 0, fileContents.length);
I don't exactly know why reading from imageio fails. My guess is that its also reading the file headers of the jpg file.
I am trying to follow the tutorial here https://developer.android.com/reference/java/net/HttpURLConnection.html
to make a post call in android. The issue I am having it I am not sure how to write the post call to the http connection.
URL url = new URL("https://chart.googleapis.com/chart");
HttpURLConnection client = null;
client = (HttpURLConnection) url.openConnection();
client.setRequestMethod("POST");
client.setRequestProperty("cht", "lc");
client.setRequestProperty("chtt", "This is | my chart");
client.setRequestProperty("chs", "300x200");
client.setRequestProperty("chxt", "x");
client.setRequestProperty("chd", "t:40,20,50,20,100");
client.setDoOutput(true);
client.setChunkedStreamingMode(0);
OutputStream outputPost = new BufferedOutputStream(client.getOutputStream());
outputPost.write(client.getRequestProperties().toString().getBytes());
outputPost.flush();
outputPost.close();
InputStream in = new BufferedInputStream(client.getInputStream());
Log.d(TAG, "Input" + in.read());
client.disconnect();
} catch (MalformedURLException error) {
//Handles an incorrectly entered URL
Log.d(TAG, "MalformedURL");
} catch (SocketTimeoutException error) {
//Handles URL access timeout.
Log.d(TAG, "Socket Timeout");
} catch (IOException error) {
//Handles input and output errors
Log.d(TAG, "IOexception");
}
The tutorial uses a custom method to write the stream but I still run into writing an unknown number of bytes for the body of the POST.
Questions to consider:
Does the Google chart API require information to be sent via the header variables?
Does the Google chart API require information to be sent in the body?
Is the information in the body being sent in the correct format?
Missing Content-Type header variable
Why is the same data being set in the header and body?
After reading the Google Chart Guide the following code will successfully make a POST request to the Google Chart API and retrieve the image as bytes.
To write the post data see the following line in the getImage code sample: con.getOutputStream().write(postDataBytes);
Take note of the following line to set the post size: con.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
public byte[] getImage() throws IOException {
URL obj = new URL("https://chart.googleapis.com/chart");
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// Store the post data
Map<String,Object> params = new LinkedHashMap<>();
params.put("cht", "lc");
params.put("chtt", "This is | my chart");
params.put("chs", "300x200");
params.put("chxt", "x");
params.put("chd", "t:40,20,50,20,100");
// Build the post data into appropriate format
StringBuilder postData = new StringBuilder();
for (Map.Entry<String,Object> param : params.entrySet()) {
if (postData.length() != 0) postData.append('&');
postData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
postData.append('=');
postData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}
byte[] postDataBytes = postData.toString().getBytes("UTF-8");
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
con.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
con.setDoOutput(true);
// post the data
con.getOutputStream().write(postDataBytes);
// opens input stream from the HTTP connection
InputStream inputStream = con.getInputStream();
// read the data from response
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] byteChunk = new byte[4096]; // Or whatever size you want to read in at a time.
int n;
while ( (n = inputStream.read(byteChunk)) > 0 ) {
baos.write(byteChunk, 0, n);
}
inputStream.close();
return baos.toByteArray();
}
I'm trying to create an android application which depends on JSON responses. Sometimes it takes a lot of time for the server to respond and ends in a time out exception. Therefore I would like to add a restriction like my webservice call should abort after 20seconds if there is no response. Can you please help me achieving this idea.
Thanks in Advance.
You're not giving much details on the actual implementation that you have.
However, messing with the timeout seems like it may be an emergency fix to an underlying problem that should be fixed.
However, using websockets for transport could be a possible (and probably more elegant) solution. They provide a persistent connection between client and server once created.
Using websockets on Android and IOS
There are several ways to achieve the goal.
We can using HttpURLConnection to do the http request.
public String doPost() {
if (!mIsNetworkAvailable) {
return null;
}
try {
URL url = new URL(mURL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
for (String key : mHeadersMap.keySet()) {
conn.setRequestProperty(key, mHeadersMap.get(key));
}
conn.setRequestProperty("User-Agent", "Android");
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setRequestProperty("Content-Type", "application/json");
conn.getOutputStream().write(mContent);
conn.getOutputStream().flush();
int rspCode = conn.getResponseCode();
if (rspCode >= 400) {
return null;
}
byte[] buffer = new byte[8 * 1024];
BufferedInputStream bis = new BufferedInputStream(conn.getInputStream());
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int len;
while ((len = bis.read(buffer)) > 0) {
baos.write(buffer, 0, len);
}
baos.flush();
final String result = new String(baos.toByteArray());
baos.close();
return result;
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
setConnectTimeout :Sets the maximum time in milliseconds to wait while connecting.
setReadTimeout:Sets the maximum time to wait for an input stream read to complete before giving up.
Reference: http://developer.android.com/reference/java/net/URLConnection.html
I'm downloading different medias files from my http server; mp3, jpg/png/ and html.
Everything worked fine when I used the now deprecated HttpClient.
I decided to use the HttpURLConnection.
But I encounter a problem with text files(html).
read() blocks on small html files, maybe waiting for a EOF or I don't know what, during few seconds and exits with the Exception "unexpected end of stream".
My code is:
URL url = new URL(urlString);
postParams = String.format("registration=%s&"....);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setDoOutput(true); // It's a POST request which replies sending a file
urlConnection.setChunkedStreamingMode(0);
if (fileName.contains(".htm")) { // Tried this to see...
urlConnection.setRequestProperty("Accept-Charset", "UTF-8");
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + "UTF-8");
}
OutputStream output = urlConnection.getOutputStream();
output.write(postParams.getBytes("UTF-8"));
is = new BufferedInputStream(urlConnection.getInputStream());
/* Get information from the HttpURLConnection automatically fires the request
* http://stackoverflow.com/questions/2793150/using-java-net-urlconnection-to-fire-and-handle-http-requests
*/
int status = urlConnection.getResponseCode();
if (status == 200) {
int len, size = 0;
byte[] buf = new byte[128 * 1024];
BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(file));
try {
while ((len = is.read(buf, 0, buf.length)) > 0) {
os.write(buf, 0, len);
size += len;
}
os.flush();
} catch (IOException e) {
Log.d(Constants.APP_TAG, "IOException." + e); // HTML files end here after few seconds
} finally {
nbFilesDownloaded++;
is.close();
os.close();
file.setReadable(true, false);
file.setWritable(true, false);
}
}
Any idea to explain why it cannot normally exit from read()??
EDIT: I verified that for the html files which cause this exception, my webserver doesn't include Content-Length in the header. Can it be the cause of the problem?
Try to use okhttp http://square.github.io/okhttp/. I have downloaded the files correctly. I hope to help.
Paul.
Im currently working on an Android app with heavy server side communication. Yesterday I got a bug report saying that the users aren't able to send (simple) special characters such as ëäï.
I searched but didn't find anything helpful
Possible duplicate ( without answer ):
https://stackoverflow.com/questions/12388974/android-httpurlconnection-post-special-charactes-to-rest-clint-in-android
My relevant code:
public void execute(String method) {
HttpsURLConnection urlConnection = null;
try {
URL url = new URL(this.url);
urlConnection = (HttpsURLConnection) url.openConnection();
urlConnection.setRequestMethod(method);
urlConnection.setReadTimeout(30 * 1000);
urlConnection.setDoInput(true);
if (secure)
urlConnection.setRequestProperty("Authorization", "Basic " + getCredentials());
if (body != null) {
urlConnection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
urlConnection.setFixedLengthStreamingMode(body.length());
urlConnection.setDoOutput(true);
DataOutputStream dos = new DataOutputStream(urlConnection.getOutputStream());
dos.writeBytes(body);
dos.flush();
dos.close();
}
responseCode = urlConnection.getResponseCode();
message = urlConnection.getResponseMessage();
InputStream in = null;
try {
in = new BufferedInputStream(urlConnection.getInputStream(), 2048);
} catch (Exception e) {
in = new BufferedInputStream(urlConnection.getErrorStream(), 2048);
}
if (in != null)
response = convertStreamToString(in);
} catch (UnknownHostException no_con) {
responseCode = 101;
}catch (ConnectException no_con_2){
responseCode = 101;
}catch(IOException io_ex){
if(io_ex.getMessage().contains("No authentication challenges found")){
responseCode = 401;
}else
responseCode = 101;
} catch (Exception e) {
e.printStackTrace();
} finally {
if (urlConnection != null)
urlConnection.disconnect();
}
}
body is a String ;-)
Hope we can solve this together
UPDATE:
Tried:
writeUTF()
need a server capable of understanding the modified UTF-8
byte[] buf = body.getBytes("UTF-8");
dos.write(buf, 0, buf.length);
strings work but no special chars
update: Got it working with StringEntity(* string, "UTF-8") then parse the result to a byte[] and write it with dos.write(byte[])!
--
Setting the encoding of the StringEntity did the trick for me:
StringEntity entity = new StringEntity(body, "UTF-8");
seen here: https://stackoverflow.com/a/5819465/570168
i am not totally sure buy try this utility for your case
URLEncoder.encode(string, "UTF-8")
I faced this problem in android while passing a json with special char (ñ).
In my WebApi method, [FromBody] param is giving null, it seems it can't parse the json.
I got it working by getting bytes as UTF-8 then writing it in DataOutputStream (Client-side fix).
byte[] b = jsonString.getBytes("UTF-8");
os.write(b, 0, b.length);