Develop Reference

Send file via HTTPS POST to remote

I need to send a zipped database (payload ~10 MB) to a PHP web service using SSL.
The code worked fine for weeks, but suddently I get a ssl broken pipe exception (SSL Broken Pipe). I don't see any reason for the error - I did not changed the code and the server config is still the same (the IIS was able to handle the data-transfer by ssl before).
Currently I use the java.net.HttpsURLConnection (code below) - which alternative would you recommend me to use when sending a file from Android to a web service? It seems that the POST is not processed on the device.
Thanks in advance
try {
final File compressedDb = new File(Environment.getExternalStorageDirectory() + "/" + Const.TMP_DIR + "/database.zip");
String sourceFileUri = compressedDb.getPath();
final long compressedDbSize = compressedDb.length();
float optimized = ((float) _dbSize / (float) compressedDbSize) * 100f;
int dbKb = (int) (((float) _dbSize) / 1024f);
int dbCompKb = (int) ((float) compressedDbSize / 1024f);
compressionInfo = getString(R.string.compression) + ": " +
dbKb + "kB " + getString(R.string.to) + " " + dbCompKb + "kB" + "<br>" +
getString(R.string.saved) + ": " + (int) optimized + "%" + "<br>";
Log.i(TAG, "DB name=" + compressedDb.getName() + " size=" + compressedDb.length() / 1024 + "kB.");
HttpsURLConnection connection;
DataOutputStream dos;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
File sourceFile = new File(sourceFileUri);
// int maxBufferSize = 1024 * 1024;
int maxBufferSize = (int) sourceFile.length();
if (sourceFile.isFile()) {
try {
// open a URL connection to the Servlet
FileInputStream fileInputStream = new FileInputStream(sourceFile);
url = new URL(params[0]);
connection = (HttpsURLConnection) url.openConnection();
connection.setDoInput(true); // Allow Inputs
connection.setUseCaches(false); // Don't use a Cached Copy
connection.setRequestMethod("POST");
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestProperty("ENCTYPE", "multipart/form-data");
connection.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
connection.setRequestProperty("the_database", sourceFileUri);
dos = new DataOutputStream(connection.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"the_database\";" +
"filename=\"" + sourceFileUri + "\"" + lineEnd);
dos.writeBytes(lineEnd);
// create a buffer of maximum size
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
int bytesTotal = bytesAvailable;
Log.i(TAG, "maxBuffer=" + maxBufferSize +
" | bytesTotal=" + bytesTotal + " | bytesAvailable=" + bytesAvailable);
while (bytesRead > 0) {
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
int progress = 100 - (int) (((float) bytesAvailable * 100 / (float) bytesTotal));
publishProgress("" + progress);
}
// send multipart form data necessary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
int serverResponseCode = connection.getResponseCode();
final String responseMessagePostDb = connection.getResponseMessage();
fileInputStream.close();
dos.flush();
dos.close();
if (responseMessagePostDb != null) {
responseStrings.add("<br>");
responseStrings.add("<b><u>" + getString(R.string.optimisation_string) + "</u></b>" + "<br>");
responseStrings.add(params[1]);
responseStrings.add("<b><u>" + getString(R.string.optimisation_database) + "</u></b>" + "<br>");
responseStrings.add(compressionInfo);
String response = (responseMessagePostDb.equals("OK") ? ("<font color=\"" + GREEN + "\">PASSED</font>") : "FAILED");
responseStrings.add(getString(R.string.server_response) + ": " + response);
return responseStrings;
}
} catch (Exception e) {
Log.e(TAG, "doInBackground: " + e.getMessage());
responseStrings.add("FAILED2"); // <- this exception is thrown
responseStrings.add(e.getMessage());
return responseStrings;
}
}
} catch (Exception ex) {
Log.e(TAG, "doInBackground: " + ex.getMessage());
responseStrings.add("FAILED3");
responseStrings.add(ex.getMessage());
return responseStrings;
}
Log.e(TAG, "doInBackground: " + "FAILED4");
responseStrings.add("FAILED4");
responseStrings.add("Unexpected Error...");
return responseStrings;

If your code worked up to this point, then logically, unless you have a race condition in your code, you're not going to see inconsistent results unless it isn't the code. In other words, if the server decided to enforce a connection timeout, your code would begin to fail seemingly without reason. Even the link to the problem involving a SSL Broken Pipe exception seems to imply that the reason is the server, not the client.
My advice would be to try to come up with various reasons for the failure which involve the server doing something it shouldn't and then testing each theory by altering your code. In other words, if the problem were a matter of timeout, time the operation and send a smaller file over the network and verify if that works. If it does, you have your problem. If it doesn't, then move on to the next theory.
Unfortunately not being a problem with the mentioned code, that makes this problem unsolveable without knowing further details.

android client parse multipart form data to non ascii

can you explain why android client parse multiparform data to non ascii chat. while file upload working good using postman
here is my app.js code
var multipart = require('connect-multiparty');
var apiRoutes = require('./routes/apiRoutes');
app.set('views', path.join(__dirname, 'views'));
app.use(logger('dev'));
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded([{extended: false},{ uploadDir:path.join(__dirname, 'uploads') }, {parameterLimit:100000}, {limit: '50mb'}]));
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));
app.use(express.static(path.join(__dirname, 'uploads')));
app.use(multipart());
app.use(apiRoutes);
and apiRoutes point my upload function contain simple print req param.using postman it working good
console.log("mediaChat called", req.body, req.files);
response
mediaChat called { apiKey: '123' } { media:
{ fieldName: 'media',
originalFilename: 'default.png',
path: '/tmp/KFnwsKGp-f4woTaBH6aPR-qa.png',
headers:
{ 'content-disposition': 'form-data; name="media"; filename="default.png"',
'content-type': 'image/png' },
size: 716,
name: 'default.png',
type: 'image/png' } }
here is my android client code (Note this code working file with php $_FILE but not working with express)
com.info.acruss.wave;
import android.os.AsyncTask;
import android.util.Log;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
/**
* Created by Naitik on 9/20/2016.
*/
public class UploadImage extends AsyncTask<Void, Void, String> {
String CallingURL;
URL url;
OnTaskCompleted myListener;
private static final String TAG = "UploadImage";
int timeoutMilli = 60000;
String sourceFileUri;
String ApiKey,Type;
public UploadImage(String sourceFileUri, String URL,String apiKey, String type,
OnTaskCompleted listener) {
Log.e("Uploading", "API:" + URL);
this.sourceFileUri = sourceFileUri;
this.CallingURL = URL;
this.myListener = listener;
this.ApiKey=apiKey;
this.Type=type;
try {
url = new URL(CallingURL);
Log.e(TAG, "Url : " + CallingURL);
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
#Override
protected String doInBackground(Void... params) {
String fileName = sourceFileUri;
HttpURLConnection conn = null;
DataOutputStream dos = null;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 10 * 1024 * 1024;
File sourceFile = new File(sourceFileUri);
if (!sourceFile.isFile()) {
Log.e("UploadImage", "Source File Does not exist";
return null;
}
String serverResponseMessage = "";
try {
// open a URL connection to the Servlet
FileInputStream fileInputStream = new FileInputStream(sourceFile);
URL url = new URL(CallingURL);
// Open a HTTP connection to the URL
conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true); // Allow Inputs
conn.setDoOutput(true); // Allow Outputs
conn.setUseCaches(false); // Don't use a Cached Copy
conn.setRequestMethod("POST";
conn.setReadTimeout(timeoutMilli);
conn.setConnectTimeout(timeoutMilli);
conn.setRequestProperty("Connection", "Keep-Alive";
conn.setRequestProperty("ENCTYPE", "multipart/form-data";
conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
conn.setRequestProperty("media", fileName);
conn.setRequestProperty("apiKey",ApiKey);
conn.setRequestProperty("media_type",Type);
conn.setRequestProperty("media", fileName);
dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name='media';filename='" + fileName + "'" + lineEnd);
dos.writeBytes(lineEnd);
// create a buffer of maximum size
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
int serverResponseCode = conn.getResponseCode();
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line = "";
StringBuilder responseOutput = new StringBuilder();
while ((line = br.readLine()) != null) {
responseOutput.append(line);
}
br.close();
serverResponseMessage = responseOutput.toString();//output.toString();
Log.e("uploadFile", "HTTP Response is : " + serverResponseMessage);
if (serverResponseCode == 200) {
//status code 200
//status ok
}
//close the streams //
fileInputStream.close();
dos.flush();
dos.close();
} catch (MalformedURLException ex) {
ex.printStackTrace();
Log.e("Upload file to server", "error: " + ex.getMessage(), ex);
} catch (Exception e) {
Log.e("Upload file to server", "error: " + e.getMessage(), e);
e.printStackTrace();
}
return serverResponseMessage;
}
#Override
protected void onPostExecute(String result) {
super.onPostExecute(result);
Log.e("Result", "" + result);
if (myListener != null)
if (result != null) {
myListener.onFileUploadComplete(result);
} else {
myListener.onFileUploadComplete("";
}
}
public interface OnTaskCompleted {
void onFileUploadComplete(String result);
}
}
using android this show wired response as below
mediaChat called { null: '����\u0000\u0010JFIF\u0000\u0001\u0001\u0000\u0000\u0001\u0000\u0001\u0000\u0000��\u0000C\u0000\u0010\u000b\f\u000e\f\n\u0010\u000e\r\u000e\u0012\u0011\u0010\u0013\u0018(\u001a\u0018\u0016\u0016\u00181#%\u001d(:3=Mqypdx\\egc��\u0000C\u0001\u0011\u0012\u0012\u0018\u0015\u0018/\u001a\u001a/cB8Bccccccccccc....
�\u001f.[���_�\u0014)M���XIjX��7�`=�/�8`��ïDʚ\u0018�D���#�V#q~m�q10L�' }
i also tried multer and other multipart handler but noting works.
please help me to out from this hell

It seems the server response is encoded as UTF-8. To properly decode and read you could try
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
Specifying the encoding part to the InputStreamReader will decode and read the stream using specified encoding scheme, which in this case is UTF-8. Check the javadocs for details.

You need to allocate buffer again based on the new size of bufferSize, I have revised the code below:
// create a buffer of maximum size
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = null;
buffer = new byte[bufferSize];
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}

Multipart form submission takes away a lot of the ambiguity that percent-encoding had: the server now can explicitly ask for certain encodings, and the client can explicitly tell the server during the form submission what encoding the fields are in.
There are two ways you go with this functionality: leave it unset and have the browser send in the same encoding as the page, or set it to UTF-8 and then do another conversion server-side. Each method has deficiencies, especially the former.
If you tell the browser to send the form in the same encoding as the page, you still have the trouble of what to do with characters that are outside of the character encoding's range. The behavior, once again, varies: Firefox 2.0 converts them to character entity references while Internet Explorer 7.0 mangles them beyond intelligibility. For serious internationalization purposes, this is not an option.
The other possibility is to set Accept-Encoding to UTF-8, which begs the question: Why aren't you using UTF-8 for everything then? This route is more palatable, but there's a notable caveat: your data will come in as UTF-8, so you will have to explicitly convert it into your favored local character encoding.
A Unicode-based encoding such as UTF-8 can support many languages and can accommodate pages and forms in any mixture of those languages. Its use also eliminates the need for server-side logic to individually determine the character encoding for each page served or each incoming form submission. This significantly reduces the complexity of dealing with a multilingual site or application.
A Unicode encoding also allows many more languages to be mixed on a single page than any other choice of encoding.
have look on Why UTF-X

URLConnection always returns 400 : Bad Request when I try to upload a .wav file

I want to implement some code that does exactly what I accomplish when I execute a CURL command in my Mac's terminal, which is to upload a .wav file to a Server.
I already accomplished this for one server before, so my general thinking works (and my code too),
....but now I want to accomplish the same but on a different server.
This time I am having trouble and all I get are "400:Bad Request"
This is he CURL command I want to do on Android, I have added a "-v" so you can see the verbose of my command.
curl -F file=#audio_life.wav 11.2.333.44:14141/transcribe -v
Output log in Terminal:
* Trying 11.2.333.44...
* Connected to 11.2.333.44 (11.2.333.44) port 14141 (#0)
> POST /transcribe HTTP/1.1
> Host: 11.2.333.44:14141
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 304898
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------62142b52672c69e0
>
* Done waiting for 100-continue
< HTTP/1.1 201 Created
< Content-Type: application/json
< Content-Length: 915
This is my code, I always get 400:Bad Request responses, as you can see I have tried to replicate the headers. What am I doing wrong? how can I close in on what the error may be?
String wavpath=Environment.getExternalStorageDirectory().getAbsolutePath()+"/ArtificialSolutions/"+StorageUtils.AUDIO_FILE_NAME+".wav";
File wavfile = new File(wavpath);
FileInputStream fileInputStream = new FileInputStream(wavpath);
connectURL = new URL(ASR_URL_VOCITEC);
String filevalue = StorageUtils.AUDIO_FILE_NAME+".wav";
HttpURLConnection conn = (HttpURLConnection)connectURL.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.setRequestProperty("Host","11.2.333.44:14141");
conn.setRequestProperty("User-Agent", "android");
conn.setRequestProperty("Accept","*/*");
conn.setRequestProperty("Transfer-Encoding", "chunked");
conn.setRequestProperty("Expect", "100-continue");
conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=------------------------f016e997e308ec07");//+boundary);
DataOutputStream dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"File\""+ lineEnd);
//dos.writeBytes("Content-Disposition: form-data; File=\""+wavpath+"\""+ lineEnd);
dos.writeBytes(lineEnd);
dos.writeBytes(wavpath);
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + lineEnd);
int bytesAvailable = fileInputStream.available();
int maxBufferSize = 1024;
int bufferSize = Math.min(bytesAvailable, maxBufferSize);
byte[ ] buffer = new byte[bufferSize];
int bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0){
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
Log.e("joshtag","BytesAvailable: "+bytesAvailable);
bufferSize = Math.min(bytesAvailable,maxBufferSize);
if(bytesAvailable>maxBufferSize){
bytesRead = fileInputStream.read(buffer, 0,bufferSize);
}
else{
bytesRead = fileInputStream.read(buffer, 0,bytesAvailable);
dos.write(buffer, 0, bytesAvailable);
break;
}
}
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
fileInputStream.close();//63532 , 64556
dos.flush();
Log.e(Tag,"File Sent, Response: "+String.valueOf(conn.getResponseCode()));
Log.e(Tag,"File Sent, ResponseMSG: "+String.valueOf(conn.getResponseMessage()));
This is a printout of the message I get back from the server:
{null=[HTTP/1.1 400 Bad Request], Content-Length=[42], Content-Type=[text/plain], X-Android-Received-Millis=[1449836192887], X-Android-Response-Source=[NETWORK 400], X-Android-Sent-Millis=[1449836192719]}
NOTE: I coded this based on the best of my knowledge and after examining related posts here on StartOverflow, the whole code is inside the doInBackground method of an AsyncTask so lets focus here. I hope can help me!! Thanks.

After much fiddling with the code, I found a way to upload a wav file with URLConnection, and using multipartentity. Here is the code, enjoy:
public int uploadWav(String sourceFileUri) {
String wavpath=Environment.getExternalStorageDirectory().getAbsolutePath()+"/myAppFolder/"+StorageUtils.AUDIO_FILE_NAME+".wav";
String filename=wavpath;
HttpURLConnection conn = null;
DataOutputStream dos = null;
String lineEnd = "\r\n";
String twoHyphens = "--";
String boundary = "------------------------afb19f4aeefb356c";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1 * 1024 * 1024;
File sourceFile = new File(fileName);
Log.e("joshtag", "Uploading: sourcefileURI, "+fileName);
if (!sourceFile.isFile()) {
Log.e("uploadFile", "Source File not exist :"+wavpath);//FullPath);
return 0; //RETURN #1
}
else{
try{
Log.v("joshtag","UPLOADING .WAV FILE");
FileInputStream fileInputStream = new FileInputStream(sourceFile);
URL url = new URL(SERVER_URL);
Log.v("joshtag","UL URL: "+url.toString());
// Open a HTTP connection to the URL
conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true); // Allow Inputs
conn.setDoOutput(true); // Allow Outputs
conn.setUseCaches(false); // Don't use a Cached Copy s
conn.setRequestMethod("POST");
// conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("ENCTYPE", "multipart/form-data");
conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
conn.setRequestProperty("file", sourceFile.getName());
//so on and so forth...
//conn.setRequestProperty("param", "value");
conn.setRequestProperty("connection", "close");
dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"file\";filename=\"" + sourceFile.getName() + "\"" + lineEnd);
dos.writeBytes(lineEnd);
// create a buffer of maximum size
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
Log.i("joshtag","->");
}
Log.i("joshtag","->->");
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
conn.connect();
Log.i("joshtag","->->->");
// Responses from the server (code and message)
serverResponseCode = conn.getResponseCode();
Log.i("joshtag","->->->->");
String serverResponseMessage = conn.getResponseMessage().toString();
Log.i("joshtag","->->->->->");
Log.i("joshtag", "HTTP Response is : " + serverResponseMessage + ": " + serverResponseCode);
// ------------------ read the SERVER RESPONSE
DataInputStream inStream;
String str="";
String response="";
try {
Log.i("joshtag","->->->->->->");
inStream = new DataInputStream(conn.getInputStream());
while ((str = inStream.readLine()) != null) {
Log.e("joshtag", "SOF Server Response" + str);
response=str;
}
inStream.close();
}
catch (IOException ioex) {
Log.e("joshtag", "SOF error: " + ioex.getMessage(), ioex);
}
conn.disconnect();
conn=null;
//close the streams //
fileInputStream.close();
dos.flush();
dos.close();
if(serverResponseCode == 201){
loge("*** SERVER RESPONSE: 201"+response);
}//END IF Response code 201
// conn.disconnect();
}//END TRY - FILE READ
catch (MalformedURLException ex) {
ex.printStackTrace();
Log.e("joshtag", "UL error: " + ex.getMessage(), ex);
} //CATCH - URL Exception
catch (Exception e) {
e.printStackTrace();
Log.e("Upload file to server Exception", "Exception : "+ e.getMessage(), e);
}
return serverResponseCode; //after try
}//END ELSE, if file exists.
}

showing a progress bar during file upload

I've got an async task that is supposed to show progress during a file upload. Everything is working except that it looks like it finishes the file upload really really fast, and then it just sits there at 100% waiting.
I traced this down to
URL url = new URL(urlServer);
connection = (HttpURLConnection) url.openConnection();
// Allow Inputs & Outputs
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
// Enable POST method
connection.setRequestMethod("POST");
connection.setRequestProperty("Connection", "Keep-Alive");
connection.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
outputStream = new DataOutputStream(connection.getOutputStream());
outputStream.writeBytes(twoHyphens + boundary + lineEnd);
outputStream.writeBytes("Content-Disposition: form-data; name=\"Filedata\";filename=\"" + pathToOurFile + "\"" + lineEnd);
outputStream.writeBytes(lineEnd);
long totalBytesWritten = 0;
while (bytesRead > 0) {
outputStream.write(buffer, 0, bufferSize);
outputStream.flush();
if (mCancel) { throw new CancelException(); }
totalBytesWritten += bufferSize;
if (mProgressDialog != null) {
mProgressDialog.setProgress(Integer.valueOf((int) (totalBytesWritten / 1024L)));
}
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
outputStream.writeBytes(lineEnd);
outputStream.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
int serverResponseCode = connection.getResponseCode();
What I noticed is that there's no real delay until the very last line where it's getting the response code. I think what's happening is that the data is getting buffered so it looks like it has uploaded it, but hasn't really - it has just buffered it. Then when i get to the getResponseCode() call, it has no choice but to finish the upload to get the upload status. is there any way to make it actually upload along the way so i can get a reasonable progress?

This is how HTTP Post is designed to work so do not expect it to give you progress details.
You can use one of the several file uploader components available in the market. They internally use flash or silverlight or iframes to show the progress.
http://dhtmlx.com/docs/products/dhtmlxVault/index.shtml
http://www.element-it.com/multiple-file-upload/flash-uploader.aspx
You will find many such others if you google a bit.
They internally use raw IO instead of http post to handle multiple files and progress notification. Yahoo and Google also uses such techniques for making attachments to mail.
If you are really feeling adventurous, you can recreate the wheel - i.e. write your own component.
Edit:
Please specify if you want to do this in a windows desktop application or a web application.

You can use Progress Dialog class as follows,
ProgressDialog progDailog = ProgressDialog.show(this,"Uploading", "Uploading File....",true,true);
new Thread ( new Runnable()
{
public void run()
{
// your loading code goes here
}
}).start();
Handler progressHandler = new Handler()
{
public void handleMessage(Message msg1)
{
progDailog.dismiss();
}
}

You can try using a AsyncTask ...
create a progress dialog in onPreExecute method and dismiss the dialog in onPostExecute method..
Keep the upload method in doInBackground()
example :
public class ProgressTask extends AsyncTask<String, Void, Boolean> {
public ProgressTask(ListActivity activity) {
this.activity = activity;
dialog = new ProgressDialog(context);
}
/** progress dialog to show user that the backup is processing. */
private ProgressDialog dialog;
protected void onPreExecute() {
this.dialog.setMessage("Progress start");
this.dialog.show();
}
#Override
protected void onPostExecute(final Boolean success) {
if (dialog.isShowing()) {
dialog.dismiss();
}
}
protected Boolean doInBackground(final String... args) {
// your upload code
return true;
}
}
}

HttpURLConnection.setFixedLengthStreamingMode(...) did the trick!

You can do like:
try { // open a URL connection to the Servlet
FileInputStream fileInputStream = new FileInputStream(
sourceFile);
URL url = new URL("http://10.0.2.2:9090/plugins/myplugin/upload");
conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true); // Allow Inputs
conn.setDoOutput(true); // Allow Outputs
conn.setUseCaches(false); // Don't use a Cached Copy
conn.setRequestMethod("POST");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("ENCTYPE", "multipart/form-data");
conn.setRequestProperty("Content-Type",
"multipart/form-data;boundary=" + boundary);
conn.setRequestProperty("uploadedfile", filename);
// conn.setFixedLengthStreamingMode(1024);
// conn.setChunkedStreamingMode(1);
dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(twoHyphens + boundary + lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\""
+ filename + "\"" + lineEnd);
dos.writeBytes(lineEnd);
bytesAvailable = fileInputStream.available();
bufferSize = (int) sourceFile.length()/200;//suppose you want to write file in 200 chunks
buffer = new byte[bufferSize];
int sentBytes=0;
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
dos.write(buffer, 0, bufferSize);
// Update progress dialog
sentBytes += bufferSize;
publishProgress((int)(sentBytes * 100 / bytesAvailable));
bytesAvailable = fileInputStream.available();
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
// Responses from the server (code and message)
serverResponseCode = conn.getResponseCode();
String serverResponseMessage = conn.getResponseMessage();
// close streams
fileInputStream.close();
dos.flush();
dos.close();
} catch (MalformedURLException ex) {
ex.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}

Android-Django Image upload

I am trying to upload image with android as frontend and django as backend.
The model:
class Photo(models.Model):
title = models.CharField(max_length=255,blank=True)
photo = models.FileField(upload_to='photos')
description = models.TextField(blank=True)
uploaded = models.DateTimeField(auto_now_add=True)
modified = models.DateTimeField(auto_now=True)
class Meta:
db_table = 'media_photos'
def __unicode__(self):
return '%s' % self.title
The view to url url(r'^photos/upload/$','upload_photo'):
def upload_photo(request):
form=PhotoForm(request.POST,request.FILES)
if request.method=='POST':
if form.is_valid():
image = request.FILES['photo']
title1 =''
new_image = Photo(title=title1,photo=image,description='')
new_image.save()
response_data=[{"success": "1"}]
return HttpResponse(simplejson.dumps(response_data), mimetype='application/json')
now i am trying to access the view here from android.
So now my android side code to upload image is:
public void doFileUpload(String path){
HttpURLConnection conn = null;
DataOutputStream dos = null;
DataInputStream inStream = null;
String lineEnd = "\r\n";
int bytesRead, bytesAvailable, bufferSize;
byte[] buffer;
int maxBufferSize = 1*1024*1024;
String urlString = "http://"; // server ip
try
{
//------------------ CLIENT REQUEST
FileInputStream fileInputStream = new FileInputStream(new File(path) );
// open a URL connection to the Servlet
URL url = new URL(urlString);
// Open a HTTP connection to the URL
conn = (HttpURLConnection) url.openConnection();
// Allow Inputs
conn.setDoInput(true);
// Allow Outputs
conn.setDoOutput(true);
// Don't use a cached copy.
conn.setUseCaches(false);
// Use a post method.
conn.setRequestMethod("POST");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.setRequestProperty("Content-Type", "multipart/form-data;boundary="+" ");
dos = new DataOutputStream( conn.getOutputStream() );
dos.writeBytes(lineEnd);
dos.writeBytes("Content-Disposition: form-data; name=\"uploadedfile\";filename=\"" + path + "\"" + lineEnd);
dos.writeBytes(lineEnd);
// create a buffer of maximum size
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
buffer = new byte[bufferSize];
// read file and write it into form...
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0)
{
dos.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
// send multipart form data necesssary after file data...
dos.writeBytes(lineEnd);
dos.writeBytes(lineEnd);
// close streams
Log.e("Debug","File is written");
fileInputStream.close();
dos.flush();
dos.close();
}
catch (MalformedURLException ex)
{
Log.e("Debug", "error: " + ex.getMessage(), ex);
}
catch (IOException ioe)
{
Log.e("Debug", "error: " + ioe.getMessage(), ioe);
}
//------------------ read the SERVER RESPONSE
try {
inStream = new DataInputStream ( conn.getInputStream() );
String str;
while (( str = inStream.readLine()) != null)
{
Log.e("Debug","Server Response "+str);
}
inStream.close();
}
catch (IOException ioex){
Log.e("Debug", "error: " + ioex.getMessage(), ioex);
}
}
}
but it gives me an error:
E/Debug(590): error: java.net.URISyntaxException: Authority expected at index 7: http://

Should be urlString = "http://192.168.1.2/photos/upload";
But then if it doesn't work, you'll get a different error, and we'll probably need that error to answer further.
Also, it looks like you don't have a real boundary string set and you aren't using it correctly.
Have a look here. Notice how he uses a unique boundary string and writes it to the output stream?

Even after you overcome these difficulties, you'll run afoul of Django's requirement that you supply a csrfmiddlewaretoken with your POST parameters. And I don't see how you can obtain that on an Android device; by design that token is there to prevent calling Django backend code (i.e. a "view") from anything other than a Django frontend (i.e. a "template"). I.e. it's designed to thwart doing exactly what you're trying to do.
You could disable the csrf feature on a particular view -- use the "#csrf_exempt" decorator. And then you can decide if you care enough about security, to work out a substitute of your own for what the csrf thing gives you.
Or, instead of uploading a picture from an Android app, write a web app to upload a picture, and have your Django project serve up that web app. Your Android app could launch the browser (as an Intent) and point it at that web app. Here's some code that'll do that: https://simpleisbetterthancomplex.com/tutorial/2016/08/01/how-to-upload-files-with-django.html (It won't win any beauty contests, but it does work.)