Everything seems to work as long as the attachment is small.
However, as I try to attach a larger file (7MB for example), the execute() method of Send just hangs.
I tried to go over the documentation and if I understand correctly I should use a send API which actually performs upload however, I didn't figure where I should provide these parameters.
Here is the email generation method :
public MimeMessage toMimeMessage(String from, Context context) throws MessagingException {
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage mimeMessage = new MimeMessage(session);
mimeMessage.setFrom(new InternetAddress(from));
mimeMessage.addRecipient(javax.mail.Message.RecipientType.TO, new InternetAddress(recipient));
mimeMessage.setSubject(subject);
MimeBodyPart mimeBodyText = new MimeBodyPart();
mimeBodyText.setContent(body, "text/html");
mimeBodyText.setHeader("Content-Type", "text/plain; charset=\"UTF-8\"");
Multipart mp = new MimeMultipart();
mp.addBodyPart(mimeBodyText);
if (attachments != null && attachments.size() > 0) {
MimeBodyPart mimeBodyAttachments = new MimeBodyPart();
for (Uri uri : attachments) {
String fileName = UriUtils.getFileName(uri, context);
String mimeType = UriUtils.getMimeType(uri, context);
Log.d(TAG, "Generating file info, uri=" + uri.getPath() + ", mimeType=" + mimeType);
FileInputStream is = UriUtils.generateFileInfo(context, uri, mimeType);
if (is == null) {
throw new MessagingException("Failed to get file for uri=" + uri.getPath());
}
try
{
DataSource source = new ByteArrayDataSource(is, mimeType);
mimeBodyAttachments.setDataHandler(new DataHandler(source));
mimeBodyAttachments.setFileName(fileName);
mimeBodyAttachments.setHeader("Content-Type", mimeType + "; name=\"" + fileName + "\"");
} catch (IOException e) {
throw new MessagingException(e.getMessage());
}
}
mimeBodyAttachments.setHeader("Content-Transfer-Encoding", "base64");
mimeBodyAttachments.setDisposition(MimeBodyPart.ATTACHMENT);
mp.addBodyPart(mimeBodyAttachments);
}
mimeMessage.setContent(mp);
return mimeMessage;
}
.
Message createMessageWithEmail(MimeMessage mimeMessage) throws MessagingException, IOException {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
mimeMessage.writeTo(bytes);
String encodedEmail = Base64.encodeBase64URLSafeString(bytes.toByteArray());
Message message = new Message();
message.setRaw(encodedEmail);
return message;
}
Followed by :
MimeMessage mimeMessage = email.toMimeMessage(userId, context);
Message message = createMessageWithEmail(mimeMessage);
Gmail.Users.Messages messages = service.users().messages();
Gmail.Users.Messages.Send send = messages.send(userId, message);
send.execute(); // method hangs when using large attachment
As you mentioned, I think you are hitting the upper limit of allowed message size. I don't think (someone please correct me if I'm wrong) that the Java Gmail API client has any built in support for messages that go over this size, so it is up to you to implement it.
Under the hood, the messages.send-method results in a regular http POST request:
POST https://www.googleapis.com/gmail/v1/users/{USER_ID}/messages/send
Content-Type: application/json
Authorization: Bearer {ACCESS_TOKEN}
{
"raw": "{MESSAGE_URL_SAFE_BASE64_ENCODED}"
}
This only works up to ~5 mb total size as you discovered. If you want to use the max limit of 35 mb you need to do the following:
POST https://www.googleapis.com/upload/gmail/v1/users/{USER_ID}/messages/send?uploadType=multipart
Content-Type: message/rfc822
Authorization: Bearer {ACCESS_TOKEN}
"{MESSAGE_IN_RFC822_FORMAT}"
Notice upload in the URL, the uploadType=multipart URL parameter, message/rfc822 as Content-Type and message non-encoded in the request body. This answer might give some inspiration.
So you probably need to go around (again, someone correct me if I'm wrong) the Java client and use some other library to make a regular http request yourself.
Related
I been using Gmail API in my app and it was working fine untill yesterday, now I'm getting
V/Error: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Request payload size exceeds the limit: 1048576 bytes.",
"reason" : "badRequest"
} ],
"message" : "Request payload size exceeds the limit: 1048576 bytes.",
"status" : "INVALID_ARGUMENT"
}
I'm using multipart to attach the file, Here is the snippet of the code
MimeMessage createEmailWithAttachment(String to,
String from,
String subject,
String bodyText,
File file)
throws MessagingException, IOException {
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage email = new MimeMessage(session);
email.setFrom(new InternetAddress(from));
email.addRecipient(javax.mail.Message.RecipientType.TO,
new InternetAddress(to));
email.setSubject(subject);
MimeBodyPart mimeBodyPart = new MimeBodyPart();
mimeBodyPart.setContent(bodyText, "text/plain");
Multipart multipart = new MimeMultipart();
multipart.addBodyPart(mimeBodyPart);
mimeBodyPart = new MimeBodyPart();
DataSource source = new FileDataSource(file);
mimeBodyPart.setDataHandler(new DataHandler(source));
mimeBodyPart.setFileName(file.getName());
multipart.addBodyPart(mimeBodyPart);
email.setContent(multipart);
return email;
}
it was working perfectly few days back and now I get the above error whenever I try to send file larger than 1 MB, what could be work around for this?
Thanks
You can make upload requests in any of the following ways. Specify the method you are using with the uploadType request parameter.
Simple upload: uploadType=media. For quick transfer of smaller files, for example, 5 MB or less.
Multipart upload: uploadType=multipart. For quick transfer of smaller files and metadata; transfers the file along with metadata that describes it, all in a single request.
Resumable upload: uploadType=resumable. For reliable transfer, especially important with larger files. With this method, you use a session initiating request, which optionally can include metadata. This is a good strategy to use for most applications, since it also works for smaller files at the cost of one additional HTTP request per upload.
Since you want to send data > 5 MB, use the multipart or resumable option.
public static MimeMessage createEmailWithAttachment(String to,
String from,
String subject,
String bodyText,
File file) throws MessagingException, IOException {
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage email = new MimeMessage(session);
email.setFrom(new InternetAddress(from));
email.addRecipient(javax.mail.Message.RecipientType.TO,
new InternetAddress(to));
email.setSubject(subject);
MimeBodyPart mimeBodyPart = new MimeBodyPart();
mimeBodyPart.setContent(bodyText, "text/plain");
Multipart multipart = new MimeMultipart();
multipart.addBodyPart(mimeBodyPart);
mimeBodyPart = new MimeBodyPart();
DataSource source = new FileDataSource(file);
mimeBodyPart.setDataHandler(new DataHandler(source));
mimeBodyPart.setFileName(file.getName());
multipart.addBodyPart(mimeBodyPart);
email.setContent(multipart);
return email;
}
Check the documentation for more information: https://developers.google.com/gmail/api/guides/uploads
I had this exact same issue. In my case, I was unable to create a Draft larger than 1 MB, despite the Gmail documentation stating that the limit is 35 MB.
After some testing, I found that the issue is a bug specific to the .NET client library for the Gmail API. My workaround was to write my own code to call the Gmail API, bypassing the .NET client library. I've posted my solution here: http://www.gmass.co/blog/call-the-gmail-api-in-net-without-using-client-library/
I use the following function in android but I get error "500 Java Heap Space" when calling uploader.upload().
If I called uploader.setDirectUploadEnabled(true) before uploader.upload() I get error "411 Length required" on upload() instead of error 500.
public static String uploadImage(Context context, String file, MediaHttpUploaderProgressListener progressListener)
throws Exception {
// The file is something like : /mnt/sdcard/wallpapers/image.jpg
File mediaFile = new File(file);
HttpTransport transport = new NetHttpTransport();
HttpRequestFactory factory = transport.createRequestFactory(null);
//obtain the upload url
GenericUrl getUploadUrl = new GenericUrl(Util.getBaseUrl(context)
+ "/myapp/blobservice?get-key");
HttpRequest request = factory.buildGetRequest(getUploadUrl);
HttpResponse response1 = request.execute();
String postUrl = convertStreamToString(response1.getContent()).trim();
// the returned post url when using the development server is something like : //http://192.168.1.4:8888/_ah/upload/agd5YWRnZXQychwLEhVfX0Jsb2JVcGxvYWRTZXNzaW9uX18Y2wUM
InputStreamContent mediaContent = new InputStreamContent("image/jpeg",
new BufferedInputStream(new FileInputStream(mediaFile)));
mediaContent.setLength(mediaFile.length());
MediaHttpUploader uploader = new MediaHttpUploader(mediaContent,transport, null);
uploader.setProgressListener(progressListener);
//the following line produces exception "500 Java heap space" on the local server
//on the remote server I get "500: Internal server error"
HttpResponse response = uploader.upload(new GenericUrl(postUrl));
if (!response.isSuccessStatusCode()) {
throw new Exception("Uploading image failed.");
} else {
String blobKey = convertStreamToString(response.getContent()).trim();
return blobKey;
}
}
It seems that MediaHttpUpload uses resumable protocol, which is not compatible with multipart/form-data file upload of GAE blobstore.
You should create a custom data upload servlet, that knows how to handle the resumable protocol.
I am use HttpClient and mime to put the image file from Android client to CouchDB.
But there are some error message like this
D/FormReviewer(4733): {"error":"bad_request","reason":"invalid UTF-8 JSON: <<45,45,103,75,66,70,69,104,121,102,121,106,72,66,101,80,\n
here is my code
final String ProfileBasicID = UUID.randomUUID().toString();
Data.postImage(IconFile, "http://spark.iriscouch.com/driver/"+ProfileBasicID,new Callback<String>())
public static void postImage(File image,String url, Callback<String> success ) throws IOException {
HttpClient httpclient = new DefaultHttpClient();
HttpPut method = new HttpPut(url);
try {
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
entity.addPart("type", new StringBody("photo"));
entity.addPart("form_file", new FileBody(image, "image/jpeg"));
method.setEntity(entity);
HttpResponse resp = httpclient.execute(method);
Log.d("httpPost", "Login form get: " + resp.getStatusLine());
StatusLine statusLine = resp.getStatusLine();
Log.d(tag, statusLine.toString());
if (entity != null) {
entity.consumeContent();
}
switch(resp.getStatusLine().getStatusCode()){
case HttpStatus.SC_CREATED:
success.call(EntityUtils.toString(resp.getEntity()));
break;
default:
throw new ClientProtocolException(statusLine.toString() +"\n"+ EntityUtils.toString(resp.getEntity()));
}
} catch (Exception ex) {
Log.d("FormReviewer", "Upload failed: " + ex.getMessage() +
" Stacktrace: " + ex.getStackTrace());
} finally {
// mDebugHandler.post(mFinishUpload);
httpclient.getConnectionManager().shutdown();
}
}
Please give me a hand,Thanks
RIGHT, forget what I posted here previously.
This is NOT as straightforward as we thought.
Some links I suggest you read:
CouchDB Document API
(Draft) Core API
Ok.
First decision is if you want "Standalone" or "inline attachments". Currently I don't know what the Pro's and Con's are, BUT based on your code, and what I did, we will go for "Standalone".
Firstly, you need the rev (revision) number of the document you want to attach your image to. As per the above link, do this by doing a Head request on that doc:
private String getParentRevision(String uuid, HttpClient httpClient) {
String rev = "";
try {
HttpHead head = new HttpHead("http://192.168.56.101/testforms/" + uuid + "/");
HttpResponse resp = httpClient.execute(head);
Header[] headers = resp.getAllHeaders();
getLog().debug("Dumping headers from head request");;
for (Header header : headers) {
getLog().debug(header.getName() + "=" + header.getValue());
if ("Etag".equals(header.getName())) {
StringBuilder arg = new StringBuilder(header.getValue());
if (arg.charAt(0) == '"') {
arg.delete(0, 1);
}
if (arg.charAt(arg.length()-1) == '"'){
arg.delete(arg.length()-1, arg.length());
}
rev = arg.toString();
break;
}
}
} catch (Exception ex) {
getLog().error("Failed to obtain DOC REV!", ex);
}
return rev;
}
I appologise for the hardcoding etc, I'm learning and experimenting here ;)
The "uuid" parameter is the UUID of the target document.
Note the removal of the wrapping '"' characters when we got the Etag (yes, the Etag header is the revision number).
THEN, when we got that, we can actually send the image:
String serveURL = "http://192.168.56.101/testforms/" + data.getString(PARENT_UUID) + "/" + imgUuid;
if (docRev != null && !docRev.trim().isEmpty()) {
//This is dumb...
serveURL += "?rev=" + docRev + "&_rev=" + docRev;
}
HttpPut post = new HttpPut(serveURL);
ByteArrayEntity entity = new ByteArrayEntity(imageData);
entity.setContentType(data.getString(MIME_TYPE));;
post.setEntity(entity);
HttpResponse formServResp = httpClient.execute(post);
With this, I was able to attache images to my docs ;)
As mentioned, please be aware that I'm also new to CouchDB, so there might be simpler ways to do this!
Something I just discovered now (but should have spotted earlier) is that there is the potential of a race condition here, if, for example, multiple clients are trying to attach images to the same document simultaneously. The reason is that the rev value changes with each change to the document.
In such a case, you will get a reply from the server like
{"error":"conflict","reason":"Document update conflict."}
Easiest solution is to just retry in such a case, until it works, or until a self imposed error limit is hit...
Cheers!
I have a rest webservice that takes a POST metod with multipart message:
#Path("transferFile")
#POST
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Produces(MediaType.APPLICATION_XML)
public String multipartTest(com.sun.jersey.multipart.MultiPart data) {
try {
// get first body part (index 0)
BodyPart bp = multiPart.getBodyParts().get(0);
etc..
Now I am trying to write a java client for that. I started with a simple jersey client:
view plaincopy to clipboardprint?
MultiPart multiPart = new MultiPart();
multiPart.bodyPart( new BodyPart(wavestream,MediaType.APPLICATION_OCTET_STREAM_TYPE));
Client c = Client.create();
WebResource r = c.resource("http://127.0.0.1:8080/webapp:);
response=r.path("transferFile").type(MediaType.MULTIPART_FORM_DATA).accept(MediaType.APPLICATION_XML).post(String.class, multiPart);
This works great - everything is ok. However I need this client working on Android and I have trouble with using jersey on that platform. So I used the normal way to send multipart message on android:
HttpClient client = new DefaultHttpClient();
client.getParams().setParameter("http.socket.timeout", new Integer(90000)); // 90 second
HttpPost httpPost = new HttpPost("http://127.0.0.1:8080/webapp/transferFile");
httpPost.setHeader("Content-Type", MediaType.MULTIPART_FORM_DATA );
//tried with and without base64
byte [] encodedWavestream = Base64.encodeBytesToBytes(wavestream);
InputStream ins = new ByteArrayInputStream(encodedWavestream);
InputStreamBody body = new InputStreamBody(ins, "test" );
int send = ins.available();
MultipartEntity requestContent = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE );
requestContent.addPart("stream", body);
httpPost.setEntity(requestContent);
HttpResponse Response = client.execute(httpPost);
An this gives an annoying response from the server :
HTTP Status 400 - Bad Request
The request sent by the client was syntactically incorrect (Bad Request).
I check the server log files but there is nothing there. So I don't know what's the origin of this error. I have wrote a simple html page with a post formula and 'multipart/form-data' content-type and it also works! An auto-generated request from soapUI also works! Why my client does not work? Can anybody help?
There is bug in Jersey. See Chunked encoding problem.
This problem appears only for few clients (iOS, Android).
If you set the Content-Type to application/octet-stream, then the Jersey MessageWriter for the application/octet-stream will set the Content-Length and
not send as chunked transport method.
There is solution for Jersey Client:
ClientConfig config = new DefaultClientConfig();
config.getProperties().put(ClientConfig.PROPERTY_CHUNKED_ENCODING_SIZE, 32 * 1024);
But it doesn't work for the iOS's or Android's client.
So I tested Apache File Upload. Threre was another bug: "Stream ended unexpectedly".
Only Oreilly upload can upload file correct for all clients.
This is my code:
public Object[] getParametersAndFiles(HttpServletRequest request) throws IOException {
log.debug("OreillyUpload");
Properties params = new Properties();
LinkedHashMap files = new LinkedHashMap();
File tempDirectory = new File(System.getProperty("java.io.tmpdir"));
MultipartParser mp = new MultipartParser(request, 1*1024*1024); // 10MB
Part part;
while ((part = mp.readNextPart()) != null) {
String name = part.getName();
if (part.isParam()) {
// it's a parameter part
ParamPart paramPart = (ParamPart) part;
String value = paramPart.getStringValue();
params.put(name, value);
log.debug("param; name=" + name + ", value=" + value);
}
else if (part.isFile()) {
// it's a file part
FilePart filePart = (FilePart) part;
String fileName = filePart.getFileName();
if (fileName != null) {
// the part actually contained a file
File file = new File(tempDirectory,fileName);
long size = filePart.writeTo(file);
files.put(name, file);
log.debug("file; name=" + name + "; filename=" + fileName +
", filePath=" + filePart.getFilePath() +
", content type=" + filePart.getContentType() +
", size=" + size);
}
else {
// the field did not contain a file
log.debug("file; name=" + name + "; EMPTY");
}
}
}
return new Object[] {params, files};
}
And this is Jersey Server code (warning all Jersey Upload anotations (like as "#FormDataParam") should be removed):
#POST
#Path("uploadMarkup")
#Produces(MediaType.APPLICATION_JSON)
// #Consumes(MediaType.MULTIPART_FORM_DATA)
//// public void uploadMarkup(
// public JSONWithPadding uploadMarkup(
// #FormDataParam("markupFile") InputStream markupFile,
// #FormDataParam("markupFile") FormDataContentDisposition details,
// #FormDataParam("slideNum") int slideNum) {
public JSONWithPadding uploadMarkup(#Context HttpServletRequest request) {
Object[] data = uploadService.getParametersAndFiles(request);
...
}
I'm sending an image from an Android phone to an AppEngine. The AppEngine is throwing content exceptions. My process of elimination has failed so I suspect I've done something larger wrong. Do I have a content type problem or a larger problem?
The AppEngine log shows the following errors depending on the content type [content type --> resulting error]:
multipart/mixed stream --> org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found the request was rejected because no multipart boundary was found
image/jpg --> org.apache.commons.fileupload.FileUploadBase$InvalidContentTypeException: the request doesn't contain a multipart/form-data or multipart/mixed stream, content type header is image/jpg
binary/octet-stream --> org.apache.commons.fileupload.FileUploadBase$InvalidContentTypeException: the request doesn't contain a multipart/form-data or multipart/mixed stream, content type header is binary/octet-stream
ANDROID CODE:
static void apachePost(File file) {
File file = new File(directory, fileName);
try {
String url = "http://chexserve.appspot.com/chexappengine";
HttpClient httpclient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
FileEntity entity = new FileEntity(file, "multipart/mixed stream");
httpPost.setEntity(entity);
Log.v(Constants.CHEX,
"executing request " + httpPost.getRequestLine());
HttpResponse response = httpclient.execute(httpPost);
Log.v(Constants.CHEX, "received http response " + response);
HttpEntity resEntity = response.getEntity();
httpclient.getConnectionManager().shutdown();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
APPENGINE CODE:
public class ChexAppEngineServlet extends HttpServlet {
private static final Logger LOG = Logger
.getLogger(ChexAppEngineServlet.class.getName());
/** Processes the incoming request */
#Override
protected void doPost(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
LOG.info("LOG: received post");
System.out.println("PRINTLN: received post " + request);
try {
ServletFileUpload upload = new ServletFileUpload();
FileItemIterator iter;
iter = upload.getItemIterator(request);
if (iter.hasNext()) {
FileItemStream imageItem = iter.next();
InputStream imgStream = imageItem.openStream();
System.out.println("PRINTLN: created imgStream " + imgStream);
// construct our entity objects
Blob imageBlob = new Blob(IOUtils.toByteArray(imgStream));
MyImage myImage = new MyImage(imageItem.getName(), imageBlob);
System.out.println("PRINTLN: created my image " + myImage);
} else {
System.out.println("PRINTLN: no file in post");
LOG.info("LOG: no file in post");
}
} catch (FileUploadException e) {
LOG.info("failed to parse post request: " + e);
e.printStackTrace();
}
}
Update:
AppEngine is still rejecting the request with "the request was rejected because no multipart boundary was found" error after I changed the Android's entity line to:
FileEntity entity = new FileEntity(file, "multipart/mixed");
According to the documentation [http://hc.apache.org/httpclient-3.x/preference-api.html],"The multipart boundary string to use in conjunction with the MultipartRequestEntity. When not set a random value will be generated for each request."
Further, there are no setter methods for the boundary.
How can I set the boundary?
The content type is "multipart/mixed", not "multipart/mixed stream". Also valid should be "multipart/form-data". If you're just uploading the image, though, you can upload it as image/jpeg, and read the raw data from the request on the server, by calling (If I recall correctly), request.getInputStream() - no need to use the Apache forms library at all.