Develop Reference

Storing additional data to mail in IMAP

Introduction
I have a xy-problem
x: High level goal
I have dream: I want to store additional data to mails which are in IMAP. I the long run I want to be able to access this data via thunderbird and k9 (android app (mail use agent)).
Use cases:
I want to store a note in html format for this mail.
Upload a PDF file for this mail.
I want to store a re-submission date on the mail. Like google inbox snooze feature: Move the mail to a "do later" folder until the date is reached. (Evaluating this date and moving the mail to inbox again is not part of this question)
But I have no clue how to store additional data
y: My current thoughts about solving it
I need a way to identify a message in IMAP. I think the message id (without folder name) should work. I know that message IDs can have duplicates, but I see no other way. Please leave a comment, if you have a better idea.
Now I need a way to store this mapping somewhere:
`user#imapserver:message-id` --> `additional-data`
Question
How to store this mapping, so that thunderbird and k9 could access it?
Of course I know that thunderbird and k9 can't access this data today. How to patch them is a different question.
Background
I like free software and I like free communication. Up to now I use WhatsApp, Threema and other tools. But in the long run I want a free (like in software) solution. Email is wide spread, and I think it makes more sense to improve email than to create something new.

You could store annotations in a parallel mailbox as MIME messages using APPEND. You'd have to figure out a way to map annotations from one message to another.
So you can find the related message easily, you'll want it easy to search for. You could do something like using the message-id of the source message as the subject of the annotation message, or a transformation of the message id as the message id of the destination message.

Here are my thoughts on it... here is the current proposed and accepted standard for IMAP4 which is the current version...
https://www.rfc-editor.org/rfc/rfc3501
Here is a wiki link to show the previous versions and the progress that has been made over time ...
https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol#IMAP4
I think the question is a good one, but maybe if your idea is good enough, contact the people that are in charge of the protocol and think about ways to potentially make IMAP5 with the kinds of functionality that you want to expand it to be able to utilize...
I would say that the best way would be to try making the IMAP protocol better. Read over all of the functionality that it currently supports and make suggestions to the group in charge of it. The additions that you are wanting to add sound great, but unless I misunderstood the question, I think that protocol updates might be the cleanest approach.
Sounds cool though.
Good luck... I hope you like my thoughts.

Have you considered Mailgun? It has an extensive API ( https://documentation.mailgun.com/api_reference.html#api-reference). Incoming messages can be stored, processed by your application and then sent via email. I believe you can add your own header field information (perhaps generating something like a GUID and using it for unique message tracking). Up to 10,000 emails/month are free (one of your preferences). If your application workflow and the API align, this might help get the job done.

Output encoding for Android and iOS

I have been working with PHP for quite a while and I have always dealt with web contents in various format. As far as I know, encoding the output of a procedure in a proper manner, according to the intended place for this output, hardens your web application against XSS attempt and, in general, injection-related vulnerabilities.
Just an example in PHP to better understand my concern: if a user provided a string which I have to display as the value of an input field, I just need to convert that string into HTML entities!
$output = \htmlspecialchars($input, \ENT_COMPAT | \ENT_HTML5, "UTF-8");
echo "<input name=\"output\" value=\"{$output}\" />";
Now, this should be enough to prevent any menace related to XSS in this specific case. Suppose that user input, when written to the database, was processed in a secure way, via prepared statements and with suitable data type binding, let's suppose using PDO. Moreover suppose that data in the value attribute is surrounded by single or double quotes like in the example above, otherwise properly dealing with this attribute becomes a much more non-trivial task!
In case this output needs to be sent to an Android or iOS client, is there any need to encode it so that no risk of manipulation, code execution nor anything else is triggered when it is displayed or processed by the application? I am not talking about cases in which the application actually tries to eval the string, I am just considering a simple output case. Is there any risk of that kind in a mobile client?
I hope my question is clear enough, if not please, feel free to ask for more details and I will either reply with a message or update the question if needed.

Android: how to integrate a 'report abuse' mechanism in my app.?

I'm currently working on my first serious app., and I would like to have some sort of contol on the data that users can enter.
Specifically, my app. allows users to write some text content (imagine something like a 'tweet'), and upload pictures.
I would like to prevent them from writing inappropriate text, and uploading offensive pictures for instance.
What I thought of doing, is to allow something like 'report abuse' button, where users who find some content offesive, can press - in which case relevent data will be saved, and later checked, to decide if indeed an inappropriate usage happened (maybe by some sort of server-side code).
As I said, I'm a beginner in android development, and I would really love to hear your suggestions and guiding. Perhaps it is something over my league for now? Maybe you know of such thing that already exists?
My app. uses Parse.com as its DB.
I would really appriciate your help.
Thank you.

I'm developing as well an app with parse and I also had to integrate in it a report button for the user.
The way I did it is simple:
In every Pf User object, I created a field of type counter named "reportCounter" while in the PFObject created by the user (it can be a string, a picture, etc.)I created a boolean field named "isReported". When a user find some inappropriate content he can report it through the dedicated button. The PFObject relative to that content gets its isReported field changed to YES and a parse background job checks every day for all the PFObjects, incrementing the reportCounter field of the owner-creator of the content and sending a report e-mail to the administrator. In this way you can keep also a record to see if a particular user is behaving badly. Just take a look at the Parse documentation about background-jobs. It's pretty good.
I hope this will help.

Possible to only use certain characters of a line to authenticate with remote database?

I'm looking to set up a remote database for user data for an Android application I am developing, but I don't want to use a server to handle the queries. Instead, I am looking to see if there is a way to make either the database recieve or the app only send parts of a generated code for each user.
This is my thought process on how it would work:
User opens app for first time ->
app generates and stores on device specific code for user and visible to user->
example of User Code: MG0CG094CF08352FBZS3042C0890432 ->
when user inputs data on app ->
data stored on device and sent to database when connection available ->
database receives only M0045328 for user identification for data input
The idea would be registration with the database without having to handle emails, passwords, etc., just the 8 digit user identification code. The purpose of the 32 character app generated code would be for security and the user in the event they they get a new device or delete the app.
The security measure I'm thinking of instead of a server is for the database to only handle certain characters of the user code or for the app to only send certain characters.
User sees this in app: MG0CG094CF08352FBZS3042C0890432
Application only sends: Mx0xx0x4xxxxx5xxxxx3xx2xxx8xxxxx
Database reads: M0045328
Obviously I don't want it to read like that in the source, so I need some discrete way for the app to exclude characters when sending information. Problem is that I can't think of some way that it's possible to code it like that. I essentially want the app to encrypt itself and only decrypt when it's sending information. I obviously also don't want each code to decrypt the same way, so maybe the 32 character code could also include which characters to send?
Anyone have some idea about how to do this or would simply a server solution be easier? Keep in mind I would prefer for it to handle it this way more than to use a server.

For all your pseudo code, in the end you are trying to achieve a DRM structure. DRM is not feasible for devices that you have no control over. So you can only obfuscate things a bit. Trying to hide code is not likely to work; it's better to try and hide some kind of data to make that more difficult to find.

Sending html email in android using <table>, etc. - is there really no relatively built-in Intent way?

I have read a good bit on the limitations of sending html email from android. All suggestions to send html email seem to be to just pass Html.fromHtml(yourHtmlString) to the intent as Intent.EXTRA_TEXT. This works for a few basic tags - bold, italic - but won't for anything like an html table.
It looks like you could try to extend some of the functionality of either Html or implement your own taghandler, but I am wondering if there is not a more fundamental limitation that will force you to do something completely different (like with the mail api or something).
The reason I suggest this is because, as far as the intent itself knows, Html.fromHtml(blah) is simply a charsequence, and if you call the methods on the charsequence interface on this object you don't see any html stuff (at least I didn't). All of the html/tag stuff seems to be wrapped up in the SpannableStringBuilder that Html.fromHtml actually returns... and I am wondering if the gmail app looks under the covers to see what the charsequence really is and then can handle a few tags, which means that there is no hope in doing anything on your app's side of things to get/trick the gmail app to handle anything more complicated than bold, italic, etc.
I have looked at the raw email the gmail app actually sends, and it automatically sends both a text/plain with no tags, and the text/html version with the limited number of tags. I even tried sticking in some escaped html tags that might ultimately get converted to actual tags in the text/html part of the email, but alas they stayed escaped... and that would of course be a bit hacky.
Anyway, for anyone who might have looked into this more, I wanted to do an additional confirmation that the default android "send html email" functionality will get you maddeningly close to what you might need, but in the end you've got to bite the bullet and implement a lot of lower level stuff yourself (such as Sending Email in Android using JavaMail API without using the default/built-in app , which means you've got to deal with the pw stuff, etc.).
Note (later):
I wrapped the SpannableStringBuilder returned from Html.fromHtml with a custom class that extended SpannableStringBuilder and passed that to the intent to listen for calls to the Spanned interface. It turns out that when things are written to the parcel that is sent to the email intent, TextUtils.writeToParcel does some special checking to root out the bold/italic stuff by first checking if the CharSequence is an instance of Spanned, and then asking for the spans (via spanned.getSpans). Nevertheless, I see no obvious hope in making the modifications to get something as simple as table/td tags handled in there. And I even tried modifying the toString() of my subclass of SpannableStringBuilder to return some raw table html to see what would happen, but it gets escaped somewhere else down there in the parcel-writing process.
And More (Later):
TextUtils.writeToParcel(CharSequence cs, Parcel p,...) will, if cs is an instance of "Spanned", write those spans only if they implement the "ParcelableSpan" interface... which is "A special kind of Parcelable for objects that will serve as text spans" and "can only be used by code in the framework; it is not intended for applications to implement their own Parcelable spans". So, even if you wanted to hook into this and write your own to handle table tags or whatever, it seems to be discouraged. Man I wish hackbod would weigh in here with something obvious I've missed.

This works for a few basic tags - bold, italic - but won't for anything like an html table.
That is a function of the email client, most likely. Not all email clients can author arbitrary HTML, on any platform. So, while Mozilla Thunderbird appears to let you create an HTML mail with a table, Gmail does not (leastways, I don't see an option for it in the message-compose window).
I am wondering if there is not a more fundamental limitation that will force you to do something completely different
Unless you write your own email client, extending the several classes needed to allow TextView and EditText to handle HTML tables (it's way more than just the Html class) will do you no good.
and I am wondering if the gmail app looks under the covers to see what the charsequence really is and then can handle a few tags
TextView and EditText can "handle a few tags", lining up roughly with what Html can parse/generate and SpannedString can represent.
None of that can handle an HTML table. Nor JavaScript. Nor CSS. Nor iframe or any number of other tags.
but in the end you've got to bite the bullet and implement a lot of lower level stuff yourself
I'd start by asking yourself whether sending HTML mail with tables from the phone directly is worth it. You could send HTML mail with tables from your server using a Web service interface, or you could send HTML mail sans tables from the phone. Neither of those would require you to collect "the pw stuff".