This has probably been delved into multiple times on SO but I still can't get my head around the flow here. Can someone please help throw some light here as to what I might be missing!
I am building a mobile messaging app, with the backend chat server in Erlang and photos getting stored in AWS S3. I want to ensure that when a user sends a photo to a group, the same is downloadable only by the said group's members and no one else.
From what I understand, I need to have pre-signed URLs (of the uploaded photos) distributed to the chat recipients (i.e. group members) so that they can download the same directly.
My question is: How do I generate these pre-signed URLs? And what role does AWS Cognito have here? Should I build a new web service that can be queried by the mobile app? So this web service would call Cognito's getIdentityId() and use the response (along with the bucket name / key passed by the mobile client) to generate the pre-signed URL, which it would then pass back to the mobile client, which can then distribute this URL to the group chat recipients?
Or should the Cognito credentials provider and the user pool ID be included in the mobile source code and the iOS / Android app itself should generate the said URL?
Apologies for the rather long rant!
The cognito credentials are used to sign the pre-signed URL which is generated by the SDK. If the pre-signed URL is not signed with valid credentials the get request for the download fails.
Then cognito pool id has to be present in the app which fetches temporary credentials for getting access to AWS resources.
Thanks,
Rohan
Related
I'm using flask as backend for REST APIs. I have a database which contains users and images information uploaded by them. All the images are stored in a S3 bucket like "bucket_name/user_1/img1.jpg". I want to allow the users to download all the images that are only specific to them i.e in folder "user_1" and they shouldn't have access to "user_2" images. Also the bucket and it's contents are private.
Image table has image_path (like "bucket_name/user_1/img1.jpg") which can be accessed by flask app only. REST APIs are used by android app.
So when the user on android app fetches his image (using GET request), he should be able to download all the images. How to deliver those images?
Using aws cloudfront and oai
flask using boto3 generate_presigned_url()
make the bucket contents public (which I definitely don't want)
How this android -- flask -- aws flow should work?
There are two ways you can achieve this.
You could use pre-signed URLs, or
Provide temporary credentials that permit access to their folder.
Pre-Signed URLs
Keep all content private. Then, when a client (eg Android app) requires access to an object, your application should verify that they are entitled to access the object by checking the database. If they are permitted access, then the application should generate a pre-signed URL that grants time-limited access to the object.
Creating a pre-signed URL only takes a couple of lines of code and does not require a call to AWS. Thus, it is very fast to create and the application retains full control over who is permitted access to which objects.
Temporary Credentials
Since your users are permitted to access all content within their folder, you could create Temporary Security Credentials using the Security Token Service. A policy can be assigned to these credentials that permits access to s3://bucket_name/user_1/*. Then, the Android app can use these credentials to directly call the AWS API to download content (or upload, or do whatever permission you have assigned).
This allows the app to directly communicate with AWS rather than having to call your servers for every object access.
If you are using federated authentication (eg trusting Facebook), you might want to use Amazon Cognito to Get Credentials.
I have a bucket in Amazon S3 and I have Lambda functions that generates JSON files for this bucket. I am using the S3 files in my mobile app. Until recently, I gave public access to these S3 files for simplicity. But now I want these S3 objects to be accessible with a simple authentication. I’ve examined all AWS tutorials but couldn’t find an easy way to implement this. I don’t want to use Cognito service since my app doesn’t need authentication and since my S3 files are not user-related, they are used for app. I want these S3 objects to be accessible by http request to a url which includes simple key like this:
https://s3.eu-central-1.amazonaws.com/<bucket name>/<object name>?<key>
where key can be a combination of region, aws access key, secret access key or other values of the user that i define (I am using Retrofit to fetch json data from S3 bucket) I’ve looked at the presigned url option but an example for android-sdk doesn’t exists there, and most of the methods that can be used for this purpose are deprecated. Isn’t there an easy way for this? Or should I host my json files in other service/place?
If you say no to Cognito then you are just complicating things for yourself. This is exactly the case where you want to use Cognito. The fact that you don't require your users to authenticate and that the S3 content is not user specific doesn't mean that Cognito is not suitable in this scenario.
All you have to do is to create Cognito identity pool and choose to support unauthenticated entities. Create an IAM policy that allows reading from that specific S3 bucket and let unauthenticated users to assume that IAM role by attaching it to those unauthenticated entities.
Authentication then happens automatically during the initialization of SDK in your application. That is all that you need to do to allow access to that S3 bucket only from your application.
And you get access metrics even for unauthenticated users as a bonus. And if you later decide that you want your application to support authentication as well, then you don't need to change almost anything in your setup.
I want to create an android application and use AWS s3 as a storage service to allow the user to upload and download files. I have studied something about S3 REST service, which can help me achieve the same, after configuration of IAM Role for Bucket etc.
Now thing is, I want only the registered users of my application/ with access control configured at my web server(username/password) should be allowed to upload/download the file and not anyone with only app access should be allowed.
Look the link below as well, to have some idea about AWS S3 upload file using REST
http://www.tothenew.com/blog/file-upload-on-amazon-s3-server-via-rest-api-call/
Putting simply, in addition to that described in the link above, I just need the answer to the following questions:
1) How to allow only registered users?
2) Is it good practice to hard code AWS S3 secretKey etc in the production application.
3) Does hardcoding these values in my app could lead to a scenario where even an unregistered user of my application could be able to upload/download the file to aws s3?
All suggestions are welcome, if-if they solve part of the puzzle, as I am completely unaware of the solution
It would be very helpful...,
Thanks in advance,
For sure it is not best practice to hard code secret keys inside your codes..
Now if you want to separate the authorized and the unauthorized users one AW services that does this job and more is AWS Cognito . You can find a lot in the aws documentation https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-console.html
A few things for Cognito :
You have Amazon Cognito User Pools tha provides :
Sign-up and sign-in services.
A built-in, customizable web UI to sign in users.
Social sign-in with Facebook, Google, and Login with Amazon, as well as sign-in with SAML identity providers from your user
pool.
User directory management and user profiles.
Security features such as multi-factor authentication (MFA), checks for compromised credentials, account takeover
protection, and phone and email verification.
You have Amazon Cognito Identity Pools (Federated Identities) that provides :
Users in an Amazon Cognito user pool
Users who authenticate with external identity providers such as
Facebook, Google, or a SAML-based identity provider
Users authenticated via your own existing authentication process
This how it works in my application
Ask user register/login using aws cognito
deploy an api via api gateway that acts as proxy for S3 services
For the api , apply AWS cognito as Authorizer in api gateway
This however has a limitation on size of object
The option I have tried is use signed urls, and this is spit out by back end application upon registered users access requests.
My implementation includes below
1. Client requests signed urls using an api with a lambda implementation
2. Lambda generates signed urls and passes on the response.
3. while creating signed urls, key/id are used from a parameter store (see EC2 parameter store) secured against role and with a KEY.
I think from your use case , EC2 parameter store (instead of database) should be sufficient to securely keep secretes.
Ok, so I am new to AWS. I want to make an app that will store a small amount of user data. S3 seems to be the way to store data. Is there a way to make multiple storage spaces automatically with S3? Lets say I make an android app and people install it on their phone. Will they each automatically get an S3 storage space? how do I do that? thanks
You can create a S3 Bucket to your project with folders (depending on the architecture, can be one for customer). On this way, you will have an instance of the s3 service with all your user data.
Amazon S3 is simply an object-storage system. How you use it is totally up to you.
If you wish to store information on a per user basis, then you need to consider security in addition to how the data is stored.
If the intention is that a user can access some information that is private to them (as opposed to being publicly visible to anyone), then you first need to control access to data.
For public information, no authentication is necessary
For private information, something needs to determine what they are allowed to access, and then grant access
You should not give permanent AWS Credentials (Access Key, Secret Key) to every user. These credentials are only for your IT operations staff (you!) and for your applications.
This leaves two options:
Your central server could generate temporary access credentials using the AWS Secure Token Service, while specifying what access rights they have (eg access to a particular S3 bucket and path, or to other AWS services such as DynamoDB). OR
Generate pre-signed URLs for specific objects stored in Amazon S3.
Based upon your use-case, it seems a better fit to use pre-signed URLs. Basically, the flow is:
Your app would send a request to your central server, requesting access to an object.
The server (or rather, the app you have written that is running on a central server) verifies their identity and confirms that they should be allowed access to the object stored in Amazon S3.
The central server then generates a pre-signed URL that grants time-limited access to an object in Amazon S3 and sends the URL back to the client app
The client app then uses the URL to retrieve the data from Amazon S3
Only the app running on your central server requires AWS credentials. It then uses those credentials to generate pre-signed URLs that can be used by the client apps.
By the way, the app on the central server doesn't actually need to be running on a server. You could use AWS API Gateway to send requests to AWS Lambda functions, which can perform the logic and send back the response. This would be a serverless solution, but still with centralized logic.
I am using AWS S3 to upload and store contents in my Android application. I am using the below code in a java file:
AmazonS3Client s3Client = new AmazonS3Client(
new BasicAWSCredentials("XYZ",
"ABC+wktN"));
But this seem to be easy to retrieve (even if the apk is signed!). I tried using progaurd but that crashes app on some mobiles devices (especially this AWS part).
Is there any approach where i can hide credentials in Code without anyone being able to retrieve it? Or has anyone used any other method? One option is to upload to a intermediate server and the server uploads it to AWS S3, but thats an extra hop. Another is make the app get credentials from server through an URL call instead of having the credentials on mobile, but if a hacker can get the credentials directly he can get the URL as well and get the key.
Amazon Cognito can help you solve the problem. You can use it to deliver temporary, limited-privilege credentials to your application. With Cognito
you don't have embed long lived credentials inside your app
you can retrieve temporary credentials from Cognito. In case credentials are intercepted, they are valid at most an hour (configurable).
you can attach access policy to the credentials to limit what resource the credentials can access. See Understanding Amazon Authentication Part 3 Roles and Policies.
Cognito supports integration with 3rd party identity providers like Facebook, Login with Amazon, etc. You can force customers authenticated before accessing your resource
Cognito also supports developer authenticated identity, so that you can build your own backend to authenticate customers.
You can find additional information their blog http://mobile.awsblog.com/blog/tag/amazon-cognito and the developer guide http://docs.aws.amazon.com/cognito/devguide/identity/.