So I'm having an issue that looks that it came straight out from the Twilight Zone.
Problem
I have to hit a REST API endpoint from a backend, the thing is that in order to hit that endpoint I need to go through a VPN. Otherwise the host is not reachable. On desktop everything works fine, I open Postman, hit the GET endpoint and get the response. However when I try to hit the same endpoint through my Android device Retrofit throws an UnknownHostException.
Context
The endpoint url is something like https://api.something.something.net/. I'm using dependency injection with Dagger, so I've a NetworkModule that looks like:
...
NetworkModule("https://api.something.something.net/")
...
#Module
class NetworkModule(
private val baseHost: String
) {
...
#Provides
#Named("authInterceptor")
fun providesAuthInterceptor(
#Named("authToken") authToken: String
): Interceptor {
return Interceptor { chain ->
var request = chain.request()
request = request.newBuilder()
.addHeader("Content-Type", "application/json")
.addHeader("Authorization", "Bearer $authToken")
.build()
val response = chain.proceed(request)
}
}
...
#Provides
#Singleton
fun provideOkHttpClient(
curlInterceptor: CurlInterceptor,
#Named("authInterceptor") authInterceptor: Interceptor
): OkHttpClient {
val builder = OkHttpClient.Builder()
builder.addInterceptor(authInterceptor)
builder.addInterceptor(curlInterceptor)
return builder.build()
}
...
#Provides
#Singleton
fun provideRetrofit(okHttpClient: OkHttpClient, gson: Gson): Retrofit {
return Retrofit.Builder()
.baseUrl(baseHost)
.addConverterFactory(GsonConverterFactory.create(gson))
.addCallAdapterFactory(RxJava2CallAdapterFactory.create())
.client(okHttpClient)
.build()
}
}
Then I've a bunch of Repositories which are the ones doing the request through Retrofit:
class MyApiRepositoryImpl(
val myRetrofitApi: MyRetrofitApi,
val uiScheduler: Scheduler,
val backgroundScheduler: Scheduler
) : MyApiRepository {
override fun getSomethingFromTheApi(): Observable<DataResource<List<ApiSomethingResponse>>> {
return myRetrofitApi.getResponseFromEndpoint()
.map {
if (it.isSuccessful) {
DataResource(it.body()?.list!!, ResourceType.NETWORK_SUCCESS)
} else {
throw RuntimeException("Network request failed code ${it.code()}")
}
}
.subscribeOn(backgroundScheduler)
.observeOn(uiScheduler)
.toObservable()
}
}
And this is the Retrofit's API interface:
interface MyRetrofitApi {
#GET("/v1/something/")
fun getResponseFromEndpoint(): Single<Response<ApiSomethingResponse>>
}
So, when I call this Repository method from my Interactor/UseCases it jumps straight through the onError and shows the UnknownHostException.
What I tried so far
I switched Retrofit by Volley and later by Ion, just to be sure that wasn't something related to the rest client. I got the same exception in all cases:
java.net.UnknownHostException: Unable to resolve host "api.something.something.net": No address associated with hostname
com.android.volley.NoConnectionError: java.net.UnknownHostException: Unable to resolve host "api.something.something.net": No address associated with hostname
I tried every configuration possible with Retrofit and the OkHttpClient:
On OkHttpClient I tried setting the followSslRedirects to true and false. followRedirects to true and false. Set hostnameVerifier to allow any hostname to pass through. Set a SSLSocketFactory to allow any unsigned certificates to pass through.
On my Manifest I set my android:networkSecurityConfig to:
https://github.com/commonsguy/cwac-netsecurity/issues/5
I tested the App on my Android Device (Android Nougat), on Emulators with Nougat, Marshmellow and Oreo, and a Genymotion emulator with Nougat.
I tried hitting a random public endpoint (https://jsonplaceholder.typicode.com/todos/1) and It worked perfectly. So this isn't an issue with the internet connection.
I've these three permissions set on my Manifest:
android.permission.INTERNET
android.permission.ACCESS_NETWORK_STATE
android.permission.ACCESS_WIFI_STATE
It's super weird because I've an Interceptor set to convert all the requests into cURL requests, I copied and pasted the same request that is failing into Postman and works perfectly.
On my laptop I'm using Cisco AnyConnect, on my Android device I'm using the Cisco AnyConnect App and AFAIK on the emulators and Genymotion it should use the same network than the hosting machine.
There's a couple of websites that are only visible through the VPN and I can see them on the devices and on the emulators. But the endpoint URL is still unreachable from the App.
Couple of weird things
Yes, this gets weirder. If I hit the endpoint through the Chrome browser in my device or in the emulator I got redirected into a login page, that's because I'm not sending the authorization token. Now If I check the network responses through chrome://inspect I can get the IP of the host. Now if I change the base url of the NetworkModule by the IP and I add this line to the authorization Interceptor:
#Provides
#Named("authInterceptor")
fun providesAuthInterceptor(
#Named("authToken") authToken: String
): Interceptor {
return Interceptor { chain ->
var request = chain.request()
request = request.newBuilder()
.addHeader("Content-Type", "application/json")
.addHeader("Authorization", "Bearer $authToken")
.build()
val response = chain.proceed(request)
response.newBuilder().code(HttpURLConnection.HTTP_SEE_OTHER).build() // <--- This one
}
}
Then I start getting:
11-13 13:56:01.084 4867-4867/com.something.myapp D/GetSomethingUseCase: javax.net.ssl.SSLPeerUnverifiedException: Hostname <BASE IP> not verified:
certificate: sha256/someShaString
DN: CN=something.server.net,OU=Title,O=Company Something\, LLC,L=Some City,ST=SomeState,C=SomeCountryCode
subjectAltNames: [someServer1.com, someServer2.com, someServer3.com, someServer4.com, andSoOn.com]
I'm not sure if this is unrelated or if it's actually one step forward to fix it.
Any tip or advice is appreciated.
Thanks,
A coworker found the issue. Basically the VPN is blocking any App except com.android.chrome.
https://www.cisco.com/c/en/us/td/docs/security/vpn_client/anyconnect/anyconnect40/administration/guide/Cisco_AnyConnect_Mobile_Administrator_Guide_4-0/mobile-anyconnect-mobile-devices.html#task_F445916BDC9649D49A98F98224D2EA7D
I think this issue may be related to ssl certificate validity. for dealing with that issue you should to set setSSLSocketFactory in HttpClient.
private SSLConnectionSocketFactory getSSLSocketFactory() {
KeyStore trustStore;
try {
trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
trustStore.load(null, null);
TrustStrategy trustStrategy = new TrustStrategy() {
#Override
public boolean isTrusted(X509Certificate[] chain, String authType)
throws CertificateException {
return true;
}
};
SSLContextBuilder sslContextBuilder = new SSLContextBuilder();
sslContextBuilder.loadTrustMaterial(trustStore, trustStrategy);
sslContextBuilder.useTLS();
SSLContext sslContext = sslContextBuilder.build();
SSLConnectionSocketFactory sslSocketFactory = new
SSLConnectionSocketFactory(sslContext);
return sslSocketFactory;
} catch (GeneralSecurityException | IOException e) {
System.err.println("SSL Error : " + e.getMessage());
}
return null;
}
HttpClientBuilder.create().setSSLSocketFactory(getSSLSocketFactory())
.build();
Related
I am writing a Unit Test to test my API calls, using Retrofit2.
I have a mock of the server that I can launch locally (using localhost:8080)
I always receive a 403 error - Forbidden but it's working great with Postman
#Config(sdk = [28])
#RunWith(RobolectricTestRunner::class)
class MockServerTest {
private lateinit var result: Response<CacheResult>
private lateinit var api: FakeAPI
#Before
fun setUp() {
api = CompanySingleton.retrofit.create(FakeAPI::class.java)
}
#Test
fun getResult() {
runBlocking {
result = api.cache()
assertThat(result.isSuccessful, equalTo(true))
}
}
}
object CompanySingleton {
private val okHttpClient: OkHttpClient = OkHttpClient.Builder()
.addInterceptor(Interceptor { chain ->
val original = chain.request()
val requestBuilder = original
.newBuilder()
.addHeader("Content-Type", "application/json")
.addHeader("Authorization", "OAuth test")
.addHeader("x-device-id", "test")
val request = requestBuilder.build()
chain.proceed(request)
}
).build()
val retrofit: Retrofit = Retrofit.Builder()
.baseUrl("http://localhost:8080/")
.client(okHttpClient)
.addConverterFactory(GsonConverterFactory.create())
.build()
}
in my case testing with emulators I put http://10.0.2.2:8000 to access my local host
I assume that the api is being served on your laptop/pc (and postman definitely running on it can reach it using localhost). I think you should use the local IP of your laptop (e.g. 192.168.1.10) on android. Also don't forget to open 8080 port on your firewall (or simply turn it off)
When I have to test some local service, I usually use ngrok. It's simple to run, just in terminal you type:
./ngrok http 3000
I'm adding DI to the existing project, in process I faced problem that header Authorization disappears from request. There is no any exceptions or logs from Retrofit/OkHttp. My dependencies are:
implementation 'com.squareup.retrofit2:retrofit:2.6.0'
implementation 'com.squareup.okhttp:okhttp:2.7.5'
implementation 'com.squareup.okhttp3:logging-interceptor:3.10.0'
implementation 'org.koin:koin-android:2.1.3'
I create http client using provideClient:
class OkHttpProvider private constructor() {
companion object {
fun provideClient(credentials: UsernamePasswordCredentials? = null, context: Context): OkHttpClient {
val client = OkHttpClient.Builder()
// logs
if (BuildConfig.DEBUG) {
client.addInterceptor(
HttpLoggingInterceptor().setLevel(HttpLoggingInterceptor.Level.BODY)
)
}
if (credentials != null) {
val creds = Credentials.basic(credentials.userName, credentials.password)
val headerInterceptor = Interceptor { chain ->
var request = chain.request()
val headers = request
.headers()
.newBuilder()
.add("Authorization", creds)
.build()
request = request.newBuilder().headers(headers).build()
chain.proceed(request)
}
//client.addInterceptor(AccessTokenInterceptor(credentials))
client.addInterceptor(headerInterceptor)
}
client
.callTimeout(60L, TimeUnit.SECONDS)
.connectTimeout(10L, TimeUnit.SECONDS)
.readTimeout(60L, TimeUnit.SECONDS)
.writeTimeout(60L, TimeUnit.SECONDS)
.sslSocketFactory(getSslContext().socketFactory).hostnameVerifier { _, _ -> true }
client.addInterceptor(ChuckInterceptor(context))
return client.build()
}
private fun getSslContext(): SSLContext {
...implementation...
}
}
}
My modules for http client and Retrofit are below:
object HttpClientModule {
val module = module {
single(named(COMMON)) {
OkHttpProvider.provideClient(
get<SharedPreferenceManager>().getUserCredentials(),
androidContext()
)
}
...other versions...
}
const val COMMON = "common"
}
object ApiModule {
val module = module {
single {
RetrofitFactory.getServiceInstance(
ApiService::class.java,
get<SharedPreferenceManager>().getString(LocalDataSource.BUILD_OPTION_API, ""),
get(named(HttpClientModule.COMMON))
)
}
...other apis...
}
}
object RetrofitFactory {
const val GEO_URL = "http://maps.googleapis.com/maps/api/"
fun <T> getServiceInstance(
clazz: Class<T>,
url: String = GEO_URL,
client: OkHttpClient
): T = getRetrofitInstance(url, client).create(clazz)
private fun getRetrofitInstance(
url: String,
client: OkHttpClient
) = Retrofit.Builder()
.baseUrl(url)
.addConverterFactory(GsonConverterFactory.create())
.client(client)
.addCallAdapterFactory(CoroutineCallAdapterFactory())
.build()
}
App starts to work with "admin" user and has some credentials saved in shared preferences, when user starts login with phone and sms and requests are sent with "admin" Authorization header, when user inputs code from sms and his new user credentials are saved in shared preferences. After that app sends two requests and Authorization header isn't presented in them. I saw it in Chuck, I even rechecked it using Charles.
To fix this problem I tried few solutions. Firstly, I changed inject for http client from single to factory, that didn't work. Secondly, I googled the problem, but I didn't mentions of this phenomenon. Thirdly, I wrote AccessTokenInterceptor according to this article and also cover everything with logs. I noticed that interceptor works fine in normal cases, but when Authorization header is missing method intercept is not called. This might be reason why default headerInterceptor also not working. Fourthly, I upgraded versions of Retrofit and OkHttp, this also didn't helped.
I noticed interesting thing about that bug: if I restart app after Retrofit lost Authorization header, app works fine test user is properly logged with correct token. Any attempts to relog without restarting the app fails. Maybe someone had similar problem or knows what is happening here, any ideas are welcomed.
I finally find solution to this problem. The problem was user credentials was passed to provideClient only once, when it's created. At that moment user was logged as admin, and standard user credentials was empty, so http client for ApiService was created without Authorization header.
To solve this I changed AccessTokenInterceptor form article (HttpClientType is a enum to select which credentials need to use):
class AccessTokenInterceptor(
private val sharedPreferenceManager: SharedPreferenceManager,
private val clientType: OkHttpProvider.HttpClientType
) : Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
val credentials = getUserCredentials(clientType)
if (credentials != null) {
val accessToken = Credentials.basic(credentials.userName, credentials.password)
val request = newRequestWithAccessToken(chain.request(), accessToken)
return chain.proceed(request)
} else {
return chain.proceed(chain.request())
}
}
private fun getUserCredentials(clientType: OkHttpProvider.HttpClientType): UsernamePasswordCredentials? {
return when (clientType) {
OkHttpProvider.HttpClientType.COMMON -> sharedPreferenceManager.getUserCredentials()
OkHttpProvider.HttpClientType.ADMIN -> ServiceCredentialsUtils.getCredentials(sharedPreferenceManager)
}
}
private fun newRequestWithAccessToken(#NonNull request: Request, #NonNull accessToken: String): Request {
return if (request.header("Authorization") == null) {
request.newBuilder()
.header("Authorization", accessToken)
.build()
} else {
request
}
}
}
Now each time request is sending, Interceptor gets user's credentials and adds header to request.
I am working on an user app for a local charitable organization, and need to access their API. The API is from wild apricot, and this is the documentation for making a token request:
Authentication tokens are obtained from Wild Apricot's authentication service, which is located at https://oauth.wildapricot.org. This service adheres to oAuth 2.0.
This is the access option I need to implement:
-----------------In order to obtain access token with API key, you have to make the following request:
POST /auth/token HTTP/1.1
Host: oauth.wildapricot.org
Authorization: Basic BASE64_ENCODED("APIKEY:YOUR_API_KEY")
Content-type: application/x-www-form-urlencoded
grant_type=client_credentials&scope=auto
-------------------------------So. finally your request will look like:
POST /auth/token HTTP/1.1
Host: oauth.wildapricot.org
Authorization: Basic QVBJS0VZOm85c2U4N3Jnb2l5c29lcjk4MDcwOS0=
Content-type: application/x-www-form-urlencoded
grant_type=client_credentials&scope=auto
I am attempting to make this call with retrofit2, and an okhttp3 interceptor, and getting a bad request response (I am very much new and learning, and have not been able to get anything other response than a 400 bad request (when I use "/auth/token" as the endpoint), or a 404 not found (when I use "/auth/token HTTP/1.1" as the endpoint). If someone could tell me where exactly I am messing this up It would be greatly appreciated, the code I have tried is below.
Interface:
interface WAApiCall {
#POST("auth/token")
fun callPost(#Body body:String ): Call<AuthToken>
}
Call Service:
object WAApiCallService {
private const val API_KEY = "xxxxxxxxIxHavexAxValidxKeyxxxx"
private const val BASE_URL = "https://oauth.wildapricot.org/"
private val AUTH = "Basic" + Base64.encodeToString(API_KEY.toByteArray(), Base64.NO_WRAP)
private const val CONTENT_TYPE = "application/x-www-form-urlencoded"
private var api:WAApiCall? = null
private fun getWAApi(context: Context) : WAApiCall {
if(api==null){
val OkHttpClient = OkHttpClient.Builder()
val logging = HttpLoggingInterceptor()
logging.level = HttpLoggingInterceptor.Level.BASIC
OkHttpClient.addInterceptor{chain ->
val request = chain.request()
Log.d("CALL", request.body.toString())
val newRequest = request.newBuilder()
.addHeader("Host", "oauth.wildapricot.org")
.addHeader("Authorization", AUTH )
.addHeader("Content-type", CONTENT_TYPE)
.method(request.method, request.body)
.build()
chain.proceed(newRequest)
}
api = Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(GsonConverterFactory.create())
.client(OkHttpClient.build())
.build()
.create(WAApiCall::class.java)
}
return api!!
}
fun call(context: Context) =
getWAApi(context)
}
Function in Main Activity to make the call:
fun testRequest(){
val call = WAApiCallService.call(this)
call.callPost("grant_type=client_credentials&scope=auto")
.enqueue(object: Callback<AuthToken>{
override fun onFailure(call: Call<AuthToken>, t: Throwable) {
Log.i("FAILURE", t.localizedMessage)
}
override fun onResponse(call: Call<AuthToken>, response: Response<AuthToken>) {
Log.i("SUCCESS", "TOKEN = ${response.body().toString()}")
Log.i("SUCCESS", "${response}")
val token = response.body()?.accessToken
Log.i("SUCCESS", "TOKEN = $token")
}
})
}
Error message:
I/SUCCESS: TOKEN = null
I/SUCCESS: Response{protocol=http/1.1, code=400, message=Bad Request, url=https://oauth.wildapricot.org/auth/token}
I think that I am just not understanding how to implement this type of request in some basic way, I could not get it to work in Postman either. I understand that I need to send the credentials to the authentication server, and receive an access token, that will expire and need to be refreshed, and that It will be included in each actual API endpoint call, I guess I'm just missing something crucial in the most important step of that process (getting the actual token, I am imagining it is a simple, forehead slapping kind of misunderstanding on my part?). The wild apricot API is on swagger hub, and I am able to gain access through that UI, with my API key, and see the responses, so I know that it is valid.
Your client credentials request looks mostly all good. The only thing I can see that looks wrong is no space character in the AUTH header between 'Basic' and the encoded credential.
If that doesn't work, could you trace the HTTP request and verify that you are sending the message you think you are.
Thank you for that observation, it led me to figuring out what ultimately was wrong in my initial attempt. After adding that space, I traced the request and found that It was actually sending two headers for content type.
The fix for that was to set the header in the retrofit call from the interface:
interface WAApiCall {
#POST("auth/token")
fun callPost(#Body Body: okhttp3.RequestBody, #Header("Content-type") type: String): Call<AuthToken>
}
As you can see the body is also slightly different, the call was getting through but was returning:
"unsupported_grant_type".
I was passing a raw string as the body parameter, which was including the quotation marks in the request. The solution there was to pass the okhttp3.Request body type rather than a raw string, in the function that makes the actual call it looks like this:
val body: "grant_type=client_credentials&scope=auto&obtain_refresh_token=true"
val requestBody = RequestBody.create("text/plain".toMediaTypeOrNull(),body)
val call = WAApiCallService.call(this)
call.callPost(requestBody,"application/x-www-form-urlencoded")
.enqueue(object: Callback<AuthToken>{
With those changes the call succeeds and my long running headache is over.
I have an Authenticator like this
#Singleton
class TokenAutheticator #Inject constructor(private val tokenHolder: Lazy<TokenHolder>,private val tokenInterceptor: TokenInterceptor):Authenticator {
override fun authenticate(route: Route?, response: Response): Request? {
val resp = tokenHolder.get().tokenService.relogin(tokenInterceptor.token).execute()
println("### res "+resp.code())
if (resp.code()==200) {
val body = tokenHolder.get().tokenService.relogin(tokenInterceptor.token).execute()?.body()
val newToken = body?.token
println("########## authenticator ########## ")
val url = route?.address()?.url()?.newBuilder()?.addQueryParameter("token", newToken)?.build()
return response.request().newBuilder().url(url).build()
}else{
return null
}
}
}
When the resp.code != 200 the Authenticator is called multiple times.
I am plugging it in Okhttp like this
#Provides
#Singleton
fun provideOkhttp(tokenInterceptor: TokenInterceptor, tokenAutheticator: TokenAutheticator): OkHttpClient {
val logging = HttpLoggingInterceptor()
logging.setLevel(HttpLoggingInterceptor.Level.BODY)
val client = OkHttpClient.Builder()
.authenticator(tokenAutheticator)
.addInterceptor(logging)
.addInterceptor(tokenInterceptor)
.build()
return client
}
So what I want to do is have the Authenticator try it only once and if it is able to get a new token then use the new token from now on and if it can't get a new token then exit. But the Authenticator is called multiple times and the API responds with Too many attempts. Any help is appreciated. Thanks
Only when you return null, Authenticator will stop getting called (stop retrying authentication).
Since you always return something, when response code is 200, it will get called a few times. Luckily, as far I understand, Authenticator detects your endless loop, and breaks out of it eventually.
Basically, you should use Authenticator to react to 401 (and similar) response code and only add authorization header in that case.
I am trying to post data to REST API server with retrofit + RxJava . When I am trying to send data to server , it said " HTTP 500 Internal Server Error Occurred". But when the data is send with POSTMAN, it succeeded.
This is the function for sending data in Model.
// Encountering with 500 server error
fun postSchedule(data : ScheduleResponse , errorLD: MutableLiveData<String>){
Log.d("POST DATA", "${data.title} ${data.remindMeAt}" )
userClient.postScheduleItem(data)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.unsubscribeOn(Schedulers.io())
.subscribe(object : io.reactivex.Observer<ServerResponse>{
override fun onComplete() {
}
override fun onSubscribe(d: Disposable) {
}
override fun onNext(t: ServerResponse) {
errorLD.value = t.status
}
override fun onError(e: Throwable) {
errorLD.value = e.message
}
})
}
This is my API interface
#Headers("Accept: application/json")
#POST("schedules")
fun postScheduleItem(#Body data: ScheduleResponse): Observable<ServerResponse>
This is the retrofit client.
val httpLoggingInterceptor = HttpLoggingInterceptor().setLevel(HttpLoggingInterceptor.Level.BODY)
val httpClient = OkHttpClient.Builder()
var dbInstance: TodoDB = TodoDB.getInstance(context)
var rxJavaAdapter = RxJava2CallAdapterFactory.createWithScheduler(Schedulers.io())
val retrofitBuilder =
Retrofit.Builder()
.baseUrl(AppConstants.BASE_URL)
.addConverterFactory(GsonConverterFactory.create())
.addCallAdapterFactory(rxJavaAdapter)
fun <T> createService(serviceClass: Class<T>, authToken: String?): T {
if (!TextUtils.isEmpty(authToken)) {
val interceptor = AuthenticationInterceptor(authToken!!)
if (!httpClient.interceptors().contains(interceptor)) {
httpClient.addInterceptor(interceptor)
retrofitBuilder.client(httpClient.build())
}
}
return retrofitBuilder.build().create(serviceClass)
}
Please help me with this.Thank you.
Client side code is not enough to determine what causes the server to respond with 500. The best you can do is start debugging the issue.
There are several directions you can go:
If you have access to the server or know someone who does, you could debug the server and determine what causes the Internal server error. Maybe the server logs can help as well and you don't have to actually step through the server code.
If you don't have access to the server, you could look at the body of the server response. Maybe there's a detailed error description there in html, json or some other format that will help you find out the root cause.
If the above steps don't help then it's very useful that you know the request works with POSTMAN. You can compare the exact POSTMAN request with the exact Retrofit request, header by header,
line by line. To do that, you should first add your httpLoggingInterceptor to your okhttp client builder with
val httpClient = OkHttpClient.Builder().addNetworkInterceptor(httpLoggingInterceptor)
and look for the request log in logcat.
If you spot the differences between the working and the not working requests, then you should work your way through all the differences, and adjust the retrofit request by adding or modifying headers using okhttp interceptors so that, at the end, the retrofit request looks exactly the same as the POSTMAN request. I suggest you remove the AuthenticationInterceptor at first and simulate it "manually" with a custom interceptor and a hard coded auth token.
Retry the request every time you eliminate a difference to isolate the cause of the internal server error.
Hope this helps!