Develop Reference

Thousands of OkHttp related issues reported daily (connection, unknown host, dns, etc)

Setup
I have an app in production used by thousands of users daily.
I'm using retrofit2 version 2.9.0 (latest)
My build.gradle below.
def retrofitVersion = '2.9.0'
api "com.squareup.retrofit2:converter-gson:${retrofitVersion}"
api "com.squareup.retrofit2:converter-scalars:${retrofitVersion}"
api "com.squareup.retrofit2:adapter-rxjava2:${retrofitVersion}"
api "com.squareup.retrofit2:retrofit:${retrofitVersion}"
I integrated Firebase Crashlytics and made it so that app would report any API related exceptions in try-catch blocks.
e.g.
viewModelScope.launch {
try {
val response = myRepository.getProfile()
if (response.isSuccessful) {
// continue with some business logic
} else {
Log.e(tag, "error", RunTimeException("some error")
}
} catch (throwable: Throwable){
Log.e(tag, "error thrown", throwable)
crashlytics.recordException(throwable)
}
}
Knowns
Now in Crashlytics, I get THOUSANDS of reports daily saying there were some errors.
Before I get to those errors, I want to assure you that users ARE connected to internet with proper network permissions. I see logs that users are opening other contents at the time. So these errors seem to be really random.
Erros
UnknownHostException
Non-fatal Exception: java.net.UnknownHostException: Unable to resolve host "my-host-address.com": No address associated with hostname
at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:156)
at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
at java.net.InetAddress.getAllByName(InetAddress.java:1152)
at okhttp3.Dns$Companion$DnsSystem.lookup(Dns.java:5)
...
Caused by android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)
at libcore.io.Linux.android_getaddrinfo(Linux.java)
at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74)
at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:200)
at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74)
at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:135)
at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
...
ConnectionException
Non-fatal Exception: java.net.ConnectException: Failed to connect to my-host-address.com/123.123.123.123:443
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:146)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:191)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:257)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:47)
...
Caused by java.net.ConnectException: failed to connect to my-host-address.com/123.123.123.123 (port 443) from /:: (port 0) after 10000ms: connect failed: ENETUNREACH (Network is unreachable)
at libcore.io.IoBridge.connect(IoBridge.java:142)
at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:142)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:390)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:230)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:212)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:436)
at java.net.Socket.connect(Socket.java:621)
...
Caused by android.system.ErrnoException: connect failed: ENETUNREACH (Network is unreachable)
at libcore.io.Linux.connect(Linux.java)
at libcore.io.ForwardingOs.connect(ForwardingOs.java:94)
at libcore.io.BlockGuardOs.connect(BlockGuardOs.java:138)
at libcore.io.ForwardingOs.connect(ForwardingOs.java:94)
at libcore.io.IoBridge.connectErrno(IoBridge.java:173)
at libcore.io.IoBridge.connect(IoBridge.java:134)
...
SocketTimeoutException
Non-fatal Exception: java.net.SocketTimeoutException: timeout
at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:4)
at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:8)
at okhttp3.internal.http2.Http2Stream.takeHeaders(Http2Stream.java:24)
at okhttp3.internal.http2.Http2ExchangeCodec.readResponseHeaders(Http2ExchangeCodec.java:5)
at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.java:2)
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:145)
...
Another SocketTimeoutException
Non-fatal Exception: java.net.SocketTimeoutException: SSL handshake timed out
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(NativeCrypto.java)
at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:387)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:234)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:72)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:52)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:196)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:257)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:47)
And lastly, what makes me think it's not my server issue is that I get this kind of error when I request banner ads to Google server as well.
I get thousands of reports of the following
{ "Message": "Error while connecting to ad server: Failed to connect to pubads.g.doubleclick.net/216.58.195.130:443", "Cause": "null", "Response Info": { "Adapter Responses": [], "Response ID": "null", "Response Extras": {}, "Mediation Adapter Class Name": "" }, "Domain": "com.google.android.gms.ads", "Code": 0 }
from Google ads SDK's onAdFailedToLoad listener.
Attempt
I tried to find some solutions in Retrofit2/OkHttp3 github issues, SO community, and everyone says there may be some network permission issues or network connection problem itself. But I know users are connected to internet and not using some sort of proxy. I worked with customer service team and they walked through with users, and they did not find any network issues.
Any insight would be helpful. Thank you in advance!

About 2 years earlier I faced an issue like this. The previous answer was not my case fortunately (I tried after clearing DNS cache). I talked to my ISP about this problem they said something about caching request and response. They also told me to wait for 24 hours the server/ISP will release the cache then try. I waited and it worked. I don't know how this happened or why it worked.

I had these issues too. After a long time, I have realized most of these problems are related to the SSL certificate validation by android in different devices. Some devices are not able to validate your certificate because it does not exist in their pre-installed system CA certificates. You need to put the certificate in your app to avoid this issue. You can do this in the following steps:
Export your SSL certificate to a file. (There are different ways to do this for example using a browser). Rename it to a simple name for example mycert.
Put this file in the raw folder in this path res > raw.
Make an XML file called network_security_config.xml in the xml folder at this path res > xml.
Paste this lines in it:
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<domain-config>
<domain includeSubdomains="true">my-host-address.com</domain>
<trust-anchors>
<certificates src="#raw/mycert"/>
</trust-anchors>
</domain-config>
</network-security-config>
change the my-host-address.com in the domain tag to your desired API address.
Add this line:
android:networkSecurityConfig="#xml/network_security_config"
to the application tag in your manifest file like so:
<application
android:allowBackup="true"
android:icon="#mipmap/app_ic_luncher"
android:label="#string/app_name"
android:theme="#style/AppCompatTheme"
android:networkSecurityConfig="#xml/network_security_config"
...
Make a new build and hopefully, it works for you.

Having the same issue with dns resolution for certain users. The url works if they for example tries to access it from a browser. Implementing an alternative DNS resolution, like DnsOverHttps using Cloudflare could work, if it doesn't make sense to invest more time finding a root cause.
val bootstrapClient = OkHttpClient.Builder().build()
val dns = DnsOverHttps.Builder().client(bootstrapClient)
.url("https://1.1.1.1/dns-query".toHttpUrl())
.includeIPv6(false)
.build()
val httpClient = bootstrapClient.newBuilder().dns(dns).build()
Alternatively wrapping that in something first trying the system one and then DnsOverHttps as a fallback if the system one doesn't behave as expected.

Firebase realtime database no longer loading in my console. Writes also failed inside my Android app

Firebase realtime database is no longer loading in my console. It only show an empty content without any error. Rules for the database fail to load as well.
Inside my Android app writes to the realtime database fail with :
E/SOL: failed to put
'https://[appid].firebaseio.com/logs/2022/3/12/0/us_0600830214/s110.json',
ex: Failed to connect to
[appid].firebaseio.com/2600:1901:0:94b6:::443
java.net.ConnectException: Failed to connect to
[appid].firebaseio.com/2600:1901:0:94b6:::443
atokhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:297)
atokhttp3.internal.connection.RealConnection.connect(RealConnection.kt:207)
atokhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226)
atokhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106)
atokhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74)
atokhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255)
atokhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
atokhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
atokhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
atokhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
atokhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
atokhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:919)
Suppressed: java.net.SocketTimeoutException: failed to connect
to [appid].firebaseio.com/35.201.97.85 (port 443) from
/10.0.2.16 (port 33500) after 10000ms
at libcore.io.IoBridge.connectErrno(IoBridge.java:191)
at libcore.io.IoBridge.connect(IoBridge.java:135)
at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:142)
atjava.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:390)
atjava.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:230)
atjava.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:212)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:436)
at java.net.Socket.connect(Socket.java:621)
atokhttp3.internal.platform.Platform.connectSocket(Platform.kt:120)
atokhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:295)
I changed nothing in my code or console. A couple of days ago the console stop loading and the write errors I think stop working way before I just didn't pay much attention to it.
I don't see any project quotas being surpassed, all values are very close to 0% of usage for the free spark plan.
I also checked the google-services.json, minor changes, ... and updating to the new version did not fix the write failures.
Has anyone else experienced this?

I have contacted Firebase support and they have confirmed that such behavior have been reported by several other users but from what they can tell it is all fine on their side. The issue lies with the ISP from my country. All other reports had the same origin.

java.net.UnknownHostException: Unable to resolve host "domain.com": No address associated with hostname

Bumped into an issue where my application using https://github.com/square/okhttp - okhttp version - 3.12.1 is not able to connect to a domain even though I've provided the Internet Permission in the AndroidManifest.xml
The user was, however, able to connect to the internet on their device and use other apps, based on their review comments.
For now, I am unable to reproduce the exception and hence if anyone who has experienced this issue before could help me understand better.
I cannot upgrade the library at the moment as I don't want to introduce a big change.
Stacktrace
Caused by: java.net.UnknownHostException: Unable to resolve host "domain.com": No address associated with hostname
at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:156)
at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
at java.net.InetAddress.getAllByName(InetAddress.java:1152)
at okhttp3.Dns$-CC.lambda$static$0(SourceFile:39)
at okhttp3.-$$Lambda$Dns$mTkNcZf2K4euny3_jks6Cac6Az0.lookup(Unknown Source:0)
at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(SourceFile:185)
at okhttp3.internal.connection.RouteSelector.nextProxy(SourceFile:149)
at okhttp3.internal.connection.RouteSelector.next(SourceFile:84)
at okhttp3.internal.connection.StreamAllocation.findConnection(SourceFile:214)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(SourceFile:135)
at okhttp3.internal.connection.StreamAllocation.newStream(SourceFile:114)
at okhttp3.internal.connection.ConnectInterceptor.intercept(SourceFile:42)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:121)
at okhttp3.internal.cache.CacheInterceptor.intercept(SourceFile:94)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:121)
at okhttp3.internal.http.BridgeInterceptor.intercept(SourceFile:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:147)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(SourceFile:125)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:121)
at com.appdynamics.eumagent.runtime.a.a$a.intercept(SourceFile:108)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:121)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(SourceFile:121)
at okhttp3.RealCall.getResponseWithInterceptorChain(SourceFile:264)
at okhttp3.RealCall.execute(SourceFile:93)
Caused by: android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)
at libcore.io.Linux.android_getaddrinfo(Native Method)
at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74)
at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:200)
at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74)
at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:135)
... 39 more

I usally have that error using Retrofit (with Okhttp as client), it happens when the device doesn't have any internet conection (both WIFI or mobile data) or my server endpoint is down. Try it yourself! But there are several factors else; like runtime request permisions or the network security config.
Hope this helps, bro.

We solved this issue by making a few DNS changes.
Reduced few CNAME records.
Reduced TTL.
No code changes were made on the android client.

Maybe, the divice use to get API don't have internet.Please connect internet and try again!

Unable to decode stream: android.graphics.ImageDecoder$DecodeException: Failed to create image decoder with message 'unimplemented'

I've recently tried implementing Pusher's Chatkit into my android app along with some Kotlin plugins and am now running into this error that I cannot resolve.
E/Drawable: Unable to decode stream: android.graphics.ImageDecoder$DecodeException: Failed to create image decoder with message 'unimplemented'Input contained an error.
the error I'm getting right below it is:
E/FirebaseInstanceId: Token retrieval failed: SERVICE_NOT_AVAILABLE
Which is through throwing me for a loop because my internet is working fine although a couple lines further down, I get:
W/Firestore: (21.3.0) [OnlineStateTracker]: Could not reach Cloud Firestore backend. Backend didn't respond within 10 seconds.
This typically indicates that your device does not have a healthy Internet connection at the moment. The client will operate in offline mode until it is able to successfully connect to the backend.
Process 15216 terminated.
Note: I've started to get this error above about not having a good connection ever since I started working more at cafes instead of at home since I needed to get out. In order to work offline, firebase asked me to opt into an offline mode I've been getting this notification of not being able to connect ever since.
I am following along to the getting started page here and have completed up to the end of the "Displaying Messages" section however when I go to the activity, it just stalls and blanks out. I have been searching StackO and the rest of the web for the past 2 days to try to resolve the conflict to no avail. All of the similar posts I have looked at dont seem to resolve the issue.
similarPost#1 but I'm not trying to save to a bitmap
Here is the entire stack trace:
E/Drawable: Unable to decode stream: android.graphics.ImageDecoder$DecodeException: Failed to create image decoder with message 'unimplemented'Input contained an error.
E/FirebaseInstanceId: Token retrieval failed: SERVICE_NOT_AVAILABLE
W/DynamiteModule: Local module descriptor class for providerinstaller not found.
I/DynamiteModule: Considering local module providerinstaller:0 and remote module providerinstaller:0
W/ProviderInstaller: Failed to load providerinstaller module: No acceptable module found. Local version is 0 and remote version is 0.
I/ghborsbookshel: The ClassLoaderContext is a special shared library.
I/chatty: uid=10086(com.example.android_myneighborsbookshelf) AsyncTask #2 identical 1 line
I/ghborsbookshel: The ClassLoaderContext is a special shared library.
V/NativeCrypto: Registering com/google/android/gms/org/conscrypt/NativeCrypto's 286 native methods...
W/ghborsbookshel: Accessing hidden method Ljava/security/spec/ECParameterSpec;->getCurveName()Ljava/lang/String; (light greylist, reflection)
I/ProviderInstaller: Installed default security provider GmsCore_OpenSSL
W/Firestore: (21.3.0) [OnlineStateTracker]: Could not reach Cloud Firestore backend. Backend didn't respond within 10 seconds
This typically indicates that your device does not have a healthy Internet connection at the moment. The client will operate in offline mode until it is able to successfully connect to the backend.
Process 15216 terminated.

It seems to have fixed itself except for the unable to decode stream error however it doesn't crash, it just shows up in the log.
I checked for updates, cleaned, rebuild, invalidated caches and restarted, and shut down my emulator. I also restarted my computer completely 2x and that seems to have fixed it.

Google Drive upload error : "I/O error during system call, Software caused connection abort"

I have developed an Android application using which the user can upload large files (~1 to 2 GB in size) to his/her Google Drive account.
I am using the Google Drive Rest API for Android. Sometimes, the upload operation fails with following exception:
javax.net.ssl.SSLException: Read error: ssl=0xcfe9ee80: I/O error during system call, Software caused connection abort
at com.android.org.conscrypt.NativeCrypto.SSL_read(Native Method)
at com.android.org.conscrypt.OpenSSLSocketImpl$SSLInputStream.read(OpenSSLSocketImpl.java:758)
at com.android.okhttp.okio.Okio$2.read(Okio.java:141)
at com.android.okhttp.okio.AsyncTimeout$2.read(AsyncTimeout.java:211)
at com.android.okhttp.okio.RealBufferedSource.indexOf(RealBufferedSource.java:316)
at com.android.okhttp.okio.RealBufferedSource.indexOf(RealBufferedSource.java:310)
at com.android.okhttp.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:206)
at com.android.okhttp.internal.http.HttpConnection.readResponse(HttpConnection.java:239)
at com.android.okhttp.internal.http.HttpTransport.readResponseHeaders(HttpTransport.java:104)
at com.android.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:1156)
at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:976)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:509)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:438)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:567)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java)
at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithoutGZip(MediaHttpUploader.java:545)
at com.google.api.client.googleapis.media.MediaHttpUploader.resumableUpload(MediaHttpUploader.java:417)
at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:336)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:427)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
I have following questions:
What are the reasons for such error?
Is this the issue related to network speed?
Is this is the problem with Google Drive API? If yes, how to avoid this error?
I testing the app with Android 7.0.
I would appreciate any suggestions and thoughts on this topic. Thank you.