I'm having a real hard time figuring out how to parse a none standard HTTP response.
The none standard response contains ICY 200 OK instead of HTTP 200 OK. Here is a sample URL that sends the none standard HTTP response.
http://50.117.121.162:80
Since Android 4.4 HttpURLConnection will no longer work with these none standard responses. I have tried using the HttpClient by Apache but it doesn't work because of the none standard HTTP response. I have then tried following the guide for adding a custom response parser, but Android doesn't seem have all the classes needed to do it.
I'm really struggling to figure out a solution. Possibly modify the none standard response before it is parsed by the HttpClient or the HttpURLConnection could work but I'm not sure if that is even possible...
Any help would be greatly appreciated.
After a lot of research for a small/lite http client library, I ran into this port of the apache httpclient for android. The library provided a complete support for http connections. Then I simply modified the source code, particularly the BasicLineParser to replace ICY with HTTP/1.0.
I had similar problem with KitKat and had a success with using two classes found here for http post. They are incredibly easy to use and you can modify the protocol params easily too.
There is another solution to this issue in Android 4.4 but it requires using Apache HttpClient. This is based on possibility of providing custom response parser into Apache Http engine that can change ICY 200 OK to HTTP/1.0 200 OK. This is based on general idea presented in:
http://hc.apache.org/httpcomponents-client-4.2.x/tutorial/html/advanced.html
I have successfully used following code.
public class IcyClientConnection extends DefaultClientConnection{
#Override
protected HttpMessageParser createResponseParser(SessionInputBuffer buffer,
HttpResponseFactory responseFactory, HttpParams params) {
return new IcyHttpResponseParser(
buffer,
new BasicLineParser (),
responseFactory,
params);
}
}
public class IcyClientConnectionOperator extends DefaultClientConnectionOperator {
public IcyClientConnectionOperator(SchemeRegistry schemes) {
super(schemes);
}
#Override
public OperatedClientConnection createConnection() {
return new IcyClientConnection();
}
}
public class IcyClientConnManager extends SingleClientConnManager {
public IcyClientConnManager(HttpParams params, SchemeRegistry schreg) {
super(params, schreg);
}
#Override
protected ClientConnectionOperator createConnectionOperator(
SchemeRegistry schreg) {
return new IcyClientConnectionOperator(schreg);
}
}
Now you have to extend parser used by default and add code that will change wrong server replay to correct one. Normally code will block on hasProtocolVersion.
public class IcyHttpResponseParser extends DefaultResponseParser{
private CharArrayBuffer icyLineBuf;
private int icyMaxGarbageLines = 1000;
private final HttpResponseFactory icyResponseFactory;
public IcyHttpResponseParser(SessionInputBuffer buffer, LineParser parser,
HttpResponseFactory responseFactory, HttpParams params) {
super(buffer, parser, responseFactory, params);
this.icyLineBuf = new CharArrayBuffer(128);
icyResponseFactory = responseFactory;
}
#Override
protected HttpMessage parseHead(SessionInputBuffer sessionBuffer)
throws IOException, HttpException {
int count = 0;
ParserCursor cursor = null;
do {
// clear the buffer
this.icyLineBuf.clear();
final int i = sessionBuffer.readLine(this.icyLineBuf);
//look for ICY and change to HTTP to provide compatibility with non standard shoutcast servers
String tmp = icyLineBuf.substring(0, this.icyLineBuf.length());
if(tmp.contains("ICY ")){
tmp = tmp.replace("ICY", "HTTP/1.0");
}
//copy
this.icyLineBuf = new CharArrayBuffer(128);
System.arraycopy(tmp.toCharArray(), 0, icyLineBuf.buffer(), 0, tmp.length());
icyLineBuf.setLength( tmp.length());
if (i == -1 && count == 0) {
// The server just dropped connection on us
throw new NoHttpResponseException("The target server failed to respond");
}
cursor = new ParserCursor(0, this.icyLineBuf.length());
if (lineParser.hasProtocolVersion(this.icyLineBuf, cursor)) {
// Got one
break;
} else if (i == -1 || count >= this.icyMaxGarbageLines) {
// Giving up
throw new ProtocolException("The server failed to respond with a " +
"valid HTTP response");
}
//if (this.log.isDebugEnabled()) {
// this.log.debug("Garbage in response: " + this.lineBuf.toString());
// }
count++;
} while(true);
//create the status line from the status string
final StatusLine statusline = lineParser.parseStatusLine(this.icyLineBuf, cursor);
return this.icyResponseFactory.newHttpResponse(statusline, null);
}
}
Plug in HttpClient:
Scheme http = new Scheme("http", PlainSocketFactory.getSocketFactory(), 80);
Scheme ftp = new Scheme("ftp", PlainSocketFactory.getSocketFactory(), 21);
SchemeRegistry sr = new SchemeRegistry();
sr.register(http);
sr.register(ftp);
HttpClient httpClient = new DefaultHttpClient(new IcyClientConnManager(params, sr), params);
This is still being tested but initial results are promising.
Thanks #Michael M, you can even make it simpler by subclassing the BasicLineParser instead of subclassing the DefaultResponseParser.
I've uploaded the code into a gist
To use it:
IcyGetRequest request = new IcyGetRequest(urlStr);
HttpResponse response = request.get();
int responseCode = response.getStatusLine().getStatusCode();
Create an runnable that creates socket proxy then you will be able to response with HTTP/1.0 instead of ICY , then just connect to this local socket proxy with your player
Here a modification of the solution from Michal M in case you don't like to create lots of subclasses just to configure already available HttpClient classes.
final SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
HttpClient httpClient = new DefaultHttpClient() {
#Override
protected ClientConnectionManager createClientConnectionManager() {
return new SingleClientConnManager(getParams(), schemeRegistry) {
#Override
protected ClientConnectionOperator createConnectionOperator(SchemeRegistry schreg) {
return new DefaultClientConnectionOperator(schreg) {
#Override
public OperatedClientConnection createConnection() {
return new DefaultClientConnection() {
#Override
protected HttpMessageParser createResponseParser(SessionInputBuffer buffer, HttpResponseFactory responseFactory, HttpParams params) {
return new IcyHttpResponseParser(buffer, new BasicLineParser(), responseFactory, params);
}
};
}
};
}
};
}
};
Probably there is a way to get the SchemeRegistry obsoleted if one could get hold somehow from within the DefaultHttpClient class.
Related
I am using olingo client android 4.0.0 library to communicate with my backend which has implemented OData protocol from android client.
I want to set request timeout to my olingo request.
Also I want to disable retry on connection failure.
ODataClient oDataClient = ODataClientFactory.getV4();
ODataEntityCreateRequest<ODataEntity> req = oDataClient.getCUDRequestFactory()
.getEntityCreateRequest(uri, oDataEntity);
I want to add timeout and disable connection retry for it.
After going through the code of Olingo client library came across a workaround for this.
ODataClient has a Configuration property, which has HttpClientFactory parameter. I had to create a class which extended DefaultHttpClientFactory and override it's methods to update HttpClient with timeout and retry policy changes.
Complete code is as below.
private class RequestRetryHttpClientFactory extends DefaultHttpClientFactory {
private final int HTTP_REQUEST_TIMEOUT = 2 * 60 * 1000;
#Override
public org.apache.http.impl.client.DefaultHttpClient create(HttpMethod method, URI uri) {
final HttpRequestRetryHandler myRetryHandler = new HttpRequestRetryHandler() {
#Override
public boolean retryRequest(IOException e, int i, HttpContext httpContext) {
Log.d(getClass().getSimpleName(), "RETRY REQUEST");
return false;
}
};
final DefaultHttpClient httpClient = super.create(method, uri);
HttpConnectionParams.setConnectionTimeout(httpClient.getParams(), HTTP_REQUEST_TIMEOUT);
HttpConnectionParams.setSoTimeout(httpClient.getParams(), HTTP_REQUEST_TIMEOUT);
httpClient.setHttpRequestRetryHandler(myRetryHandler);
return httpClient;
}
}
I added this to my oDataClient as bellow.
ODataClient oDataClient = ODataClientFactory.getV4();
oDataClient.getConfiguration().setHttpClientFactory(new RequestRetryHttpClientFactory());
ODataEntityCreateRequest<ODataEntity> req = oDataClient.getCUDRequestFactory()
.getEntityCreateRequest(uri, oDataEntity);
I hope this will help someone.
I'm developing a program that sends some Logs to a database.
As logs, the main idea is to save as much information as it can, I want to avoid the part if the server where I store logs is down. What I'm trying to do is to make an http request and totally ignore server response, it doesn't matter if is online or offline. What I'm done so far is to set a timeout to my request but that doesn't resolve my problem. Here is my code:
public synchronized void LOG_sendERROR(String token, String channelnumber, String type)
{
HttpResponse response = null;
HttpParams httpParameters = new BasicHttpParams();
int timeout = 3000;
HttpConnectionParams.setConnectionTimeout(httpParameters,timeout);
int timeoutsocket = 5000;
HttpConnectionParams.setSoTimeout(httpParameters, timeoutsocket);
DefaultHttpClient httpclient = new DefaultHttpClient(httpParameters);
try
{
if (ep.getString("username", "").length() == 0)
return ;
String url = Server.Log_Server + "/LOG_CHANNEL_LOGS.aspx?params=1";
log.i("LOGS", url);
HttpGet c = new HttpGet(url);
response = httpclient.execute(c);
return ;
}
catch (Exception e)
{
e.printStackTrace();
try
{
if (response != null)
response.getEntity().consumeContent();
}
catch (Exception f)
{
}
return ;
}
}
If server is down my application stuck for 3 seconds. I want to send the logs, it doesn't matter for the client application to know whether the server saved the logs the client just send. How can I make a http request and ignore response?
After i tried many methods, the best way is to execute as a parallel task. In this way client wont ignore the response of the server but at least the application will be running normaly.
private Runnable LOG_CHANNEL_LOGS = new Runnable()
{
public void run()
{
new LOG_CHANNEL_LOGS().executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR);
}
};
I have this code so far:
private class DownloadWebPageTask extends AsyncTask<String, Void, String>
{
#Override
protected String doInBackground(String... theParams)
{
String myUrl = theParams[0];
String myEmail = theParams[1];
String myPassword = theParams[2];
HttpPost post = new HttpPost(myUrl);
post.addHeader("Authorization","Basic "+ Base64.encodeToString((myEmail+":"+myPassword).getBytes(), 0 ));
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String response = null;
try
{
response = client.execute(post, responseHandler);
InputStream content = execute.getEntity().getContent();
BufferedReader buffer = new BufferedReader(
new InputStreamReader(content));
String s = "";
while ((s = buffer.readLine()) != null)
{
response += s;
}
}
catch (Exception e)
{
e.printStackTrace();
}
return response;
}
#Override
protected void onPostExecute(String result)
{
}
}
This code does not compile because I am running into confusion at the point of:
response = client.execute(post, responseHandler);
InputStream content = execute.getEntity().getContent();
I got that code from tinkering with various examples, and not sure what Object the client is supposed to be, and whether the first line will just get me the server response, or I have to go the route of getting the InputStream and reading the server response in?
Please help me understand how to do this correctly.
Thank you!
I have managed to use Digest authentication using OkHttp. In this code sample I also use Dagger and Robospice-retrofit. What I did was creating an OkHttp Authenticator and assign it to my custom OkHttp client.
The authenticator class implements an authenticate method that will be called whenever the server encounters a 401 error and expects an Authorization header back (if it expects Proxy-Authorization you should implement the authenticateProxy method.
What it basically does is wrapping calls to the HttpClient DigestScheme and make it usable for OkHttp. Currently it does not increase the nc counter. This could cause problems with your server as it could be interpreted as a replay attack.
public class DigestAuthenticator implements com.squareup.okhttp.Authenticator {
#Inject DigestScheme mDigestScheme;
#Inject org.apache.http.auth.Credentials mCredentials;
#Override
public Request authenticate(Proxy proxy, Response response) throws IOException {
String authHeader = buildAuthorizationHeader(response);
if (authHeader == null) {
return null;
}
return response.request().newBuilder().addHeader("Authorization", authHeader).build();
}
#Override
public Request authenticateProxy(Proxy proxy, Response response) throws IOException {
return null;
}
private String buildAuthorizationHeader(Response response) throws IOException {
processChallenge("WWW-Authenticate", response.header("WWW-Authenticate"));
return generateDigestHeader(response);
}
private void processChallenge(String headerName, String headerValue) {
try {
mDigestScheme.processChallenge(new BasicHeader(headerName, headerValue));
} catch (MalformedChallengeException e) {
Timber.e(e, "Error processing header " + headerName + " for DIGEST authentication.");
}
}
private String generateDigestHeader(Response response) throws IOException {
org.apache.http.HttpRequest request = new BasicHttpRequest(
response.request().method(),
response.request().uri().toString()
);
try {
return mDigestScheme.authenticate(mCredentials, request).getValue();
} catch (AuthenticationException e) {
Timber.e(e, "Error generating DIGEST auth header.");
return null;
}
}
}
The authenticator will then be used in an OkHttpClient built with a provider:
public class CustomClientProvider implements Client.Provider {
#Inject DigestAuthenticator mDigestAuthenticator;
#Override
public Client get() {
OkHttpClient client = new OkHttpClient();
client.setAuthenticator(mDigestAuthenticator);
return new OkClient(client);
}
}
Finally the client is set to the RetrofitRobospice server in the function createRestAdapterBuilder:
public class ApiRetrofitSpiceService extends RetrofitJackson2SpiceService {
#Inject Client.Provider mClientProvider;
#Override
public void onCreate() {
App.get(this).inject(this);
super.onCreate();
addRetrofitInterface(NotificationRestInterface.class);
}
#Override
protected String getServerUrl() {
return Constants.Url.BASE;
}
#Override
protected RestAdapter.Builder createRestAdapterBuilder() {
return super.createRestAdapterBuilder()
.setClient(mClientProvider.get());
}
}
You might want to switch to HttpURLConnection. According to this article its API is simpler than HttpClient's and it's better supported on Android. If you do choose to go with HttpURLConnection, authenticating is pretty simple:
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("username", "password".toCharArray());
}
});
After that, continue using HttpURLConnection as usual. A simple example:
final URL url = new URL("http://example.com/");
final HttpURLConnection conn = (HttpURLConnection) url.openConnection();
final InputStream is = conn.getInputStream();
final byte[] buffer = new byte[8196];
int readCount;
final StringBuilder builder = new StringBuilder();
while ((readCount = is.read(buffer)) > -1) {
builder.append(new String(buffer, 0, readCount));
}
final String response = builder.toString();
The version of Apache's HttpClient shipped with Android is based on an old, pre-BETA version of HttpClient. Google has long recommended against using it and removed it in Android 6.0. Google's replacement HttpURLConnection does not support HTTP digest authentication, only basic.
This leaves you with a few options, including:
Migrate to HttpURLConnection (as Google recommends) and use a library, bare-bones-digest, for digest authentication. Example below.
Use the OkHttp library instead of HttpURLConnection or HttpClient. OkHttp does not support digest out of the box, but there's a library okhttp-digest that implements a digest authenticator. Example below.
Continue using the (deprecated) HttpClient by explicitly adding the 'org.apache.http.legacy' library to your build, as mentioned in the changelist for Android 6.0.
There is an Apache project for porting newer versions of HttpClient to Android, but the project has been discontinued. Read more on Apache's page on HttpClient for Android.
Implement HTTP digest yourself.
Here is a verbose example of how to authenticate a request using bare-bones-digest and HttpURLConnection (copied from the project's github page):
// Step 1. Create the connection
URL url = new URL("http://httpbin.org/digest-auth/auth/user/passwd");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
// Step 2. Make the request and check to see if the response contains
// an authorization challenge
if (connection.getResponseCode() == HttpURLConnection.HTTP_UNAUTHORIZED) {
// Step 3. Create a authentication object from the challenge...
DigestAuthentication auth = DigestAuthentication.fromResponse(connection);
// ...with correct credentials
auth.username("user").password("passwd");
// Step 4 (Optional). Check if the challenge was a digest
// challenge of a supported type
if (!auth.canRespond()) {
// No digest challenge or a challenge of an unsupported
// type - do something else or fail
return;
}
// Step 5. Create a new connection, identical to the original
// one..
connection = (HttpURLConnection) url.openConnection();
// ...and set the Authorization header on the request, with the
// challenge response
connection.setRequestProperty(
DigestChallengeResponse.HTTP_HEADER_AUTHORIZATION,
auth.getAuthorizationForRequest("GET", connection.getURL().getPath()));
}
Here is an example using OkHttp and okhttp-digest (copied from the okhttp-digest page):
client = new OkHttpClient();
final DigestAuthenticator authenticator = new DigestAuthenticator(new Credentials("username", "pass"));
final Map<String, CachingAuthenticator> authCache = new ConcurrentHashMap<>();
client.interceptors().add(new AuthenticationCacheInterceptor(authCache));
client.setAuthenticator(new CachingAuthenticatorDecorator(authenticator, authCache));
Request request = new Request.Builder()
.url(url);
.get()
.build();
Response response = client.newCall(request).execute();
I have the following problem:
My application needs to inform a website of a change as fast as possible
This can happen so fast that the previous webrequest hasn't been completely dealt with
The application should always send at least the last webrequest (it doesn't matter if previous ones are lost).
I'm not sure how to do this optimally. My current method is below, but gives me a warning ("Invalid use of SingleClientConnManager: connection still allocated."). I suspect I can reuse this connection, but have no clue how. Using threadSafeConnManager doesn't seem to be the solution, since I only need one connection (I think :) ).
How should I optimize my code for my needs?
The runnable in the code below is in a thread (webThread) and webrequest is a global variable that gets set to a certain url. After setting the variable, webThread.run() is fired.
private Runnable mSyncInternet = new Runnable() {
HttpClient httpClient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
public void run() {
HttpGet httpGet = new HttpGet(webrequest);
try {
HttpResponse response = httpClient.execute(httpGet, localContext);
} catch (ClientProtocolException e) {
sendMessageToUI(MSG_NO_INTERNET, 1);
} catch (IOException e) {
sendMessageToUI(MSG_NO_INTERNET, 1);
}
}
};
Thanks so much in advance!
Just today I found a blog with the solution to the SingleClientConnManager issue:
public static DefaultHttpClient getThreadSafeClient() {
DefaultHttpClient client = new DefaultHttpClient();
ClientConnectionManager mgr = client.getConnectionManager();
HttpParams params = client.getParams();
client = new DefaultHttpClient(new ThreadSafeClientConnManager(params,
mgr.getSchemeRegistry()), params);
return client;
}
I tried it, and it worked perfectly!
I have an Android app that sends commands to a robot via HTTP. (The robot server is written in Python using the BaseHttpServer class.) I'm setting up my connection in the app correctly as far as I can tell, but most requests fail to return and if they do there is a very long delay. Here is the initialization code:
private void setupHttpStuff() {
HttpParams params = new BasicHttpParams();
ConnManagerParams.setMaxTotalConnections(params, 10);
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(
new Scheme("http", PlainSocketFactory.getSocketFactory(), 1504));
ClientConnectionManager cm = new ThreadSafeClientConnManager(params, schemeRegistry);
this.HttpClient = new DefaultHttpClient(cm, params);
}
The GET request takes place in an AsyncTask and the code looks like:
HttpContext localContext = new BasicHttpContext();
HttpGet httpGet = new HttpGet(url);
mClient.execute(httpGet, localContext);
Where mClient is a variable shared from the Activity.
By using Log.d messages I've determined that the first two requests work fine but then all subsequent requests (in separate AsyncTasks) hang and never return from execute. I've also tried to manually create a Socket and send a HttpRequest:
Socket socket = new Socket("192.168.1.45", 1504);
HttpParams params = new BasicHttpParams();
DefaultHttpClientConnection conn = new DefaultHttpClientConnection();
conn.bind(socket, params);
HttpRequest request = new BasicHttpRequest("GET", "/");
conn.sendRequestHeader(request);
HttpResponse response = conn.receiveResponseHeader(); // Hangs here
conn.receiveResponseEntity(response);
socket.close();
But this hangs on the conn.receiveResponseHeader() line and the server never sees the request.
The Python server works fine with requests from a browser, just not with my Android app. I've tried this single-threaded as well in the main UI but the same effect occurs.
EDIT
Here is the Python code that causes the problems:
class MyHand(BaseHTTPServer.BaseHTTPRequestHandler):
def do_GET(self):
qv = parse_qs(get_qs(urlparse(self.path)))
if("method" not in qv):
self.send_nack()
elif(qv["method"][0] == "drive"):
createbot.Drive(int(qv["velocity"][0]), int(qv["radius"][0]))
self.send_ack()
else:
self.send_nack()
def send_ack(self, content_type='text/html'):
self.send_response(200)
self.send_header('Content-type', content_type)
self.end_headers()
def send_nack(self):
self.send_response(500)
self.send_header('Content-type', 'text/html')
self.end_headers()
This may be related to keep-alive. Android sends Connection: Keep-Alive and Python sends back a Connection: close. I've looked at the HTTP headers that are being sent from Firefox and HttpClient and there is no difference (other than Firefox sending some additional Accept headers). Somehow Android is expecting the Python server to send something other than HTTP/1.1 200 OK\nContent-type: text/html and therefore keeping the socket open. I ended up writing my own basic HTTP server and client.
Python code:
class SingleTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
data = self.request.recv(1024)
parts = data.rsplit(" ")
if (parts[0] != "GET"):
self.request.close()
return
url = parts[1].split("?")[1]
qv = parse_qs(url)
...
Android:
Socket socket = null;
PrintWriter output = null;
try {
socket = new Socket(this.ipAddress, this.portNo);
output = new PrintWriter(new BufferedWriter
(new OutputStreamWriter(socket.getOutputStream())), true);
output.println("GET " + url);
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if (socket != null)
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
if (output != null)
output.close();
}