I've set up a bidirectional stream construct in an Android app, where I am currently using the mechanism to send chunks of large files. The problem I am having is that my app will receive a request message for a file, then I'll respond with potentially hundreds of MBs worth of response GRPC messages, frequently causing an OOM. Pseudo code:
public class Myclass implements StreamObserver<CameraRequest>, Closeable {
...
public void onNext(Request req) {
for (Chunk chunk : getChunks(req))
this.requestObserver.onNext(builder.setChunk(chunk).build());
}
...
}
Is there some good way to rate limit the number of outstanding calls to onNext based on what has actually been put on the wire (and corresponding memory made freeable)? IE only allowing 10 calls to onNext to be made, then subsequent ones block until the data for the preceding calls has successfully been sent by the underlying protocol stack? I could implement a full e2e acknowledgement window in my wire protocol TCP style, but was hoping there was an easier/built in technique others were using.
Thanks!
Cast requestObserver to ClientCallStreamObserver. You can then call clientCallStreamObserver.isReady() to check if you should stop sending.
You will then need notifications for when the RPC is ready for more messages, to resume sending. For that, implement ClientResponseObserver and call clientCallStreamObserver.setOnReadyHandler(Runnable) within beforeStart().
Putting that all together, gets you something like:
public class MyClass implements
ClientResponseObserver<CameraRequest,CameraResponse> {
private ClientCallStreamObserver<CameraRequest> requestObserver;
private Iterable<Chunk> chunks;
public void beforeStart(ClientCallStreamObserver<CameraRequest> requestObserver) {
this.requestObserver = requestObserver;
requestObserver.setOnReadyHandler(MyClass::drain);
}
public void onNext(CameraRequest req) {
// I don't know if this assert valid for your protocol
assert chunks == null || !chunks.hasNext();
chunks = getChunks(req);
drain();
}
public void drain() {
while (requestObserver.isReady() && chunks.hasNext()) {
Chunk chunk = chunks.next();
requestObserver.onNext(builder.setChunk(chunk).build());
}
}
...
}
You can check out the flow control example here.
Related
Where is the documentation/sample for all overloads of invokeApi function for Azure Mobile Service client SDK for Android?
I found this article and tried following code, which does not work. There are no compile time or run time errors, invokeApi gets called, but it does not come back to onSuccess or onFailure. If I call invokeApi without order object, everything works as expected
PizzaOrder order = new PizzaOrder();
order.Size = "Large";
order.Flavor = "Four cheeses";
order.UserPhone = "555-555-1234";
ListenableFuture<PizzaOrderResponse> testresult = mClient.invokeApi("bookservice", order, PizzaOrderResponse.class);
Futures.addCallback(testresult, new FutureCallback<PizzaOrderResponse>() {
#Override
public void onFailure(Throwable exc) {
// failure handling code here
}
#Override
public void onSuccess(PizzaOrderResponse testresult) {
// success handling code here
}
});
One of the properties in the data object being returned by the custom API had incorrect data type. I am still not sure where the good documentation is and why custom API call did not fail but at least it is working now.
Friends!
I'm getting occasional and unexpected HTTP 400 responses from nanohttpd in my Android app. The error is following a specific pattern. I've been looking at this for some time now but I've come to the point where I need a different angle or some other help pointing me in the right direction.
Could you please have a look and share your thoughts or even direct points and suggestions?
Why am I getting this HTTP 400 status code?
And why only under the given circumstances? (I don't want it at all!)
Some Background
I'm running nanohttpd in my Android project as a temporary isolation layer (due to server side not being mature enough yet). I have isolated the nanohttpd server in an Android Service, which I start from my custom Application object once it's created. This way nanohttpd is not bound to the lifecycle of any particular Activity but can live rather independent of the overall application logic and component life cycles.
The Problem
Now, (almost) everything is working nice and dandy: I can start nanohttpd and perform some initial login requests, my expected mock response is even delivered. When I perform my first "GET" request, though, nanohttpd throws a 400 Bad request status at me, but only the first time. If I back out of the Activity being responsible for the particular "GET" request, and launch it again (from the home screen), it delivers the expected payload with a 200 status, flawlessly.
What Have I Done So Far
I have had a closer look at the nanohttpd source code, trying to track down where and why this 400 status is set. It's not that many places this status code is used. Roughly speaking only here, here and here. Since I'm not dealing with multipart content, I'm left with the first and third "here". But - of course - I can not for my life find neither the root cause of the 400 status, nor which exact block is causing the state for me. When I debug the code, everything works just peachy.
Some Code
This is roughly what my nanohttpd Service (MyNanoHttpdService) looks like:
#Override
public int onStartCommand(Intent intent, int flags, int startId) {
if (ACTION_START.equals(intent.getAction())) {
String errorMessage = null;
if (myNanoHttpd == null) {
String hostUrl = intent.getStringExtra(EXTRA_HOST);
Uri uri = Utils.notEmpty(hostUrl) ? Uri.parse(hostUrl) : Uri.EMPTY;
myNanoHttpd = new MyNanoHttpd(this, uri.getHost(), uri.getPort(), null);
}
if (!myNanoHttpd.isAlive()) {
try {
myNanoHttpd.start();
} catch (IOException e) {
StringWriter stringWriter = new StringWriter();
PrintWriter printWriter = new PrintWriter(stringWriter);
e.printStackTrace(printWriter);
errorMessage = stringWriter.toString();
stopSelf();
}
}
final ResultReceiver resultReceiver = intent.getParcelableExtra(EXTRA_RESULT_LISTENER);
if (resultReceiver != null) {
int status = myNanoHttpd.isAlive() ? CODE_SUCCESS : CODE_FAILURE;
Bundle bundle = new Bundle();
bundle.putString(EXTRA_MESSAGE, errorMessage);
resultReceiver.send(status, bundle);
}
}
return Service.START_STICKY;
}
And this is how I start the service from my custom Application object, initialize my client side state and fetch some content:
#Override
public void onCreate() {
super.onCreate();
// Yes, that is a Java 8 Lambda you see there!
MyNanoHttpdService
.start(this, "http://localhost:8080")
.withStartupListener((status, message) -> {
if (status == 0) {
// POST REQUEST: Works like a charm
myNetworkHelper.login();
// GET REQUEST: Always fails on first launch
myNetworkHelper.getContent();
} else {
Log.e("LOG_TAG", "Couldn't start MyNanoHttpd: " + message);
}
});
}
It's safe to assume that the wrapping convenience code (the .withStartupListener(...) - which essentially wraps a ResultReceiver used by the above Service - and the myNetworkHelper object) works as expected. Also, in production, the getContent() call would be made from an Activity or Fragment, but for the sake ease I have moved it to the Application for now.
I may have found the root cause for my issue, and possibly even a workaround for the moment.
If I'm correct in my investigation, the issue was caused by unconsumed data from a previous (POST) request, contaminating the current (POST) request.
This line in the NanoHTTPD code base (the header parsing block in the NanoHTTPD.HTTPSession.execute() method, just before calling through to any custom serve(...) method - the third "here" in my question above) was the very line where the HTTP 400 status code was thrown, and just as the code suggests, there was no proper value for the "method" header.
The value - which I expected to be "POST" in clear text - was contaminated with parts of the JSON content body from the previous request. As soon as I realized this, I tried to consume the entire request body in my custom MyNanoHttpd.serve(IHTTPSession session) method, like so:
#Override
public Response serve(IHTTPSesion session) {
InputStream inputStream = session.getInputStream();
inputStream.skip(inputStream.available());
// or
// inputStream.skip(Long.MAX_VALUE);
// or even
// inputStream.close();
...
}
This didn't work, though, as I kept getting various exceptions. I ended up gently modifying the NanoHTTPD code, safely closing the input stream in the finally block of the very NanoHTTPD.HTTPSession.execute() method instead.
I'm, nonetheless, considering reaching out to the NanoHTTPD community to discuss a suitable and sustainable solution.
I'm develop an android app that communicates with the server using a protocol WebSocket. I use AsyncAndroid lib. WebSocket has listener, that receive data from server.
Something like this
private WebSocket.StringCallback mStringCallback = new WebSocket.StringCallback() {
#Override
public void onStringAvailable(String s) {
}
};
I need write api this, and i want use rxjava. But i cannot imagine, that should be Observable, and that Observer. i tried to do Observable string received from server into WebSocket listener, but i think it is bad idea.
Are there any ideas how correctly build api.
Thank u.
You could wrap WebSocket into a new interface like this:
interface RxWebSocket {
void send(byte[]); // send data
Observable<String> stringObservable(); // receive string data
}
Then in a RxWebSocket implementation you can use PublishSubject with an internal WebSocket.StringCallback to publish the Strings.
I'm trying to use rx-java on Android to do few sequential http requests, each of which is dependent of the response of the former one.
This does not quite fit the map() / doFinall() model and so I'm not sure what would be the best way to do this without getting into "callback hell" as well as writing concise code.
More concretely:
do http GET "/x"
do http GET "/y" if (2) was successfully
do calculation on the result of GET /y
Any suggestions on how to go about this?
I think flatMap is what you're looking for. For example, assuming you have the following methods:
Observable<Foo> getFoo();
Observable<Bar> getBar(Foo foo); //needs a Foo first
You could effectively chain them this way:
getFoo().flatMap(new Func1<Foo, Observable<Bar>>() {
#Override
public Observable<Bar> call(Foo foo) {
return getBar(foo);
}
});
You could then perform some calculation with the final result Bar by subscribing to the resulting Observable<Bar> (full example shown for clarity):
getFoo().flatMap(new Func1<Foo, Observable<Bar>>() {
#Override
public Observable<Bar> call(Foo foo) {
return getBar(foo);
}
}).subscribe(new Action1<Bar>() {
#Override
public void call(Bar bar) {
//everything succeeded, so perform calculation to the Bar
}
}, new Action1<Throwable>() {
#Override
public void call(Throwable throwable) {
//handle an error that occurred anywhere in the chain
}
});
Note that an error anywhere in the process of getting the Foo or the Bar will be handled by the Action1 that we provide when subscribing to the Observable. It is, of course, painfully verbose because Java, but at least it avoids nesting Observables/callback hell.
I'm using the In App Billing sample app to add this feature to my application.
After I finished adding it to my app, and tested all working, I noticed the comment in this Security class:
Security-related methods. For a secure implementation, all of
this code should be implemented on a server that communicates with
the application on the device. For the sake of simplicity and
clarity of this example, this code is included here and is executed
on the device. If you must verify the purchases on the phone, you
should obfuscate this code to make it harder for an attacker to
replace the code with stubs that treat all purchases as verified.
As Google suggests, I do the purchase verification on the server side so I really don't need the Security class in my project.
The problem is, I can't figure out how to remove the BillingService class dependency in the Security class.
I started by deleting the Security class and following the errors in the BillingService and most places it's being used I can remove easily, except in one place:
private void purchaseStateChanged(int startId, String signedData, String signature) {
ArrayList<Security.VerifiedPurchase> purchases;
purchases = Security.verifyPurchase(signedData, signature);
if (purchases == null) {
return;
}
ArrayList<String> notifyList = new ArrayList<String>();
for (VerifiedPurchase vp : purchases) {
if (vp.notificationId != null) {
notifyList.add(vp.notificationId);
}
ResponseHandler.purchaseResponse(this, vp.purchaseState, vp.productId,
vp.orderId, vp.purchaseTime, vp.developerPayload);
}
if (!notifyList.isEmpty()) {
String[] notifyIds = notifyList.toArray(new String[notifyList.size()]);
confirmNotifications(startId, notifyIds);
}
}
Would love if someone can share his/hers purchaseStateChanged method (based on the in app billing sample app) without the use of the Security class.
So here's what I did. First the calls to BillingService occur on the applications main thread, so you need to issue your server calls in a background thread. I chose to finish up processing on the main thread, since I wasn't sure what impact calling methods like 'confirmNotifications' on a background thread might have.
I created a callback interface VerifyTransactionCompletion which could be dispatched back to the main thread after the remote call completed.
I keep around the Security class and have it manage the call to the server now, instead of what it originally performed in the sample. So when you see the call to Security, that's where I call out to my server and perform signature validation.
/**
* Callback interface to <em>finish</em> processing a transaction once the remote
* servers have processed it.
*/
public interface VerifyTransactionCompletion {
public void transactionVerified(List<Security.VerifiedPurchase> purchases);
}
private void purchaseStateChanged(final int startId, String signedData, String signature) {
// verifyPurchase issues remote call to server (in a background thread), then
// calls transactionVerified on the main thread to continue processing.
Security.verifyPurchase(signedData, signature, new VerifyTransactionCompletion() {
#Override
public void transactionVerified(List<VerifiedPurchase> purchases) {
if (purchases == null) {
return;
}
ArrayList<String> notifyList = new ArrayList<String>();
for (VerifiedPurchase vp : purchases) {
if (vp.notificationId != null) {
notifyList.add(vp.notificationId);
}
ResponseHandler.purchaseResponse(BillingService.this, vp.purchaseState, vp.productId,
vp.orderId, vp.purchaseTime, vp.developerPayload);
}
if (!notifyList.isEmpty()) {
String[] notifyIds = notifyList.toArray(new String[notifyList.size()]);
confirmNotifications(startId, notifyIds);
}
}
});
}