Develop Reference

AWS DynamoDB Scan failing on iOS but works on Android

I am using DynamoDB as back-end database for my mobile app, and the schema etc are identical across Android & iOS. For a particular use-case, I have to perform a Scan, based on two attributes which are not indexed. For iOS Objective C, I am using the following code:
AWSDynamoDBScanExpression *scanExpression = [AWSDynamoDBScanExpression new];
scanExpression.limit = [NSNumber numberWithInt:maxCount];
scanExpression.filterExpression = #"#l = :location AND event = :event";
scanExpression.expressionAttributeNames = #{#"#l":#"location"};
scanExpression.expressionAttributeValues = #{#":location":location,
#":event":EVENT_TASTING};
Both location and event are Strings. EVENT_TASTING is a String constant. This scan keeps returning zero results, even though I have validated that for the provided entries I should be receiving the results. I use the following code in Android Java:
DynamoDBScanExpression scanExpression = new DynamoDBScanExpression();
scanExpression.setLimit(maxCount);
scanExpression.addFilterCondition("location",
new Condition()
.withComparisonOperator(ComparisonOperator.EQ)
.withAttributeValueList(new AttributeValue().withS(location)));
scanExpression.addFilterCondition("event",
new Condition()
.withComparisonOperator(ComparisonOperator.EQ)
.withAttributeValueList(new AttributeValue().withS(Constants.EVENT_TASTING)));
The scan works as expected in Android. What needs to change in iOS to make it work there too? I updated iOS SDK to 2.3.6 but it has not made a difference. This is the only scan operation I am doing in my code.
Is there an error in my scanExpression for iOS? Is there a way I can use the Android-style syntax to make this work on iOS?
Update
I tried the following changes:
AWSDynamoDBScanExpression *scanExpression = [AWSDynamoDBScanExpression new];
AWSDynamoDBAttributeValue *locationVal = [AWSDynamoDBAttributeValue new];
locationVal.S = location;
AWSDynamoDBAttributeValue *eventVal = [AWSDynamoDBAttributeValue new];
eventVal.S = EVENT_TASTING;
scanExpression.limit = [NSNumber numberWithInt:maxCount];
scanExpression.filterExpression = #"#l = :location AND event = :event";
scanExpression.expressionAttributeNames = #{#"#l":#"location"};
scanExpression.expressionAttributeValues = #{#":location":locationVal,
#":event":eventVal};
But now I am getting an error:
The request failed. Error: [Error Domain=com.amazonaws.AWSDynamoDBErrorDomain Code=0 "(null)" UserInfo={message=ExpressionAttributeValues contains invalid value: Supplied AttributeValue is empty, must contain exactly one of the supported datatypes for key :location, __type=com.amazon.coral.validate#ValidationException}]

Thanks to the hint from #YosukeMatsuda, I was able to fix this by calling Scan repeatedly until LastEvaluatedKey is empty. I am posting this as answer because unfortunately Mike's answer is not pointing out the correct issue and is misleading.
Here's how I changed the code in iOS:
// In a different method (for first call):
AWSDynamoDBScanExpression *scanExpression = // See code in original question
// In a new method that can be called recursively:
// DynamoDBObjectMapper scan:class-for-model expression:scanExpression
// continueWithBlock -> if (task.result):
AWSDynamoDBPaginatedOutput *paginatedOutput = task.result;
if (paginatedOutput.items.count != 0)
// Append the paginatedOutput.items to the cumulative array
else
// Replace the cumulative array with paginatedOutput.items
if (paginatedOutput.lastEvaluatedKey.count == 0) {
// Scan is complete - handle results
} else {
// Check if you have sufficient results
// In my case I had asked for 25 results but was getting 39
// So it doesn't seem to obey the scanExpression.limit value
// If more results are needed, continue the scan
[scanExpression setExclusiveStartKey:paginatedOutput.lastEvaluatedKey];
// Call this method recursively
}
If there is a more elegant solution I'd love to hear it. But at least it works now.

There are several differences between the Android code you're using and the ObjectiveC version.
in the Android Version you're using the older Filter Condition API while in the ObjectiveC you're using the more modern Filter Expression API; this doesn't necessarily make the newer one fail but it's just something to point out
in the case of ExpressionAttributeValues, the values for location and event that you're passing in should be of type AWSDynamoDBAttributeValue *, not strings; if you make this change your query will most likely start working.
I hope this answers your question but can't be certain because you only say "this works as expected in Android - how can I make it work in iOS" but you're not telling us what's broken.

Android Logcat very strange behavior when getting empty string

So I came across something strange that made me loose some time. I have been trying to print the content of an ArrayList containing string elements, sometimes, an element might contain an empty string, which is fine and absolutely my intention.
So I have something like this:
List<String> l = new ArrayList<String>();
//adding strings in l, sometimes it's an empty string
for (int i=0; i < l.size(); i++) {
Log.w("element in l : ", l.get(i));
}
So here, when the loop is gonna hit the empty string, logcat is simply NOT going to print it out BUT (and here is the root of my confusion), if you have a message following the one that failed to display, suddenly the failed message is going to show up as if it contained the new logcat message. For example if you try logging an empty string like this
Log.w(TAG, <empty string here>);
Logcat is going to output nothing at first, then, when it has a NEW message to display this is what it prints out (in this case the new message is some warning about AudioTrack):
08-21 17:06:02.265 13047-13047/company.myapp W/TAG﹕ [ 08-21 17:06:05.411 766: 937 W/AudioTrack ]
AUDIO_OUTPUT_FLAG_FAST denied by client
I'm interested in knowing how this happens, maybe it can help someone else not getting super confused like I did. I suppose trying to log an empty string triggers some kind of buffer that sits there until it gets something to print, is this a bug?

That is an interesting question. I just tried this in LogRabbit and am able to see the same result.
I took a quick browse through the android source and see that Log.W(...) ends up in native code and getting handled in logd_write.c
This basically writes the data to /dev/log/main (or one of the other logs)
You can get those logs like this:
adb pull /dev/log/events .
adb pull /dev/log/main .
adb pull /dev/log/radio .
adb pull /dev/log/system .
You will need to press cntl-C otherwise the copy will happen forever.
Looking in the raw log in /dev/log/main I see the message does get logged:
<8b>F×U^_<8c>^Y^U^Emfl_MessageList^#Before Empty^#^R^#^#^#!z^#^#!z^#^#
<8b>F×U^_<8c>^Y^U^Emfl_MessageList^#^#^]^#^#^#!z^#^#!z^#^#
<8b>F×U^_ <8c>^Y^U^Emfl_MessageList^#After Empty^#7^#^#^#^#^E^#^#^Z^E^#^#
That gets decoded by struct found in logger.h So I think this is a problem in adb. pull the source code from here: (looks like quite a few of undocumented commands there)
This is the primary function
static int logcat(TransportType transport, const char* serial, int argc, const char** argv) {
char* log_tags = getenv("ANDROID_LOG_TAGS");
std::string quoted = escape_arg(log_tags == nullptr ? "" : log_tags);
std::string cmd = "shell:export ANDROID_LOG_TAGS=\"" + quoted + "\"; exec logcat";
if (!strcmp(argv[0], "longcat")) {
cmd += " -v long";
}
--argc;
++argv;
while (argc-- > 0) {
cmd += " " + escape_arg(*argv++);
}
return send_shell_command(transport, serial, cmd);
}
Looking in there I see that all logcat does is basically this:
adb shell
> exec logcat
So I think the root of the problem is in logcat itself. Logcat.cpp calls into log_read.c
Based on my quick read through things what I think is happening is the message is not terminated properly. The empty message does not show up until another message is appended and the first message overruns and shows the second message because it has the appropriate termination.

error with pymtp to work on python 3

I want to access a android device from python to download some photos.
libmtp works from the CLI.
Than pymtp. It's been around for a while but it's designed for python 2 and i'm using python 3. Meanwhile fixed several minor issues but i'm stuck at an error from function get_filelisting
specially this section:
ret = []
next = files
while next:
ret.append(next.contents)
if (next(next.contents) is None):
break
next = next(next.contents)
The error is related to the "next".
That section looks strange to me, i've been coding in python for a while but i'm new to ctypes. Tried a lot of variants, they all failed. The "next" could be confusing with python buildin function so i renamed it to nextpointer and came to this code:
ret = []
nextpointer = files
while nextpointer:
ret.append(nextpointer.contents)
nextpointer = nextpointer.contents.next
It seems to work but did it work by accident ? does it have any design flaws ? Could anyone with experience on python ctypes confirm this a solution ? Any suggestion welcome.

From python2.7 documentation
next(iterator[, default])
Retrieve the next item from the iterator by calling its next() method. If default is given, it is returned if the iterator is
exhausted, otherwise StopIteration is raised.
from python3 documentation
next(iterator[, default])
Retrieve the next item from the iterator by calling its __next__() method. If default is given, it is returned if the iterator is
exhausted, otherwise StopIteration is raised.
Notice that next() method was removed from python3 but the function still exists.
This is all I can say about the next function and .next()/__next__() methods.
I downloaded the pymtp module and get_filelisting() is slightly different from what you posted in your ported code, here it is:
ret = []
next = files
while next:
ret.append(next.contents)
if (next.contents.next == None):
break
next = next.contents.next
If none of this helped you (which probably didn't :D), the version of pymtp library that I am using is 0.0.6 download using pip.

Android Extract url with specific domain name from String

I am developing a JSON application. I am able to download all of the data but I'm running into an interesting issue. I am trying to grab a string with the domain name:
http://www.prindlepost.org/
When grabbing all of the JSON, I get an extremely large string which I am unable to paste in there. The part I am trying to parse out is:
<p>The road through Belgrade was quiet at 4 A.M. Besides the occasional whir of another car speeding by, my taxi was largely alone on the road. Through the windshield I could see the last traces of apartment blocks pass by as we left the outskirts of the city. Somewhere beyond the limits of my vision, I knew the airport waited, its converging neon runway lines already lighting up the pre-dawn darkness.</p>
<div class="more-link-wrap wpb_button"> Read more</div>
where I am focusing on:
Read more</div>
I'm unfamiliar with extracting strings like this. In the end, I want to be able to save the URL as its own string. For example, the above would be converted into:
String url = "http://www.prindlepost.org/2015/06/this-is-a-self-portrait/";
One thing to note, there are A LOT of URLs to narrowing down by class name may help me a bunch.
My initial guess was:
// <READ MORE>
Pattern p = Pattern.compile("href=\"(.*?)\"");
Matcher m = p.matcher(content);
String urlTemp = null;
if (m.find()) {
urlTemp = m.group(1); // this variable should contain the link URL
}
Log.d("LINK WITHIN TEXT", ""+urlTemp);
// </READ MORE>
Any help is appreciated!

It may be work trying to use something like: http://jsoup.org/
If you check out their example for parsing out links:
String html = "<p>The road through Belgrade was quiet at 4 A.M. Besides the occasional whir of another car speeding by, my taxi was largely alone on the road. Through the windshield I could see the last traces of apartment blocks pass by as we left the outskirts of the city. Somewhere beyond the limits of my vision, I knew the airport waited, its converging neon runway lines already lighting up the pre-dawn darkness.</p>"
+ "<div class=\"more-link-wrap wpb_button\">"
+ "<a href=\"http://www.prindlepost.org/2015/06/this-is-a-self-portrait/\" class=\"more-link\">"
+ "Read more</a></div>";
Document doc = Jsoup.parse(html);
Element link = doc.select("a").first();
String relHref = link.attr("href"); // == "/2015/06/this-is-a-self-portrait/"
String absHref = link.attr("abs:href"); // "http://www.prindlepost.org/2015/06/this-is-a-self-portrait/"

why just after initializing the zram read is issued before write?

I am newbie to Linux kernel and just started to know how zram works. Initial testing, I am seeing that READ is issued before WRITE just after the zram is being initialized. But I am just eager to know, why this is so ?
As an activity I took the dump_stack() and followed the path form where to how this zram read is being performed.
zram get to know this info whether it has to do READ or WRITE operation on issued bio->bi_rw. Code flow is like that zram_make_request API is being called from create_device in zram driver. And zram_make_request internally called __zram_make_request which called the zram_bvec_rw API.
In zram_bvec_rw API check the available info of bio->bi_rw and correspondingly issued the READ and WRITE call.
Now, in this case what is happening: READ is being encapsulated inside bio struct itself. As triage I found that submit_bh fills all the entry of bio and issued the submit_bio.
I was wondering who is actually sets the bio->bi_rw as READ. By enabling the few prints I found that ll_rw_block API is being called by __block_write_begin with READ, later ll_rw_block calls the submit_bh API where rest of bio struct entries are filled.
But I am still not getting the answer why READ is issued for ll_rw_block from __block_write_begin ?
zram driver:
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/drivers/block/zram/zram_drv.c?id=refs/tags/v3.18.14
in file: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/fs/buffer.c?id=refs/tags/v3.18.14
if (!buffer_uptodate(bh) && !buffer_delay(bh) &&
!buffer_unwritten(bh) &&
(block_start < from || block_end > to)) {
ll_rw_block(READ, 1, &bh);
*wait_bh++=bh;
}
buffer_uptodate(bh), /* Contains valid data */
buffer_delay(bh), /* Buffer is not yet allocated on disk */
buffer_unwritten(bh), /* Buffer is allocated on disk but not written */
Please can someone give an explaination/answer to my question ?
How I am concluding that read is perfomed before write ??
I just check the num_reads and num_writes count. And num_reads count is set to 1 while num_writes is found 0 when we do mkswap /dev/block/zram0 and after calling the swapon /dev/block/zram0 the final counts are num_reads = 2 and num_writes=1.
NOTE: This is the case when we don't performing any additional zram activity. We got this behavior in case as explained above.

Because of this as far as i can see: block_start < from || block_end > to (respecting other conditions of course, buffer_uptodate() etc.) .
i.e. bio will write a whole block so if a region to be updated smaller than submited block you obviously need a fresh copy.