CursorWindow size need to grow - android

I am asking this question because some answers on StackOverflow are not the ones I am looking for.
My issue is that in early versions of Android like 2.3, the query is very slow and I get this message.
My Error Message
06:37:25.521: ERROR/CursorWindow(322): need to grow: mSize = 1048576, size = 45, freeSpace() = 43, numRows = 8928
07-12 06:37:25.521: ERROR/CursorWindow(322): not growing since there are already 8928 row(s), max size 1048576This message does not happen on Android 3.0+ for me I think, but it still takes a long time to do the query, so I am looking into using limits. It works almost flawlessly on Android 4.0+ though without limits. I am using SQLite FTS3 MATCH queries as well.
The reason I have 8928 rows is because I'm doing search suggestions and someone can input a "t" and get 8928 rows or something even larger, which freezes up Android 2.3 apparently. I cannot limit my search suggestion threshold because some results with only two characters may have 20 results or 20000+ depending on what it is.
Android SQLite and huge data sets This link is probably the best place for answers. The last answer in the given link seems promising, but there is no way to get the number of rows unless you do cursor.getCount(), which requires you to get the total number of rows from a big query (ex. 20000+ results), unless I'm thinking about it wrong. Is there some workaround to this? How would you do this in code to solve the CursorWindow problem?

Related

Kotlin , code enters if statemant even when condition is false

I'm really confused here ... looks like a really silly mistake but I don't know what's happening. Here's little snippet of my code :
if (tempDeltaDeviation > standardDeltaDeviation) {
Log.e(TAG, "handleMessage: plus $tempDeltaDeviation : $standardDeltaDeviation")
scaleUpAnimation(deltaAnimStep, "Delta", binding.deltaImg)
}
Really basic stuff, right ? Well checking the logs I can see :
handleMessage: plus 1.57756888292539E14 : 7.8364593205657E13
Dunno but last time I was checking 1 was much smaller than 7 so why app enters if statement ?
1.57756888292539E14 is in fact larger than 7.8364593205657E13.
These numbers are represented in scientific notation which is used to work with very small or very large numbers. 1.57756888292539E14 means: 1.57756888292539 * 10^14. By increasing the number after E by one we actually increase the resulting number 10 times. By increasing it by 6, we increase the resulting number million times (10^6 = 1000000).
Making things simple, your numbers are really:
157756888292539 (1.57756888292539 * 100000000000000)
78364593205657 (7.8364593205657 * 10000000000000)
As you can see, the first number is actually bigger.

how to set the Shinobicharts column width fixed in Android?

I am facing a strange issue with ShinobiCharts[Android] column width for the below scenarios:
XAxis : Date <br>
YAxis : Double
Input 1: DataAdapter= {[00:00,0.0], [02:00,360] , [24:00,0.0] }
Output : below output#1 screenshot shows expected column width wit bar.
Input 2: DataAdapter={[00,0.0], [15:150.0], [24:00,0.0]}
Output : screenshot 2 shows a bar with more length of column which does not fit inside that bounds.
Input 3: DataAdapter={[00,0.0], [23:00,360.0] , [24:00,0.0]}
Output : screenshot 3 shows even bigger and bigger and column width.
If adapter all has 3 entries for that day, then every column width showed like screenshot 1.
Question is: I just wanted to know what am i missing here, and what api should I use to restrict this column width to shown only inside that bounds where it fits?
Please help me.
private Series createSeries(#NonNull final DataAdapter<Date, Integer> dataAdapter, #NonNull final String title, final int color) {
final ColumnSeries series = new ColumnSeries();
series.setDataAdapter(dataAdapter);
series.setShownInLegend(true);
series.setTitle(title);
series.setSelectionMode(Series.SelectionMode.POINT_MULTIPLE);
ColumnSeriesStyle style = series.getStyle();
style.setAreaColor(color);
style.setAreaColorGradient(color);
style.setLineColor(color);
series.setStackId(1);
return series;
}
Here is the solution to my own question.
Problem:
shinoby-version: shinobicharts-android-premium-1.7.2-0.jar
from the below link , got to know that shinobycharts does not have api to control the width of the column (vertical bar) (scroll down and see (rippling posted this 22 January 2016)) its backlog item .
[shinobicontrols.com/forum/shinobicontrols/2013/4/how-are-the-widths-of-columns-set]
as per the attached screenshots, if there is no entry point for the first partition midnight 12- to early morning 4 [12-4] or [00-04] per a day on X-axis, the width of the column is uncontrollable.
Solution:
As a workaround or a proper fix, I just added a dummy entry in the first partition if there is no actual data. and the final adapter looks like below.
If actual data presents for the 1st partition i.e [12-4] in screenshot. no need to add dummy here:
earlier: DataAdapter= {[00:00,0.0],[02:00,360],[24:00,0.0] }
now: DataAdapter= {[00:00,0.0],[02:00,360],[24:00,0.0] }
//no change as it was earlier.
if not:
earlier: DataAdapter={[00,0.0], [15:150.0], [24:00,0.0]}
now: DataAdapter= {[00:00,0.0],[02:00,0],[15:150.0],[24:00,0.0] }
end entry [02:00, 0] is the dummy entry here. and does not mess up the output as the VALUE is set to Zero here.
Output:#
Without entry in the first partition 12-4
With entry in the first partition 12-4
I am glad that you were able to resolve your issue.
As you have seen, the data itself can have a significant impact on the width at which the columns are drawn.
When the chart calculates the width at which to draw the column, amongst other things it looks at the distance between each data value for the relevant axis. It will choose the shortest distance between two data point values and base it’s column width calculation on this. This approach usually results in evenly spaced columns with a consistent width, except when the data point values have a varied frequency. For example if most of your data points have an x value 9 hours from it’s adjacent ones, but then one pair of x values are only 2 hours apart, you may experience this type of issue. Furthermore, a DateTime axis, when used with a column or bar chart can also result in varying column or bar widths because date frequencies are often varied. For example, for monthly data, some months have 31 days whereas some have 30, 28 or 29.
In the future we may look to provide additional api methods to allow manual setting of column width. I cannot of course give any firm indication of when such api will be available. In the meantime if you still experience issues with your column chart, please do get in touch by contacting info#shinobicontrols.com. In this instance please do include a code sample where possible, to help us to get up to speed with your issue quickly.
Thanks and kind regards,
Kai
Disclaimer: I work for Shinobicontrols.

Cursor window: window is full

I've created a ListView populated by the data returned from a query.
It works, but in the LogCat I've got the message:
Cursor Window: Window is full: requested allocation 444 bytes, free space 363 bytes, window size 2097152 bytes
and it uses a couple of minutes for loading / visualizing the ListView.
My query returns about 3700 rows of String/Int/Double, each of which with 30 columns; no images or particular datatypes
What does this message exactly mean and how can I avoid it?
Can you improve performances by changing this Cursor Window?
From my experience this means that the query results are too large for the cursor's window and it requests more memory. Most times this request is honored, but on low end devices it could throw exceptions.
I don't know the specifics of the app in question but you referred to a ListView. A ListView cannot show 3700 rows at once and a endless list could help to load the data on demand
My advise is to break up the query into a multiple queries that return smaller results and close them before running the next query. After each successive query combine the results.
Short version:
After some investigation, it appears that this message is part of normal operation, and not a cause for concern. It is logged at the "Warning" level, but I think this is simply overeager.
Longer version:
This is (clearly labelled as) a "Windowed" cursor, which means that old records will be discarded as new records are obtained. In the simplest form, such a "window" implementation may contain up to N rows total, possibly with some read-ahead. In this implementation, however, the window size is defined instead by the total size. The number of rows kept in memory is instead based on how many would fit in the overall window, and will vary at runtime (This could perhaps be considered more of a "buffered" Cursor than "windowed" Cursor).
As a buffered implementation with a (soft-?)capped size, the earliest rows will be discarded only when the buffer is too full to accommodate the next row. In this case, 1 or more older rows are dropped. This "keep allocating rows as-needed until we can no longer have room for more, at which point we free up the oldest record(s) in our buffer and try again" process appears to be completely normal and expected, as a normal part of the process to keep the memory space confined.
I based this conclusion on reading the source here, combined with some inference:
https://android.googlesource.com/platform/frameworks/base/+/master/libs/androidfw/CursorWindow.cpp
Why are people talking about images and other massive LOBs?
If the size of a single row is larger than the entire "window" (buffer), then this strategy breaks down and you have an actual problem.
This was the message #op was getting:
Cursor Window: Window is full: requested allocation 444 bytes, free space 363 bytes, window size 2097152 bytes
This was the message #vovahost was getting:
CursorWindow: Window is full: requested allocation 2202504 bytes, free space 2076560 bytes, window size 2097152 bytes
In the first case, requested allocation is much smaller than the windows size. I expect that similar messages are issued repeatedly, with the same window size and varying requested allocation sizes. Each time this is printed, memory is freed from the larger window, and new allocations are made. This is normal and healthy operation.
In the second case, requested allocation size exceeds the overall window size. This is an actual problem, requiring storing and reading data in a more streamable way.
The difference is "length" (total number of rows) vs "width" (memory cost of the largest single row). The former (#tirrel's issue) is not an issue, but the latter (#vovahost's issue) is.
I also got this problem. In my case I saved a 2.2 MB image in database. When loading the data from the database using Cursor.getBlob() I would see this message in the Log:
CursorWindow: Window is full: requested allocation 2202504 bytes, free space 2076560 bytes, window size 2097152 bytes
After I would get this message if I try to retrieve any data (String, number, etc) for successive rows it is returned as null without any errors.
The solution was to remove the 2.2 MB blob. I don't know if it's possible to load bigger blobs from database in Android.
Also, note that changing the window has overhead of IPC.
So, if the cursor has large number of items and is used with a listview, fast navigation results in change of window and hence frequent IPCs. This might result in ANR if the system is loaded.

GreenDao - determine database size on runtime

I would like to know please how to get my database(which is off course *.sqlite file) size in bytes?
My current way to do it(which isn't working) is:
new File(DataManager.getInstance().db.getPath()).length()
but I'm just getting here the same number every time 53,676~ , which is irrelevant to the database's content, I'm getting this number even when it's empty.
Thank you.
OK the solution is pretty simple, my recent way to check the database file was good.
But I didn't take in account that greenDao adds to the database another 53 KB. So an empty DB size would be 53± KB and after some insertions it would get bigger and bigger.

Doing order by using the Jaro-Winkler distance algorithm?

I am wondering how would I be able to run a SQLite order by in this manner
select * from contacts order by jarowinkler(contacts.name,'john smith');
I know Android has a bottleneck with user defined functions, do I have an alternative?
Step #1: Do the query minus the ORDER BY portion
Step #2: Create a CursorWrapper that wraps your Cursor, calculates the Jaro-Winkler distance for each position, sorts the positions, then uses the sorted positions when overriding all methods that require a position (e.g., moveToPosition(), moveToNext()).
Pre calculate string lengths and add them into separate column. Then sort entired table by that that length. Add indexes (if you can). Then add extra filters for example you don't want to compare "Srivastava Brahmaputra" to "John Smith". The length are out of wack by way too much so exclude these kind of comparison by length as a percentage of the total length. So if your word is 10 characters compare it only to words with 10+-2 or 10+-3 characters.
This way you will significantly reduce the number of times this algorithm needs to run.
Typically in the vocalbulary of 100 000 entries such filters reduce the number of comparisons to about 300. Unless you are doing a full blown record linkage and then I would wonder why use Android for that. You would still need to apply probabilistic methods for that and calculate scores and this is not a job for Android (at least not for now).
Also in MS SQL Server Jaro Winkler string distance wrapped into CLR function perform much better, since SQL Server doesn't supprt arays natively and much of the processing is around arrays. So implementation in T-SQL add too much overhead, but SQL-CLR works extremely fast.

Categories

Resources