I'm developing an app in Android and I need to traverse a xml file.
I need to traverse a xml - backwards and forward from a given position. It means i start to parse the file, but at each instant i can stop and go backwards or continue.
I was thinking in using DOM, with its for cycle i could control it and do what i wanted. But the xml file that i want to parse has at least 8 Mb and since DOM is very memory intensive, don't seem to be a good solution.
A solution to this problem was not to load the whole document for parsing. Like to split the document in several parts and only load one part to memory and parse. When i come to the end of this part, i load another. The same when i want to rewind.
My question is, how can i achieve to split the file in several pieces. Since it is a xml file and the childs don't have all the same size ?
For example:
<root>
<child time="A">
<sub1>1</sub1>
<sub2>2</sub2>
</child>
<child time="B">
<sub1>3</sub1>
</child>
<child time="C">
<sub2>4</sub2>
</child>
</root>
As you can see, their childs have different sizes and i don't know how I can split a file like this in an efficient way in several parts.
Can anyone give me a clue ?
Best regards.
With XML you typically have to make a choice. DOM is memory intensive, SAX cannot go backward, and hand made parsers are tedious to create and maintain.
If you can afford consuming tens of MB of memory, go simply with DOM.
The decision between SAX and manual parsing depends on how often you actually need to go backward and whether you can afford a delay at this point.
If you cannot, you will have to implement a hand made parser with precomputation. Precomputation can be done, for example, using SAX, used in conjunction with CountingInputStream, or also manually. You would precompute starting and ending offsets of each n-th child element and store that as an array of intervals like these:
public class Interval {
public int startOffset;
public int endOffset;
}
Interval[] precomputedOffsets;
The value of n, the page size, could be something like 20. Balance that to control the tradeoff between memory consumption and performance of going back.
Now, if you know that you need to go to item i at runtime, you will call reset and skip(precomputedOffsets[i / n]) on the input stream, and hand parse of i % n remaining child elements from there.
Related
I'm just getting started and trying to make a simple app after going through some of the Google Codelabs. I wrote a Choose your own Adventure type of game. It consists of 500 paragraphs, and at the end of each paragraph is usually a choice of the next action to take (ie paragraph to load) and possibly some game logic. For now I am concentrating on the text only.
It seems simpler to make 500 text files and load up whatever file number is necessary, however it makes more sense to load ALL the paragraphs as the entire text takes up roughly 300Kb as a text file.
How would I even attempt this? I know I would need a text array and I file reader. I think it would be more work to format the paragraphs with a delineator to separate them, as it may be simpler to just create 500 files and write a While loop feeding my array.
That should fulfill my current objective, but when I want to add the choices at the end of the paragraphs, it seems as if a JSON or XML file that includes the text and choices would be better. I could not figure out how to even attempt this.
Example Paragraph: "You hear a horseman galloping and spin towards the sound. Do you have the skill of Forest Stealth? If you do, turn to 95. If you do not, turn to 234."
I would like to get a list of files whose line count >=400. How to do that?
I have tried to use search with regex options, but in vain.
Any idea on how to do this in Android Studio?
Try by installing Statistic plugin in your Android studio. So that you can view the individual and all the file information (like number of lines of code and size etc.) in one place like below.
The only way to read it is to read each file fully or at least until you reach the 400 lines. The second solution is better, because that way you don't always need to read the whole file, so it's faster.
To do that, you should use a buffer byte array and analyze each read character.
Using a regex for it is not always the best idea, because it's way slower, than checking each character manually.
So the regex for that will be something like this: ([^\n]*\n){400,}
I'm making an ListView in my app which over time could contain hundreds of items. Are there any "best" methods of loading lots of data?
My idea is to load it in chunks (say 10-20 items). Load the first chunk, then when the user is about halfway through scrolling, load the next chunk, add it to the bottom of the list (and make sure the list scroll offset doesn't jump about).
Some other ideas I had just didn't like so much were accepting the cost of a large http call and load all the data at once, but just load it in chunks as they scroll, or maybe add a "Next x items" button at the bottom, or loading all the items into the list at once and having one large list I don't need to keep track of.
I personally like my original idea, I was just wondering if there is a preferred method or doing this, and if there are any performance issues I could have.
The data in question will be a JSON string, and each item will display some title text, a date, the author of the item, and an image which will be downloaded using the Picasso library.
Your initial idea is my preferred approach because it works very well in most situations.
The second one may work well, but the problem is, the "large" data concept is relative across devices. For powerful devices you may load 2000 items at once, but it will kill older, slower phones. Also, if you're loading 2000 items when the use case of that ListView is to choose one in the first 100, you are wasting bandwith.
The first approach is very scalable: You really don't care if there are 5 items or 50 million, you just load chunks as the user consumes them. The memory usage is consistent. Coupled with ListView's view recycling, this will have a small memory footprint.
To say something positive about the second approach: Maybe in a use case when the ListView always has the same data, and it rarely changes, for example, an image library, you may want to load all the data at application start and cache it, so you never have to do network requests while the user is using the app. If the data size is not huge, I'd go for this second approach. But always having in mind that there's a critical size after which you will need to page!
Basically you can load all items once and show them all (if you will use ViewHolder pattern and lazy image loading using Picasso - everything should be ok). If you have some business logic which force you to show data by pages - you can follow the way that you described first.
If you will show all data - you can add search by list for better UX(you also can do it when you have pages but it will be more tricky).
Some time ago I wrote an article on similar topic:
http://developer.samsung.com/android/technical-docs/Batch-loading-list
The article comes also with a small library of common classes.
Maybe you will find it helpful.
Cheers,
I have been looking for a while but haven't been able to find an answer. I have a text view in my APK that needs to potentially handle large amounts of text being appended to it a little at a time. So it could grow over time as stuff streams to it. So far I haven't seen any issues and I have let the underlying Android implementation take care of the data. Does anyone know if Android caches these all in memory or that if it passes a watermark level it could then write to file for the TextView? What if it gets too huge, would the APK run out of memory and get killed by Dalvik? If that is the case I am open for any suggestion as how to mitigate this possiblity. One solution in my mind is that have a custom textview that does exactly what I explained and caches data to file if it passes a water mark. However, I am not sure how tricky it would get to detect where the user is navigating within the TextView to pull the data back and forth from the underlying data file and populate the actual TextView object.
Thanks
While I was unable to find the actual threshold, it has been discussed and suggested that extremely long text be displayed in smaller chunks. You can also write to a file which you can then read back in as necessary for your project.
Edit: regarding the TextView bookmark (for lack of a better term), you could always programmatically check to see where the user is in the TextView (checking for a certain character or string at the end of the TextView) and reading the next few lines of text into the TextView. You would have to match the getText().toString() value of the TextView against the readable text in your file.
I'm wondering now too if you would be able to do that in either direction. To save space, you could write your text to file as the user needs it, and update the TextView to only display 5 or 6 lines (arbitrary number). You could trim the TextView in either direction to save memory. You could ellipsize both sides of your text, scroll enable it, and trim/read those points as necessary.
I'm using the code outlined in the following post:
Draw text in OpenGL ES
I thought I could use this technique in order to dynamically display text (say an FPS counter). I realised that calls to resources to get the drawable slows down this process quite a lot, and I didn't need a bitmap background, so I removed it.
The technique works, but after a while (~2000 frames) the whole phone locks up. I suspect there's some memory which is not being freed in this code but I don't know where. I tried offloading the Canvas, Paint and Bitmap object creations which worked (so they aren't created every single frame) but the same problem still occurs.
I suspect therefore, that the generated GL texture is to blame, but I'm unsure how to remove it, or if this is even the case.
Any help would be appreciated.
EDIT: As an alternative, can someone please point out an easy way to render text to the screen dynamically (e.g. should be able to render the # of frames since starting for example, continually being updated and increasing). All the ways I can think of are either extremely tedious (make individual quads for each digit, store the textures for 0-9 in memory, parse the number and render each digit onto each quad), cannot be updated in good time (overlay Views) or can't get the required positioning in the glSurfaceView.
CBFG - http://www.codehead.co.uk/cbfg
This really is exactly what I've been wanting. You build a bitmap file from a font file using CBFG which can then be loaded and displayed with only a few lines of code (after importing his packages). It's literally as easy as fnt.PrintAt(gl,"Hello world!", 50, 160); in onDraw and more importantly, it handles dynamic text really well. I strongly advise anyone who is the same situation to try this.
two things I can guess you'll want to try:
1) dont' recreate the number of your frs every frame, generate number 1 to 60 and always reuse those.
2) there is an issue I found when generating text for my textures is that the font loader code of android never frees the memory space so avoid loading the font all the time, do it once and store a reference to it
I just wrote an entire tutorial on creating exactly what you are looking for.
The idea is basically to use font files and then generate a font bitmap (or atlas) at run-time instead of using a tool like CBFG to generate it offline. The benefit of this is that you can ship a small font file instead of multiple large bitmaps with your app, and never have to sacrifice font quality by using scaling.
The tutorial includes full working source (that can be dropped into any project). If you are interested go have a look here.