Develop Reference

Reading a large text file with over 130000 line of text

How can i read a large text file into my Application?
This is my code but it does not work. My code must read a file called list.txt. The code worked only with a file with only 10.000 lines.
can someone helps me?
Thanks!
My code:(Worked with small files, but not with large files)
private void largefile(){
String strLine2="";
wwwdf2 = new StringBuffer();
InputStream fis2 = this.getResources().openRawResource(R.raw.list);
BufferedReader br2 = new BufferedReader(new InputStreamReader(fis2));
if(fis2 != null) {
try {
LineNumberReader lnr = new LineNumberReader(br2);
String linenumber = String.valueOf(lnr);
while ((strLine2 = br2.readLine()) != null) {
wwwdf2.append(strLine2 + "\n");
}
// Toast.makeText(getApplicationContext(), linenumber, Toast.LENGTH_LONG).show();
Toast.makeText(getApplicationContext(), wwwdf2, Toast.LENGTH_LONG).show();
} catch (IOException e) {
e.printStackTrace();
}
}
}

Since you are processing a large file, you should process the data in chunks . Here your file reading is fine but then you keep adding all rows in string buffer and finally passing to Toast.makeText(). It creates a big foot-print in memory. Instead you can read 100-100 lines and call Toast.makeText() to process in chunks. One more thing, use string builder instead of string buffer go avoid unwanted overhead of synchronization. You initializing wwwdf2 variable inside the method but looks it is a instance variable which I think is not required. Declare it inside method to make it's scope shorter.

Android - Save and Read from file

I have a listview lv_operationListand I'm trying to save it to a file and read it after. I can save and read files but I dont know if what I'm saving and reading is correct.
Basically I want to save the list rows and when I load a program I want to fill the same listview with the saved data.
Saving:
for (int i = 0; i < lv_operationList.getAdapter().getCount() - 1; i++) {
fileOutputStream.write(lv_operationList.getAdapter().toString().getBytes());
}
fileOutputStream.close();
Load?? Maybe something like this?
`fileInputStream = getContext().openFileInput(programName);
Scanner scanner = new Scanner(new DataInputStream(fileInputStream));
while(scanner.hasNext()) {
//Read??
}
//and display on lv_operationList. How?

i think you need to append the contents to the same file so you need to use append_mode like this.
OutputStreamWriter out = new OutputStreamWriter(openFileOutput("save.txt", Context.MODE_APPEND));
out.write("text");
out.write('\n');
in order to read them you can use this method.
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
text.append(line);
Log.i("reading line by line",""+text);
}

Android app too slow. Ways to accelerate?

I've just wrote my first Android app with Android Studio. It's a vocabulary trainer and it reads in a text file in my assets folder when starting, which contains all the words (until now, I have only ~1000) like this: english$japanese$category. So, I thought this shouldn't be much work, even though I have an old Samsung S2. But It takes like 10 secs to start and sometimes it crashes.
Here's the critical code:
static String word;
static String[] listOfWords;
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
readWords();
generateRandomWord();
}
public void readWords() {
try {
InputStream is = getAssets().open("words.txt");
String ww = "";
int data = is.read();
while(data != -1){
ww += (char) data;
data = is.read();
}
listOfWords = ww.split("\n");
} catch (IOException e) {
e.printStackTrace();
}
}
public void generateRandomWord() {
TextView textView = new TextView(this);
textView.setTextSize(40);
textView = (TextView) findViewById(R.id.text_id);
Random random = new Random();
int randomKey = random.nextInt(listOfWords.length-1);
String line = listOfWords[randomKey];
String[] parts = line.split("/");
Log.d("Tango-renshuu", "line: "+line+" "+parts.length+" "+parts[1]);
textView.setText(parts[1]);
word = parts[2];
}
The same thing happens when I try to go back to that Activity from another one, even though I'm using Intent.FLAG_ACTIVITY_CLEAR_TOP Like this:
public void back(View view) {
Intent intent = new Intent(this, MainActivity.class);
intent.setFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP);
startActivity(intent);
}
Any idea or do you think its just my device?
Thank

You are reading the asset on the Main-Thread, you need to start a Task to load it, while the Activity is rendered the asset loading happens in background.

Your readWords method is fairly inefficient: you are creating a new string at every loop iteration, and you're reading the file character by character. Consider using a BufferedReader to read the strings line by line directly:
InputStream stream = getAssets().open("words.txt");
BufferedReader reader = new BufferedReader(new InputStreamReader(stream));
ArrayList<String> lines = new ArrayList<String>();
String line;
while ((line = reader.readLine()) != null) {
lines.add(line);
}
listOfWords = lines.toArray(new String[lines.size()]);
reader.close();
If your code is still too slow after this optimization, then you should move this code to an AsyncTask so at least it doesn't freeze the UI, and you can display a loading spinner in the meantime.

read file from assets android

I'm trying to build a card viewer in android for a trading card game. My plan was to use an arraylist to populate the cards into the database. I would read in all values from file and would be displayed via listview. My listview code and arraylist code is fine (after testing without using file input). The problem occurs when I attempt to read in from file. I'm at a loss of what to do. When I run the program like this, it crashes. When I remove the "parseInt" method (because i'm reading in an integer from file), it runs, but none of my data is populated. What is wrong with my io code?
import android.content.Intent;
import android.support.v7.app.ActionBarActivity;
import android.os.Bundle;
import java.util.ArrayList;
import java.io.*;
import android.widget.ArrayAdapter;
import android.widget.ListView;
public class DeckBuilderActivity extends ActionBarActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
Intent intent = getIntent();
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_deck_builder);
//populate database************************************************************************
ArrayList<Card> database = new ArrayList<Card>(); //create new arraylist of datatype card
//value holders
String cardName, expansion, cardNum, unitType, unitClan, unitRace, trigger, unitSkill, cardLore,
imageId;
int unitGrade, unitPower, unitShield;
//exception handling
try{
String myfile = "cardmaindatabase.txt";
InputStream inputReader = getAssets().open(myfile);
BufferedReader reader = new BufferedReader(new InputStreamReader(inputReader)); //open file
//begin populating with a for loop
int i = 0; //incrementer
while(reader.readLine() != null) //while not end of file
{
//assign values to placeholders
//add new card and take parameters
database.add(new Card());
database.get(i).setCardName(reader.readLine());
database.get(i).setExpansion(reader.readLine());
database.get(i).setCardNum(reader.readLine());
database.get(i).setUnitGrade(Integer.parseInt(reader.readLine()));
}
} catch (IOException e) {
e.printStackTrace();
}
//******************************************************************************************
//populate arraylist
ArrayAdapter<Card> adapter = new ArrayAdapter(this, android.R.layout.simple_list_item_1, database);
ListView lv = (ListView) findViewById(R.id.card_database_listview);
lv.setAdapter(adapter);
}

StringBuffer stringBuffer = new StringBuffer();
try {
InputStream is = mContext.getResources().getAssets().open("your-file-path");
BufferedReader br = new BufferedReader(new InputStreamReader(is, "UTF-8"));
String line;
while ((line = br.readLine()) != null) {
//play with each line here
}
}...
Try this code to get data from the assets.
OR if you don't want to clear your code. try modify your code like this. I think the problem is in your loop.
String line = null;
while((line = reader.readLine()) != null) //while not end of file
{
//play with each line here
}
but collecting/storing data like this is unstable and hard to maintain. i'll suggest to use JSON. to store data in asset,so you could be able to collect and store data as fast as possible.

I keep getting ArrayIndexOutOfBoundsException

Keep getting ArrayIndexOutOfBoundsException when trying to display a specific item form array list data displays fine in logcat but then after displaying throws exception at top please help is the code kind of new to android
public void loadData(){
InputStream file = ourContext.getResources().openRawResource(R.raw.cashpot);
BufferedReader reader = new BufferedReader(new InputStreamReader(file));
String line ="";
ArrayList<String> values = new ArrayList<String>();
try {
while((line = reader.readLine()) != null) {
values.add(line);
// String[] strings = line.split(",");
/*for(String s : strings) {
values.add(s);
}*/
}
reader.close();
//for(String s : values) Log.d("CSV Test", s);
for(int i = 0; i < values.size();i++){
String[] data = values.get(i).split(",");
Log.d("Test String",data[2] );
}
} catch (IOException e) {
e.printStackTrace();
}
}

Your some lines do not have comma(,)(or 1 comma) and you are trying to split them by comma and storing it in to data array. and you are accessing data[2] location. which might not been created that's why you are getting this exception.

There might not be a 3rd element in your array ,that is created after the split on ",".
You can print value of data before accessing data[2](that might not exist).
Or you can Debug your program and proceed step by step and monitoring the value of each variable.