亚洲国产日韩欧美一区二区三区,精品亚洲国产成人av在线,国产99视频精品免视看7,99国产精品久久久久久久成人热,欧美日韩亚洲国产综合乱

java如何高效讀寫10G以上大文件
黃舟
黃舟 2017-04-18 10:52:01
0
5
775

有一份10G以上大文本文件,需要替換里面的一些文本信息(每一行都有),如何高效讀并替換掉生成新的文件

黃舟
黃舟

人生最曼妙的風(fēng)景,竟是內(nèi)心的淡定與從容!

reply all(5)
伊謝爾倫
  1. Split into multiple files first

  2. Multiple threads operate multiple files to avoid two threads operating the same file

  3. Read files line by line and write new files line by line

  4. Merge all files

1,4 Just use linux commands~

劉奇

File file = new File(filepath);
BufferedInputStream fis = new BufferedInputStream(new FileInputStream(file));
BufferedReader reader = new BufferedReader(new InputStreamReader(fis,"utf-8"),510241024);
String line = "";
while((line = reader.readLine()) != null){

//進(jìn)行替換操作和其他業(yè)務(wù)    

}

迷茫

In order to improve performance, you may need mapped IO. For details, please refer to:

  1. Why use Memory Mapped File or MappedByteBuffer in Java

  2. java large file read and write operations, java nio's MappedByteBuffer, efficient file/memory mapping

  3. A simple comparison of the performance of java.io and java.nio

Peter_Zhu

If it is a simple text replacement, just use the sed command of Linux.

If it is a more complex text replacement, see below:

  1. http://stackoverflow.com/ques...

  2. http://www.baeldung.com/java-...

劉奇

用spark分析、
lines=sc.textFile("your_file");
filterlines=lines.filter(your_filter_function)
filterlines.xxx()

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template