Read a File as fast as possible with High speed
I compare 3 algorithms to read file from groovy.
TODO: Compare with JAVA
: Measure memory heap
In order to run all tests, I'm using a closure to evaluate the runtime.
def benchmark = { closure -> start = System.currentTimeMillis() closure.call() now = System.currentTimeMillis() now - start }
Solution 1
def index = 0 file.eachLine(startLine){ line -> content.add line index++ if(index > (endLine - startLine)) // Out of range return }
Solution 2
content2 = file.readLines().getAt(startLine..endLine)
Solution 3
def number = 0 file.eachLine { line -> number++ if(number > startLine) content.add line if(number > endLine) return }
Complete example
/*** * How to read a file as fast as possible * * @author vicben01 */ def benchmark = { closure -> start = System.currentTimeMillis() closure.call() now = System.currentTimeMillis() now - start } // Configuration def file = new File ('CellBuilder_2010_06_29_20_01_17_012.log') def startLine = 500 //def endLine = 2000 for(endLine in [502, 800, 1000, 2000,10000,24000]){ println ">> read ${endLine - startLine} lines" /*** * */ // Algo 3 def content = [] def duration = benchmark { def index = 0 file.eachLine(startLine){ line -> content.add line index++ if(index > (endLine - startLine)) // Out of range return } } content = null println "1> execution took ${duration} ms / ${(endLine - startLine)} lines = ${duration / (endLine - startLine) }" sleep(500) // Algo 2 def content2 = [] def duration2 = benchmark { content2 = file.readLines().getAt(startLine..endLine) } //assert content2.size() == (endLine - startLine) content2 = null println "2> execution took ${duration2} ms / ${(endLine - startLine)} lines = ${duration2 / (endLine - startLine) }" sleep(500) // Algo 1 def content3 = [] def duration3 = benchmark { def number = 0 file.eachLine { line -> number++ if(number > startLine) content3.add line if(number > endLine) return } } //assert content.size() == (endLine - startLine) content3 = null println "3> execution took ${duration3} ms / ${(endLine - startLine)} lines = ${duration3 / (endLine - startLine) }" sleep(500) // TODO: compute means }