Files. Parallel version of walkfiletree (Java or Scala)

Does anyone know java files Any parallel equivalent of walkfiletree or something like that? It can be a Java or Scala library

Solution

We assume that it is sufficient to execute a callback on each file

This code doesn't handle loops in the file system - you need a registry to indicate where you've been (for example, Java. Util. Concurrent. Concurrent HashMap) You can add various improvements, such as reporting exceptions instead of silently ignoring them

import java.io.File
import scala.util._
def walk(f: File,callback: File => Unit,pick: File => Boolean = _ => true) {
  Try {
    val (dirs,fs) = f.listFiles.partition(_.isDirectory)
    fs.filter(pick).foreach(callback)
    dirs.par.foreach(f => walk(f,callback,pick))
  }
}

Using folding instead of foreach to collect files is not very difficult, but I leave it to the reader as an exercise (concurrentlinkedqueue may be fast enough to accept them in callbacks anyway, unless your thread is very slow and the file system is great.)