Bad use of very large strings? (JAVA)

Is there any negative creation of huge strings? For example, if we read text from a potentially large text file:

while (scanner.hasNext()) {
  someString += scanner.next();
}
// do something cool with someString

A better solution would be to process files line by line (generally), why?

thank you

Solution

Streaming vs not

When you can stream, you can process files of any size (assuming you can really forget all the data you have seen) You will eventually have a natural o (n) complexity, which is a very good thing You won't interrupt because of insufficient memory

Streaming is cute... But it doesn't work in every case

StringBuilder

There seems to be some controversy about the StringBuilder proposal. Here is the benchmark for displaying the effect I had to reduce the size of the benchmark so that the slow version could even be completed in a reasonable time

Results first, then codes This is a very rough and prepared benchmark, but the result is dramatic enough to make the point

c:\Users\Jon\Test>java Test slow
Building a string of length 120000 without StringBuilder took 21763ms

c:\Users\Jon\Test>java Test fast
Building a string of length 120000 with StringBuilder took 7ms

And code

class FakeScanner
{
    private int linesLeft;
    private final String line;

    public FakeScanner(String line,int count)
    {
        linesLeft = count;
        this.line = line;
    }

    public boolean hasNext()
    {
        return linesLeft > 0;
    }

    public String next()
    {
        linesLeft--;
        return line;
    }
}

public class Test
{    
    public static void main(String[] args)
    {
        FakeScanner scanner = new FakeScanner("test",30000);

        boolean useStringBuilder = "fast".equals(args[0]);

        // Accurate enough for this test
        long start = System.currentTimeMillis();

        String someString;
        if (useStringBuilder)
        {
            StringBuilder builder = new StringBuilder();
            while (scanner.hasNext())
            {
                builder.append(scanner.next());
            }
            someString = builder.toString();
        }
        else
        {
            someString = "";     
            while (scanner.hasNext())
            {
                someString += scanner.next();
            }        
        }
        long end = System.currentTimeMillis();

        System.out.println("Building a string of length " 
                           + someString.length()
                           + (useStringBuilder ? " with" : " without")
                           + " StringBuilder took " + (end - start) + "ms");
    }
}