Java – advanced tag generator for complex mathematical expressions

I want to mark a string of integers, floating point numbers, operators, functions, variables, and parentheses The following examples should highlight the nature of the problem:

Current status:

String infix = 4*x+5.2024*(Log(x,y)^z)-300.12

Desired state:

String tokBuf[0]=4 
 String tokBuf[1]=* 
 String tokBuf[2]=x 
 String tokBuf[3]=+ 
 String tokBuf[4]=5.2024 
 String tokBuf[5]=* 
 String tokBuf[6]=( 
 String tokBuf[7]=Log
 String tokBuf[8]=( 
 String tokBuf[9]=x
 String tokBuf[10]=,String tokBuf[11]=y 
 String tokBuf[12]=) 
 String tokBuf[13]=^ 
 String tokBuf[14]=z 
 String tokBuf[15]=) 
 String tokBuf[16]=- 
 String tokBuf[17]=300.12

All tips and solutions will be appreciated

Solution

Use the Java stream tag generator The interface is a little strange, but people will get used to it:

http://docs.oracle.com/javase/7/docs/api/java/io/StreamTokenizer.html

Sample code for parsing the requested string list (you may want to use tokenizer directly or at least use the object list so that you can directly store numbers as double):

public static List<String> tokenize(String s) throws IOException {
  StreamTokenizer tokenizer = new StreamTokenizer(new StringReader(s));
  tokenizer.ordinaryChar('-');  // Don't parse minus as part of numbers.
  tokenizer.ordinaryChar('/');  // Don't treat slash as a comment start.
  List<String> tokBuf = new ArrayList<String>();
  while (tokenizer.nextToken() != StreamTokenizer.TT_EOF) {
    switch(tokenizer.ttype) {
      case StreamTokenizer.TT_NUMBER:
        tokBuf.add(String.valueOf(tokenizer.nval));
        break;
      case StreamTokenizer.TT_WORD:
        tokBuf.add(tokenizer.sval);
        break;
      default:  // operator
        tokBuf.add(String.valueOf((char) tokenizer.ttype));
    }
  }
  return tokBuf; 
}

Test run:

System.out.println(tokenize("4*x+5.2024*(Log(x,y)^z)-300.12"));
[4.0,*,x,+,5.2024,(,Log,y,),^,z,-,300.12]