Java Utililty Methods String Tokenize

List of utility methods to do String Tokenize

  1. HOME
  2. Java
  3. S
  4. String Tokenize

Description

The list of methods to do String Tokenize are organized into topic(s).

Method

String[]tokenizeToStringArray(String str, String seperators)
tokenize To String Array 
StringTokenizer tokenlizer = new StringTokenizer(str, seperators);
List result = new ArrayList();
while (tokenlizer.hasMoreElements()) {
    Object s = tokenlizer.nextElement();
    result.add(s);
return (String[]) result.toArray(new String[result.size()]);
String[]tokenizeWhitespace(String input)
tokenize Whitespace 
String[] result = input.split("\\s+");
return result;
ArrayListtokens(String exp)
tokens 
ArrayList<String> array = new ArrayList<String>();
String subexp = "";
int numPar = 0;
int start = 0;
boolean open = false;
for (int i = 0; i < exp.length(); i++) {
    if (exp.charAt(i) == '(') {
        open = true;
...
String[]toStringToArray(String str, String token)
to String To Array 
if (token == null || token.trim().length() == 0)
    return new String[] { str };
if (str == null)
    return null;
if (str.trim().length() == 0)
    return new String[0];
int tokenLen = token.length();
int start = 0;
...
ListtoTokens(String s)
to Tokens 
StringTokenizer tokenizer = new StringTokenizer(s, " ");
List l = new ArrayList();
while (tokenizer.hasMoreTokens())
    l.add(tokenizer.nextToken());
return l;
List>unpackConllSentenceToTokens(String input)
unpack Conll Sentence To Tokens 
List<List<String>> list = new ArrayList();
for (String word : unpackConllSentence(input)) {
    list.add(unpackConllWord(word));
return list;