Java Utililty Methods String Tokenize

List of utility methods to do String Tokenize

  1. HOME
  2. Java
  3. S
  4. String Tokenize

Description

The list of methods to do String Tokenize are organized into topic(s).

Method

ListconservativeTokenize(String text)
Conservatively normalize a string while tokenizing it 
String[] token_arr = text.toLowerCase()
        .split("[ \t~`@#$%^&\\*\\(\\)_\\+-=\\{\\}\\[\\]:\";'<>\\?,./\\|\\\\]+");
return Arrays.asList(token_arr);
intcountCommonTokens(String string1, String string2)
count Common Tokens 
List<String> string1Tokens = new ArrayList<String>(Arrays.asList(string1.split("\\s+")));
List<String> string2Tokens = new ArrayList<String>(Arrays.asList(string2.split("\\s+")));
string1Tokens.retainAll(string2Tokens);
return string1Tokens.size();
ListescapedTokens(String s, char separator)
Tokenize a String using the specified separator character and the backslash as an escape character (see OGC WFS 1.1.0 14.2.2). 
if (s == null) {
    throw new IllegalArgumentException("The String to parse may not be null.");
if (separator == '\\') {
    throw new IllegalArgumentException("The separator may not be a backslash.");
List<String> ret = new ArrayList<String>();
StringBuilder sb = new StringBuilder();
...
ArrayListextractTokens(String strStartToken, String strEndToken, String strExpression)
extract Tokens 
ArrayList<String> Result = new ArrayList<String>();
if (strExpression.contains(strStartToken)) {
    String strTmp = strExpression;
    int intStartIndex = strTmp.indexOf(strStartToken); 
    int intEndIndex = strTmp.indexOf(strEndToken); 
    while (intStartIndex > 0 && intEndIndex > intStartIndex) {
        if (intEndIndex > intStartIndex) {
            String strCallToAdd = strTmp.substring(intStartIndex + strStartToken.length(), intEndIndex);
...
ListextractTokens(String text, String delim)
This method will extract tokens from a string containing multiple tokens and place the tokens in a collection. 
List<String> tokenList = new ArrayList<String>();
if (text != null) {
    String[] tokens = text.split(delim);
    for (int i = 0; i < tokens.length; i++) {
        tokenList.add(tokens[i].trim());
return tokenList;
...
String[]getAllTokens(final String string)
Applies StringTokenizer to the string and returns all the tokens found. 
return getAllTokens(string, LIST_SEPARATOR);
String[]getCommandTokens(String commandString)
get Command Tokens 
boolean escaped = false;
ArrayList<String> tokens = new ArrayList<String>();
int length = commandString.length();
int startIx = 0;
int endIx = length;
final char NONE = 'x';
char startChar = NONE;
for (int i = 0; i < length; i++) {
...
StringgetFileName(StringTokenizer s)

Gets the name for the file, eliminating "" and skiping "=" 

String val = s.nextToken(); 
val = s.nextToken();
val = val.replace('"', ' ').trim();
return val;
ListgetNameTokens(final String names)
Return a List of name tokens 
List<String> listNames = new ArrayList<String>();
if (fieldIsPresent(names)) {
    StringTokenizer stkn = new StringTokenizer(names, " ,");
    while (stkn.hasMoreTokens()) {
        String aName = stkn.nextToken();
        if (aName.trim().length() > 0) {
            listNames.add(aName);
return listNames;
floatgetParamFloat(StringTokenizer s)

Gets an float from param file, skiping "=" 

String val = s.nextToken(); 
val = s.nextToken();
return Float.parseFloat(val);