Java String Tokenize tokenize(String source, char separator)

Description

Break up a string into tokens This method will take a source string and break it into substrings.

License

Open Source License

Parameter

Parameter	Description
source	The string we will break into tokens
separator	The separator character

Return

an array of tokens, or null if source is null

Declaration

public static String[] tokenize(String source, char separator) 

Method Source Code

//package com.java2s;

public class Main {
    /**/* w  ww.  ja v  a2  s.c o m*/
    Break up a string into tokens
    This method will take a source string and break it into substrings.  The
    substrings are separated from each other by a single separator char in the
    source.  If there are two adjacent separator charactors, or the first or
    last character in source is a separator, then a token of an empty string
    will result.
    @param source The string we will break into tokens
    @param separator The separator character
    @return an array of tokens, or null if source is null
    */
    public static String[] tokenize(String source, char separator) {
        if (source == null) {
            return null;
        }

        // we'll allocate an overlarge array and then copy later
        int slen = source.length();
        String[] scratch = new String[slen + 1]; // maximum possible size
        int ntokens = 0;

        int start = 0;
        while (true) {
            if (start >= slen) {
                // the last character must have been a separator
                // add an empty token
                scratch[ntokens++] = "";
                break;
            }
            int sepx = source.indexOf(separator, start);
            if (sepx < 0) {
                // we've run off the end, no more separators
                scratch[ntokens++] = source.substring(start);
                break;
            }

            // the normal case
            scratch[ntokens++] = source.substring(start, sepx);
            start = sepx + 1;
        }

        // reallocate to the size we've actually got
        String[] result = new String[ntokens];
        System.arraycopy(scratch, 0, result, 0, ntokens);

        return result;
    }
}

Related

1. tokenize(String s, char separator, int maxTokens)
2. tokenize(String s, String delimiters)
3. tokenize(String s1)
4. Tokenize(String sent)
5. tokenize(String sentence)
6. tokenize(String source, char separator)
7. tokenize(String str, char delim)
8. tokenize(String str, String delims)
9. tokenize(String str, String delims, boolean returnDelims)