1. Filtering Java comments using StreamTokenizer stackoverflow.comMy goal is to analyze java source files to find line numbers containing non-comment code. Since StreamTokenizer has slashStarComments() and slashSlashComments(), I figured I'll use it to filter out the lines ... |
2. How can I ensure that there's whitespace between specific tokens with StreamTokenizer? stackoverflow.comI am working with |
3. How to deal with operators while tokenizing in Java (StreamTokenizer) stackoverflow.comI'm writing a tokenizer in Java which has to deal with operators, and whitespace chars between tokens is not necessary. I need to recognize something like "<=" as a token, while also ... |
4. streamtokenizer coderanch.comHi friends, Please do send me some suggestions to solve this problem. The inputfile i'm trying to read has 4 fields and looks like this Tg5_Tg0 Genename1 I 3.5 Tg5_Tg2 Genename2 I 2.5 I have used a stream tokenizer to read each field for me. I read the first token as a string.However the nexttoken() method has recognised the Tg5 and ... |
5. StreamTokenizer coderanch.com |
6. StreamTokenizer 'word' tokens coderanch.comHi there. I am using a StreamTokenizer to parse a data file, containing multiple entries which look like this: @article{walker1980, author="Daniel Walker", title="BibtexingAndStuff", year="1980" } Things are working OK, until I get to any entry which has it's value spread over multiple lines. The StreamTokenizer will tokenize from a quote character (") to either another quote character or an end of ... |
7. StreamTokenizer help coderanch.comhi mike, here is a sample code i wrote... import java.io.*; public class StreamTokenizerTest { public void tokenizeFile(String file) { StreamTokenizer st; try { FileReader fr = new FileReader(file);// Line-1 st = new StreamTokenizer(fr);// Line-2 while( StreamTokenizer.TT_EOF != st.nextToken()) { // Line-3 switch(st.ttype) { case StreamTokenizer.TT_NUMBER:// Line-4 System.out.println(st.nval); break; case StreamTokenizer.TT_WORD:// Line-5 System.out.println("Ignoring WORD:"+st.sval+":"); break; case StreamTokenizer.TT_EOL:// Line-6 System.out.println(); break; } ... |
8. Problem in StreamTokenizer coderanch.comHello, I am using StreamTokenizer to generate tokens. But the confusion I am having is that there is a field TT_NUMBER in StreamTokenizer which recognizes numbers. But I want to differentiate between Integers and Floating point numbers. Can anyone tell me how is that possible. I have written the following code: BufferedReader readInput = new BufferedReader(new InputStreamReader(is)); str = new StreamTokenizer(readInput); ... |
9. StreamTokenizer coderanch.comI have a few questions about StreamTokenizer 1. Is it possible to make streamtokenizer not distinguish between numbers and words? Is it possible to treat numbers as words for nextToken purposes? 2. Is it possible to make spaces treated as words so that nextToken reads many words until a stopping ordinary character is reached? 3. In one of my code i ... |
10. StreamTokenizer and hard returns coderanch.comHello. If somebody has a minute, I am trying to read the input from a semicolon delimited text file with the following format... The Dog;The Cat;The Mouse The Motorcycle;The Car;The Truck The Square;The Circle;The Oval I eventually want to convert each line to a row in a db table. What I'm having trouble with is reading the hard return in the ... |
11. problem StreamTokenizer?? coderanch.comi wish to design a class as a tool for solving polynomial equations. user interaction: user enters the data through the keyboard. the program finds the roots of the equation and displays the results. the program is divided into two significant processes. 1. scanning the equation for corect format 2. finding the roots. its just to practice streamtokenizers the user would ... |
12. StreamTokenizer coderanch.comHi, I'm using StreamTokenizer to read contents from a text file. I want to retain the spaces in the text file, but streamtokenizer removes the spaces and appends all the characters in the file. I tried using ordinaryChar() method, but that didn't really help either. Any suggestions would really help... Reader read = new BufferedReader(new InputStreamReader(myFile));//read the contents using buffered reader ... |
13. StreamTokenizer coderanch.com |
14. Issue with StreamTokenizer? coderanch.comI was fooling around with StreamTokenizer to create a SubClass, which can return comments as tokens. In the process, I stumbled into something very puzzling. See the code. public static void main(String[] args) throws IOException { parse("hello"); parse("\"hello\""); parse("\"hello"); parse("hello\""); } public static void parse(String string) throws IOException { System.out.println(""); System.out.println("parsed: " + string); StreamTokenizer tokenizer = new StreamTokenizer(new StringReader(string)); tokenizer.slashSlashComments(false); ... |
15. StreamTokenizer problem again... coderanch.com |
16. How to use StreamTokenizer? coderanch.com |
17. StreamTokenizer coderanch.com |
18. StreamTokenizer and EOL problem coderanch.comI am having a problem with this code. When I run it , can get the numbers fine, but I am not reading the StreamTokenizer.TT_EOL so I am stuck in the while loop forever. public class ReadingNumbersMain { public static void main(String[] args)throws IOException { readfile("puzzle1.txt"); } static void readfile(String filename) throws IOException { int count =0; int[] intArray; ArrayList |
19. StreamTokenizer forums.oracle.comI want to give a String (from the keyboard), and after using class StreamTokenizer, i want to handle each letter/number/operand alone... For example i want to give input "1st test in streamtokenizer" and then to push "st" + "token" (or sth else) into a stack, and the other letters into something else...? |
20. StreamTokenizer forums.oracle.comIs there any way to determine what has been set in wordChars, quoteChar, ordinaryChar and/or whitespaceChars without having to go back and look at the code? What I need to do is, save off the current settings of the token, reset them to what I need them to be, and then reset them back to what they were. Any help, many ... |
21. StreamTokenizer problem forums.oracle.comI wa trying to study about StreamTokenizer, but I couldn't find any clear document Please tell me how the StreamTokenizer tokenize the stream, I mean what delimeter characters are used?? Im using it in this way StreamTokenizer st = new StreamTokenizer(new BufferedReader(new FileReader("input.txt"))); so when so far as i know it tokenize the stream by using space character and many other ... |
22. StreamTokenizer NVAL bug? 16 digits not reading correctly? forums.oracle.comFrom what I read from the docs, that class wont help you much anymore. Must say that I was surprised how it looked disgustingly unlike the Java I'm used to use... maybe because this class is mostly unchanged since Java 1.0. I suggest you change your file reading classes... use something like Scanner class. |
23. StreamTokenizer and MappedByteBuffer forums.oracle.com |
24. StreamTokenizer '/' read problem forums.oracle.com |
25. StreamTokenizer forums.oracle.com |
26. what's Streamtokenizer for ? forums.oracle.comReally this question is just too vague. Perhaps you've read the API documentation (or other stuff) and can't understand it. If so, say what you've read and what it is about it that you don't understand. Perhaps you have some specific task in mind, and wonder if StreamTokenizer is appropriate. If so, what is that task? Perhaps you are wanting to ... |
27. StreamTokenizer's ugly behaviour forums.oracle.comGreetings, a StreamTokenizer can read numbers and 'words'. The 'word' is considered a 'word character' followed by 'word character's or any character that is considered a digit character. A dot and a minus sign are considered a digit too, so "x-y" is considered a 'word' while 'x+y' are considered three separate tokens. I find this very ugly and I'm going to ... |
28. Need help with streamtokenizer forums.oracle.comall those instructions are in a file. What I need to know is how do i use these tokens to actually carry out functions such as deposit, withdraw etc.. if you can give me an example on how to use streamtokenizer to carry aout a function, lets say a line "D 100.00 S" which means "deposit 100 into savings account" |
29. StreamTokenizer forums.oracle.comThat's essentially what it does, but you'll need to clarify to it what you mean by "words" and so on. Is "0x230" a word? Is "new-item"? Bear in mind that StreamTokenizer is really designed to tokenise programming, not natural languages. I've found it quite useful in the past, but it does have limits. If you want to parse input with maximum ... |