nio « API « Java I/O Q&A





1. How to avoid OutOfMemoryError when using Bytebuffers and NIO?    stackoverflow.com

I'm using ByteBuffers and FileChannels to write binary data to a file. When doing that for big files or successively for multiple files, I get a OutOfMemoryError exception. I've read elsewhere that ...

2. FileChannel & RandomAccessFile don't seem to work    stackoverflow.com

To put it simple: a swing app that uses sqlitejdbc as backend. Currently, there's no problem launching multiple instances that work with the same database file. And there should be. The file ...

3. transferring bytes from one ByteBuffer to another    stackoverflow.com

What's the most efficient way to put as many bytes as possible from a ByteBuffer bbuf_src into another ByteBuffer bbuf_dest (as well as know how many bytes were transferred)? I'm trying ...

4. Extending ByteBuffer class    stackoverflow.com

Is there any way to create class that extends ByteBuffer class? Some abstract methods from ByteBuffer are package private, and if I create package java.nio, security exception is thrown. I would want to ...

5. Java Large Files Disk IO Performance    stackoverflow.com

I have two (2GB each) files on my harddisk and want to compare them with each other:

  • Copying the original files with Windows explorer takes approx. 2-4 minutes (that is reading and ...

6. Is it possible to read the Process stdout InputStream into an NIO ByteBuffer?    stackoverflow.com

Is it possible to use NIO to process the stdout from a Process? I have it working with java.io, but this is something of an exercise to learn a bit more ...

7. java.nio channels buffers streams terminology    stackoverflow.com

Does anybody have a good analogy (or, failing that, a good resource) for describing the relationships between buffers, streams, readers, channels, selectors etc. in java.io and java.nio? Thanks

8. BufferedReader for large ByteBuffer?    stackoverflow.com

Is there a way to read a ByteBuffer with a BufferedReader without having to turn it into a String first? I want to read through a fairly large ByteBuffer as lines ...

9. Java: Converting String to and from ByteBuffer and associated problems    stackoverflow.com

I am using Java NIO for my socket connections, and my protocol is text based, so I need to be able to convert Strings to ByteBuffers before writing them to the ...





10. Java Serializable, ObjectInputstream, Non-blocking I/O    stackoverflow.com

I'm just starting out with Java serialization, and I'm not clear on how you are supposed to get objects from a source in a scenario with non-blocking I/O . All the documentation ...

11. Java: Generalization possible between: streams, readers, char buffers, stringbuilder, ...?    stackoverflow.com

Background story:

There are these Source and Result interfaces for XML. These are adapters between different XML technologies in Java. Instances of these classes represent DOM, SAX, JAXB, XML streams, XML events (and even more?).

The ...

12. Java NIO FileChannel versus FileOutputstream performance / usefulness    stackoverflow.com

I am trying to figure out if there is any difference in performance (or advantages) when we use nio FileChannel versus normal FileInputStream/FileOuputStream to read and write files to filesystem. I ...

13. In-memory version of Java's FileChannel    stackoverflow.com

I'm in the process of making some changes to a library that I'm using. In order to reduce memory usage the library is writing its temporary data to disk instead of ...

14. get FileChannel without using java.io.* (use pure NIO)    stackoverflow.com

Recently I got a comment to this answer that I should stay away from java.io if I want to use "pure NIO".
This is the simplified code (copy a file):

private static ...

15. nio FileChannel.transferFrom transferring 0?    stackoverflow.com

I'm trying to use NIO to assemble a file out of several smaller files, using transferFrom. The call to transferFrom returns 0. No exception. Nothing done to turn on synchronous behavior.

  ...

16. Write a stream into a file with NIO and the Channel system    stackoverflow.com

I have an inputStream and i want to write it to a file. I saw NIO and the FileChannel which has the method "transferTo" ou "transferFrom" and i know how to create ...





17. In Java What is the guaranteed way to get a FileLock from FileChannel while accessing a RandomAccessFile?    stackoverflow.com

I am trying to use

FileLock lock(long position, long size,boolean shared)
in FileChannel object As per the javadoc it can throw OverlappingFileLockException. When I create a test program with 2 threads lock ...

18. java: converting part of a ByteBuffer to a string    stackoverflow.com

I have a ByteBuffer containing bytes that were derived by String.getBytes(charsetName), where "containing" means that the string comprises the entire sequence of bytes between the ByteBuffer's position() and limit(). ...

19. How can I tell file deletion from directory deletion using JDK 7?    stackoverflow.com

I am using JDK 7's WatchService to monitor directories. The ENTRY_DELETE event tells me an entry has been deleted. I can get the name of that entry doing something similar to:

WatchEvent<Path> ev ...

20. Using Java's ByteBuffer to read millions of messages    stackoverflow.com

Here's my problem: one big gzipped file; millions of messages. Each message consists of:

***************** *************** ****************** 
* 2-byte LENGTH * * 1-byte TYPE * * N-byte PAYLOAD * , where N ...

21. Deep copy duplicate() of Java's ByteBuffer    stackoverflow.com

java.nio.ByteBuffer#duplicate() returns a new byte buffer that shares the old buffer's content. Changes to the old buffer's content will be visible in the new buffer, and vice versa. What if I ...

22. Java Rolling File Creation Fails when attempting to read simultaneously    stackoverflow.com

I am using java.util logging classes to create a rolling file appender. I want to create a log reader that reads from these logs as data is written to them. The ...

23. Why the odd performance curve differential between ByteBuffer.allocate() and ByteBuffer.allocateDirect()    stackoverflow.com

I'm working on some SocketChannel-to-SocketChannel code which will do best with a direct byte buffer--long lived and large (tens to hundreds of megabytes per connection.) While hashing out the exact ...

24. Why FileChannel in Java is not non-blocking?    stackoverflow.com

I wanted to write a program which writes to multiple files simultaneously; thought it will be possible with one thread by using non-blocking mode. But FileChannel does not support non-blocking mode. ...

25. Would FileChannel.read read less bytes than specified if there's enough data?    stackoverflow.com

For example I have a file whose content is?

abcdefg
then i use the following code to read 'defg'.
ByteBuffer bb = ByteBuffer.allocate(4);
int read = channel.read(bb, 3);
assert(read == 4);
Because there's adequate data in the ...

26. Could ByteBuffer implement DataOutput/DataInput?    stackoverflow.com

Is there some subtle reason why java.nio.ByteBuffer does not implement java.io.DataOutput or java.io.DataInput, or did the authors just not choose to do this? It ...

27. Difference between ByteBuffer and CharBuffer in Java NIO    stackoverflow.com

What is difference between ByteBuffer and CharBuffer in case java.nio package. Is it the same difference as byte and char has ?

28. Java: reading strings from a random access file with buffered input    stackoverflow.com

I've never had close experiences with Java IO API before and I'm really frustrated now. I find it hard to believe how strange and complex it is and how hard it ...

29. Wrapping a ByteBuffer with an InputStream    stackoverflow.com

I have a method that takes an InputStream and reads data from it. I would like to use this method with a ByteBuffer also. Is there a way to wrap a ...

30. Efficient method to read String lines from file    stackoverflow.com

Assuming I have 15GB log records file, and I would like to iterate over \n terminated lines from this file. What java standard lib / 3rd parties provide clean interface for ...

31. Java Circular Byte Buffer that Extends java.nio.ByteBuffer    stackoverflow.com

Every Java circular byte buffer implementation I have seen referenced on SO and elsewhere does not extend java.nio.ByteBuffer, which for me is necessary for use with a SocketChannel. Does anyone ...

32. Are FileChannel.force and FileDescriptor.sync both needed?    stackoverflow.com

In Really force file sync/flush in Java, the author writes in the summary of the answers:

Use c.force(true) followed by s.getFD().sync() for Java NIO
My question is: ...

33. ByteBuffer.allocate() vs. ByteBuffer.allocateDirect()    stackoverflow.com

To allocate() or to allocateDirect(), that is the question. Hi. For some years now I've just stuck to the thought that since DirectByteBuffers are a direct memory mapping at OS level, that it ...

34. What method is more efficient for concatenating large files in Java using FileChannels    stackoverflow.com

I want to find out what method is better of two that I have come up with for concatenating my text files in Java. If someone has some insight they can ...

35. How to use java.nio.channels.FileChannel to read to ByteBuffer achieve similiar behavior like BufferedReader#readLine()    stackoverflow.com

I want to use java.nio.channels.FileChannel to read from a file, but I want to read line per line like BufferedReader#readLine() does. The reason why I need to use java.nio.channels.FileChannel instead of ...

36. What Charset does ByteBuffer.asCharBuffer() use?    stackoverflow.com

What Charset does ByteBuffer.asCharBuffer() use? It seems to convert 3 bytes to one character on my system. On a related note, how does CharsetDecoder relate to ByteBuffer.asCharBuffer()? UPDATE: With ...

37. ByteBuffer getInt() question    stackoverflow.com

We are using Java ByteBuffer for socket communication with a C++ server. We know Java is Big-endian and Socket communication is also Big-endian. So whenever the byte stream received and put ...

38. Need explanation of transferring binary data using Thrift rpc    stackoverflow.com

Lets say I defined following Thrift service

service FileResource {      
binary get_file(1:string file_name)
}
Here is the generated implementation which I cannot understand
public ByteBuffer recv_get_file() throws org.apache.thrift.TException
{
  org.apache.thrift.protocol.TMessage ...

39. How to set position properly in NIO.2's AsynchronousFileChannel.write(ByteBuffer src, long position)?    stackoverflow.com

My Java-program does a multithreaded XSLT Transformation and writes the results to a file. To avoid the IO bottleneck I am experimenting with the new AsynchronousFileChannel. The XSLT-Threads should submit their output ...

40. What is the diff between Java FileChannel.force() and StandardOpenOption.Sync?    stackoverflow.com

I'm not sure if the new Java 7 nio.file.StandardOpenOption is different from the old FileChannel.force() method. Is there a way to do O_DIRECT also?

41. Extracting -Strings from a ByteBuffer - I'm getting all confused    stackoverflow.com

I've been sitting with this problem for some hours now without getting it to work the way I want it to work. I'm creating a server which reads data from a telnet ...

42. How to avoid making defensive copies of ByteBuffer?    stackoverflow.com

I've got a class that takes a ByteBuffer as a constructor argument. Is there a way to avoid making defensive copies in order to ensure that the buffer doesn't get modified ...

43. ByteBuffer and FileChannel reading only the specified number of bytes    stackoverflow.com

I have a situation where in I keep reading with a ByteBuffer as below.

 ByteBuffer buffer = MappedByteBuffer.allocateDirect(Constants.BUFFER_SIZE);
But when the reading reaches the boundary (when the remaining bytes to read is ...

44. Using FileChannel to fsync a directory with NIO.2    stackoverflow.com

I just discovered that with NIO.2, at least under Linux, I can open a FileChannel on a directory, and calling force(true) calls fsync() on the underlying file descriptor. Without ...

45. how can I subclass ByteBuffer?    stackoverflow.com

So the Java NIO architects didn't make a ByteBuffer interface, but rather a ByteBuffer class, which isn't a final class, but it has no package-public constructors, and therefore it ...

46. ByteBuffer (java) performance issue    stackoverflow.com

While processing multiple gigabyte files I noticed something odd: it seems that reading from a file using a filechannel into a re-used ByteBuffer object allocated with allocateDirect is much slower than ...

47. java.nio.file.WatchEvent gives me only relative path. How can I get the absolute path of the modified file?    stackoverflow.com

I am using Java 7, java.nio.file.WatchEvent along with the WatchService. After registering, when I poll for ENTRY_MODIFY events, i cannot get to the absolute path of the file for the event. ...

48. Read specific bytes from RandomAccessFile using FileChannel : Java    stackoverflow.com

I have a RandomAccessFile and its FileChannel. What I'm trying to do is read a specific section of the bytes from said file; however, while looking over the FileChannel read methods, ...

49. Java: create a bytearray-backed FileChannel    stackoverflow.com

I have a class for IO that uses ByteBuffer to buffer access to a FileChannel (so it basically accepts a FileChannel at the constructor). I'd like to unittest it, so it'd ...

50. Java nio FileSystem Watcher locks directories. Deletion becomes impossible    stackoverflow.com

I'm using the new feature of Java7 for watching directories for changes (as it is described in the following tutorial: http://download.oracle.com/javase/tutorial/essential/io/notification.html ) The watching itself works (nearly) without problems. More or ...

51. Is it possible to Fork (clone) a stream in java.nio (or regular java io)?    stackoverflow.com

Is it possible (using the standard java.nio api, without major hacking or proxying or facading) to take an output stream, and clone it, so that every write to the stream gets ...

52. ObjectOutputStream faster than nio?    coderanch.com

I thought that nio was faster than the standard io for Java. But today as I was writing a rather large int[] to disk, I discovered that using ObjectOutputStream was significantly faster. Twice as fast in fact. Can someone take a look at this code and please tell me if I did something wrong with the nio part? If I didn't ...

53. FileChannel, MappedByteByffer, NIO questions    coderanch.com

I have a fairly large CSV file, say about 40,000+ lines that I get from a client. I also have a CSV file that is an export from a database that is around the same number of lines, as it should be the same data. I am reading the client file and needing to determine what records need to be updated ...

54. How to implement readLine in java.nio ?    coderanch.com

55. File Size Issue while delivering file more than 2gb using java.nio.channels.FileChannel.transferFrom    coderanch.com

Hi, I am facing an issue while delivering a file more than 2gb using the API java.nio.channels.FileChannel.transferFrom. The size of the file copied from source is only till 2gb. Kindly let me know if anyone could help me out with this. Basically I need to find out about some JAva api which could help in coping files more than 2 Gb. ...

56. IndexOutOfBoundsException thrown by java.nio bytebuffer.put(byte[] arsrc, int offset , int length)    coderanch.com

I'm curious as to why I'm getting this runtime error when from my perspective I shouldn't here's the code section: public static void SendMesg() { // Send Message to the Message Log String mesg_str = message_data.toString() ; // convert message data string buffer to a string int msgstr_len = mesg_str.length(); // determine actual message length int array_len = mesgwork.length ; // ...

57. use NIO FileChannel Question    forums.oracle.com

I know nothing is free, and I do not even care the cost of parsing the data. Two pieces of code are doing the same thing which will readLine() except one is using Channel to create the BufferedReader. If the file content is the same, one should be faster if the Channel has better I/O performance, right?

58. java.nio.FileChannel position() problem    forums.oracle.com

59. Reading ZIP/GZIP files from a FileChannel (NIO)    forums.oracle.com

I need to read/unpack a zip file given a FileChannel as part of a multi-task process. I've played around with extracting ZIP archives using ZipInputStream (FileInputStream) and ZipFile (File), but neither one of those will take a FileChannel. I don't have access to the original FileInputStream that the FileChannel was taken from. If someone could tell me a good way (or ...

60. java.nio, FileChannel, FileLock conundrum...    forums.oracle.com

because when you obtain an exclusive lock, no one else can even read it. So until that person is done, the file is inaccessible, which is bad for my particular need: Most of the time people will only need to read my file, but occassionally someone may write to it. I want the first user to open it to obtain the ...

61. NIO ByteBuffer and write()    forums.oracle.com

Hello, I'm having a slight difficulty using non blocking IO when performing a socketChannel.write(ByteBuffer) operation. I recently wrote a server that wrapped a large amount of data in a ByteBuffer and then attempted to write that buffer to the socket channel. However, it became clear that the entire buffer was not being transferred. Specifically, the result of the write operation (the ...

62. I/O Question -- ByteBuffer, NIO StreamTokenizer and speed    forums.oracle.com

All I get printed on the screen are '?'. What I am I doing wrong? Also, is there anyway to use StreamTokenizer with this construct (instead of what I am doing, i.e get char and process it). One last thing: speed is the most important factor for what I want to do. Can I read the file and parse it any ...

63. Java NIO with ByteBuffer    forums.oracle.com

First of all, it's pretty silly to be using a 1 byte buffer to copy a stream. Why would you use that as an example? One advantage of ByteBuffer is that it can use direct allocation, which reduces overhead when transferring large amounts of data. Another is that ByteBuffer encapsulates a lot of operations that are normally done with a byte[] ...

64. nio ByteBuffer and memory-mapped file size limitation    forums.oracle.com

I have a question/issue regarding ByteBuffer and memory-mapped file size limitations. I recently started using NIO FileChannels and ByteBuffers to store and process buffers of binary data. Until now, the maximum individual ByteBuffer/memory-mapped file size I have needed to process was around 80MB. However, I need to now begin processing larger buffers of binary data from a new source. Initial testing ...

65. java.nio: how to write IntBuffer to a FileChannel    forums.oracle.com

Hi, i have opened a FileChannel to a newly created file. How can i write I (the data of) an IntBuffer to it? I know how to write ByteBuffers by using aFileChannel.write(aByteBuffer), but this doesn't work in this case, since "write" method requires a ByteBuffer as a parameter, not a IntBuffer. Can somebody post an example or a link to one? ...

66. nio: how do i extract objects from a ByteBuffer buffer?    forums.oracle.com

Well you can do it but it's painful. At the sending end you have to serialize to a ByteArrayOutputStream so you can get the serialized object's length, then transmit first the length then the byte[] array of the BAOS. At the receiving end you have to read the length, then read that many bytes, create a ByteArrayInputStream, and deserialize the object ...

67. Speed using java.nio.ByteBuffer vs byte[]    forums.oracle.com

I have a potentially huge array of bytes that will be created at runtime which my program will then need to access frequently at random locations. I'm not sure whether it is best to store the array in a traditional byte[] and simply dereference the index I need, or to create a ByteBuffer and seek then read. I've heard that ByteBuffer ...