Package nom.tam.util

Class BufferedDataInputStream

  • All Implemented Interfaces:
    java.io.Closeable, java.io.DataInput, java.lang.AutoCloseable, ArrayDataInput, FitsIO

    public class BufferedDataInputStream
    extends java.io.BufferedInputStream
    implements ArrayDataInput
    This class is intended for high performance I/O in scientific applications. It combines the functionality of the BufferedInputStream and the DataInputStream as well as more efficient handling of arrays. This minimizes the number of method calls that are required to read data. Informal tests of this method show that it can be as much as 10 times faster than using a DataInputStream layered on a BufferedInputStream for writing large arrays. The performance gain on scalars or small arrays will be less but there should probably never be substantial degradation of performance.

    Many new read calls are added to allow efficient reading off array data. The read(Object o) call provides for reading a primitive array of arbitrary type or dimensionality. There are also reads for each type of one dimensional array.

    Note that there is substantial duplication of code to minimize method invocations. E.g., the floating point read routines read the data as integer values and then convert to float. However the integer code is duplicated rather than invoked. There has been considerable effort expended to ensure that these routines are efficient, but they could easily be superceded if an efficient underlying I/O package were ever delivered as part of the basic Java libraries. [This has subsequently happened with the NIO package and in an ideal universe these classes would be rewritten to take advantage of NIO.]

    Testing and timing routines are provided in the nom.tam.util.test.BufferedFileTester class. Version 1.1: October 12, 2000: Fixed handling of EOF to return partially read arrays when EOF is detected. Version 1.2: July 20, 2009: Added handling of very large Object arrays. Additional work is required to handle very large arrays generally.

    • Field Detail

      • LOG

        private static final java.util.logging.Logger LOG
      • SKIP_BUFFER_SIZE

        private static final int SKIP_BUFFER_SIZE
        size of the skip buffer (if it exists) skipBuf.
        See Also:
        Constant Field Values
      • skipBuf

        private byte[] skipBuf
        Skip the requested number of bytes. This differs from the skip call in that it takes an long argument and will throw an end of file if the full number of bytes cannot be skipped.
    • Constructor Detail

      • BufferedDataInputStream

        public BufferedDataInputStream​(java.io.InputStream o)
        Create a BufferedInputStream based on an input stream.
        Parameters:
        o - the input stream to use for reading.
      • BufferedDataInputStream

        public BufferedDataInputStream​(java.io.InputStream o,
                                       int bufLength)
        Create a BufferedInputStream based on a input stream with a specified buffer size.
        Parameters:
        o - the input stream to use for reading.
        bufLength - the buffer length to use.
    • Method Detail

      • eofCheck

        private int eofCheck​(java.io.EOFException e,
                             int i,
                             int start,
                             int length)
                      throws java.io.EOFException
        For array reads return an EOF if unable to read any data.
        Parameters:
        e - the eof exception that happened.
        i - the current index
        start - the start index
        length - the element length
        Returns:
        the number of bytes read before the end of file exception.
        Throws:
        java.io.EOFException - if no extra bytes could be read
      • read

        public int read​(boolean[] b)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read an array of boolean's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        b - array of boolean's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(boolean[] b,
                        int start,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of boolean's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        b - array of boolean's.
        start - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(byte[] obuf,
                        int offset,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of byte's.
        Specified by:
        read in interface ArrayDataInput
        Overrides:
        read in class java.io.BufferedInputStream
        Parameters:
        obuf - array of byte's.
        offset - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
        See Also:
        ArrayDataInput.readFully(byte[], int, int)
      • read

        public int read​(char[] c)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read an array of char's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        c - array of char's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(char[] c,
                        int start,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of char's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        c - array of char's.
        start - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(double[] d)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read an array of double's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        d - array of double's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(double[] d,
                        int start,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of double's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        d - array of double's.
        start - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(float[] f)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read an array of float's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        f - array of float's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(float[] f,
                        int start,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of float's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        f - array of float's.
        start - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(int[] i)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read an array of int's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        i - array of int's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(int[] i,
                        int start,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of int's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        i - array of int's.
        start - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(long[] l)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of long's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        l - array of long's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(long[] l,
                        int start,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of long's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        l - array of long's.
        start - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(short[] s)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read an array of short's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        s - array of short's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • read

        public int read​(short[] s,
                        int start,
                        int length)
                 throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a segment of an array of short's.
        Specified by:
        read in interface ArrayDataInput
        Parameters:
        s - array of short's.
        start - start index in the array
        length - number of array elements to read
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if one of the underlying read operations failed
      • readArray

        @Deprecated
        public int readArray​(java.lang.Object o)
                      throws java.io.IOException
        Deprecated.
        Description copied from interface: ArrayDataInput
        Read a generic (possibly multidimensional) primitive array. An Object[] array is also a legal argument if each element of the array is a legal.

        The ArrayDataInput classes do not support String input since it is unclear how one would read in an Array of strings.

        Specified by:
        readArray in interface ArrayDataInput
        Parameters:
        o - A [multidimensional] primitive (or Object) array.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if the underlying stream failed
      • readBoolean

        public boolean readBoolean()
                            throws java.io.IOException
        Specified by:
        readBoolean in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readByte

        public byte readByte()
                      throws java.io.IOException
        Specified by:
        readByte in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readBytesIntoSharedBuffer

        private void readBytesIntoSharedBuffer​(int bytes)
                                        throws java.io.IOException,
                                               java.io.EOFException
        Throws:
        java.io.IOException
        java.io.EOFException
      • readChar

        public char readChar()
                      throws java.io.IOException
        Specified by:
        readChar in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readDouble

        public double readDouble()
                          throws java.io.IOException
        Specified by:
        readDouble in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readFloat

        public float readFloat()
                        throws java.io.IOException
        Specified by:
        readFloat in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readFully

        public void readFully​(byte[] b)
                       throws java.io.IOException
        Specified by:
        readFully in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readFully

        public void readFully​(byte[] b,
                              int off,
                              int len)
                       throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read a buffer and signal an EOF if the requested elements cannot be read. This differs from read(b,off,len) since that call will not signal and end of file unless no bytes can be read. However both of these routines will attempt to fill their buffers completely.
        Specified by:
        readFully in interface ArrayDataInput
        Specified by:
        readFully in interface java.io.DataInput
        Parameters:
        b - The input buffer.
        off - The requested offset into the buffer.
        len - The number of bytes requested.
        Throws:
        java.io.IOException
      • readInt

        public int readInt()
                    throws java.io.IOException
        Specified by:
        readInt in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readLArray

        public long readLArray​(java.lang.Object o)
                        throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Read an object. An EOF will be signaled if the object cannot be fully read. This version works even if the underlying data is more than 2 Gigabytes.
        Specified by:
        readLArray in interface ArrayDataInput
        Parameters:
        o - The object to be read. This object should be a primitive (possibly multi-dimensional) array.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if the underlying stream failed
      • readLine

        @Deprecated
        public java.lang.String readLine()
                                  throws java.io.IOException
        Deprecated.
        Use BufferedReader methods.
        Emulate the deprecated DataInputStream.readLine() method. Originally we used the method itself, but Alan Brighton suggested using a java.io.BufferedReader to eliminate the deprecation warning. This was used for a long time, but more recently we noted that this doesn't work. We now use a simple method that largely ignores character encodings and only uses the "\n" as the line separator. This method is slow regardless. In the current version
        Specified by:
        readLine in interface java.io.DataInput
        Returns:
        The String read.
        Throws:
        java.io.IOException
      • readLong

        public long readLong()
                      throws java.io.IOException
        Specified by:
        readLong in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readPrimitiveArray

        @Deprecated
        public int readPrimitiveArray​(java.lang.Object o)
                               throws java.io.IOException
        Deprecated.
        use readLArray(Object) instead
        This routine provides efficient reading of arrays of any primitive type. It is an error to invoke this method with an object that is not an array of some primitive type. Note that there is no corresponding capability to writePrimitiveArray in BufferedDataOutputStream to read in an array of Strings.
        Parameters:
        o - The object to be read. It must be an array of a primitive type, or an array of Object's.
        Returns:
        number of bytes read.
        Throws:
        java.io.IOException - if the underlying read operation fails
      • readShort

        public short readShort()
                        throws java.io.IOException
        Specified by:
        readShort in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readUnsignedByte

        public int readUnsignedByte()
                             throws java.io.IOException
        Specified by:
        readUnsignedByte in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readUnsignedShort

        public int readUnsignedShort()
                              throws java.io.IOException
        Specified by:
        readUnsignedShort in interface java.io.DataInput
        Throws:
        java.io.IOException
      • readUTF

        public java.lang.String readUTF()
                                 throws java.io.IOException
        Specified by:
        readUTF in interface java.io.DataInput
        Throws:
        java.io.IOException
      • skipAllBytes

        public void skipAllBytes​(int toSkip)
                          throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Skip the number of bytes. This differs from the skip method in that it will throw an EOF if a forward skip cannot be fully accomplished... (However that isn't supposed to happen with a random access file, so there is probably no operational difference).
        Specified by:
        skipAllBytes in interface ArrayDataInput
        Parameters:
        toSkip - the number of bytes to skip
        Throws:
        java.io.IOException - if the underlying stream failed
      • skipAllBytes

        public void skipAllBytes​(long toSkip)
                          throws java.io.IOException
        Description copied from interface: ArrayDataInput
        Skip the number of bytes. This differs from the skip method in that it will throw an EOF if a forward skip cannot be fully accomplished... (However that isn't supposed to happen with a random access file, so there is probably no operational difference).
        Specified by:
        skipAllBytes in interface ArrayDataInput
        Parameters:
        toSkip - the number of bytes to skip
        Throws:
        java.io.IOException - if the underlying stream failed
      • handleExceptionInSkip

        private long handleExceptionInSkip​(long skip,
                                           java.io.IOException e)
                                    throws java.io.IOException
        Throws:
        java.io.IOException
      • skipBytes

        public int skipBytes​(int toSkip)
                      throws java.io.IOException
        Specified by:
        skipBytes in interface java.io.DataInput
        Throws:
        java.io.IOException
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object