3.9 Streams and Iterators

Streams are objects that act as a sequential channel of elements from (or even to) some source.

3.9.1 Basic Protocol

Streams respond to a number of common messages. However, many of these only work on some of the stream types, usually according to good sense:

next &onExhaustion:

reads and answers the next element in the stream. This causes the stream reader to advance one element. If no further elements are available to return, then the block supplied to &onExhaustion: is invoked (with no arguments), or if the optional argument was not supplied, an Exhaustion condition is signalled.

peek &onExhaustion:

reads and answers the next element in the stream. This does not advance the stream reader. If no further elements are available to return, the &onExhaustion: argument is processed as for #next.

next:

draws the next n number of elements from the stream and delivers them in a Sequence of the appropriate type. If fewer than n elements are available before the end of stream (when #next would have signalled Exhaustion), then a shorter Sequence than asked for, possibly even empty, is returned.

next:putInto:

reads the next N elements into the given Sequence starting from index 0. Returns the number of elements actually read and inserted. As for next:, the returned number may be smaller than actually asked for.

next:putInto:startingAt:

reads the N elements into the given Sequence starting from the given index. Returns the number of elements read and inserted, exactly as for next:putInto:.

nextPutInto:

reads into the given Sequence the number of elements which will fit into it. Returns a result exactly as for next:putInto:.

nextPut: &onExhaustion:

writes the object to the stream. If the stream is full, and will not accept the object, the &onExhaustion: optional argument is processed as for #next.

nextPutAll:

alias stream ; sequence writes all the objects in the Sequence to the stream. The ; selector allows the user to cascade several sequences into the stream as though they were concatenated. If the stream fills up (signalling Exhaustion) during the write, a PartialWrite condition containing the number of successfully written elements is signalled.

do:

applies a Block to each element of the stream.

flush

synchronizes the total state of the stream with any pending requests made by the user.

isAtEnd

answers whether or not the stream has reached some input limit: returns true if next or peek would signal Exhaustion at the time isAtEnd was called; returns false otherwise (modulo signalled Conditions). Note that, for some streams (notably sockets), even though isAtEnd has returned false, next or peek may still signal Exhaustion because the underlying input-source has changed its state asynchronously. Use the &onExhaustion: optional argument to next and peek for reliable notifications in these cases.

isWritable

answers whether the stream is prepared to accept further objects via nextPut:. Returns false if nextPut: would signal Exhaustion at the time isWritable was called; returns true otherwise (modulo signalled Conditions). See caveat as for isAtEnd.

upToEnd

collects all the elements of the stream up to its limit into an ExtensibleSequence, and returns the sequence.

contents

answers a collection of the output of the argument WriteStream.

3.9.2 Basic Stream Variants

Figure 3 shows the major stream types and their relationships.

Figure 3: Stream Inheritance

Stream

provides the basic protocol for instantiating streams.

ReadStream

provides the basic protocol for input access from a source.

WriteStream

provides the basic protocol for output access to a target.

ReadWriteStream

provides the basic protocol for both read and write access, and caches its input as necessary.

PeekableStream

extends Stream to provide the ability to look forward on the stream without advancing it.

PositionableStream

extends PeekableStream to provide a basic protocol to iterate over a sequence of elements from a Sequence or a file or other source. These streams store their position in the sequence as they iterate, and are re-positionable. It also has its own variants, -Read-, -Write-, and -ReadWrite-.

DummyStream

is a (singleton) ReadWriteStream that just answers Nil repeatedly (and does nothing on writing). It is best created by applying the reader, writer, or iterator methods to Nil.

EchoStream

is a wrapper for a Stream which copies all stream input/output interactions to another Stream for logging purposes. It is best created by applying the echo (goes to the Console) or echoTo: anotherStream methods to any stream. It answers the original stream, so that further processing can be chained.

Method ReadStream

is a ReadStream that targets a no-input block and returns its output each time. It is best created by applying the reader or iterator method to any block.

Method WriteStream

is a WriteStream that targets a single-input block and applies its input each time. It is best created by applying the writer method to any block.

StreamProcessor

is an abstract kind of ReadStream that has a source ReadStream which it processes in some way. Derivatives specialize it in various useful ways.

FilterStream

is a StreamProcessor that returns the elements of a wrapped ReadStream that satisfy the logical test of a single-argument block being applied to each element. It is created by applying the select: or reject: method to any Stream.

CollectStream

is a StreamProcessor that returns the results of applying a single-argument block to each element of a ReadStream that it wraps. It is created by applying the collect: method to any Stream.

InjectStream

is a StreamProcessor that returns the results of successively applying a two-argument block to an initial seed and each stream element. The result of the block is used as seed for the next block application. It is created by applying the inject:into: method to any Stream.

Tokenizer

is a StreamProcessor that returns the sequences of elements between a given set of separators. It is created by applying the split (using whitespace separators) or splitWith: methods to any Stream.

GeneratorStream

is a ReadStream that returns the results of successively applying a single-argument block to an initial seed. The result of the block is used as seed for the next block. The method generate:until: creates such a stream with a termination block which takes each element as the argument to determine where the stream should end.

ReadBufferStream

wraps another stream with a special Buffer object for reading large chunks from the stream at a time while handing out the elements as requested. This also minimizes stress on the memory-allocator by avoiding unnecessary allocation of arrays. It is created by applying the readBuffered method to any Stream.

WriteBufferStream

wraps another stream with a special Buffer object for writing large chunks to the stream at a time while accepting new elements as requested. This also minimizes stress on the memory-allocator by avoiding unnecessary allocation of arrays. It is created by applying the writeBuffered method to any Stream.

3.9.3 Basic Instantiation

There are a number of ways to create Streams, and a large number of implementations, so some methods exist to simplify the process of making a new one:

newOn:

creates a new Stream of the same type as the first argument, targeting it to the second as a source. This should not be overridden. Instead, the re-targeting method on: is overridden.

newTo:

creates a new WriteStream of the appropriate type on the specified target. This should be overridden for derived types, and the first argument should apply to the generic Stream type to allow any instance to know this protocol.

newFrom:

creates a new ReadStream of the appropriate type on the specified target. This should be overridden for derived types, and the first argument should apply to the generic Stream type to allow any instance to know this protocol.

buffered

creates and returns a new BufferStream whose type corresponds to the argument and wraps the argument Stream.

readBuffered

creates and returns a new ReadBufferStream which wraps the argument Stream.

writeBuffered

creates and returns a new WriteBufferStream which wraps the argument Stream.

echoTo:

creates and returns a new EchoStream which wraps the first argument Stream and echoes to the second.

echo

creates and returns a new EchoStream to the Console.

>>

performs a looping iterative transfer of all elements of the first stream to the second. The second argument may be any WriteStream, or a StreamProcessor, or a single-argument Method in which case it has the same semantics as collect:. For targets to ExternalResources, it will perform a buffered transfer. This method always returns the target stream so that the results may be further processed.

3.9.4 Collecting Protocols

Mirroring the collection protocols, streams support a mirror of that interface (do:, select:, collect:, reject:, inject:into:). The difference is that where collections would answer other collections, streams return corresponding streams.

3.9.5 Iterator Streams

Many types (typically collections) define their own Stream type which goes over its elements in series, even if the collection is not ordered, and only visits each element once. This type's prototype is accessed via the slot ReadStream within each collection (located on its traits object). So ``Set ReadStream'' refers to the prototype suitable for iterating over Sets.

In order to create a new iterator for a specific collection, the iterator message is provided, which clones the prototype for that collection's type and targets it to the receiver of the message. The protocol summary:

iterator

will return a ReadStream or preferably a ReadWriteStream if one is available for the type, targetted to the argument of the message.

reader

and writer get streams with only ReadStream and WriteStream capabilities for the type, when available, targetted to the argument of the message.

The stream capabilities supported for each basic collection type are usually limited by the behavior that the type supports. The capabilities per basic type are as follows; types not mentioned inherit or specialize the capabilities of their ancestors:

Type	Capabilities
Collection	none
ExtensibleCollection	Write
Bag	Read and Write separately
Sequence	Positionables (R, W, RW); copy for extension
ExtensibleSequence	Positionables (R, W, RW)