Streams

Streaming data. More...

Macros
#define	using_stream(s)
	Use a stream in a scope, freeing the stream when exiting. More...
#define	using_bstream(s)
	Use a buffered stream in a scope, freeing the stream when exiting. More...

Typedefs
typedef struct _nodec_stream_t	nodec_stream_t
	Basic unbuffered streams.
typedef struct _nodec_bstream_t	nodec_bstream_t
	The type of buffered streams. More...
typedef struct _uv_stream_t	uv_stream_t
	A libuv stream. More...

Functions
void	nodec_stream_free (nodec_stream_t *stream)
	Free a basic stream. More...
void	nodec_stream_freev (lh_value streamv)
	Free a basic stream as an lh_value. More...
void	async_shutdown (nodec_stream_t *stream)
	Shutdown a stream. More...
uv_buf_t	async_read_bufx (nodec_stream_t *stream, bool *buf_owned)
	Asynchronously read a buffer from a stream. More...
uv_buf_t	async_read_buf (nodec_stream_t *stream)
	Asynchronously read a buffer from a stream. More...
char *	async_read (nodec_stream_t *stream)
	Asynchronously read a string from a stream.
void	async_write_bufs (nodec_stream_t *stream, uv_buf_t bufs[], size_t count)
	Write an array of buffers to a stream. More...
void	async_write_buf (nodec_stream_t *stream, uv_buf_t buf)
	Write a buffer to a stream.
void	async_write (nodec_stream_t *stream, const char *s)
	Write a string to a stream. More...
void	async_vprintf (nodec_stream_t *stream, const char *fmt, va_list args)
	Write a formatted string to a stream. More...
void	async_printf (nodec_stream_t *stream, const char *fmt,...)
	Write a formatted string to a stream. More...
nodec_stream_t *	as_stream (nodec_bstream_t *stream)
	Safely cast a buffered stream to a basic stream.
uv_buf_t	async_read_buf_all (nodec_bstream_t *bstream, size_t read_max)
	Read the entire stream as a single buffer. More...
char *	async_read_all (nodec_bstream_t *bstream, size_t read_max)
	Read the entire stream as a string. More...
size_t	async_read_into (nodec_bstream_t *bstream, uv_buf_t buf)
	Read a stream into a pre-allocated buffer. More...
uv_buf_t	async_read_buf_including (nodec_bstream_t *bstream, size_t *toread, const void *pat, size_t pat_len, size_t read_max)
	Read a stream until some pattern is encountered. More...
uv_buf_t	async_read_buf_upto (nodec_bstream_t *bstream, const void *pat, size_t pat_len, size_t read_max)
	Read a stream until some pattern is encountered. More...
uv_buf_t	async_read_buf_line (nodec_bstream_t *bstream)
	Read the first line of a buffered stream.
char *	async_read_line (nodec_bstream_t *bstream)
	Read the first line as a string from a buffered stream.
void	nodec_pushback_buf (nodec_bstream_t *bstream, uv_buf_t buf)
	Push back a buffer onto a read stream. More...
nodec_bstream_t *	nodec_bstream_alloc_on (nodec_stream_t *source)
	Create a buffered stream from a plain stream.
nodec_bstream_t *	nodec_bstream_alloc (uv_stream_t *stream)
	Low level: Create a buffered stream from an internal uv_stream_t. More...
nodec_bstream_t *	nodec_bstream_alloc_read (uv_stream_t *stream)
	Low level: Create a buffered stream from an internal uv_stream_t for reading. More...
nodec_bstream_t *	nodec_bstream_alloc_read_ex (uv_stream_t *stream, size_t alloc_init, size_t alloc_max)
	Low level: Create a buffered stream from an internal uv_stream_t for reading. More...
nodec_bstream_t *	nodec_zstream_alloc (nodec_stream_t *stream, bool own_stream)
	Create a stream over another stream that is gzip'd. More...
nodec_bstream_t *	nodec_zstream_alloc_ex (nodec_stream_t *stream, bool own_stream, int compress_level, bool gzip)
	Create a stream over another stream that is gzip'd. More...

Detailed Description

Streaming data.

The main interfaces are plain streams as nodec_stream_t and buffered streams nodec_bstream_t.

Macro Definition Documentation

using_bstream

#define using_bstream

(

s

)

Use a buffered stream in a scope, freeing the stream when exiting.

This safely uses the stream in a scope and frees the stream afterwards (even if an exception is thrown). Uses async_shutdown(stream) too if no exception was thrown.

Parameters

s	the nodec_bstream_t to free after use.

Example

nodec_bstream_t s = nodec_tcp_connect("http://www.google.com");
{using_bstream(s){
   uv_buf_t buf = async_read_buf_all(s);
   {using_buf(&buf){
      printf("received: %s\n", buf.base);
   }}
}}

using_stream

#define using_stream

(

s

)

Use a stream in a scope, freeing the stream when exiting.

This safely uses the stream in a scope and frees the stream afterwards (even if an exception is thrown). Uses async_shutdown(stream) too if no exception was thrown.

Parameters

s	the nodec_stream_t to free after use.

Example

nodec_stream_t s = ...
{using_stream(s){
   async_write(s, "hello");
}}

Typedef Documentation

nodec_bstream_t

typedef struct _nodec_bstream_t nodec_bstream_t

The type of buffered streams.

Derives from a basic nodec_stream_t and can be cast to it using as_stream(). Buffered streams have added functionality, like being able to read a stream up to a certain pattern is encountered, or reading the full stream as a single buffer.

uv_stream_t

typedef struct _uv_stream_t uv_stream_t

A libuv stream.

Try not to use these directly but wrap it into a buffered stream, See nodec_bstream_alloc() and nodec_bstream_alloc_read().

Function Documentation

async_printf()

void async_printf	(	nodec_stream_t *	stream,
		const char *	fmt,
			...
	)

Write a formatted string to a stream.

Writes at most 511 bytes after formatting.

Parameters

stream	stream to write to.
fmt	the format string.
...	the arguments for the format string.

async_read_all()

char* async_read_all	(	nodec_bstream_t *	bstream,
		size_t	read_max
	)

Read the entire stream as a string.

Parameters

bstream	the stream to read from.
read_max	maximum number of bytes to read. Use 0 (or SIZE_MAX) for unlimited.

Returns

the data read in a callee owned string (see using_str()).

async_read_buf()

uv_buf_t async_read_buf

(

nodec_stream_t *

stream

)

Asynchronously read a buffer from a stream.

Parameters

stream

stream to read from.

Returns

uv_buf_t a buffer with the data. This buffer is becomes the callee's responsibility (see using_buf()) and is passed without copying from the lower level read routines.

async_read_buf_all()

uv_buf_t async_read_buf_all	(	nodec_bstream_t *	bstream,
		size_t	read_max
	)

Read the entire stream as a single buffer.

Parameters

bstream	the stream to read from.
read_max	maximum number of bytes to read. Use 0 (or SIZE_MAX) for unlimited.

Returns

the data read in a callee owned buffer (see using_buf()).

async_read_buf_including()

uv_buf_t async_read_buf_including	(	nodec_bstream_t *	bstream,
		size_t *	toread,
		const void *	pat,
		size_t	pat_len,
		size_t	read_max
	)

Read a stream until some pattern is encountered.

The returned buffer will contain pat unless the end-of-stream was encountered or read_max bytes were read. The return buffer may contain more data following the the pattern pat. This is more efficient than async_read_buf_upto which may involve some memory copying to split buffers.

Parameters

	bstream	the stream to read from.
[out]	toread	the number of bytes just including the pattern pat. 0 on failure.
[in]	pat	the pattern to scan for.
[in]	pat_len	the length of the patter. For efficiency reasons, this can be at most 8 (bytes).
[in]	read_max	stop reading after read_max bytes have been seen. Depending on the internal buffering, the returned buffer might still contain more bytes.

Returns

buffer with the read bytes where buf.len >= *toread. Returns a null buffer (see nodec_buf_is_null()) if an end-of-stream is encountered.

async_read_buf_upto()

uv_buf_t async_read_buf_upto	(	nodec_bstream_t *	bstream,
		const void *	pat,
		size_t	pat_len,
		size_t	read_max
	)

Read a stream until some pattern is encountered.

The returned buffer will contain pat at its end unless the end-of-stream was encountered or read_max bytes were read.

Parameters

	bstream	the stream to read from.
[in]	pat	the pattern to scan for.
[in]	pat_len	the length of the patter. For efficiency reasons, this can be at most 8 (bytes).
[in]	read_max	stop reading after read_max bytes have been seen. Depending on the internal buffering, the returned buffer might still contain more bytes.

Returns

buffer with the read bytes. Returns a null buffer (see nodec_buf_is_null()) if an end-of-stream is encountered.

async_read_bufx()

uv_buf_t async_read_bufx	(	nodec_stream_t *	stream,
		bool *	buf_owned
	)

Asynchronously read a buffer from a stream.

This is the most efficient form of reading since it can even read by providing unowned view buffers but it is also most complex as returned unowned buffers are only valid until the next read call. See async_read_buf() for more robust reading that always returns callee owned buffers.

Parameters

	stream	stream to read from.
[out]	buf_owned	this is set to true if the returned buffer is owned, i.e. should be freed by the callee.

Returns

a buffer with the data. This buffer is becomes the callee's responsibility if buf_owned is true; otherwise the buffer should not be freed by the callee. See using_buf_owned() to do this automatically. Moreover, if a buffer is returned with *buf_owned false, the buffer only stays valid until the next read from that stream.

Example

nodec_stream_t stream = http_req_body();
{using_stream(stream){
   uv_buf_t buf;
   bool owned;
   while( (buf = async_read_bufx(stream,&owned), !nodec_buf_is_null(buf)) ) {
     {using_buf_owned(owned,buf){
        printf("data read: %s\n", buf.base);
     }}
   }
}}

async_read_into()

size_t async_read_into	(	nodec_bstream_t *	bstream,
		uv_buf_t	buf
	)

Read a stream into a pre-allocated buffer.

Parameters

bstream	the stream to read from.
buf	the buffer to read into; reads up to either the end of the stream or up to the buf.len.

Returns

the number of bytes read.

async_shutdown()

void async_shutdown

(

nodec_stream_t *

stream

)

Shutdown a stream.

This closes the stream gracefully for writes (nodec_stream_free() does this too but closes the stream abruptly). Usually this is not used directly but instead using_stream() is used.

async_vprintf()

void async_vprintf	(	nodec_stream_t *	stream,
		const char *	fmt,
		va_list	args
	)

Write a formatted string to a stream.

Writes at most 511 bytes after formatting.

Parameters

stream	stream to write to.
fmt	the format string.
args	the arguments for the format string.

async_write()

void async_write	(	nodec_stream_t *	stream,
		const char *	s
	)

Write a string to a stream.

Parameters

stream	stream to write to.
s	the string to write.

async_write_bufs()

void async_write_bufs	(	nodec_stream_t *	stream,
		uv_buf_t	bufs[],
		size_t	count
	)

Write an array of buffers to a stream.

Parameters

stream	the stream to write to.
bufs	the array of buffers to write.
count	the number of buffers in the array.

nodec_bstream_alloc()

nodec_bstream_t* nodec_bstream_alloc

(

uv_stream_t *

stream

)

Low level: Create a buffered stream from an internal uv_stream_t.

Parameters

stream

the underlying uv_stream_t. Freed when the nodec_bstream_t is freed.

Returns

a buffered stream.

nodec_bstream_alloc_read()

nodec_bstream_t* nodec_bstream_alloc_read

(

uv_stream_t *

stream

)

Low level: Create a buffered stream from an internal uv_stream_t for reading.

Parameters

stream

the underlying uv_stream_t. Freed when the nodec_bstream_t is freed.

Returns

a buffered stream.

nodec_bstream_alloc_read_ex()

nodec_bstream_t* nodec_bstream_alloc_read_ex	(	uv_stream_t *	stream,
		size_t	alloc_init,
		size_t	alloc_max
	)

Low level: Create a buffered stream from an internal uv_stream_t for reading.

Parameters

stream	the underlying uv_stream_t. Freed when the nodec_bstream_t is freed.
alloc_init	the initial allocation size for read buffers. Use 0 for default (8k). Doubles on further data until alloc_max.
alloc_max	the maximal allocation size for read buffers.

Returns

a buffered stream.

nodec_pushback_buf()

void nodec_pushback_buf	(	nodec_bstream_t *	bstream,
		uv_buf_t	buf
	)

Push back a buffer onto a read stream.

The data pushed back will be read on the next read call.

nodec_stream_free()

void nodec_stream_free

(

nodec_stream_t *

stream

)

Free a basic stream.

Usually this is not used directly but instead using_stream() is used.

nodec_stream_freev()

void nodec_stream_freev

(

lh_value

streamv

)

Free a basic stream as an lh_value.

Usually this is not used directly but instead using_stream() is used.

nodec_zstream_alloc()

nodec_bstream_t* nodec_zstream_alloc	(	nodec_stream_t *	stream,
		bool	own_stream
	)

Create a stream over another stream that is gzip'd.

Reading unzips, while writing zips again. Uses a default compression level of 6 and gzip format (instead of deflate).

Parameters

stream	the gzipped stream
own_stream	If true, then the returned stream owns the input stream and takes care of shutdown and free.

Returns

a new buffered stream that automatically (de)compresses.

nodec_zstream_alloc_ex()

nodec_bstream_t* nodec_zstream_alloc_ex	(	nodec_stream_t *	stream,
		bool	own_stream,
		int	compress_level,
		bool	gzip
	)

Create a stream over another stream that is gzip'd.

Reading unzips automatically, while writing zips again.

Parameters

stream	the gzipped stream – after allocation the returned stream owns this stream and takes care of shutdown and free.
own_stream	If true, then the returned stream owns the input stream and takes care of shutdown and free.
compress_level	the compression level to use between 1 and 9 (6 by default, which is a good balance between speed and compression ratio).
gzip	if true uses the gzip format for compression while reading both deflate and gzip streams. If false, uses deflate for both compression and decompression.

Returns

a new buffered stream that automatically (de)compresses.