Struct StreamReader

struct StreamReader<S, B> { ... }

Convert a Stream of byte chunks into an AsyncRead.

This type performs the inverse operation of ReaderStream.

This type also implements the AsyncBufRead trait, so you can use it to read a Stream of byte chunks line-by-line. See the examples below.

Example

use bytes::Bytes;
use tokio::io::{AsyncReadExt, Result};
use tokio_util::io::StreamReader;
# #[tokio::main(flavor = "current_thread")]
# async fn main() -> std::io::Result<()> {

// Create a stream from an iterator.
let stream = tokio_stream::iter(vec![
    Result::Ok(Bytes::from_static(&[0, 1, 2, 3])),
    Result::Ok(Bytes::from_static(&[4, 5, 6, 7])),
    Result::Ok(Bytes::from_static(&[8, 9, 10, 11])),
]);

// Convert it to an AsyncRead.
let mut read = StreamReader::new(stream);

// Read five bytes from the stream.
let mut buf = [0; 5];
read.read_exact(&mut buf).await?;
assert_eq!(buf, [0, 1, 2, 3, 4]);

// Read the rest of the current chunk.
assert_eq!(read.read(&mut buf).await?, 3);
assert_eq!(&buf[..3], [5, 6, 7]);

// Read the next chunk.
assert_eq!(read.read(&mut buf).await?, 4);
assert_eq!(&buf[..4], [8, 9, 10, 11]);

// We have now reached the end.
assert_eq!(read.read(&mut buf).await?, 0);

# Ok(())
# }

If the stream produces errors which are not std::io::Error, the errors can be converted using StreamExt to map each element.

use bytes::Bytes;
use tokio::io::AsyncReadExt;
use tokio_util::io::StreamReader;
use tokio_stream::StreamExt;
# #[tokio::main(flavor = "current_thread")]
# async fn main() -> std::io::Result<()> {

// Create a stream from an iterator, including an error.
let stream = tokio_stream::iter(vec![
    Result::Ok(Bytes::from_static(&[0, 1, 2, 3])),
    Result::Ok(Bytes::from_static(&[4, 5, 6, 7])),
    Result::Err("Something bad happened!")
]);

// Use StreamExt to map the stream and error to a std::io::Error
let stream = stream.map(|result| result.map_err(|err| {
    std::io::Error::new(std::io::ErrorKind::Other, err)
}));

// Convert it to an AsyncRead.
let mut read = StreamReader::new(stream);

// Read five bytes from the stream.
let mut buf = [0; 5];
read.read_exact(&mut buf).await?;
assert_eq!(buf, [0, 1, 2, 3, 4]);

// Read the rest of the current chunk.
assert_eq!(read.read(&mut buf).await?, 3);
assert_eq!(&buf[..3], [5, 6, 7]);

// Reading the next chunk will produce an error
let error = read.read(&mut buf).await.unwrap_err();
assert_eq!(error.kind(), std::io::ErrorKind::Other);
assert_eq!(error.into_inner().unwrap().to_string(), "Something bad happened!");

// We have now reached the end.
assert_eq!(read.read(&mut buf).await?, 0);

# Ok(())
# }

Using the AsyncBufRead impl, you can read a Stream of byte chunks line-by-line. Note that you will usually also need to convert the error type when doing this. See the second example for an explanation of how to do this.

use tokio::io::{Result, AsyncBufReadExt};
use tokio_util::io::StreamReader;
# #[tokio::main(flavor = "current_thread")]
# async fn main() -> std::io::Result<()> {

// Create a stream of byte chunks.
let stream = tokio_stream::iter(vec![
    Result::Ok(b"The first line.\n".as_slice()),
    Result::Ok(b"The second line.".as_slice()),
    Result::Ok(b"\nThe third".as_slice()),
    Result::Ok(b" line.\nThe fourth line.\nThe fifth line.\n".as_slice()),
]);

// Convert it to an AsyncRead.
let mut read = StreamReader::new(stream);

// Loop through the lines from the `StreamReader`.
let mut line = String::new();
let mut lines = Vec::new();
loop {
    line.clear();
    let len = read.read_line(&mut line).await?;
    if len == 0 { break; }
    lines.push(line.clone());
}

// Verify that we got the lines we expected.
assert_eq!(
    lines,
    vec![
        "The first line.\n",
        "The second line.\n",
        "The third line.\n",
        "The fourth line.\n",
        "The fifth line.\n",
    ]
);
# Ok(())
# }

Implementations

impl<S, B> StreamReader<S, B>

fn get_ref(self: &Self) -> &S

Gets a reference to the underlying stream.

It is inadvisable to directly read from the underlying stream.

fn get_mut(self: &mut Self) -> &mut S

Gets a mutable reference to the underlying stream.

It is inadvisable to directly read from the underlying stream.

fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut S>

Gets a pinned mutable reference to the underlying stream.

It is inadvisable to directly read from the underlying stream.

fn into_inner(self: Self) -> S

Consumes this BufWriter, returning the underlying stream.

Note that any leftover data in the internal buffer is lost. If you additionally want access to the internal buffer use into_inner_with_chunk.

impl<S, B, E> StreamReader<S, B>

fn new(stream: S) -> Self

Convert a stream of byte chunks into an AsyncRead.

The item should be a Result with the ok variant being something that implements the Buf trait (e.g. Cursor<Vec<u8>> or Bytes). The error should be convertible into an io error.

fn into_inner_with_chunk(self: Self) -> (S, Option<B>)

Consumes this StreamReader, returning a Tuple consisting of the underlying stream and an Option of the internal buffer, which is Some in case the buffer contains elements.

impl<R> AsyncBufReadExt for StreamReader<S, B>

impl<R> AsyncReadExt for StreamReader<S, B>

impl<S, B> Freeze for StreamReader<S, B>

impl<S, B> RefUnwindSafe for StreamReader<S, B>

impl<S, B> Send for StreamReader<S, B>

impl<S, B> Sync for StreamReader<S, B>

impl<S, B> UnsafeUnpin for StreamReader<S, B>

impl<S, B> UnwindSafe for StreamReader<S, B>

impl<S, B, E> AsyncBufRead for StreamReader<S, B>

fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<&[u8]>>
fn consume(self: Pin<&mut Self>, amt: usize)

impl<S, B, E> AsyncRead for StreamReader<S, B>

fn poll_read(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<Result<()>>

impl<S: $crate::fmt::Debug, B: $crate::fmt::Debug> Debug for StreamReader<S, B>

fn fmt(self: &Self, f: &mut Formatter<'_>) -> Result

impl<S: Sink<T, Error = E>, B, E, T> Sink for StreamReader<S, B>

fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), <Self as >::Error>>
fn start_send(self: Pin<&mut Self>, item: T) -> Result<(), <Self as >::Error>
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), <Self as >::Error>>
fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), <Self as >::Error>>

impl<S: Unpin, B> Unpin for StreamReader<S, B>

impl<T> Any for StreamReader<S, B>

fn type_id(self: &Self) -> TypeId

impl<T> Borrow for StreamReader<S, B>

fn borrow(self: &Self) -> &T

impl<T> BorrowMut for StreamReader<S, B>

fn borrow_mut(self: &mut Self) -> &mut T

impl<T> From for StreamReader<S, B>

fn from(t: T) -> T

Returns the argument unchanged.

impl<T, U> Into for StreamReader<S, B>

fn into(self: Self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of [From]<T> for U chooses to do.

impl<T, U> TryFrom for StreamReader<S, B>

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

impl<T, U> TryInto for StreamReader<S, B>

fn try_into(self: Self) -> Result<U, <U as TryFrom<T>>::Error>