Struct `AnyDelimiterCodec`

struct AnyDelimiterCodec { ... }

A simple Decoder and Encoder implementation that splits up data into chunks based on any character in the given delimiter string.

Example

Decode string of bytes containing various different delimiters.

use tokio_util::codec::{AnyDelimiterCodec, Decoder};
use bytes::{BufMut, BytesMut};

#
# #[tokio::main(flavor = "current_thread")]
# async fn main() -> Result<(), std::io::Error> {
let mut codec = AnyDelimiterCodec::new(b",;\r\n".to_vec(),b";".to_vec());
let buf = &mut BytesMut::new();
buf.reserve(200);
buf.put_slice(b"chunk 1,chunk 2;chunk 3\n\r");
assert_eq!("chunk 1", codec.decode(buf).unwrap().unwrap());
assert_eq!("chunk 2", codec.decode(buf).unwrap().unwrap());
assert_eq!("chunk 3", codec.decode(buf).unwrap().unwrap());
assert_eq!("", codec.decode(buf).unwrap().unwrap());
assert_eq!(None, codec.decode(buf).unwrap());
# Ok(())
# }

Implementations

`impl AnyDelimiterCodec`

fn new(seek_delimiters: Vec<u8>, sequence_writer: Vec<u8>) -> AnyDelimiterCodec

Returns a AnyDelimiterCodec for splitting up data into chunks.

Note

The returned AnyDelimiterCodec will not have an upper bound on the length of a buffered chunk. See the documentation for new_with_max_length for information on why this could be a potential security risk.

fn new_with_max_length(seek_delimiters: Vec<u8>, sequence_writer: Vec<u8>, max_length: usize) -> Self

Returns a AnyDelimiterCodec with a maximum chunk length limit.

If this is set, calls to AnyDelimiterCodec::decode will return a AnyDelimiterCodecError when a chunk exceeds the length limit. Subsequent calls will discard up to limit bytes from that chunk until a delimiter character is reached, returning None until the delimiter over the limit has been fully discarded. After that point, calls to decode will function as normal.

Note

Setting a length limit is highly recommended for any AnyDelimiterCodec which will be exposed to untrusted input. Otherwise, the size of the buffer that holds the chunk currently being read is unbounded. An attacker could exploit this unbounded buffer by sending an unbounded amount of input without any delimiter characters, causing unbounded memory consumption.

fn max_length(self: &Self) -> usize

Returns the maximum chunk length when decoding.

use std::usize;
use tokio_util::codec::AnyDelimiterCodec;

let codec = AnyDelimiterCodec::new(b",;\n".to_vec(), b";".to_vec());
assert_eq!(codec.max_length(), usize::MAX);

use tokio_util::codec::AnyDelimiterCodec;

let codec = AnyDelimiterCodec::new_with_max_length(b",;\n".to_vec(), b";".to_vec(), 256);
assert_eq!(codec.max_length(), 256);