Struct BoundedBacktracker
struct BoundedBacktracker { ... }
A backtracking regex engine that bounds its execution to avoid exponential blow-up.
This regex engine only implements leftmost-first match semantics and
only supports leftmost searches. It effectively does the same thing as a
PikeVM, but typically does it faster because
it doesn't have to worry about copying capturing group spans for most NFA
states. Instead, the backtracker can maintain one set of captures (provided
by the caller) and never needs to copy them. In exchange, the backtracker
bounds itself to ensure it doesn't exhibit worst case exponential time.
This results in the backtracker only being able to handle short haystacks
given reasonable memory usage.
Searches may return an error!
By design, this backtracking regex engine is bounded. This bound is
implemented by not visiting any combination of NFA state ID and position
in a haystack more than once. Thus, the total memory required to bound
backtracking is proportional to haystack.len() * nfa.states().len().
This can obviously get quite large, since large haystacks aren't terribly
uncommon. To avoid using exorbitant memory, the capacity is bounded by
a fixed limit set via Config::visited_capacity. Thus, if the total
capacity required for a particular regex and a haystack exceeds this
capacity, then the search routine will return an error.
Unlike other regex engines that may return an error at search time (like the DFA or the hybrid NFA/DFA), there is no way to guarantee that a bounded backtracker will work for every haystack. Therefore, this regex engine only exposes fallible search routines to avoid the footgun of panicking when running a search on a haystack that is too big.
If one wants to use the fallible search APIs without handling the
error, the only way to guarantee an error won't occur from the
haystack length is to ensure the haystack length does not exceed
BoundedBacktracker::max_haystack_len.
Example: Unicode word boundaries
This example shows that the bounded backtracker implements Unicode word boundaries correctly by default.
# if cfg! // miri takes too long
use ;
let re = new?;
let mut cache = re.create_cache;
let mut it = re.try_find_iter;
assert_eq!;
assert_eq!;
assert_eq!;
# Ok::
Example: multiple regex patterns
The bounded backtracker supports searching for multiple patterns simultaneously, just like other regex engines. Note though that because it uses a backtracking strategy, this regex engine is unlikely to scale well as more patterns are added. But then again, as more patterns are added, the maximum haystack length allowed will also shorten (assuming the visited capacity remains invariant).
use ;
let re = new_many?;
let mut cache = re.create_cache;
let mut it = re.try_find_iter;
assert_eq!;
assert_eq!;
assert_eq!;
assert_eq!;
assert_eq!;
assert_eq!;
assert_eq!;
# Ok::
Implementations
impl BoundedBacktracker
fn try_is_match<'h, I: Into<Input<'h>>>(self: &Self, cache: &mut Cache, input: I) -> Result<bool, MatchError>Returns true if and only if this regex matches the given haystack.
In the case of a backtracking regex engine, and unlike most other regex engines in this crate, short circuiting isn't practical. However, this routine may still be faster because it instructs backtracking to not keep track of any capturing groups.
Errors
This routine only errors if the search could not complete. For this backtracking regex engine, this only occurs when the haystack length exceeds
BoundedBacktracker::max_haystack_len.When a search cannot complete, callers cannot know whether a match exists or not.
Example
use BoundedBacktracker; let re = new?; let mut cache = re.create_cache; assert!; assert!; # Ok::Example: consistency with search APIs
is_matchis guaranteed to returntruewheneverfindreturns a match. This includes searches that are executed entirely within a codepoint:use ; let re = new?; let mut cache = re.create_cache; assert!; # Ok::Notice that when UTF-8 mode is disabled, then the above reports a match because the restriction against zero-width matches that split a codepoint has been lifted:
use ; let re = builder .thompson .build?; let mut cache = re.create_cache; assert!; # Ok::fn try_find<'h, I: Into<Input<'h>>>(self: &Self, cache: &mut Cache, input: I) -> Result<Option<Match>, MatchError>Executes a leftmost forward search and returns a
Matchif one exists.This routine only includes the overall match span. To get access to the individual spans of each capturing group, use
BoundedBacktracker::try_captures.Errors
This routine only errors if the search could not complete. For this backtracking regex engine, this only occurs when the haystack length exceeds
BoundedBacktracker::max_haystack_len.When a search cannot complete, callers cannot know whether a match exists or not.
Example
use ; let re = new?; let mut cache = re.create_cache; let expected = must; assert_eq!; # Ok::fn try_captures<'h, I: Into<Input<'h>>>(self: &Self, cache: &mut Cache, input: I, caps: &mut Captures) -> Result<(), MatchError>Executes a leftmost forward search and writes the spans of capturing groups that participated in a match into the provided
Capturesvalue. If no match was found, thenCaptures::is_matchis guaranteed to returnfalse.Errors
This routine only errors if the search could not complete. For this backtracking regex engine, this only occurs when the haystack length exceeds
BoundedBacktracker::max_haystack_len.When a search cannot complete, callers cannot know whether a match exists or not.
Example
use ; let re = new?; let = ; re.try_captures?; assert!; assert_eq!; assert_eq!; assert_eq!; # Ok::fn try_find_iter<'r, 'c, 'h, I: Into<Input<'h>>>(self: &'r Self, cache: &'c mut Cache, input: I) -> TryFindMatches<'r, 'c, 'h>Returns an iterator over all non-overlapping leftmost matches in the given bytes. If no match exists, then the iterator yields no elements.
If the regex engine returns an error at any point, then the iterator will yield that error.
Example
use ; let re = new?; let mut cache = re.create_cache; let text = "foo1 foo12 foo123"; let result: = re .try_find_iter .collect; let matches = result?; assert_eq!; # Ok::fn try_captures_iter<'r, 'c, 'h, I: Into<Input<'h>>>(self: &'r Self, cache: &'c mut Cache, input: I) -> TryCapturesMatches<'r, 'c, 'h>Returns an iterator over all non-overlapping
Capturesvalues. If no match exists, then the iterator yields no elements.This yields the same matches as
BoundedBacktracker::try_find_iter, but it includes the spans of all capturing groups that participate in each match.If the regex engine returns an error at any point, then the iterator will yield that error.
Tip: See
util::iter::Searcherfor how to correctly iterate over all matches in a haystack while avoiding the creation of a newCapturesvalue for every match. (Which you are forced to do with anIterator.)Example
use ; let re = new?; let mut cache = re.create_cache; let text = "foo1 foo12 foo123"; let mut spans = vec!; for result in re.try_captures_iter assert_eq!; # Ok::
impl BoundedBacktracker
fn new(pattern: &str) -> Result<BoundedBacktracker, BuildError>Parse the given regular expression using the default configuration and return the corresponding
BoundedBacktracker.If you want a non-default configuration, then use the
Builderto set your own configuration.Example
use ; let re = new?; let mut cache = re.create_cache; assert_eq!; # Ok::fn new_many<P: AsRef<str>>(patterns: &[P]) -> Result<BoundedBacktracker, BuildError>Like
new, but parses multiple patterns into a single "multi regex." This similarly uses the default regex configuration.Example
use ; let re = new_many?; let mut cache = re.create_cache; let mut it = re.try_find_iter; assert_eq!; assert_eq!; assert_eq!; assert_eq!; assert_eq!; assert_eq!; assert_eq!; # Ok::fn new_from_nfa(nfa: NFA) -> Result<BoundedBacktracker, BuildError>Example
This shows how to hand assemble a regular expression via its HIR, compile an NFA from it and build a BoundedBacktracker from the NFA.
use ; use ; let hir = class; let config = NFAconfig.nfa_size_limit; let nfa = NFAcompiler.configure.build_from_hir?; let re = new_from_nfa?; let = ; let expected = Some; re.try_captures?; assert_eq!; # Ok::fn always_match() -> Result<BoundedBacktracker, BuildError>Create a new
BoundedBacktrackerthat matches every input.Example
use ; let re = always_match?; let mut cache = re.create_cache; let expected = Some; assert_eq!; assert_eq!; # Ok::fn never_match() -> Result<BoundedBacktracker, BuildError>Create a new
BoundedBacktrackerthat never matches any input.Example
use BoundedBacktracker; let re = never_match?; let mut cache = re.create_cache; assert_eq!; assert_eq!; # Ok::fn config() -> ConfigReturn a default configuration for a
BoundedBacktracker.This is a convenience routine to avoid needing to import the
Configtype when customizing the construction of aBoundedBacktracker.Example
This example shows how to disable UTF-8 mode. When UTF-8 mode is disabled, zero-width matches that split a codepoint are allowed. Otherwise they are never reported.
In the code below, notice that
""is permitted to match positions that split the encoding of a codepoint.use ; let re = builder .thompson .build?; let mut cache = re.create_cache; let haystack = "a☃z"; let mut it = re.try_find_iter; assert_eq!; assert_eq!; assert_eq!; assert_eq!; assert_eq!; assert_eq!; assert_eq!; # Ok::fn builder() -> BuilderReturn a builder for configuring the construction of a
BoundedBacktracker.This is a convenience routine to avoid needing to import the
Buildertype in common cases.Example
This example shows how to use the builder to disable UTF-8 mode everywhere.
# if cfg! // miri takes too long use ; let re = builder .syntax .thompson .build?; let = ; let haystack = b"\xFEfoo\xFFarzz\xE2\x98\xFF\n"; let expected = Some; re.try_captures?; assert_eq!; # Ok::fn create_cache(self: &Self) -> CacheCreate a new cache for this regex.
The cache returned should only be used for searches for this regex. If you want to reuse the cache for another regex, then you must call
Cache::resetwith that regex (or, equivalently,BoundedBacktracker::reset_cache).fn create_captures(self: &Self) -> CapturesCreate a new empty set of capturing groups that is guaranteed to be valid for the search APIs on this
BoundedBacktracker.A
Capturesvalue created for a specificBoundedBacktrackercannot be used with any otherBoundedBacktracker.This is a convenience function for
Captures::all. See theCapturesdocumentation for an explanation of its alternative constructors that permit theBoundedBacktrackerto do less work during a search, and thus might make it faster.fn reset_cache(self: &Self, cache: &mut Cache)Reset the given cache such that it can be used for searching with the this
BoundedBacktracker(and only thisBoundedBacktracker).A cache reset permits reusing memory already allocated in this cache with a different
BoundedBacktracker.Example
This shows how to re-purpose a cache for use with a different
BoundedBacktracker.# if cfg! // miri takes too long use ; let re1 = new?; let re2 = new?; let mut cache = re1.create_cache; assert_eq!; // Using 'cache' with re2 is not allowed. It may result in panics or // incorrect results. In order to re-purpose the cache, we must reset // it with the BoundedBacktracker we'd like to use it with. // // Similarly, after this reset, using the cache with 're1' is also not // allowed. cache.reset; assert_eq!; # Ok::fn pattern_len(self: &Self) -> usizeReturns the total number of patterns compiled into this
BoundedBacktracker.In the case of a
BoundedBacktrackerthat contains no patterns, this returns0.Example
This example shows the pattern length for a
BoundedBacktrackerthat never matches:use BoundedBacktracker; let re = never_match?; assert_eq!; # Ok::And another example for a
BoundedBacktrackerthat matches at every position:use BoundedBacktracker; let re = always_match?; assert_eq!; # Ok::And finally, a
BoundedBacktrackerthat was constructed from multiple patterns:use BoundedBacktracker; let re = new_many?; assert_eq!; # Ok::fn get_config(self: &Self) -> &ConfigReturn the config for this
BoundedBacktracker.fn get_nfa(self: &Self) -> &NFAReturns a reference to the underlying NFA.
fn max_haystack_len(self: &Self) -> usizeReturns the maximum haystack length supported by this backtracker.
This routine is a function of both
Config::visited_capacityand the internal size of the backtracker's NFA.Example
This example shows how the maximum haystack length can vary depending on the size of the regex itself. Note though that the specific maximum values here are not an API guarantee. The default visited capacity is subject to change and not covered by semver.
# if cfg! // miri takes too long use ; // If you're only using ASCII, you get a big budget. let re = new?; let mut cache = re.create_cache; assert_eq!; // Things work up to the max. let mut haystack = "a".repeat; let expected = Some; assert_eq!; // But you'll get an error if you provide a haystack that's too big. // Notice that we use the 'try_find_iter' routine instead, which // yields Result<Match, MatchError> instead of Match. haystack.push; let expected = Some; assert_eq!; // Unicode inflates the size of the underlying NFA quite a bit, and // thus means that the backtracker can only handle smaller haystacks, // assuming that the visited capacity remains unchanged. let re = new?; assert!; // But we can increase the visited capacity to handle bigger haystacks! let re = builder .configure .build?; assert!; assert!; # Ok::
impl BoundedBacktracker
fn try_search(self: &Self, cache: &mut Cache, input: &Input<'_>, caps: &mut Captures) -> Result<(), MatchError>Executes a leftmost forward search and writes the spans of capturing groups that participated in a match into the provided
Capturesvalue. If no match was found, thenCaptures::is_matchis guaranteed to returnfalse.This is like
BoundedBacktracker::try_captures, but it accepts a concrete&Inputinstead of anInto<Input>.Errors
This routine only errors if the search could not complete. For this backtracking regex engine, this only occurs when the haystack length exceeds
BoundedBacktracker::max_haystack_len.When a search cannot complete, callers cannot know whether a match exists or not.
Example: specific pattern search
This example shows how to build a multi bounded backtracker that permits searching for specific patterns.
use ; let re = new_many?; let = ; let haystack = "foo123"; // Since we are using the default leftmost-first match and both // patterns match at the same starting position, only the first pattern // will be returned in this case when doing a search for any of the // patterns. let expected = Some; re.try_search?; assert_eq!; // But if we want to check whether some other pattern matches, then we // can provide its pattern ID. let expected = Some; let input = new .anchored; re.try_search?; assert_eq!; # Ok::Example: specifying the bounds of a search
This example shows how providing the bounds of a search can produce different results than simply sub-slicing the haystack.
# if cfg! // miri takes too long use ; let re = new?; let = ; let haystack = "foo123bar"; // Since we sub-slice the haystack, the search doesn't know about // the larger context and assumes that `123` is surrounded by word // boundaries. And of course, the match position is reported relative // to the sub-slice as well, which means we get `0..3` instead of // `3..6`. let expected = Some; re.try_search?; assert_eq!; // But if we provide the bounds of the search within the context of the // entire haystack, then the search can take the surrounding context // into account. (And if we did find a match, it would be reported // as a valid offset into `haystack` instead of its sub-slice.) let expected = None; re.try_search?; assert_eq!; # Ok::fn try_search_slots(self: &Self, cache: &mut Cache, input: &Input<'_>, slots: &mut [Option<NonMaxUsize>]) -> Result<Option<PatternID>, MatchError>Executes a leftmost forward search and writes the spans of capturing groups that participated in a match into the provided
slots, and returns the matching pattern ID. The contents of the slots for patterns other than the matching pattern are unspecified. If no match was found, thenNoneis returned and the contents of allslotsis unspecified.This is like
BoundedBacktracker::try_search, but it accepts a raw slots slice instead of aCapturesvalue. This is useful in contexts where you don't want or need to allocate aCaptures.It is legal to pass any number of slots to this routine. If the regex engine would otherwise write a slot offset that doesn't fit in the provided slice, then it is simply skipped. In general though, there are usually three slice lengths you might want to use:
- An empty slice, if you only care about which pattern matched.
- A slice with
pattern_len() * 2slots, if you only care about the overall match spans for each matching pattern. - A slice with
slot_len()slots, which permits recording match offsets for every capturing group in every pattern.
Errors
This routine only errors if the search could not complete. For this backtracking regex engine, this only occurs when the haystack length exceeds
BoundedBacktracker::max_haystack_len.When a search cannot complete, callers cannot know whether a match exists or not.
Example
This example shows how to find the overall match offsets in a multi-pattern search without allocating a
Capturesvalue. Indeed, we can put our slots right on the stack.# if cfg! // miri takes too long use ; let re = new_many?; let mut cache = re.create_cache; let input = new; // We only care about the overall match offsets here, so we just // allocate two slots for each pattern. Each slot records the start // and end of the match. let mut slots = ; let pid = re.try_search_slots?; assert_eq!; // The overall match offsets are always at 'pid * 2' and 'pid * 2 + 1'. // See 'GroupInfo' for more details on the mapping between groups and // slot indices. let slot_start = pid.unwrap.as_usize * 2; let slot_end = slot_start + 1; assert_eq!; assert_eq!; # Ok::
impl Clone for BoundedBacktracker
fn clone(self: &Self) -> BoundedBacktracker
impl Debug for BoundedBacktracker
fn fmt(self: &Self, f: &mut Formatter<'_>) -> Result
impl Freeze for BoundedBacktracker
impl RefUnwindSafe for BoundedBacktracker
impl Send for BoundedBacktracker
impl Sync for BoundedBacktracker
impl Unpin for BoundedBacktracker
impl UnsafeUnpin for BoundedBacktracker
impl UnwindSafe for BoundedBacktracker
impl<T> Any for BoundedBacktracker
fn type_id(self: &Self) -> TypeId
impl<T> Borrow for BoundedBacktracker
fn borrow(self: &Self) -> &T
impl<T> BorrowMut for BoundedBacktracker
fn borrow_mut(self: &mut Self) -> &mut T
impl<T> CloneToUninit for BoundedBacktracker
unsafe fn clone_to_uninit(self: &Self, dest: *mut u8)
impl<T> From for BoundedBacktracker
fn from(t: T) -> TReturns the argument unchanged.
impl<T> ToOwned for BoundedBacktracker
fn to_owned(self: &Self) -> Tfn clone_into(self: &Self, target: &mut T)
impl<T, U> Into for BoundedBacktracker
fn into(self: Self) -> UCalls
U::from(self).That is, this conversion is whatever the implementation of
[From]<T> for Uchooses to do.
impl<T, U> TryFrom for BoundedBacktracker
fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>
impl<T, U> TryInto for BoundedBacktracker
fn try_into(self: Self) -> Result<U, <U as TryFrom<T>>::Error>