Struct Compiler

struct Compiler { ... }

A builder for compiling an NFA from a regex's high-level intermediate representation (HIR).

This compiler provides a way to translate a parsed regex pattern into an NFA state graph. The NFA state graph can either be used directly to execute a search (e.g., with a Pike VM), or it can be further used to build a DFA.

This compiler provides APIs both for compiling regex patterns directly from their concrete syntax, or via a regex_syntax::hir::Hir.

This compiler has various options that may be configured via thompson::Config.

Note that a compiler is not the same as a thompson::Builder. A Builder provides a lower level API that is uncoupled from a regex pattern's concrete syntax or even its HIR. Instead, it permits stitching together an NFA by hand. See its docs for examples.

Example: compilation from concrete syntax

This shows how to compile an NFA from a pattern string while setting a size limit on how big the NFA is allowed to be (in terms of bytes of heap used).

use regex_automata::{
    nfa::thompson::{NFA, pikevm::PikeVM},
    Match,
};

let config = NFA::config().nfa_size_limit(Some(1_000));
let nfa = NFA::compiler().configure(config).build(r"(?-u)\w")?;

let re = PikeVM::new_from_nfa(nfa)?;
let mut cache = re.create_cache();
let mut caps = re.create_captures();
let expected = Some(Match::must(0, 3..4));
re.captures(&mut cache, "!@#A#@!", &mut caps);
assert_eq!(expected, caps.get_match());

# Ok::<(), Box<dyn std::error::Error>>(())

Example: compilation from HIR

This shows how to hand assemble a regular expression via its HIR, and then compile an NFA directly from it.

use regex_automata::{nfa::thompson::{NFA, pikevm::PikeVM}, Match};
use regex_syntax::hir::{Hir, Class, ClassBytes, ClassBytesRange};

let hir = Hir::class(Class::Bytes(ClassBytes::new(vec![
    ClassBytesRange::new(b'0', b'9'),
    ClassBytesRange::new(b'A', b'Z'),
    ClassBytesRange::new(b'_', b'_'),
    ClassBytesRange::new(b'a', b'z'),
])));

let config = NFA::config().nfa_size_limit(Some(1_000));
let nfa = NFA::compiler().configure(config).build_from_hir(&hir)?;

let re = PikeVM::new_from_nfa(nfa)?;
let mut cache = re.create_cache();
let mut caps = re.create_captures();
let expected = Some(Match::must(0, 3..4));
re.captures(&mut cache, "!@#A#@!", &mut caps);
assert_eq!(expected, caps.get_match());

# Ok::<(), Box<dyn std::error::Error>>(())

Implementations

impl Compiler

fn new() -> Compiler

Create a new NFA builder with its default configuration.

fn build(self: &Self, pattern: &str) -> Result<NFA, BuildError>

Compile the given regular expression pattern into an NFA.

If there was a problem parsing the regex, then that error is returned.

Otherwise, if there was a problem building the NFA, then an error is returned. The only error that can occur is if the compiled regex would exceed the size limits configured on this builder, or if any part of the NFA would exceed the integer representations used. (For example, too many states might plausibly occur on a 16-bit target.)

Example

use regex_automata::{nfa::thompson::{NFA, pikevm::PikeVM}, Match};

let config = NFA::config().nfa_size_limit(Some(1_000));
let nfa = NFA::compiler().configure(config).build(r"(?-u)\w")?;

let re = PikeVM::new_from_nfa(nfa)?;
let mut cache = re.create_cache();
let mut caps = re.create_captures();
let expected = Some(Match::must(0, 3..4));
re.captures(&mut cache, "!@#A#@!", &mut caps);
assert_eq!(expected, caps.get_match());

# Ok::<(), Box<dyn std::error::Error>>(())

fn build_many<P: AsRef<str>>(self: &Self, patterns: &[P]) -> Result<NFA, BuildError>

Compile the given regular expression patterns into a single NFA.

When matches are returned, the pattern ID corresponds to the index of the pattern in the slice given.

Example

use regex_automata::{nfa::thompson::{NFA, pikevm::PikeVM}, Match};

let config = NFA::config().nfa_size_limit(Some(1_000));
let nfa = NFA::compiler().configure(config).build_many(&[
    r"(?-u)\s",
    r"(?-u)\w",
])?;

let re = PikeVM::new_from_nfa(nfa)?;
let mut cache = re.create_cache();
let mut caps = re.create_captures();
let expected = Some(Match::must(1, 1..2));
re.captures(&mut cache, "!A! !A!", &mut caps);
assert_eq!(expected, caps.get_match());

# Ok::<(), Box<dyn std::error::Error>>(())

fn build_from_hir(self: &Self, expr: &Hir) -> Result<NFA, BuildError>

Compile the given high level intermediate representation of a regular expression into an NFA.

If there was a problem building the NFA, then an error is returned. The only error that can occur is if the compiled regex would exceed the size limits configured on this builder, or if any part of the NFA would exceed the integer representations used. (For example, too many states might plausibly occur on a 16-bit target.)

Example

use regex_automata::{nfa::thompson::{NFA, pikevm::PikeVM}, Match};
use regex_syntax::hir::{Hir, Class, ClassBytes, ClassBytesRange};

let hir = Hir::class(Class::Bytes(ClassBytes::new(vec![
    ClassBytesRange::new(b'0', b'9'),
    ClassBytesRange::new(b'A', b'Z'),
    ClassBytesRange::new(b'_', b'_'),
    ClassBytesRange::new(b'a', b'z'),
])));

let config = NFA::config().nfa_size_limit(Some(1_000));
let nfa = NFA::compiler().configure(config).build_from_hir(&hir)?;

let re = PikeVM::new_from_nfa(nfa)?;
let mut cache = re.create_cache();
let mut caps = re.create_captures();
let expected = Some(Match::must(0, 3..4));
re.captures(&mut cache, "!@#A#@!", &mut caps);
assert_eq!(expected, caps.get_match());

# Ok::<(), Box<dyn std::error::Error>>(())

fn build_many_from_hir<H: Borrow<Hir>>(self: &Self, exprs: &[H]) -> Result<NFA, BuildError>

Compile the given high level intermediate representations of regular expressions into a single NFA.

When matches are returned, the pattern ID corresponds to the index of the pattern in the slice given.

Example

use regex_automata::{nfa::thompson::{NFA, pikevm::PikeVM}, Match};
use regex_syntax::hir::{Hir, Class, ClassBytes, ClassBytesRange};

let hirs = &[
    Hir::class(Class::Bytes(ClassBytes::new(vec![
        ClassBytesRange::new(b'\t', b'\r'),
        ClassBytesRange::new(b' ', b' '),
    ]))),
    Hir::class(Class::Bytes(ClassBytes::new(vec![
        ClassBytesRange::new(b'0', b'9'),
        ClassBytesRange::new(b'A', b'Z'),
        ClassBytesRange::new(b'_', b'_'),
        ClassBytesRange::new(b'a', b'z'),
    ]))),
];

let config = NFA::config().nfa_size_limit(Some(1_000));
let nfa = NFA::compiler().configure(config).build_many_from_hir(hirs)?;

let re = PikeVM::new_from_nfa(nfa)?;
let mut cache = re.create_cache();
let mut caps = re.create_captures();
let expected = Some(Match::must(1, 1..2));
re.captures(&mut cache, "!A! !A!", &mut caps);
assert_eq!(expected, caps.get_match());

# Ok::<(), Box<dyn std::error::Error>>(())

fn configure(self: &mut Self, config: Config) -> &mut Compiler

Apply the given NFA configuration options to this builder.

Example

use regex_automata::nfa::thompson::NFA;

let config = NFA::config().nfa_size_limit(Some(1_000));
let nfa = NFA::compiler().configure(config).build(r"(?-u)\w")?;
assert_eq!(nfa.pattern_len(), 1);

# Ok::<(), Box<dyn std::error::Error>>(())

fn syntax(self: &mut Self, config: Config) -> &mut Compiler

Set the syntax configuration for this builder using syntax::Config.

This permits setting things like case insensitivity, Unicode and multi line mode.

This syntax configuration only applies when an NFA is built directly from a pattern string. If an NFA is built from an HIR, then all syntax settings are ignored.

Example

use regex_automata::{nfa::thompson::NFA, util::syntax};

let syntax_config = syntax::Config::new().unicode(false);
let nfa = NFA::compiler().syntax(syntax_config).build(r"\w")?;
// If Unicode were enabled, the number of states would be much bigger.
assert!(nfa.states().len() < 15);

# Ok::<(), Box<dyn std::error::Error>>(())

impl Clone for Compiler

fn clone(self: &Self) -> Compiler

impl Debug for Compiler

fn fmt(self: &Self, f: &mut Formatter<'_>) -> Result

impl Freeze for Compiler

impl RefUnwindSafe for Compiler

impl Send for Compiler

impl Sync for Compiler

impl Unpin for Compiler

impl UnsafeUnpin for Compiler

impl UnwindSafe for Compiler

impl<T> Any for Compiler

fn type_id(self: &Self) -> TypeId

impl<T> Borrow for Compiler

fn borrow(self: &Self) -> &T

impl<T> BorrowMut for Compiler

fn borrow_mut(self: &mut Self) -> &mut T

impl<T> CloneToUninit for Compiler

unsafe fn clone_to_uninit(self: &Self, dest: *mut u8)

impl<T> From for Compiler

fn from(t: T) -> T

Returns the argument unchanged.

impl<T> ToOwned for Compiler

fn to_owned(self: &Self) -> T
fn clone_into(self: &Self, target: &mut T)

impl<T, U> Into for Compiler

fn into(self: Self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of [From]<T> for U chooses to do.

impl<T, U> TryFrom for Compiler

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

impl<T, U> TryInto for Compiler

fn try_into(self: Self) -> Result<U, <U as TryFrom<T>>::Error>