Module `intrinsics`

Compiler intrinsics.

The functions in this module are implementation details of core and should not be used outside of the standard library. We generally provide access to intrinsics via stable wrapper functions. Use these instead.

These are the imports making intrinsics available to Rust code. The actual implementations live in the compiler. Some of these intrinsics are lowered to MIR in https://github.com/rust-lang/rust/blob/HEAD/compiler/rustc_mir_transform/src/lower_intrinsics.rs. The remaining intrinsics are implemented for the LLVM backend in https://github.com/rust-lang/rust/blob/HEAD/compiler/rustc_codegen_ssa/src/mir/intrinsic.rs and https://github.com/rust-lang/rust/blob/HEAD/compiler/rustc_codegen_llvm/src/intrinsic.rs, and for const evaluation in https://github.com/rust-lang/rust/blob/HEAD/compiler/rustc_const_eval/src/interpret/intrinsics.rs.

Intrinsics don't need a body. However, they optionally can have a body, which we call the "fallback body". This will be used by codegen backends that do not have a dedicated implementation of the intrinsic, making it easier to add new intrinsics for specific operations without having to implement them in each codegen backend. The fallback body obviously has to be a valid implementation of the documented specification of the intrinsic. In some cases, the fallback body will be equivalent to the specification. Note that this is a strong requirement: if the spec says "UB if input x is even", then a valid implementation can just ignore this and do whatever it wants in that case; an equivalent implementation needs to actually check this condition and trigger UB in that case (e.g. by using hint::assert_unchecked()). Similar, if the spec says "returns x or y non-deterministically", then an equivalent implementation must actually do non-deterministic choice and return either value (e.g. by invoking some other language operation that has the same non-determinism). Intrinsics with such a fallback body that is equivalent to the spec may be marked with #[miri::intrinsic_fallback_is_spec]; the fallback body will then also be used by Miri for UB checking. When in doubt, do not use this attribute or ask the Miri maintainers for advice.

Intrinsics are, in general, language extensions. Therefore, t-lang should be involved whenever a new intrinsic is exposed to stable code. However, if an intrinsic is marked #[miri::intrinsic_fallback_is_spec] with a fallback body that only uses stable features (or if such a fallback body could be written, but for one reason or another the actual fallback body is different), and if it also does not make other promises that go beyond observable program behavior (such as steering the optimizer in a particular direction), then an intrinsic may be used without t-lang involvement.

Const intrinsics

In order to make an intrinsic unstable usable at compile-time, copy the implementation from https://github.com/rust-lang/miri/blob/master/src/intrinsics to https://github.com/rust-lang/rust/blob/HEAD/compiler/rustc_const_eval/src/interpret/intrinsics.rs and make the intrinsic declaration below a const fn. This should be done in coordination with wg-const-eval.

If an intrinsic is supposed to be used from a const fn with a rustc_const_stable attribute, #[rustc_intrinsic_const_stable_indirect] needs to be added to the intrinsic. Such a change requires T-lang approval, because it may bake a feature into the language that cannot be replicated in user code without compiler support. The same exception as above applies for #[miri::intrinsic_fallback_is_spec] intrinsics.

Volatiles

The volatile intrinsics provide operations intended to act on I/O memory, which are guaranteed to not be reordered by the compiler across other volatile intrinsics. See [read_volatile][ptr::read_volatile] and [write_volatile][ptr::write_volatile].

Atomics

The atomic intrinsics provide common atomic operations on machine words, with multiple possible memory orderings. See the [atomic types][atomic] docs for details.

Unwinding

Rust intrinsics may, in general, unwind. If an intrinsic can never unwind, add the #[rustc_nounwind] attribute so that the compiler can make use of this fact.

However, even for intrinsics that may unwind, rustc assumes that a Rust intrinsics will never initiate a foreign (non-Rust) unwind, and thus for panic=abort we can always assume that these intrinsics cannot unwind.

Modules

bounds Various traits used to restrict intrinsics to not-completely-wrong types.
cos
exp
exp2
fallback
gpu Intrinsics for GPU targets.
log
log10
log2
macros Macros used to declare intrinsics.
mir Rustc internal tooling for hand-writing MIR.
reflection Intrinsics for compile time reflection
simd SIMD compiler intrinsics.
sin

Enums

AtomicOrdering A type for atomic ordering parameters for intrinsics. This is a separate type from atomic::Ordering so that we can make it ConstParamTy and fix the values used here without a risk of leaking that to stable code.

Functions

abort_immediate Ungracefully aborts the execution of the process.
add_with_overflow Performs checked integer addition.
aggregate_raw_ptr Lowers in MIR to Rvalue::Aggregate with AggregateKind::RawPtr.
align_of The minimum alignment of a type.
align_of_val The required alignment of the referenced value.
arith_offset Calculates the offset from a pointer, potentially wrapping.
assert_inhabited A guard for unsafe functions that cannot ever be executed if T is uninhabited: This will statically either panic, or do nothing. It does not guarantee to ever panic, and should only be called if an assertion failure will imply language UB in the following code.
assert_mem_uninitialized_valid A guard for std::mem::uninitialized. This will statically either panic, or do nothing. It does not guarantee to ever panic, and should only be called if an assertion failure will imply language UB in the following code.
assert_zero_valid A guard for unsafe functions that cannot ever be executed if T does not permit zero-initialization: This will statically either panic, or do nothing. It does not guarantee to ever panic, and should only be called if an assertion failure will imply language UB in the following code.
assume Informs the optimizer that a condition is always true. If the condition is false, the behavior is undefined.
atomic_and Bitwise and with the current value, returning the previous value. T must be an integer or pointer type. U must be the same as T if that is an integer type, or usize if T is a pointer type.
atomic_cxchg Stores a value if the current value is the same as the old value, with the returned T being the previous value and the returned bool being whether the exchange was successful. T must be an integer or pointer type.
atomic_cxchgweak Stores a value if the current value is the same as the old value, with the returned T being the previous value and the returned bool being whether the exchange was successful. T must be an integer or pointer type. The comparison may spuriously fail.
atomic_fence An atomic fence.
atomic_load Loads the current value of the pointer. T must be an integer or pointer type.
atomic_max Maximum with the current value using a signed comparison. T must be a signed integer type.
atomic_min Minimum with the current value using a signed comparison. T must be a signed integer type.
atomic_nand Bitwise nand with the current value, returning the previous value. T must be an integer or pointer type. U must be the same as T if that is an integer type, or usize if T is a pointer type.
atomic_or Bitwise or with the current value, returning the previous value. T must be an integer or pointer type. U must be the same as T if that is an integer type, or usize if T is a pointer type.
atomic_singlethreadfence An atomic fence for synchronization within a single thread.
atomic_store Stores the value at the specified memory location. T must be an integer or pointer type.
atomic_umax Maximum with the current value using an unsigned comparison. T must be an unsigned integer type.
atomic_umin Minimum with the current value using an unsigned comparison. T must be an unsigned integer type.
atomic_xadd Adds to the current value, returning the previous value. T must be an integer or pointer type. U must be the same as T if that is an integer type, or usize if T is a pointer type.
atomic_xchg Stores the value at the specified memory location, returning the old value. T must be an integer or pointer type.
atomic_xor Bitwise xor with the current value, returning the previous value. T must be an integer or pointer type. U must be the same as T if that is an integer type, or usize if T is a pointer type.
atomic_xsub Subtract from the current value, returning the previous value. T must be an integer or pointer type. U must be the same as T if that is an integer type, or usize if T is a pointer type.
autodiff Generates the LLVM body for the automatic differentiation of f using Enzyme, with df as the derivative function and args as its arguments.
bitreverse Reverses the bits in an integer type T.
black_box See documentation of std::hint::black_box for details.
breakpoint Executes a breakpoint trap, for inspection by a debugger.
bswap Reverses the bytes in an integer type T.
caller_location Gets a reference to a static Location indicating where it was called.
carrying_mul_add Performs full-width multiplication and addition with a carry: multiplier * multiplicand + addend + carry.
carryless_mul Carryless multiply.
catch_unwind Rust's "try catch" construct for unwinding. Invokes the function pointer try_fn with the data pointer data, and calls catch_fn if unwinding occurs while try_fn runs. Returns true if unwinding occurred and catch_fn was called; returns false otherwise.
ceilf128 Returns the smallest integer greater than or equal to an f128.
ceilf16 Returns the smallest integer greater than or equal to an f16.
ceilf32 Returns the smallest integer greater than or equal to an f32.
ceilf64 Returns the smallest integer greater than or equal to an f64.
cold_path Hints to the compiler that current code path is cold.
compare_bytes Lexicographically compare [left, left + bytes) and [right, right + bytes) as unsigned bytes, returning negative if left is less, zero if all the bytes match, or positive if left is greater.
const_allocate Allocates a block of memory at compile time. At runtime, just returns a null pointer.
const_deallocate Deallocates a memory which allocated by intrinsics::const_allocate at compile time. At runtime, it does nothing.
const_eval_select Selects which function to call depending on the context.
const_make_global Convert the allocation this pointer points to into immutable global memory. The pointer must point to the beginning of a heap allocation. This operation only makes sense during compile time. At runtime, it does nothing.
contract_check_ensures Check if the post-condition cond has been met.
contract_check_requires Check if the pre-condition cond has been met.
copy This is an accidentally-stable alias to ptr::copy; use that instead.
copy_nonoverlapping This is an accidentally-stable alias to ptr::copy_nonoverlapping; use that instead.
copysignf128 Copies the sign from y to x for f128 values.
copysignf16 Copies the sign from y to x for f16 values.
copysignf32 Copies the sign from y to x for f32 values.
copysignf64 Copies the sign from y to x for f64 values.
cos Returns the cosine of a floating-point value.
ctlz Returns the number of leading unset bits (zeroes) in an integer type T.
ctlz_nonzero Like ctlz, but extra-unsafe as it returns undef when given an x with value 0.
ctpop Returns the number of bits set in an integer type T
cttz Returns the number of trailing unset bits (zeroes) in an integer type T.
cttz_nonzero Like cttz, but extra-unsafe as it returns undef when given an x with value 0.
discriminant_value Returns the value of the discriminant for the variant in 'v'; if T has no discriminant, returns 0.
disjoint_bitor Combine two values which have no bits in common.
exact_div Performs an exact division, resulting in undefined behavior where x % y != 0 or y == 0 or x == T::MIN && y == -1
exp Returns the exponential of a floating-point value.
exp2 Returns 2 raised to the power of a floating-point value.
fabs Returns the absolute value of a floating-point value.
fadd_algebraic Float addition that allows optimizations based on algebraic rules.
fadd_fast Float addition that allows optimizations based on algebraic rules. Requires that inputs and output of the operation are finite, causing UB otherwise.
fdiv_algebraic Float division that allows optimizations based on algebraic rules.
fdiv_fast Float division that allows optimizations based on algebraic rules. Requires that inputs and output of the operation are finite, causing UB otherwise.
field_offset The offset of a field queried by its field representing type.
field_representing_type_actual_type_id Gets the actual field TypeId of the FieldRepresentingType's TypeId.
field_representing_type_name Gets the name of the field represented by the FieldRepresentingType's TypeId.
field_representing_type_offset Gets the name of the field represented by the FieldRepresentingType's TypeId.
float_to_int_unchecked Converts with LLVM’s fptoui/fptosi, which may return undef for values out of range (https://github.com/rust-lang/rust/issues/10184)
floorf128 Returns the largest integer less than or equal to an f128.
floorf16 Returns the largest integer less than or equal to an f16.
floorf32 Returns the largest integer less than or equal to an f32.
floorf64 Returns the largest integer less than or equal to an f64.
fmaf128 Returns a * b + c without rounding the intermediate result for f128 values.
fmaf16 Returns a * b + c without rounding the intermediate result for f16 values.
fmaf32 Returns a * b + c without rounding the intermediate result for f32 values.
fmaf64 Returns a * b + c without rounding the intermediate result for f64 values.
fmul_algebraic Float multiplication that allows optimizations based on algebraic rules.
fmul_fast Float multiplication that allows optimizations based on algebraic rules. Requires that inputs and output of the operation are finite, causing UB otherwise.
fmuladdf128 Returns a * b + c for f128 values, non-deterministically executing either a fused multiply-add or two operations with rounding of the intermediate result.
fmuladdf16 Returns a * b + c for f16 values, non-deterministically executing either a fused multiply-add or two operations with rounding of the intermediate result.
fmuladdf32 Returns a * b + c for f32 values, non-deterministically executing either a fused multiply-add or two operations with rounding of the intermediate result.
fmuladdf64 Returns a * b + c for f64 values, non-deterministically executing either a fused multiply-add or two operations with rounding of the intermediate result.
forget Moves a value out of scope without running drop glue.
frem_algebraic Float remainder that allows optimizations based on algebraic rules.
frem_fast Float remainder that allows optimizations based on algebraic rules. Requires that inputs and output of the operation are finite, causing UB otherwise.
fsub_algebraic Float subtraction that allows optimizations based on algebraic rules.
fsub_fast Float subtraction that allows optimizations based on algebraic rules. Requires that inputs and output of the operation are finite, causing UB otherwise.
integer_max Integer maximum, signed or unsigned depending on T.
integer_min Integer minimum, signed or unsigned depending on T.
is_val_statically_known Returns whether the argument's value is statically known at compile-time.
likely Hints to the compiler that branch condition is likely to be true. Returns the value passed to it.
log Returns the natural logarithm of a floating-point value.
log10 Returns the base 10 logarithm of a floating-point value.
log2 Returns the base 2 logarithm of a floating-point value.
maximum_number_nsz_f128 Returns the maximum of two f128 values, ignoring NaN.
maximum_number_nsz_f16 Returns the maximum of two f16 values, ignoring NaN.
maximum_number_nsz_f32 Returns the maximum of two f32 values, ignoring NaN.
maximum_number_nsz_f64 Returns the maximum of two f64 values, ignoring NaN.
maximumf128 Returns the maximum of two f128 values, propagating NaN.
maximumf16 Returns the maximum of two f16 values, propagating NaN.
maximumf32 Returns the maximum of two f32 values, propagating NaN.
maximumf64 Returns the maximum of two f64 values, propagating NaN.
minimum_number_nsz_f128 Returns the minimum of two f128 values, ignoring NaN.
minimum_number_nsz_f16 Returns the minimum of two f16 values, ignoring NaN.
minimum_number_nsz_f32 Returns the minimum of two f32 values, ignoring NaN.
minimum_number_nsz_f64 Returns the minimum of two f64 values, ignoring NaN.
minimumf128 Returns the minimum of two f128 values, propagating NaN.
minimumf16 Returns the minimum of two f16 values, propagating NaN.
minimumf32 Returns the minimum of two f32 values, propagating NaN.
minimumf64 Returns the minimum of two f64 values, propagating NaN.
mul_with_overflow Performs checked integer multiplication
needs_drop Returns true if the actual type given as T requires drop glue; returns false if the actual type provided for T implements Copy.
nontemporal_store Emits a nontemporal store, which gives a hint to the CPU that the data should not be held in cache. Except for performance, this is fully equivalent to ptr.write(val).
offload Generates the LLVM body of a wrapper function to offload a kernel f.
offload_get_num_devices Returns the number of offload devices available on the system.
offset Calculates the offset from a pointer.
offset_of The offset of a field inside a type.
overflow_checks Returns whether we should perform some overflow-checking at runtime. This eventually evaluates to cfg!(overflow_checks), but behaves different from cfg! when mixing crates built with different flags: if the crate has overflow checks enabled or carries the #[rustc_inherit_overflow_checks] attribute, evaluation is delayed until monomorphization (or until the call gets inlined into a crate that does not delay evaluation further); otherwise it can happen any time.
powf128 Raises an f128 to an f128 power.
powf16 Raises an f16 to an f16 power.
powf32 Raises an f32 to an f32 power.
powf64 Raises an f64 to an f64 power.
powif128 Raises an f128 to an integer power.
powif16 Raises an f16 to an integer power.
powif32 Raises an f32 to an integer power.
powif64 Raises an f64 to an integer power.
prefetch_read_data The prefetch intrinsic is a hint to the code generator to insert a prefetch instruction for the given address if supported; otherwise, it is a no-op. Prefetches have no effect on the behavior of the program but can change its performance characteristics.
prefetch_read_instruction The prefetch intrinsic is a hint to the code generator to insert a prefetch instruction for the given address if supported; otherwise, it is a no-op. Prefetches have no effect on the behavior of the program but can change its performance characteristics.
prefetch_write_data The prefetch intrinsic is a hint to the code generator to insert a prefetch instruction for the given address if supported; otherwise, it is a no-op. Prefetches have no effect on the behavior of the program but can change its performance characteristics.
prefetch_write_instruction The prefetch intrinsic is a hint to the code generator to insert a prefetch instruction for the given address if supported; otherwise, it is a no-op. Prefetches have no effect on the behavior of the program but can change its performance characteristics.
ptr_guaranteed_cmp See documentation of <*const T>::guaranteed_eq for details. Returns 2 if the result is unknown. Returns 1 if the pointers are guaranteed equal. Returns 0 if the pointers are guaranteed inequal.
ptr_mask Masks out bits of the pointer according to a mask.
ptr_metadata Lowers in MIR to Rvalue::UnaryOp with UnOp::PtrMetadata.
ptr_offset_from See documentation of <*const T>::offset_from for details.
ptr_offset_from_unsigned See documentation of <*const T>::offset_from_unsigned for details.
raw_eq Determines whether the raw bytes of the two values are equal.
read_via_copy This is an implementation detail of crate::ptr::read and should not be used anywhere else. See its comments for why this exists.
return_address Returns the return address of the caller function (after inlining) in a best-effort manner or a null pointer if it is not supported on the current backend. Returning an accurate value is a quality-of-implementation concern, but no hard guarantees are made about the return value: formally, the intrinsic non-deterministically returns an arbitrary pointer without provenance.
rotate_left Performs rotate left.
rotate_right Performs rotate right.
round_ties_even_f128 Returns the nearest integer to an f128. Rounds half-way cases to the number with an even least significant digit.
round_ties_even_f16 Returns the nearest integer to an f16. Rounds half-way cases to the number with an even least significant digit.
round_ties_even_f32 Returns the nearest integer to an f32. Rounds half-way cases to the number with an even least significant digit.
round_ties_even_f64 Returns the nearest integer to an f64. Rounds half-way cases to the number with an even least significant digit.
roundf128 Returns the nearest integer to an f128. Rounds half-way cases away from zero.
roundf16 Returns the nearest integer to an f16. Rounds half-way cases away from zero.
roundf32 Returns the nearest integer to an f32. Rounds half-way cases away from zero.
roundf64 Returns the nearest integer to an f64. Rounds half-way cases away from zero.
rustc_peek Magic intrinsic that derives its meaning from attributes attached to the function.
saturating_add Computes a + b, saturating at numeric bounds.
saturating_sub Computes a - b, saturating at numeric bounds.
select_unpredictable Returns either true_val or false_val depending on condition b with a hint to the compiler that this condition is unlikely to be correctly predicted by a CPU's branch predictor (e.g. a binary search).
sin Returns the sine of a floating-point value.
size_of The size of a type in bytes.
size_of_val The size of the referenced value in bytes.
slice_get_unchecked Projects to the index-th element of slice_ptr, as the same kind of pointer as the slice was provided -- so &mut [T] → &mut T, &[T] → &T, *mut [T] → *mut T, or *const [T] → *const T -- without a bounds check.
sqrtf128 Returns the square root of an f128
sqrtf16 Returns the square root of an f16
sqrtf32 Returns the square root of an f32
sqrtf64 Returns the square root of an f64
sub_with_overflow Performs checked integer subtraction
three_way_compare Does a three-way comparison between the two arguments, which must be of character or integer (signed or unsigned) type.
transmute Reinterprets the bits of a value of one type as another type.
transmute_unchecked Like transmute, but even less checked at compile-time: rather than giving an error for size_of::<Src>() != size_of::<Dst>(), it's Undefined Behavior at runtime.
truncf128 Returns the integer part of an f128.
truncf16 Returns the integer part of an f16.
truncf32 Returns the integer part of an f32.
truncf64 Returns the integer part of an f64.
type_name Gets a static string slice containing the name of a type.
typed_swap_nonoverlapping Non-overlapping typed swap of a single value.
ub_checks Returns whether we should perform some UB-checking at runtime. This eventually evaluates to cfg!(ub_checks), but behaves different from cfg! when mixing crates built with different flags: if the crate has UB checks enabled or carries the #[rustc_preserve_ub_checks] attribute, evaluation is delayed until monomorphization (or until the call gets inlined into a crate that does not delay evaluation further); otherwise it can happen any time.
unaligned_volatile_load Performs a volatile load from the src pointer The pointer is not required to be aligned.
unaligned_volatile_store Performs a volatile store to the dst pointer. The pointer is not required to be aligned.
unchecked_add Returns the result of an unchecked addition, resulting in undefined behavior when x + y > T::MAX or x + y < T::MIN.
unchecked_div Performs an unchecked division, resulting in undefined behavior where y == 0 or x == T::MIN && y == -1
unchecked_funnel_shl Funnel Shift left.
unchecked_funnel_shr Funnel Shift right.
unchecked_mul Returns the result of an unchecked multiplication, resulting in undefined behavior when x * y > T::MAX or x * y < T::MIN.
unchecked_rem Returns the remainder of an unchecked division, resulting in undefined behavior when y == 0 or x == T::MIN && y == -1
unchecked_shl Performs an unchecked left shift, resulting in undefined behavior when y < 0 or y >= N, where N is the width of T in bits.
unchecked_shr Performs an unchecked right shift, resulting in undefined behavior when y < 0 or y >= N, where N is the width of T in bits.
unchecked_sub Returns the result of an unchecked subtraction, resulting in undefined behavior when x - y > T::MAX or x - y < T::MIN.
unlikely Hints to the compiler that branch condition is likely to be false. Returns the value passed to it.
unreachable Informs the optimizer that this point in the code is not reachable, enabling further optimizations.
va_arg Loads an argument of type T from the va_list ap and increment the argument ap points to.
va_copy Duplicates a variable argument list. The returned list is initially at the same position as the one in src, but can be advanced independently.
va_end Destroy the variable argument list ap after initialization with va_start (part of the desugaring of ...) or va_copy.
variant_count Returns the number of variants of the type T cast to a usize; if T has no variants, returns 0. Uninhabited variants will be counted.
volatile_copy_memory Equivalent to the appropriate llvm.memmove.p0i8.0i8.* intrinsic, with a size of count * size_of::<T>() and an alignment of align_of::<T>().
volatile_copy_nonoverlapping_memory Equivalent to the appropriate llvm.memcpy.p0i8.0i8.* intrinsic, with a size of count * size_of::<T>() and an alignment of align_of::<T>().
volatile_load Performs a volatile load from the src pointer.
volatile_set_memory Equivalent to the appropriate llvm.memset.p0i8.* intrinsic, with a size of count * size_of::<T>() and an alignment of align_of::<T>().
volatile_store Performs a volatile store to the dst pointer.
vtable_align The intrinsic will return the alignment stored in that vtable.
vtable_size The intrinsic will return the size stored in that vtable.
wrapping_add Wrapping (modular) addition. Computes a + b, wrapping around at the boundary of the type.
wrapping_mul Wrapping (modular) multiplication. Computes a * b, wrapping around at the boundary of the type.
wrapping_sub Wrapping (modular) subtraction. Computes a - b, wrapping around at the boundary of the type.
write_bytes This is an accidentally-stable alias to ptr::write_bytes; use that instead.
write_via_move This is an implementation detail of crate::ptr::write and should not be used anywhere else. See its comments for why this exists.

Macros

const_eval_select A macro to make it easier to invoke const_eval_select. Use as follows: