Summary: Query an attribute of a surface.

Query an attribute of a surface. Operand a is a.surfref variable or a.u64 register.
Query Returns.width.height.depth value in elements. (see the official PTX ISA docs for the full description)

Query an attribute of a surface. Operand a is a.surfref variable or a.u64 register.
Query Returns.width.height.depth value in elements.channel_data_type Unsigned integer corresponding to source language’s channel data
type enumeration. If the source language combines channel data
type and channel order into a single enumeration type, that value
is returned for both channel_data_type and channel_order queries..channel_order Unsigned integer corresponding to source language’s channel order
enumeration. (see the official PTX ISA docs for the full description)


sured.b.op.geom.ctype.clamp  [a,b],c; // byte addressing

Reduction to surface memory using a surface coordinate vector. (see the official PTX ISA docs for the full description)

sured.b.add.2d.u32.trap  [surf_A, {x,y}], r1;
sured.p.min.1d.u32.trap  [surf_B, {x}], r1;
sured.b.max.1d.u64.trap  [surf_C, {x}], r1;
sured.p.min.1d.b64.trap  [surf_D, {x}], r1;

Reduction to surface memory using a surface coordinate vector. The instruction performs a reduction
operation with data from operand c to the surface named by operand a at coordinates given by
operand b. Operand a is a.surfref variable or.u64 register. Operand b is a
scalar or singleton tuple for 1d surfaces; is a two-element vector for 2d surfaces; and is a
four-element vector for 3d surfaces, where the fourth element is ignored. Coordinate elements are of
type.s32.
sured.b performs an unformatted reduction on.u32,.s32,.b32,.u64, or.s64 data. The lowest dimension coordinate represents a byte offset into the surface and is not
scaled. (see the official PTX ISA docs for the full description)


sust.b.{1d,2d,3d}{.cop}.vec.ctype.clamp  [a, b], c;  // unformatted

sust.{1d,2d,3d}
Store to surface memory using a surface coordinate vector. The instruction stores data from operand c to the surface named by operand a at coordinates given by operand b. (see the official PTX ISA docs for the full description)

sust.p.1d.v4.b32.trap  [surf_B, {x}], {f1,f2,f3,f4};
sust.b.3d.v2.b64.trap  [surf_A, {x,y,z,w}], {r1,r2};
sust.b.a1d.v2.b64      [surf_C, {idx,x}], {r1,r2};
sust.b.a2d.b32         [surf_D, {idx,x,y,z}], r0;  // z ignored

sust.{1d,2d,3d}
Store to surface memory using a surface coordinate vector. The instruction stores data from operand c to the surface named by operand a at coordinates given by operand b. Operand a is
a.surfref variable or.u64 register. Operand b is a scalar or singleton tuple for 1d
surfaces; is a two-element vector for 2d surfaces; and is a four-element vector for 3d surfaces,
where the fourth element is ignored. Coordinate elements are of type.s32.
sust.b performs an unformatted store of binary data. The lowest dimension coordinate represents
a byte offset into the surface and is not scaled. (see the official PTX ISA docs for the full description)


Sign-extends or zero-extends an N-bit value from operand a where N is specified in operand b. The resulting value is stored in the destination operand d.
For the. (see the official PTX ISA docs for the full description)

b1        = b & 0x1f;
too_large = (b >= 32 && .mode == .clamp) ? true : false;
mask      = too_large ? 0 : (~0) << b1;
sign_pos  = (b1 - 1) & 0x1f;

if (b1 == 0 || too_large || .type != .s32) {
    sign_bit = false;
} else {
    sign_bit = (a >> sign_pos) & 1;
}
d = (a & ~mask) | (sign_bit ? mask | 0);

szext.clamp.s32 rd, ra, rb;
szext.wrap.u32  rd, 0xffffffff, 0; // Result is 0.

Sign-extends or zero-extends an N-bit value from operand a where N is specified in operand b. The resulting value is stored in the destination operand d.
For the.s32 instruction type, the value in a is treated as an N-bit signed value and the
most significant bit of this N-bit value is replicated up to bit 31. For the.u32 instruction
type, the value in a is treated as an N-bit unsigned number and is zero-extended to 32
bits. Operand b is an unsigned 32-bit value.
If the value of N is 0, then the result of szext is 0. (see the official PTX ISA docs for the full description)


Half Precision Floating-Point Instructions

Summary: Take hyperbolic tangent value of a.

Take hyperbolic tangent value of a.
The operands d and a are of type.f32.

Take hyperbolic tangent value of a.
The type of operands d and a are as specified by.type.
For.f16x2 or. (see the official PTX ISA docs for the full description)

tanh.approx.f32 ta, a;

tanh.approx.f16    h1, h0;
tanh.approx.f16x2  hd1, hd0;
tanh.approx.bf16   b1, b0;
tanh.approx.bf16x2 hb1, hb0;

Take hyperbolic tangent value of a.
The type of operands d and a are as specified by.type.
For.f16x2 or.bf16x2 instruction type, each of the half-word operands are operated in
parallel and the results are packed appropriately into a.f16x2 or.bf16x2.
For.f16 instruction type, operands d and a have.f16 or.b16 type.
For.f16x2 instruction type, operands d and a have.f16x2 or.b32 type.
For.bf16 instruction type, operands d and a have.b16 type.
For.bf16x2 instruction type, operands d and a have.b32 type.


Tensor Memory Allocation and Management Instructions

tcgen05.alloc.cta_group.sync.aligned{.shared::cta}.b32  [dst], nCols;

tcgen05.alloc is a blocking instruction which dynamically allocates the specified number of columns in the Tensor Memory and writes the address of the allocated Tensor Memory into shared memory at the… (see the official PTX ISA docs for the full description)

// Example 1:

tcgen05.alloc.cta_group::1.sync.aligned.shared::cta.b32 [sMemAddr1], 32;
ld.shared.b32 taddr, [sMemAddr1];
// use taddr ...
// more allocations and its usages ...
// (truncated - see the official PTX ISA docs for the full example)

tcgen05.alloc is a blocking instruction which dynamically allocates
the specified number of columns in the Tensor Memory and writes
the address of the allocated Tensor Memory into shared memory
at the location specified by address operand dst. The tcgen05.alloc blocks if the
requested amount of Tensor Memory is not available and unblocks
as soon as the requested amount of Tensor Memory becomes
available for allocation.
tcgen05.dealloc is a potentially blocking instruction which deallocates the Tensor Memory specified by the Tensor Memory address taddr. (see the official PTX ISA docs for the full description)


TensorCore 5th Generation Family Instructions

The instruction tcgen05.commit is an asynchronous instruction which makes the mbarrier object, specified by the address operand mbar, track the completion of all the prior asynchronous tcgen05 operati… (see the official PTX ISA docs for the full description)

Example 1:
tcgen05.cp.cta_group::1.128x256b                      [taddr0], sdesc0;
tcgen05.commit.cta_group::1.mbarrier::arrive::one.b64 [mbarObj1];

loop:
mbarrier.try_wait.parity.b64 p, [mbarObj1], 0;
// (truncated - see the official PTX ISA docs for the full example)

The instruction tcgen05.commit is an asynchronous instruction which makes the mbarrier object,
specified by the address operand mbar, track the completion of all the prior asynchronous tcgen05 operations, as listed in mbarrier based completion mechanism,
initiated by the executing thread. Upon the completion of the tracked asynchronous tcgen05 operations, the signal specified by the.completion_mechanism is triggered by the system
on the mbarrier object.
This instruction accesses its mbarrier operand using generic-proxy.
The instruction tcgen05.commit.cta_group::1 tracks for the completion of all prior
asynchronous tcgen05 operations with.cta_group::1 issued by the current thread. (see the official PTX ISA docs for the full description)


Tensor Memory Data Movement Instructions

Instruction tcgen05.cp initiates an asynchronous copy operation from shared memory to the
location specified by the address operand taddr in the Tensor Memory. (see the official PTX ISA docs for the full description)

tcgen05.cp.cta_group::1.128x256b                 [taddr0], sdesc0;
tcgen05.cp.cta_group::2.128x128b.b8x16.b6x16_p32 [taddr1], sdesc1;
tcgen05.cp.cta_group::1.64x128b.warpx2::02_13    [taddr2], sdesc2;

Instruction tcgen05.cp initiates an asynchronous copy operation from shared memory to the
location specified by the address operand taddr in the Tensor Memory.
The 64-bit register operand s-desc is the matrix descriptor which represents the source
matrix in the shared memory that needs to be copied. The format of the matrix descriptor is
described in Matrix Descriptors.
The.shape qualifier indicates the dimension of data to be copied as described in the Data Movement Shape.
Qualifier.cta_group specifies the number of CTAs whose Tensor Memory is
accessed when a single thread of a single CTA executes the tcgen05.cp instruction.
When. (see the official PTX ISA docs for the full description)


Summary: tcgen05.dealloc is a blocking instruction which de-allocates the Tensor Memory specified by the Tensor Memory address taddr. The operand nCols specifies the number of columns to be de-allocated.

tcgen05.dealloc.cta_group.sync.aligned.b32  taddr, nCols;

// de-allocate the columns previously allocated at taddr:
tcgen05.dealloc.cta_group::1.sync.aligned.b32  taddr, 32;


The instruction tcgen05.fence::before_thread_sync orders all the prior asynchronous tcgen05 operations with respect to the subsequent tcgen05 and the execution
ordering operations. (see the official PTX ISA docs for the full description)

// Producer thread:

tcgen05.cp.cta_group::1.128x256b  [taddr0], sdesc0;

tcgen05.fence::before_thread_sync;
st.relaxed.b32 [flag], 1;
// (truncated - see the official PTX ISA docs for the full example)

The instruction tcgen05.fence::before_thread_sync orders all the prior asynchronous tcgen05 operations with respect to the subsequent tcgen05 and the execution
ordering operations.
The instruction tcgen05.fence::after_thread_sync orders all the subsequent asynchronous tcgen05 operations with respect to the prior tcgen05 and the execution ordering
operations.
The tcgen05.fence::* instructions compose with execution ordering instructions across
a thread scope and provide ordering between tcgen05 instructions across the same scope.
The tcgen05.fence::before_thread_sync instructions behave as code motion fence for prior tcgen05 instructions as they cannot be hoisted across. (see the official PTX ISA docs for the full description)


Tensor Memory and Register Load/Store Instructions

// Base load instruction:
tcgen05.ld.sync.aligned.shape1.num{.pack}.b32    r, [taddr];

Instruction tcgen05.ld asynchronously loads data from the Tensor Memory at the location specified by the 32-bit address operand taddr into the destination register r, collectively across all threads o… (see the official PTX ISA docs for the full description)

tcgen05.ld.sync.aligned.32x32b.x2.b32     {r0, r1}, [taddr1];

tcgen05.ld.sync.aligned.16x128b.x4.b32    {r0, r1, r2, r3, r4, r5, r6, r7}, [taddr2];

tcgen05.ld.red.sync.aligned.16x32bx2.x8.u32.max {r0, r1, r2, r3, r4, r5, r6, r7},
                                                 redVal, [taddr3], 16;

Instruction tcgen05.ld asynchronously loads data from the Tensor Memory at the location specified by the 32-bit address operand taddr into the destination
register r, collectively across all threads of the warps.
All the threads in the warp must specify the same value of taddr, which must be the
base address of the collective load operation. Otherwise, the behavior is undefined.
The.shape qualifier and the.num qualifier together determines the total
dimension of the data which is loaded from the Tensor Memory. The.shape qualifier indicates the base dimension of data to be accessed as described in the Data Movement Shape. (see the official PTX ISA docs for the full description)


TensorCore 5th Generation of MMA Instructions

// 1. Floating-point type without block scaling:
tcgen05.mma.cta_group.kind   [d-tmem],  a-desc,  b-desc, idesc,
{ disable-output-lane }, enable-input-d {, scale-input-d};

Instruction tcgen05.mma is an asynchronous instruction which initiates an MxNxK matrix multiply and accumulate operation, D = A*B+D where the A matrix is MxK, the B matrix is KxN, and the D matrix is… (see the official PTX ISA docs for the full description)

tcgen05.mma.cta_group::1.kind::tf32      [taddr0],  adesc,  bdesc, idesc, {m0, m1, m2, m3}, p;
tcgen05.mma.cta_group::1.kind::mxf8f6f4  [taddr2],  [taddr1],  bdesc, idesc,
                                         [tmem_scaleA], [tmem_scaleB], p;

tcgen05.commit.cta_group::1.mbarrier::arrive::one.b64 [mbarObj0];
// (truncated - see the official PTX ISA docs for the full example)

Instruction tcgen05.mma is an asynchronous instruction which initiates an MxNxK matrix
multiply and accumulate operation, D = A*B+D where the A matrix is MxK, the B matrix is KxN, and the D matrix is MxN.
The operation of the form D = A*B is issued when the input predicate argument enable-input-d is false.
The optional immediate argument scale-input-d can be specified to scale the input
matrix D as follows: D = A*B+D * (2 ^ - scale-input-d)
The valid range of values for argument scale-input-d is [0, 15]. (see the official PTX ISA docs for the full description)


// 1. Floating-point type without block scaling:
tcgen05.mma.sp.cta_group.kind  [d-tmem],  a-desc,  b-desc, [sp-meta-tmem] ,  idesc,
{ disable-output-lane }, enable-input-d{, scale-input-d};

Instruction tcgen05.mma.sp is an asynchronous instruction which initiates an MxNxK matrix multiply and accumulate operation of the form D = A*B+D where the A matrix is Mx(K/2), the B matrix is KxN, an… (see the official PTX ISA docs for the full description)

tcgen05.mma.sp.cta_group::1.kind::f16      [taddr0],  adesc,  bdesc, [tmem_spmeta0], idesc, p;

tcgen05.mma.sp.cta_group::1.kind::mxf8f6f4.collector::a:fill
                                           [taddr2],  [taddr1],  bdesc, [tmem_spmeta1], idesc,
                                           [tmem_scaleA], [tmem_scaleB], p;
// (truncated - see the official PTX ISA docs for the full example)

Instruction tcgen05.mma.sp is an asynchronous instruction which initiates an MxNxK matrix multiply and accumulate operation of the form D = A*B+D where the A matrix is Mx(K/2), the B matrix is KxN, and the D matrix is MxN. Sparse Matrices describes the details of the sparsity.
The operation of the form D = A*B is issued when the input predicate argument enable-input-d is false.
The optional immediate argument scale-input-d can be specified to scale the
input matrix D as follows: D = A*B+D * (2 ^ - scale-input-d)
The valid range of values for argument scale-input-d is [0, 15]. (see the official PTX ISA docs for the full description)


// 1. Floating-point type without block scaling:
tcgen05.mma.ws.cta_group::1.kind{.collector_usage}    [d-tmem],  a-desc,  b-desc,  idesc,
enable-input-d {, zero-column-mask-desc };

Instruction tcgen05.mma.ws is an asynchronous instruction which initiates an MxNxK matrix multiply and accumulate operation, D = A*B+D where the A matrix is MxK, the B matrix is KxN, and the D matrix… (see the official PTX ISA docs for the full description)

tcgen05.mma.ws.cta_group::1.kind::i8.collector::b2:use [taddr2], [taddr1], bdesc, idesc, p;
tcgen05.commit.cta_group::1.mbarrier::arrive::one.b64 [mbarObj0];

loop:
mbarrier.try_wait.parity.b64 p, [mbarObj0], 0;
@!p bra loop;

Instruction tcgen05.mma.ws is an asynchronous instruction which initiates an MxNxK matrix multiply and accumulate operation, D = A*B+D where the A matrix is MxK, the B matrix is KxN, and the D matrix is MxN.
The operation of the form D = A*B is issued when the input predicate argument enable-input-d is false.
The 32-bit register operand idesc is the instruction descriptor as described in Instruction descriptor, specifies the shapes, exact
types, sparsity and other details of the input matrices, output matrix and the matrix
multiply and accumulate operation.
The qualifier. (see the official PTX ISA docs for the full description)


// 1. Floating-point type without block scaling:
tcgen05.mma.ws.sp.cta_group::1.kind{.collector_usage} [d-tmem],  a-desc,  b-desc,
[sp-meta-tmem] ,  idesc,
enable-input-d {, zero-column-mask-desc};

Instruction tcgen05.mma.ws.sp is an asynchronous instruction which initiates
an MxNxK matrix multiply and accumulate operation, D = A*B+D where the A matrix is Mx(K/2), the B matrix is KxN, and the D (see the official PTX ISA docs for the full description)

tcgen05.mma.ws.sp.cta_group::1.kind::tf32.collector::b1::fill  [taddr1], [taddr0], bdesc,
                                                               [tmem_spmeta0], idesc, p;

tcgen05.commit.cta_group::1.mbarrier::arrive::one.b64 [mbarObj0];

loop:
// (truncated - see the official PTX ISA docs for the full example)

Instruction tcgen05.mma.ws.sp is an asynchronous instruction which initiates
an MxNxK matrix multiply and accumulate operation, D = A*B+D where the A matrix is Mx(K/2), the B matrix is KxN, and the D matrix
is MxN. Sparse Matrices describes the details of the
sparsity.
The operation of the form D = A*B is issued when the input predicate argument enable-input-d is false.
The 32-bit register operand idesc is the instruction descriptor as described in Instruction descriptor, specifies the shapes, exact
types, sparsity and other details of the input matrices, output matrix and the matrix
multiply and accumulate operation.
The qualifier. (see the official PTX ISA docs for the full description)


Summary: tcgen05.relinquish_alloc_permit specifies that the CTA of the executing thread is relinquishing the right to allocate Tensor Memory.


Instruction tcgen05.shift is an asynchronous instruction which initiates the shifting of 32-byte
elements downwards across all the rows, except the last, by one row. (see the official PTX ISA docs for the full description)

tcgen05.shift.down.cta_group::1 [taddr0];
tcgen05.shift.down.cta_group::2 [taddr1];

Instruction tcgen05.shift is an asynchronous instruction which initiates the shifting of 32-byte
elements downwards across all the rows, except the last, by one row. The address operand taddr specifies the base address of the matrix in the Tensor Memory whose rows must
be down shifted.
The lane of the address operand taddr must be aligned to 32.
Qualifier.cta_group specifies the number of CTAs whose Tensor Memory is touched when a single thread of a single CTA executes the tcgen05.shift instruction.
When.cta_group::1 is specified, the shift operation is performed in the Tensor Memory of the current CTA. (see the official PTX ISA docs for the full description)


tcgen05.st.sync.aligned.shape1.num{.unpack}.b32    [taddr], r;

Instruction tcgen05.st asynchronously stores data from the source register r into the Tensor Memory at the location specified by the 32-bit address operand taddr, collectively across all threads of th… (see the official PTX ISA docs for the full description)

tcgen05.st.sync.aligned.16x64b.x4.b32               [taddr0], {r0,  r1,  r2,  r3};

tcgen05.st.sync.aligned.16x128b.x1.unpack::16b.b32  [taddr1], {r0,  r1};

Instruction tcgen05.st asynchronously stores data from the source register r into
the Tensor Memory at the location specified by the 32-bit address operand taddr,
collectively across all threads of the warps.
All the threads in the warp must specify the same value of taddr, which must be the base
address of the collective store operation. Otherwise, the behavior is undefined.
The.shape qualifier and the.num qualifier together determines the total dimension
of the data which is stored to the Tensor Memory. The.shape qualifier indicates the base
dimension of data to be accessed as described in the Data Movement Shape. (see the official PTX ISA docs for the full description)


Instruction tcgen05.wait::st causes the executing thread to block until all prior tcgen05.st operations issued by the executing thread have completed.
Instruction tcgen05. (see the official PTX ISA docs for the full description)

Example 1:

tcgen05.ld.sync.aligned.32x32b.x2.b32     {r0, r1}, [taddr0];

// Prevents subsequent tcgen05.mma from racing ahead of the tcgen05.ld
// (truncated - see the official PTX ISA docs for the full example)

Instruction tcgen05.wait::st causes the executing thread to block until all prior tcgen05.st operations issued by the executing thread have completed.
Instruction tcgen05.wait::ld causes the executing thread to block until all prior tcgen05.ld operations issued by the executing thread have completed.
The mandatory.sync qualifier indicates that tcgen05.wait_operation causes the
executing thread to wait until all threads in the warp execute the same tcgen05.wait_operation instruction before resuming execution.
The mandatory.aligned qualifier indicates that all threads in the warp must execute the
same tcgen05.wait_operation instruction.


Parallel Synchronization and Communication Instructions

Summary: The tensormap.cp_fenceproxy instructions perform the following operations in order: Copies data of size specified by the size argument, in bytes, from the location specified by the address operand…

tensormap.cp_fenceproxy.cp_qualifiers.fence_qualifiers.sync.aligned  [dst], [src], size;

The tensormap.cp_fenceproxy instructions perform the following operations in order: Copies data of size specified by the size argument, in bytes, from the location specified by the address operand src… (see the official PTX ISA docs for the full description)

// Example: manipulate a tensor-map object and then consume it in cp.async.bulk.tensor

.reg .b64 new_addr;
.global .align 128 .b8 gbl[128];
.shared .align 128 .b8 sMem[128];
// (truncated - see the official PTX ISA docs for the full example)

The tensormap.cp_fenceproxy instructions perform the following operations in order:
Copies data of size specified by the size argument, in bytes, from the location specified
by the address operand src in shared memory to the location specified by the address operand dst in the global memory, in the generic proxy. Establishes a uni-directional proxy release pattern on the ordering from the copy operation
to the subsequent access performed in the tensormap proxy on the address dst.
The valid value of immediate operand size is 128.
The operands src and dst specify non-generic addresses in shared::cta and global state space respectively.
The. (see the official PTX ISA docs for the full description)


Data Movement and Conversion Instructions

tensormap.replace.mode.field1{.ss}.b1024.type  [addr], new_val;

The tensormap.replace instruction replaces the field, specified by.field qualifier,
of the tensor-map object at the location specified by the address operand addr with a
new value. (see the official PTX ISA docs for the full description)

tensormap.replace.tile.global_address.shared::cta.b1024.b64   [sMem], new_val;

The tensormap.replace instruction replaces the field, specified by.field qualifier,
of the tensor-map object at the location specified by the address operand addr with a
new value. The new value is specified by the argument new_val.
Qualifier.mode specifies the mode of the tensor-map object
located at the address operand addr.
Instruction type.b1024 indicates the size of the tensor-map object, which is 1024 bits.
Operand new_val has the type.type. When.field is specified as.global_address or.global_stride,.type must be.b64. (see the official PTX ISA docs for the full description)


testp.op.type  p, a;  // result is .pred

testp tests common properties of floating-point numbers and returns a predicate value of 1 if True and 0 if False.
testp.finite True if the input is not infinite or NaN testp. (see the official PTX ISA docs for the full description)

testp.notanumber.f32  isnan, f0;
testp.infinite.f64    p, X;

testp tests common properties of floating-point numbers and returns a predicate value of 1 if True and 0 if False.
testp.finite True if the input is not infinite or NaN testp.infinite True if the input is positive or negative infinity testp.number True if the input is not NaN testp.notanumber True if the input is NaN testp.normal True if the input is a normal number (not NaN, not infinity) testp.subnormal True if the input is a subnormal number (not NaN, not infinity)
As a special case, positive and negative zero are considered normal numbers.


tex.geom.v4.dtype.ctype  d, [a, c] {, e} {, f};

tex.{1d,2d,3d}
Texture lookup using a texture coordinate vector. The instruction loads data from the texture named
by operand a at coordinates given by operand c into destination d. (see the official PTX ISA docs for the full description)

// Example of unified mode texturing
 // - f4 is required to pad four-element tuple and is ignored
 tex.3d.v4.s32.s32  {r1,r2,r3,r4}, [tex_a,{f1,f2,f3,f4}];

 // Example of independent mode texturing
 tex.1d.v4.s32.f32  {r1,r2,r3,r4}, [tex_a,smpl_x,{f1}];
// (truncated - see the official PTX ISA docs for the full example)

tex.{1d,2d,3d}
Texture lookup using a texture coordinate vector. The instruction loads data from the texture named
by operand a at coordinates given by operand c into destination d. Operand c is a
scalar or singleton tuple for 1d textures; is a two-element vector for 2d textures; and is a
four-element vector for 3d textures, where the fourth element is ignored. An optional texture
sampler b may be specified. If no sampler is specified, the sampler behavior is a property of
the named texture. The optional destination predicate p is set to True if data from texture
at specified coordinates is resident in memory, False otherwise. (see the official PTX ISA docs for the full description)


Texture fetch of the 4-texel bilerp footprint using a texture coordinate vector. (see the official PTX ISA docs for the full description)

//Example of unified mode texturing
tld4.r.2d.v4.s32.f32  {r1,r2,r3,r4}, [tex_a,{f1,f2}];

// Example of independent mode texturing
tld4.r.2d.v4.u32.f32  {u1,u2,u3,u4}, [tex_a,smpl_x,{f1,f2}];
// (truncated - see the official PTX ISA docs for the full example)

Texture fetch of the 4-texel bilerp footprint using a texture coordinate vector. The instruction
loads the bilerp footprint from the texture named by operand a at coordinates given by operand c into vector destination d. The texture component fetched for each texel sample is
specified by.comp. The four texel samples are placed into destination vector d in
counter-clockwise order starting at lower left.
An optional texture sampler b may be specified. If no sampler is specified, the sampler behavior
is a property of the named texture.
The optional destination predicate p is set to True if data from texture at specified
coordinates is resident in memory, False otherwise. (see the official PTX ISA docs for the full description)


Summary: Abort execution and generate an interrupt to the host CPU.


txq.tquery.b32         d, [a];       // texture attributes

Query an attribute of a texture or sampler. Operand a is either a.texref or.samplerref variable, or a.u64 register.
Query Returns.width.height.depth value in elements. (see the official PTX ISA docs for the full description)

txq.width.b32       %r1, [tex_A];
txq.filter_mode.b32 %r1, [tex_A];   // unified mode
txq.addr_mode_0.b32 %r1, [smpl_B];  // independent mode
txq.level.width.b32 %r1, [tex_A], %r_lod;

Query an attribute of a texture or sampler. Operand a is either a.texref or.samplerref variable, or a.u64 register.
Query Returns.width.height.depth value in elements.channel_data_type Unsigned integer corresponding to source language’s channel data type
enumeration. If the source language combines channel data type and channel
order into a single enumeration type, that value is returned for both channel_data_type and channel_order queries..channel_order Unsigned integer corresponding to source language’s channel order
enumeration. (see the official PTX ISA docs for the full description)


Summary: Perform scalar arithmetic operation with optional saturate, and optional secondary arithmetic operation or subword data merge.

// 32-bit scalar operation, with optional secondary operation
vadd.dtype.atype.btype{.sat}       d, a{.asel}, b{.bsel};

// extract byte/half-word/word and sign- or zero-extend
// based on source operand type
ta = partSelectSignExtend( a, atype, asel );
tb = partSelectSignExtend( b, btype, bsel );

switch ( vop ) {
    case vadd:     tmp = ta + tb;
    case vsub:     tmp = ta - tb;
    case vabsdiff: tmp = | ta - tb |;
    case vmin:     tmp = MIN( ta, tb );
    case vmax:     tmp = MAX( ta, tb );
}
// saturate, taking into account destination type and merge operations
tmp = optSaturate( tmp, sat, isSigned(dtype), dsel );
d = optSecondaryOp( op2, tmp, c );  // optional secondary operation
d = optMerge( dsel, tmp, c );       // optional merge with c operand


// SIMD instruction with secondary SIMD merge operation
vadd2.dtype.atype.btype{.sat}  d{.mask}, a{.asel}, b{.bsel}, c;

Two-way SIMD parallel arithmetic operation with secondary operation. (see the official PTX ISA docs for the full description)

// extract pairs of half-words and sign- or zero-extend
// based on operand type
Va = extractAndSignExt_2( a, b, .asel, .atype );
Vb = extractAndSignExt_2( a, b, .bsel, .btype );
Vc = extractAndSignExt_2( c );

for (i=0; i<2; i++) {
    switch ( vop2 ) {
       case vadd2:             t[i] = Va[i] + Vb[i];
       case vsub2:             t[i] = Va[i] - Vb[i];
       case vavrg2:            if ( ( Va[i] + Vb[i] ) >= 0 ) {
                                   t[i] = ( Va[i] + Vb[i] + 1 ) >> 1;
                               } else {
                                   t[i] = ( Va[i] + Vb[i] ) >> 1;
                               }
       case vabsdiff2:         t[i] = | Va[i] - Vb[i] |;
       case vmin2:             t[i] = MIN( Va[i], Vb[i] );
       case vmax2:             t[i] = MAX( Va[i], Vb[i] );
    }
    if (.sat) {
        if ( .dtype == .s32 )  t[i] = CLAMP( t[i], S16_MAX, S16_MIN ); (see the official PTX ISA docs for the full semantics)

Two-way SIMD parallel arithmetic operation with secondary operation.
Elements of each dual half-word source to the operation are selected from any of the four half-words
in the two source operands a and b using the asel and bsel modifiers.
The selected half-words are then operated on in parallel.
The results are optionally clamped to the appropriate range determined by the destination type
(signed or unsigned). Saturation cannot be used with the secondary accumulate operation.
For instructions with a secondary SIMD merge operation:
For half-word positions indicated in mask, the selected half-word results are copied into
destination d. (see the official PTX ISA docs for the full description)


// SIMD instruction with secondary SIMD merge operation
vadd4.dtype.atype.btype{.sat}  d{.mask}, a{.asel}, b{.bsel}, c;

Four-way SIMD parallel arithmetic operation with secondary operation. (see the official PTX ISA docs for the full description)

// extract quads of bytes and sign- or zero-extend
// based on operand type
Va = extractAndSignExt_4( a, b, .asel, .atype );
Vb = extractAndSignExt_4( a, b, .bsel, .btype );
Vc = extractAndSignExt_4( c );
for (i=0; i<4; i++) {
    switch ( vop4 ) {
        case vadd4:            t[i] = Va[i] + Vb[i];
        case vsub4:            t[i] = Va[i] - Vb[i];
        case vavrg4:           if ( ( Va[i] + Vb[i] ) >= 0 ) {
                                   t[i] = ( Va[i] + Vb[i] + 1 ) >> 1;
                               } else {
                                   t[i] = ( Va[i] + Vb[i] ) >> 1;
                               }
        case vabsdiff4:        t[i] = | Va[i] - Vb[i] |;
        case vmin4:            t[i] = MIN( Va[i], Vb[i] );
        case vmax4:            t[i] = MAX( Va[i], Vb[i] );
    }
    if (.sat) {
        if ( .dtype == .s32 )  t[i] = CLAMP( t[i], S8_MAX, S8_MIN ); (see the official PTX ISA docs for the full semantics)

Four-way SIMD parallel arithmetic operation with secondary operation.
Elements of each quad byte source to the operation are selected from any of the eight bytes in the
two source operands a and b using the asel and bsel modifiers.
The selected bytes are then operated on in parallel.
The results are optionally clamped to the appropriate range determined by the destination type
(signed or unsigned). Saturation cannot be used with the secondary accumulate operation.
For instructions with a secondary SIMD merge operation:
For byte positions indicated in mask, the selected byte results are copied into destination d. (see the official PTX ISA docs for the full description)


// 32-bit scalar operation
vmad.dtype.atype.btype{.sat}{.scale}     d, {-}a{.asel}, {-}b{.bsel},
{-}c;

Calculate (a*b) + c, with optional operand negates, plus one mode, and scaling.
The source operands support optional negation with some restrictions. (see the official PTX ISA docs for the full description)

// extract byte/half-word/word and sign- or zero-extend
// based on source operand type
ta = partSelectSignExtend( a, atype, asel );
tb = partSelectSignExtend( b, btype, bsel );
signedFinal = isSigned(atype) || isSigned(btype) ||
                                 (a.negate ^ b.negate) || c.negate;
tmp[127:0] = ta * tb;

lsb = 0;
if ( .po )                  {              lsb = 1; } else
if ( a.negate ^ b.negate )  { tmp = ~tmp;  lsb = 1; } else
if ( c.negate )             { c   = ~c;    lsb = 1; }

c128[127:0] = (signedFinal) sext32( c ) : zext ( c );
tmp = tmp + c128 + lsb;
switch( scale ) {
   case .shr7:   result = (tmp >>  7) & 0xffffffffffffffff;
   case .shr15:  result = (tmp >> 15) & 0xffffffffffffffff;
}
if ( .sat ) {
     if (signedFinal) result = CLAMP(result, S32_MAX, S32_MIN);
     else             result = CLAMP(result, U32_MAX, U32_MIN);
}

Calculate (a*b) + c, with optional operand negates, plus one mode, and scaling.
The source operands support optional negation with some restrictions. Although PTX syntax allows
separate negation of the a and b operands, internally this is represented as negation of the
product (a*b). That is, (a*b) is negated if and only if exactly one of a or b is
negated. PTX allows negation of either (a*b) or c.
The plus one mode (.po ) computes (a*b) + c + 1, which is used in computing averages. Source
operands may not be negated in.po mode.
The intermediate result of (a*b) is unsigned if atype and btype are unsigned and the product (a*b) is not negated; otherwise, the intermediate result is signed. (see the official PTX ISA docs for the full description)


Summary: Combine a per-lane predicate across the warp using any/all/ballot reduction.

Reduce predicate a across the warp: any true, all true, or all-lanes-agree.

Collect per-lane predicate a into a 32-bit bitmask, one bit per lane.

d = reduction of predicate a across the active lanes of the warp, per the selected mode.

vote.sync.all.pred    p,q,0xffffffff;
vote.sync.ballot.b32  r1,p,0xffffffff;  // get 'ballot' across warp

vote.sync will cause executing thread to wait until all non-exited threads corresponding to membermask have executed vote.sync with the same qualifiers and same membermask value
before resuming execution.
Operand membermask specifies a 32-bit integer which is a mask indicating threads participating
in this instruction where the bit position corresponds to thread’s laneid. Operand a is a
predicate register.
In the mode form, vote.sync performs a reduction of the source predicate across all non-exited
threads in membermask. (see the official PTX ISA docs for the full description)


// 32-bit scalar operation, with optional secondary operation
vset.atype.btype.cmp       d, a{.asel}, b{.bsel};

Compare input values using specified comparison, with optional secondary arithmetic operation or
subword data merge. (see the official PTX ISA docs for the full description)

// extract byte/half-word/word and sign- or zero-extend
// based on source operand type
ta = partSelectSignExtend( a, atype, asel );
tb = partSelectSignExtend( b, btype, bsel );
tmp = compare( ta, tb, cmp ) ? 1 : 0;
d = optSecondaryOp( op2, tmp, c );    // optional secondary operation
d = optMerge( dsel, tmp, c );         // optional merge with c operand

Compare input values using specified comparison, with optional secondary arithmetic operation or
subword data merge.
The intermediate result of the comparison is always unsigned, and therefore destination d and
operand c are also unsigned.


// SIMD instruction with secondary SIMD merge operation
vset2.atype.btype.cmp  d{.mask}, a{.asel}, b{.bsel}, c;

Two-way SIMD parallel comparison with secondary operation. (see the official PTX ISA docs for the full description)

// extract pairs of half-words and sign- or zero-extend
// based on operand type
Va = extractAndSignExt_2( a, b, .asel, .atype );
Vb = extractAndSignExt_2( a, b, .bsel, .btype );
Vc = extractAndSignExt_2( c );
for (i=0; i<2; i++) {
    t[i] = compare( Va[i], Vb[i], .cmp ) ? 1 : 0;
}
// secondary accumulate or SIMD merge
mask = extractMaskBits( .mask );
if (.add) {
    d = c;
    for (i=0; i<2; i++) {  d += mask[i] ? t[i] : 0;  }
} else {
    d = 0;
    for (i=0; i<2; i++)  {  d |= mask[i] ? t[i] : Vc[i];  }
}

Two-way SIMD parallel comparison with secondary operation.
Elements of each dual half-word source to the operation are selected from any of the four half-words
in the two source operands a and b using the asel and bsel modifiers.
The selected half-words are then compared in parallel.
The intermediate result of the comparison is always unsigned, and therefore the half-words of
destination d and operand c are also unsigned.
For instructions with a secondary SIMD merge operation:
For half-word positions indicated in mask, the selected half-word results are copied into
destination d. (see the official PTX ISA docs for the full description)


// SIMD instruction with secondary SIMD merge operation
vset4.atype.btype.cmp  d{.mask}, a{.asel}, b{.bsel}, c;

Four-way SIMD parallel comparison with secondary operation. (see the official PTX ISA docs for the full description)

// extract quads of bytes and sign- or zero-extend
// based on operand type
Va = extractAndSignExt_4( a, b, .asel, .atype );
Vb = extractAndSignExt_4( a, b, .bsel, .btype );
Vc = extractAndSignExt_4( c );
for (i=0; i<4; i++) {
    t[i] = compare( Va[i], Vb[i], cmp ) ? 1 : 0;
}
// secondary accumulate or SIMD merge
mask = extractMaskBits( .mask );
if (.add) {
    d = c;
    for (i=0; i<4; i++) {  d += mask[i] ? t[i] : 0;  }
} else {
    d = 0;
    for (i=0; i<4; i++)  {  d |= mask[i] ? t[i] : Vc[i];  }
}

Four-way SIMD parallel comparison with secondary operation.
Elements of each quad byte source to the operation are selected from any of the eight bytes in the
two source operands a and b using the asel and bsel modifiers.
The selected bytes are then compared in parallel.
The intermediate result of the comparison is always unsigned, and therefore the bytes of destination d and operand c are also unsigned.
For instructions with a secondary SIMD merge operation:
For byte positions indicated in mask, the selected byte results are copied into destination d. (see the official PTX ISA docs for the full description)


// 32-bit scalar operation, with optional secondary operation
vshl.dtype.atype.u32{.sat}.mode       d, a{.asel}, b{.bsel};

vshl Shift a left by unsigned amount in b with optional saturate, and optional secondary
arithmetic operation or subword data merge. Left shift fills with zero. (see the official PTX ISA docs for the full description)

// extract byte/half-word/word and sign- or zero-extend
// based on source operand type
ta = partSelectSignExtend( a,atype, asel );
tb = partSelectSignExtend( b, .u32, bsel );
if ( mode == .clamp  && tb > 32 )  tb = 32;
if ( mode == .wrap )                       tb = tb & 0x1f;
switch ( vop ){
   case vshl:  tmp = ta << tb;
   case vshr:  tmp = ta >> tb;
}
// saturate, taking into account destination type and merge operations
tmp = optSaturate( tmp, sat, isSigned(dtype), dsel );
d = optSecondaryOp( op2, tmp, c );  // optional secondary operation
d = optMerge( dsel, tmp, c );       // optional merge with c operand

vshl Shift a left by unsigned amount in b with optional saturate, and optional secondary
arithmetic operation or subword data merge. Left shift fills with zero. vshr Shift a right by unsigned amount in b with optional saturate, and optional secondary
arithmetic operation or subword data merge. Signed shift fills with the sign bit, unsigned shift
fills with zero.


// 32-bit scalar operation, with optional secondary operation
vshr.dtype.atype.u32{.sat}.mode       d, a{.asel}, b{.bsel};


// 32-bit scalar operation, with optional secondary operation
vsub.dtype.atype.btype{.sat}       d, a{.asel}, b{.bsel};

Perform scalar arithmetic operation with optional saturate, and optional secondary arithmetic operation or subword data merge.


// SIMD instruction with secondary SIMD merge operation
vsub2.dtype.atype.btype{.sat}  d{.mask}, a{.asel}, b{.bsel}, c;

Two-way SIMD parallel arithmetic operation with secondary operation. (see the official PTX ISA docs for the full description)

Two-way SIMD parallel arithmetic operation with secondary operation.
Elements of each dual half-word source to the operation are selected from any of the four half-words
in the two source operands a and b using the asel and bsel modifiers.
The selected half-words are then operated on in parallel.
The results are optionally clamped to the appropriate range determined by the destination type
(signed or unsigned). Saturation cannot be used with the secondary accumulate operation.
For instructions with a secondary SIMD merge operation:
For half-word positions indicated in mask, the selected half-word results are copied into
destination d. (see the official PTX ISA docs for the full description)


// SIMD instruction with secondary SIMD merge operation
vsub4.dtype.atype.btype{.sat}  d{.mask}, a{.asel}, b{.bsel}, c;

Four-way SIMD parallel arithmetic operation with secondary operation. (see the official PTX ISA docs for the full description)

Four-way SIMD parallel arithmetic operation with secondary operation.
Elements of each quad byte source to the operation are selected from any of the eight bytes in the
two source operands a and b using the asel and bsel modifiers.
The selected bytes are then operated on in parallel.
The results are optionally clamped to the appropriate range determined by the destination type
(signed or unsigned). Saturation cannot be used with the secondary accumulate operation.
For instructions with a secondary SIMD merge operation:
For byte positions indicated in mask, the selected byte results are copied into destination d. (see the official PTX ISA docs for the full description)


Asynchronous Warpgroup Level Matrix Multiply-Accumulate Instructions

wgmma.commit_group instruction creates a new wgmma-group per warpgroup and batches all prior wgmma. (see the official PTX ISA docs for the full description)

wgmma.commit_group instruction creates a new wgmma-group per warpgroup and batches all prior wgmma.mma_async instructions initiated by the executing warp but not committed to any
wgmma-group into the new wgmma-group. If there are no uncommitted wgmma.mma_async instructions
then wgmma.commit_group results in an empty wgmma-group.
An executing thread can wait for the completion of all wgmma.mma_async operations in a
wgmma-group by using wgmma.wait_group.
The mandatory.sync qualifier indicates that wgmma.commit_group instruction causes the
executing thread to wait until all threads in the warp execute the same wgmma.commit_group instruction before resuming execution.
The mandatory. (see the official PTX ISA docs for the full description)


wgmma.fence instruction establishes an ordering between prior accesses to any warpgroup
registers and subsequent accesses to the same registers by a wgmma.mma_async instruction. (see the official PTX ISA docs for the full description)

// Example 1, first use example:
wgmma.fence.sync.aligned;    // Establishes an ordering w.r.t. prior accesses to the registers s32d<0-3>
wgmma.mma_async.sync.aligned.m64n8k32.s32.u8.u8  {s32d0, s32d1, s32d2, s32d3},
                                                  descA, descB, scaleD;
wgmma.commit_group.sync.aligned;
wgmma.wait_group.sync.aligned 0;
// (truncated - see the official PTX ISA docs for the full example)

wgmma.fence instruction establishes an ordering between prior accesses to any warpgroup
registers and subsequent accesses to the same registers by a wgmma.mma_async instruction. Only
the accumulator register and the input registers containing the fragments of matrix A require this
ordering.
The wgmma.fence instruction must be issued by all warps of the warpgroup at the following
locations:
Before the first wgmma.mma_async operation in a warpgroup. (see the official PTX ISA docs for the full description)


wgmma.mma_async.sync.aligned.shape.dtype.f16.f16  d, a-desc, b-desc, scale-d, imm-scale-a, imm-scale-b, imm-trans-a, imm-trans-b;

Instruction wgmma.mma_async issues a MxNxK matrix multiply and accumulate operation, D = A*B+D, where the A matrix is MxK, the B matrix is KxN, and the D matrix is MxN.
The operation of the form D = A (see the official PTX ISA docs for the full description)

Instruction wgmma.mma_async issues a MxNxK matrix multiply and accumulate operation, D = A*B+D, where the A matrix is MxK, the B matrix is KxN, and the D matrix is MxN.
The operation of the form D = A*B is issued when the input predicate argument scale-d is
false.
wgmma.fence instruction must be used to fence the register accesses of wgmma.mma_async instruction from their prior accesses. (see the official PTX ISA docs for the full description)


wgmma.mma_async.sp.sync.aligned.shape.dtype.f16.f16  d, a-desc, b-desc, sp-meta, sp-sel, scale-d, imm-scale-a, imm-scale-b, imm-trans-a, imm-trans-b;

Instruction wgmma.mma_async issues a MxNxK matrix multiply and accumulate operation, D = A*B+D, where the A matrix is MxK, the B matrix is KxN, and the D matrix is MxN. (see the official PTX ISA docs for the full description)

Instruction wgmma.mma_async issues a MxNxK matrix multiply and accumulate operation, D = A*B+D, where the A matrix is MxK, the B matrix is KxN, and the D matrix is MxN.
The matrix A is stored in the packed format Mx(K/2) as described in Sparse matrix storage.
The operation of the form D = A*B is issued when the input predicate argument scale-d is
false.
wgmma.fence instruction must be used to fence the register accesses of wgmma.mma_async instruction from their prior accesses. (see the official PTX ISA docs for the full description)


wgmma.wait_group instruction will cause the executing thread to wait until only N or fewer of the most recent wgmma-groups are pending and all the prior wgmma-groups committed by the executing threads… (see the official PTX ISA docs for the full description)

wgmma.fence.sync.aligned;

wgmma.mma_async.sync.aligned.m64n8k32.s32.u8.u8  {s32d0, s32d1, s32d2, s32d3},
                                                  descA, descB, scaleD;
wgmma.commit_group.sync.aligned;
// (truncated - see the official PTX ISA docs for the full example)

wgmma.wait_group instruction will cause the executing thread to wait until only N or fewer of
the most recent wgmma-groups are pending and all the prior wgmma-groups committed by the executing
threads are complete. For example, when N is 0, the executing thread waits on all the prior
wgmma-groups to complete. Operand N is an integer constant.
Accessing the accumulator register or the input register containing the fragments of matrix A of a wgmma.mma_async instruction without first performing a wgmma.wait_group instruction that
waits on a wgmma-group including that wgmma.mma_async instruction is undefined behavior.
The mandatory.sync qualifier indicates that wgmma. (see the official PTX ISA docs for the full description)


Warp Level Matrix Multiply-Accumulate Instructions

Summary: Higher-level warp matrix-multiply-accumulate built from explicit load/mma/store steps.

Load a matrix fragment cooperatively across the warp.

wmma.mma.sync.aligned.layout.shape.dtype.ctype d, a, b, c;

Perform the accumulate step on already-loaded fragments.

Store an accumulator fragment cooperatively across the warp.

Together, the load/mma/store triplet compute D = A * B + C for a fixed tile shape, cooperatively across the warp.

// Load elements from f16 row-major matrix B
.reg .b32 x<8>;

wmma.load.b.sync.aligned.m16n16k16.row.f16 {x0,x1,x2,x3,x4,x5,x,x7}, [ptr];
// Now use {x0, ..., x7} for the actual wmma.mma
// (truncated - see the official PTX ISA docs for the full example)

.global .align 32 .f16 A[256], B[256];
.global .align 32 .f32 C[256], D[256];
.reg .b32 a<8> b<8> c<8> d<8>;

wmma.load.a.sync.aligned.m16n16k16.global.row.f16
        {a0, a1, a2, a3, a4, a5, a6, a7}, [A];
// (truncated - see the official PTX ISA docs for the full example)

// Storing f32 elements computed by a wmma.mma
.reg .b32 x<8>;

wmma.mma.sync.m16n16k16.row.col.f32.f32
              {d0, d1, d2, d3, d4, d5, d6, d7}, ...;
wmma.store.d.sync.m16n16k16.row.f32
// (truncated - see the official PTX ISA docs for the full example)

Perform a warp-level matrix multiply-and-accumulate computation D = A * B + C using matrices A,
B and C loaded in registers a, b and c respectively, and store the result matrix in
register d. The register arguments a, b, c and d hold unspecified fragments of
the corresponding matrices as described in Matrix Fragments for WMMA
The qualifiers.dtype,.atype,.btype and.ctype indicate the data-type of the
elements in the matrices D, A, B and C respectively.
For wmma.mma without explicit.atype and.btype:.atype and.btype are
implicitly set to.f16.
For integer wmma,.ctype and.dtype must be specified as.s32. (see the official PTX ISA docs for the full description)


Summary: Bitwise XOR of two operands.

Bitwise XOR, including a predicate form.

Compute the bit-wise exclusive-or operation for the bits in a and b.


Add a double-precision float value in the data register to a location in a buffer surface. Store the original value from buffer surface into a vector register iff the SC0 bit is set.

tmp = MEM[ADDR].f64;
MEM[ADDR].f64 += DATA.f64;
RETURN_DATA = tmp

Floating-point addition handles NAN/INF/denorm.


Add two unsigned 32-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_add_u32 v5, off, s[8:11], s3


Add two unsigned 64-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_add_u64 v[5:6], off, s[8:11], s3


Calculate bitwise AND given two unsigned 32-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_and_b32 v5, off, s[8:11], s3


Calculate bitwise AND given two unsigned 64-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_and_b64 v[5:6], off, s[8:11], s3


Compare two unsigned 32-bit integer values stored in the data comparison register and a location in a buffer surface. Modify the memory location with a value in the data source register iff the comparison is equal. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_cmpswap_b32 v[5:6], off, s[8:11], s3


Compare two unsigned 64-bit integer values stored in the data comparison register and a location in a buffer surface. Modify the memory location with a value in the data source register iff the comparison is equal. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_cmpswap_b64 v[5:8], off, s[8:11], s3


Compare two single-precision float values stored in the data comparison register and a location in a buffer surface. Modify the memory location with a value in the data source register iff the comparison is equal. Store the original value from buffer surface into a vector register iff the GLC bit is set.

buffer_atomic_cmpswap_f32 v[5:6], off, s[8:11], s3


Summary: Decrement an unsigned 32-bit integer value from a location in a buffer surface with wraparound to a value in the data register if the decrement…

Decrement an unsigned 32-bit integer value from a location in a buffer surface with wraparound to a value in the data register if the decrement yields a negative value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_dec_u32 v5, off, s[8:11], s3


Summary: Decrement an unsigned 64-bit integer value from a location in a buffer surface with wraparound to a value in the data register if the decrement…

Decrement an unsigned 64-bit integer value from a location in a buffer surface with wraparound to a value in the data register if the decrement yields a negative value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_dec_u64 v[5:6], off, s[8:11], s3


Select the maximum of two double-precision float inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the GLC bit is set.


Select the minimum of two double-precision float inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the GLC bit is set.


Increment an unsigned 32-bit integer value from a location in a buffer surface with wraparound to 0 if the value exceeds a value in the data register. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_inc_u32 v5, off, s[8:11], s3


Increment an unsigned 64-bit integer value from a location in a buffer surface with wraparound to 0 if the value exceeds a value in the data register. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_inc_u64 v[5:6], off, s[8:11], s3


Select the maximum of two double-precision float inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the SC0 bit is set.

addr = CalcBufferAddr(VADDR.b32, SRSRC.b32, SOFFSET.b32, OFFSET.b32);
tmp = MEM[addr].f64;
src = DATA.f64;
MEM[addr].f64 = src > tmp ? src : tmp;
RETURN_DATA.f64 = tmp


Select the maximum of two signed 32-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_max_i32 v5, off, s[8:11], s3


Select the maximum of two signed 64-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_max_i64 v[5:6], off, s[8:11], s3


Select the IEEE maximumNumber() of two single-precision float inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.


Summary: AMDGPU MUBUF vector instruction operating on f64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Select the maximum of two unsigned 32-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_max_u32 v5, off, s[8:11], s3


Select the maximum of two unsigned 64-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_max_u64 v[5:6], off, s[8:11], s3


Select the minimum of two double-precision float inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the SC0 bit is set.

addr = CalcBufferAddr(VADDR.b32, SRSRC.b32, SOFFSET.b32, OFFSET.b32);
tmp = MEM[addr].f64;
src = DATA.f64;
MEM[addr].f64 = src < tmp ? src : tmp;
RETURN_DATA.f64 = tmp


Select the minimum of two signed 32-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_min_i32 v5, off, s[8:11], s3


Select the minimum of two signed 64-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_min_i64 v[5:6], off, s[8:11], s3


Select the IEEE minimumNumber() of two single-precision float inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.


Select the minimum of two unsigned 32-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_min_u32 v5, off, s[8:11], s3


Select the minimum of two unsigned 64-bit integer inputs, given two values stored in the data register and a location in a buffer surface. Update the buffer surface with the selected value. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_min_u64 v[5:6], off, s[8:11], s3


Calculate bitwise OR given two unsigned 32-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_or_b32 v5, off, s[8:11], s3


Calculate bitwise OR given two unsigned 64-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_or_b64 v[5:6], off, s[8:11], s3


Summary: AMDGPU MUBUF vector instruction. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Subtract an unsigned 32-bit integer location in a buffer surface from a value in the data register and clamp the result to zero. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.


Subtract an unsigned 32-bit integer value stored in the data register from a value stored in a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_sub_u32 v5, off, s[8:11], s3


Subtract an unsigned 64-bit integer value stored in the data register from a value stored in a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_sub_u64 v[5:6], off, s[8:11], s3


Swap an unsigned 32-bit integer value in the data register with a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_swap_b32 v5, off, s[8:11], s3


Swap an unsigned 64-bit integer value in the data register with a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_swap_b64 v[5:6], off, s[8:11], s3


Calculate bitwise XOR given two unsigned 32-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_xor_b32 v5, off, s[8:11], s3


Calculate bitwise XOR given two unsigned 64-bit integer values stored in the data register and a location in a buffer surface. Store the original value from buffer surface into a vector register iff the temporal hint enables atomic return.

buffer_atomic_xor_b64 v[5:6], off, s[8:11], s3


Summary: Write back and invalidate the shader L0. Returns ACK to shader.


Summary: Invalidate the GL1 cache only. Returns ACK to shader.


Summary: Invalidate CU and/or L2 cache depending on sc0 and sc1 bits. Returns ACK to shader.


Summary: Invalidate L2 cache. Returns ACK to shader.


Summary: Load 128 bits of data from a buffer surface into a vector register.


Summary: Load 32 bits of data from a buffer surface into a vector register.


Summary: Load 64 bits of data from a buffer surface into a vector register.


Summary: Load 96 bits of data from a buffer surface into a vector register.


Summary: Load 16 bits of unsigned data from a buffer surface and store the result into the low 16 bits of a 32-bit vector register.

buffer_load_d16_b16 v5, off, s[8:11], s3


Summary: Load 1-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result…

Load 1-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result into the low 16 bits of a 32-bit vector register. The resource descriptor specifies the data format of the surface.

buffer_load_d16_format_x v5, off, s[8:11], s3


Summary: Load 2-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result…

Load 2-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result into a vector register. The resource descriptor specifies the data format of the surface.

buffer_load_d16_format_xy v5, off, s[8:11], s3


Summary: Load 3-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result…

Load 3-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result into a vector register. The resource descriptor specifies the data format of the surface.

buffer_load_d16_format_xyz v[5:6], off, s[8:11], s3


Summary: Load 4-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result…

Load 4-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result into a vector register. The resource descriptor specifies the data format of the surface.

buffer_load_d16_format_xyzw v[5:6], off, s[8:11], s3


Summary: Load 16 bits of unsigned data from a buffer surface and store the result into the high 16 bits of a 32-bit vector register.

buffer_load_d16_hi_b16 v5, off, s[8:11], s3


Load 1-component formatted data from a buffer surface, convert the data to packed 16 bit integral or floating point format, then store the result into the high 16 bits of a 32-bit vector register. The resource descriptor specifies the data format of the surface.

buffer_load_d16_hi_format_x v5, off, s[8:11], s3


Summary: Load 8 bits of signed data from a buffer surface, sign extend to 16 bits and store the result into the high 16 bits of a 32-bit vector register.

buffer_load_d16_hi_i8 v5, off, s[8:11], s3


Summary: Load 8 bits of unsigned data from a buffer surface, zero extend to 16 bits and store the result into the high 16 bits of a 32-bit vector register.

buffer_load_d16_hi_u8 v5, off, s[8:11], s3


Summary: Load 8 bits of signed data from a buffer surface, sign extend to 16 bits and store the result into the low 16 bits of a 32-bit vector register.


Summary: Load 8 bits of unsigned data from a buffer surface, zero extend to 16 bits and store the result into the low 16 bits of a 32-bit vector register.


Summary: Load one 32-bit dword per lane through a buffer (raw/structured) resource descriptor.

buffer_load_dword VDST, VADDR, SRSRC, offset

VDST[lane] = *(resource-relative address computed from SRSRC, VADDR[lane], offset) for each active lane; bounds-checked against the descriptor.

buffer_load_dword  v2, v0, s[4:7], 0 offen   // v2 = buffer[s[4:7]](v0)


Summary: Load 16 bits of signed data from a buffer surface, sign extend to 32 bits and store the result into a vector register.


Summary: Load 8 bits of signed data from a buffer surface, sign extend to 32 bits and store the result into a vector register.


Summary: Load 16 bits of unsigned data from a buffer surface, zero extend to 32 bits and store the result into a vector register.


Summary: Load 8 bits of unsigned data from a buffer surface, zero extend to 32 bits and store the result into a vector register.


Summary: Store 128 bits of data from vector input registers into a buffer surface.


Summary: Store 16 bits of data from a vector register into a buffer surface.


Summary: Store 32 bits of data from vector input registers into a buffer surface.


Summary: Store 64 bits of data from vector input registers into a buffer surface.


Summary: Store 8 bits of data from a vector register into a buffer surface.


Summary: Store 96 bits of data from vector input registers into a buffer surface.


Summary: Convert 16 bits of data from the low 16 bits of a 32-bit vector input register into 1-component formatted data and store the data into a buffer…

Convert 16 bits of data from the low 16 bits of a 32-bit vector input register into 1-component formatted data and store the data into a buffer surface. The instruction specifies the data format of the surface, overriding the resource descriptor.

buffer_store_d16_format_x v1, off, s[12:15], s4


Convert 32 bits of data from vector input registers into 2-component formatted data and store the data into a buffer surface. The instruction specifies the data format of the surface, overriding the resource descriptor.

buffer_store_d16_format_xy v1, off, s[12:15], s4


Convert 48 bits of data from vector input registers into 3-component formatted data and store the data into a buffer surface. The instruction specifies the data format of the surface, overriding the resource descriptor.

buffer_store_d16_format_xyz v[1:2], off, s[12:15], s4


Convert 64 bits of data from vector input registers into 4-component formatted data and store the data into a buffer surface. The instruction specifies the data format of the surface, overriding the resource descriptor.

buffer_store_d16_format_xyzw v[1:2], off, s[12:15], s4


Summary: Store 16 bits of data from the high 16 bits of a 32-bit vector register into a buffer surface.

buffer_store_d16_hi_b16 v1, off, s[12:15], s4


Summary: Store 8 bits of data from the high 16 bits of a 32-bit vector register into a buffer surface.

buffer_store_d16_hi_b8 v1, off, s[12:15], s4


Summary: Convert 16 bits of data from the high 16 bits of a 32-bit vector input register into 1-component formatted data and store the data into a buffer…

Convert 16 bits of data from the high 16 bits of a 32-bit vector input register into 1-component formatted data and store the data into a buffer surface. The instruction specifies the data format of the surface, overriding the resource descriptor.

buffer_store_d16_hi_format_x v1, off, s[12:15], s4


Summary: Store one DWORD from LDS memory to system memory without utilizing VGPRs.


Summary: AMDGPU MUBUF vector instruction. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: Write back and invalidate the shader L1 only for lines that are marked volatile. Returns ACK to shader.


Summary: Write back L2 cache. Returns ACK to shader.


Summary: AMDGPU FLAT vector instruction operating on b128 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU FLAT vector instruction operating on b32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU FLAT vector instruction operating on b64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU FLAT vector instruction operating on b8 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: Add two single-precision float values stored in the data register and a location in a data share.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].f32;
MEM[addr].f32 += DATA.f32;
RETURN_DATA.f32 = tmp


Summary: Add a double-precision float value in the data register to a location in a data share.


Perform an atomic add to data in specific registers embedded in GDS rather than operating on GDS memory directly. This instruction returns the pre-op value. This instruction is only used by the GS stage and is used to facilitate streamout.


Add two single-precision float values stored in the data register and a location in a data share. Store the original value from data share into a vector register.


Add a double-precision float value in the data register to a location in a data share. Store the original value from data share into a vector register.


Add two unsigned 32-bit integer values stored in the data register and a location in a data share. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u32;
MEM[addr].u32 += DATA.u32;
RETURN_DATA.u32 = tmp


Add two unsigned 64-bit integer values stored in the data register and a location in a data share. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u64;
MEM[addr].u64 += DATA.u64;
RETURN_DATA.u64 = tmp


Summary: AMDGPU DS vector instruction operating on f32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on u32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on u64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: Atomically add a per-lane value to an LDS location.

Add two unsigned 32-bit integer values stored in the data register and a location in a data share.

old = LDS[ADDR[lane] + offset]; LDS[...] = old + DATA[lane]; indivisible with respect to other lanes/waves in the workgroup.


Add two unsigned 64-bit integer values stored in the data register and a location in a data share.


Summary: Calculate bitwise AND given two unsigned 32-bit integer values stored in the data register and a location in a data share.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].b32;
MEM[addr].b32 = (tmp & DATA.b32);
RETURN_DATA.b32 = tmp


Summary: Calculate bitwise AND given two unsigned 64-bit integer values stored in the data register and a location in a data share.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].b64;
MEM[addr].b64 = (tmp & DATA.b64);
RETURN_DATA.b64 = tmp


Calculate bitwise AND given two unsigned 32-bit integer values stored in the data register and a location in a data share. Store the original value from data share into a vector register.


Calculate bitwise AND given two unsigned 64-bit integer values stored in the data register and a location in a data share. Store the original value from data share into a vector register.


Summary: AMDGPU DS vector instruction operating on b32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on b64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Add (count_bits(exec_mask)) to the value stored in DS memory at (M0.base + instr_offset) if GDS, or at instr_offset if LDS. Return the pre-operation value to VGPRs.


Summary: Backward permute.

Backward permute. This does not access LDS memory and may be called even if no LDS memory is allocated to the wave. It uses LDS hardware to implement an arbitrary swizzle across threads in a wavefront. Note the address passed in is the thread ID multiplied by 4. Note that EXEC mask is applied to both VGPR read and write. If src_lane selects a disabled thread then zero is returned. See also DS_PERMUTE_B32.

// VGPR[laneId][index] is the VGPR RAM
// VDST, ADDR and DATA0 are from the microcode DS encoding
declare tmp : 32'B[64];
declare OFFSET : 16'U;
declare DATA0 : 32'U;
declare VDST : 32'U;
for i in 0 : 63 do
tmp[i] = 0x0
endfor;
for i in 0 : 63 do
// ADDR needs to be divided by 4.
// High-order bits are ignored.
src_lane = (VGPR[i][ADDR].u32 + OFFSET.u32) / 4U % 64U;
// EXEC is applied to the source VGPR reads.
if EXEC[src_lane].u1 then
tmp[i] = VGPR[src_lane][DATA0]
endif
endfor;
// Copy data into destination VGPRs. Some source
// data may be broadcast to multiple lanes.
for i in 0 : 63 do
if EXEC[i].u1 then
VGPR[i][VDST] = tmp[i]
endif
endfor

Examples (simplified 4-thread wavefronts): VGPR[SRC0] = { A, B, C, D }
VGPR[ADDR] = { 0, 0, 12, 4 }
EXEC = 0xF, OFFSET = 0
VGPR[VDST] = { A, A, D, B }
VGPR[SRC0] = { A, B, C, D }
VGPR[ADDR] = { 0, 0, 12, 4 }
EXEC = 0xA, OFFSET = 0
VGPR[VDST] = { -, 0, -, B }


Backward permute and fetch data for invalid lanes. This does not access LDS memory and may be called even if no LDS memory is allocated to the wave. It uses LDS hardware to implement an arbitrary swizzle across threads in a wavefront.


Ray tracing involves traversing a BVH which is a kind of tree where nodes have up to 4 children. Each shader thread processes one child at a time, and overflow nodes are stored temporarily in LDS using a stack. This instruction supports pushing/popping the stack to reduce the number of VALU instructions required per traversal and reduce VMEM bandwidth requirements.


Summary: Compare an unsigned 32-bit integer value in the data comparison register with a location in a data share, and modify the memory location with a value…

Compare an unsigned 32-bit integer value in the data comparison register with a location in a data share, and modify the memory location with a value in the data source register if the comparison is equal.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].b32;
src = DATA2.b32;
cmp = DATA.b32;
MEM[addr].b32 = tmp == cmp ? src : tmp;
RETURN_DATA.b32 = tmp

Caution, the order of src and cmp are the opposite of the BUFFER_ATOMIC_CMPSWAP opcode.


Summary: Compare an unsigned 64-bit integer value in the data comparison register with a location in a data share, and modify the memory location with a value…

Compare an unsigned 64-bit integer value in the data comparison register with a location in a data share, and modify the memory location with a value in the data source register if the comparison is equal.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].b64;
src = DATA2.b64;
cmp = DATA.b64;
MEM[addr].b64 = tmp == cmp ? src : tmp;
RETURN_DATA.b64 = tmp


Summary: Compare a single-precision float value in the data comparison register with a location in a data share, and modify the memory location with a value…

Compare a single-precision float value in the data comparison register with a location in a data share, and modify the memory location with a value in the data source register if the comparison is equal.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].f32;
src = DATA2.f32;
cmp = DATA.f32;
MEM[addr].f32 = tmp == cmp ? src : tmp;
RETURN_DATA.f32 = tmp


Summary: Compare a double-precision float value in the data comparison register with a location in a data share, and modify the memory location with a value…

Compare a double-precision float value in the data comparison register with a location in a data share, and modify the memory location with a value in the data source register if the comparison is equal.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].f64;
src = DATA2.f64;
cmp = DATA.f64;
MEM[addr].f64 = tmp == cmp ? src : tmp;
RETURN_DATA.f64 = tmp


Summary: Subtract an unsigned 32-bit integer value in the data register from a location in a data share only if the memory value is greater than or equal to…

Subtract an unsigned 32-bit integer value in the data register from a location in a data share only if the memory value is greater than or equal to the data register value. Store the original value from data share into a vector register.


Subtract an unsigned 32-bit integer value in the data register from a location in a data share only if the memory value is greater than or equal to the data register value.


Summary: Perform 2 conditional write exchanges, where each conditional write exchange writes a 32 bit value from a data register to a location in data share…

Perform 2 conditional write exchanges, where each conditional write exchange writes a 32 bit value from a data register to a location in data share iff the most significant bit of the data value is set.

declare OFFSET0 : 8'U;
declare OFFSET1 : 8'U;
declare RETURN_DATA : 32'U[2];
ADDR = S0.u32;
DATA = S1.u64;
offset = { OFFSET1, OFFSET0 };
ADDR0 = ((ADDR + offset.u32) & 0xfff8U);
ADDR1 = ADDR0 + 4U;
RETURN_DATA[0] = LDS[ADDR0].u32;
if DATA[31] then
LDS[ADDR0] = { 1'0, DATA[30 : 0] }
endif;
RETURN_DATA[1] = LDS[ADDR1].u32;
if DATA[63] then
LDS[ADDR1] = { 1'0, DATA[62 : 32] }
endif


Subtract (count_bits(exec_mask)) from the value stored in DS memory at (M0.base + instr_offset) if GDS, or at instr_offset if LDS. Return the pre-operation value to VGPRs.


Summary: Decrement an unsigned 32-bit integer value from a location in a data share with wraparound to a value in the data register if the decrement yields a…

Decrement an unsigned 32-bit integer value from a location in a data share with wraparound to a value in the data register if the decrement yields a negative value. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u32;
src = DATA.u32;
MEM[addr].u32 = ((tmp == 0U) || (tmp > src)) ? src : tmp - 1U;
RETURN_DATA.u32 = tmp


Summary: Decrement an unsigned 64-bit integer value from a location in a data share with wraparound to a value in the data register if the decrement yields a…

Decrement an unsigned 64-bit integer value from a location in a data share with wraparound to a value in the data register if the decrement yields a negative value. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u64;
src = DATA.u64;
MEM[addr].u64 = ((tmp == 0ULL) || (tmp > src)) ? src : tmp - 1ULL;
RETURN_DATA.u64 = tmp


Summary: AMDGPU DS vector instruction operating on u32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on u64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Decrement an unsigned 32-bit integer value from a location in a data share with wraparound to a value in the data register if the decrement yields a negative value.


Decrement an unsigned 64-bit integer value from a location in a data share with wraparound to a value in the data register if the decrement yields a negative value.


Read a single 32-bit value from LDS to all lanes. A single DWORD is read from LDS memory at ADDR[M0[15:0]], where M0[15:0] is a byte address and is dword-aligned. M0[18:16] specify the data type for the read and may be 0=UBYTE, 1=USHORT, 2=DWORD, 4=SBYTE, 5=SSHORT.


GDS Only: The GWS resource indicated processes this opcode by queueing it until barrier is satisfied. The number of waves needed is passed in as DATA of first valid thread.

//Determine the GWS resource to work on
rid[5:0] = gds_base[5:0] + OFFSET0[5:0];
index =  find first valid (vector mask);
value = DATA[thread: index];
// Input Decision Machine
state.type[rid] = BARRIER;
if(state[rid].counter <= 0) then
thread[rid].flag = state[rid].flag;
ENQUEUE;
state[rid].flag = !state.flag;
state[rid].counter = value;
return rd_done;
else
state[rid].counter -= 1;
thread.flag = state[rid].flag;
ENQUEUE;
endif.
Since the waves deliver the count for the next barrier, this function can have a different size barrier for each
occurrence.
// Release Machine
if(state.type == BARRIER) then
if(state.flag != thread.flag) then
return rd_done;
endif;
endif.
CAUTION: The VGPR operand MUST be even-aligned for this instruction. Only 32 bits are used but hardware
treats this instruction as a 64 bit read.


Summary: GDS Only: Initialize a barrier or semaphore resource.

// Determine the GWS resource to work on
rid[5:0] = gds_base[5:0] + offset0[5:0];
// Get the value to use in init
index = find_first_valid(vector mask)
value = DATA[thread: index]
// Set the state of the resource
state.counter[rid] = lsb(value); //limit #waves
state.flag[rid] = 0;
return rd_done; //release calling wave
CAUTION: The VGPR operand MUST be even-aligned for this instruction. Only 32 bits are used but hardware
treats this instruction as a 64 bit read.


Summary: GDS Only: The GWS resource indicated processes this opcode by updating the counter by the bulk release delivered count and labeling the resource as a…

GDS Only: The GWS resource indicated processes this opcode by updating the counter by the bulk release delivered count and labeling the resource as a semaphore.

//Determine the GWS resource to work on
rid[5:0] = gds_base[5:0] + offset0[5:0];
index =  find first valid (vector mask)
count = DATA[thread: index];
//Add count to the resource state counter
state.counter[rid] += count;
state.type = SEMAPHORE;
return rd_done; //release calling wave
This action releases count number of waves, promptly if queued, or as they arrive from the noted resource.
CAUTION: The VGPR operand MUST be even-aligned for this instruction. Only 32 bits are used but hardware
treats this instruction as a 64 bit read.


Summary: GDS Only: The GWS resource indicated processes this opcode by queueing it until counter enables a release and then decrementing the counter of the…

GDS Only: The GWS resource indicated processes this opcode by queueing it until counter enables a release and then decrementing the counter of the resource as a semaphore.

//Determine the GWS resource to work on
rid[5:0] = gds_base[5:0] + offset0[5:0];
state.type = SEMAPHORE;
ENQUEUE until(state[rid].counter > 0)
state[rid].counter -= 1;
return rd_done;


Summary: GDS Only: The GWS resource (rid) indicated processes this opcode by updating the counter and labeling the specified resource as a semaphore.

// Determine the GWS resource to work on
rid[5:0] = gds_base[5:0] + offset0[5:0];
// Incr the state counter of the resource
state.counter[rid] = state.wave_in_queue;
state.type = SEMAPHORE;
return rd_done; //release calling wave
This action releases ALL queued waves; it has no effect if no waves are present.


Summary: GDS Only: The GWS resource indicated processes this opcode by updating the counter and labeling the resource as a semaphore.

//Determine the GWS resource to work on
rid[5:0] = gds_base[5:0] + offset0[5:0];
//Incr the state counter of the resource
state.counter[rid] += 1;
state.type = SEMAPHORE;
return rd_done; //release calling wave
This action releases one wave if any are queued in this resource.


Increment an unsigned 32-bit integer value from a location in a data share with wraparound to 0 if the value exceeds a value in the data register. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u32;
src = DATA.u32;
MEM[addr].u32 = tmp >= src ? 0U : tmp + 1U;
RETURN_DATA.u32 = tmp


Increment an unsigned 64-bit integer value from a location in a data share with wraparound to 0 if the value exceeds a value in the data register. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u64;
src = DATA.u64;
MEM[addr].u64 = tmp >= src ? 0ULL : tmp + 1ULL;
RETURN_DATA.u64 = tmp


Increment an unsigned 32-bit integer value from a location in a data share with wraparound to 0 if the value exceeds a value in the data register.


Increment an unsigned 64-bit integer value from a location in a data share with wraparound to 0 if the value exceeds a value in the data register.


Summary: Load 32 bits of data from one location in a data share and then 32 bits of data from a second location in a data share and store the results into a…

Load 32 bits of data from one location in a data share and then 32 bits of data from a second location in a data share and store the results into a 64-bit vector register.


Summary: Load 64 bits of data from one location in a data share and then 64 bits of data from a second location in a data share and store the results into a…

Load 64 bits of data from one location in a data share and then 64 bits of data from a second location in a data share and store the results into a 128-bit vector register.


Load 32 bits of data from one location in a data share and then 32 bits of data from a second location in a data share and store the results into a 64-bit vector register. Treat each offset as an index and multiply by a stride of 64 elements (256 bytes) to generate an offset for each DS address.


Load 64 bits of data from one location in a data share and then 64 bits of data from a second location in a data share and store the results into a 128-bit vector register. Treat each offset as an index and multiply by a stride of 64 elements (256 bytes) to generate an offset for each DS address.


Load 32 bits of data from a data share into a vector register. The memory base address is provided as an immediate value and the lane ID is used as an offset.


Summary: Load 128 bits of data from a data share into a vector register.


Load 32 bits of data from a data share into a vector register.


Summary: Load 64 bits of data from a data share into a vector register.


Summary: Load 96 bits of data from a data share into a vector register.


Summary: Load 16 bits of signed data from a data share, sign extend to 32 bits and store the result into a vector register.


Summary: Load 8 bits of signed data from a data share, sign extend to 32 bits and store the result into a vector register.


Summary: Load 8 bits of signed data from a data share, sign extend to 16 bits and store the result into the low 16 bits of a vector register.


Summary: Load 8 bits of signed data from a data share, sign extend to 16 bits and store the result into the high 16 bits of a vector register.


Summary: AMDGPU DS vector instruction operating on b128 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on b64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on b96 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: Load 16 bits of unsigned data from a data share, zero extend to 32 bits and store the result into a vector register.


Summary: Load 16 bits of unsigned data from a data share and store the result into the low 16 bits of a vector register.


Summary: Load 16 bits of unsigned data from a data share and store the result into the high 16 bits of a vector register.


Summary: Load 8 bits of unsigned data from a data share, zero extend to 32 bits and store the result into a vector register.


Summary: Load 8 bits of unsigned data from a data share, zero extend to 16 bits and store the result into the low 16 bits of a vector register.


Summary: Load 8 bits of unsigned data from a data share, zero extend to 16 bits and store the result into the high 16 bits of a vector register.


Select the maximum of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].f32;
src = DATA.f32;
MEM[addr].f32 = src > tmp ? src : tmp;
RETURN_DATA.f32 = tmp

Floating-point compare handles NAN/INF/denorm.


Select the maximum of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].f64;
src = DATA.f64;
MEM[addr].f64 = src > tmp ? src : tmp;
RETURN_DATA.f64 = tmp


Select the maximum of two signed 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].i32;
src = DATA.i32;
MEM[addr].i32 = src >= tmp ? src : tmp;
RETURN_DATA.i32 = tmp


Select the maximum of two signed 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].i64;
src = DATA.i64;
MEM[addr].i64 = src >= tmp ? src : tmp;
RETURN_DATA.i64 = tmp


Select the IEEE maximumNumber() of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.


Select the IEEE maximumNumber() of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.


Select the IEEE maximumNumber() of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the IEEE maximumNumber() of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the maximum of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the maximum of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the maximum of two signed 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the maximum of two signed 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the maximum of two unsigned 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u32;
src = DATA.u32;
MEM[addr].u32 = src >= tmp ? src : tmp;
RETURN_DATA.u32 = tmp


Select the maximum of two unsigned 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u64;
src = DATA.u64;
MEM[addr].u64 = src >= tmp ? src : tmp;
RETURN_DATA.u64 = tmp


Summary: AMDGPU DS vector instruction operating on f32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on f64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on i32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on i64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on u32 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Summary: AMDGPU DS vector instruction operating on u64 data. (Format and name extracted from LLVM's AMDGPU backend source - semantics not yet curated.)


Select the maximum of two unsigned 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.


Select the maximum of two unsigned 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.


Select the minimum of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].f32;
src = DATA.f32;
MEM[addr].f32 = src < tmp ? src : tmp;
RETURN_DATA.f32 = tmp


Select the minimum of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].f64;
src = DATA.f64;
MEM[addr].f64 = src < tmp ? src : tmp;
RETURN_DATA.f64 = tmp


Select the minimum of two signed 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].i32;
src = DATA.i32;
MEM[addr].i32 = src < tmp ? src : tmp;
RETURN_DATA.i32 = tmp


Select the minimum of two signed 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].i64;
src = DATA.i64;
MEM[addr].i64 = src < tmp ? src : tmp;
RETURN_DATA.i64 = tmp


Select the IEEE minimumNumber() of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.


Select the IEEE minimumNumber() of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.


Select the IEEE minimumNumber() of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the IEEE minimumNumber() of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the minimum of two single-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the minimum of two double-precision float inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the minimum of two signed 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the minimum of two signed 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.


Select the minimum of two unsigned 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u32;
src = DATA.u32;
MEM[addr].u32 = src < tmp ? src : tmp;
RETURN_DATA.u32 = tmp


Select the minimum of two unsigned 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value. Store the original value from data share into a vector register.

addr = CalcDsAddr(ADDR.b32, OFFSET0.b32, OFFSET1.b32);
tmp = MEM[addr].u64;
src = DATA.u64;
MEM[addr].u64 = src < tmp ? src : tmp;
RETURN_DATA.u64 = tmp


Select the minimum of two unsigned 32-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.


Select the minimum of two unsigned 64-bit integer inputs, given two values stored in the data register and a location in a data share. Update the data share with the selected value.