The call
Six lines become one
The arguments, drawn
Which kernel runs
The dispatcher's decision
What the math fallback costs
Masks
Boolean masks: True means attend
is_causal when L ≠ S
Shapes and control
Getting into [B, h, T, d] and back
Forcing and checking a backend
Exceptions and sharp edges
attn_mask and is_causal together
fold causal into your explicit mask, or use only is_causal.dropout_p at eval time
dropout_p = self.p if self.training else 0.0.Rows with nothing to see
make sure every query can see at least one key.fp32 skips Flash
use autocast or cast the model to bf16.The default scale
check the reference implementation's scale.