Measured greedy-generation latency on one CPU core, with analytic serial steps, attention FLOPs (key-value cached), and cache bytes at the measured mean trace length. Inputs of length 64.
series
setting
measured latency (normalized)attention FLOPs (normalized)1/c and 1/c² reference