Speculative decoding accelerates large language model generation by allowing multiple tokens to be drafted swiftly by a lightweight model before being verified by a larger, more powerful one. This ...

In a new paper titled Principled Coarse-Grained Acceptance for Speculative Decoding in Speech, Apple researchers detail an interesting approach to generating speech from text. While there are ...