SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices
Introduces SpecExec, a new speculative decoding method for accelerating LLM inference on consumer devices through parallel processing and optimized tree structures.