Posts

Showing posts with the label continuous batching

Benchmarking Dynamic Quantization for Larger Language Models