Projects
Efficient inference · with Huawei Research, 2026
CRE-Router, cost-aware multi-LLM serving
CRE-Router is a two-stage framework that first routes clusters of queries to the most
cost-effective model in a pool, and then escalates queries with low-quality outputs to
a stronger model, saving costs while retaining quality.
ClusteringCost-aware routingEscalation
Code
Paper
Speech · IWSLT 2025
Halving an 8B speech translation model
Compressed Qwen2-Audio to half its parameters and storage by combining
iterative layer pruning, QLoRA, and knowledge distillation, while
holding on to almost all of the teacher model's quality.
50% fewer parameters
97 to 100% of teacher quality
Code
Paper
Speech · IWSLT 2026
Cross-lingual voice cloning for scientific speech
Multi-model distillation for cross-lingual voice cloning applied to
scientific speech, alongside synthetic audio to improve low-resource
speech translation.
Voice cloningTTSDistillation
Paper
Findings
Model compression · WMT 2025
Pruning a multilingual LLM without losing quality
Iterative layer pruning applied to Aya-Expanse 8B. Removing between a
fifth and nearly half of the model left translation quality almost intact
and made it substantially faster to serve, with a further speedup once
deployed through vLLM.
22 to 44% smaller
99% of COMET retained
up to 84% higher throughput
4.3× faster serving
Code
Paper
Open models · AfricaNLP 2026, Spotlight
AfriNLLB, faster open translation for African languages
Compressed translation models covering 15 language pairs in both
directions, released openly alongside a curated training set. They run
considerably faster than the NLLB-200 baseline at equal or better
quality.
30 directions
2.4× faster inference
1.6M open segments
Code
Models and dataset
Paper