Microdosing: Knowledge Distillation for GAN based Compression
Leonhard Helminger, Roberto Azevedo, Abdelaziz Djelouah, Markus Gross, Christopher Schroers
arXiv Preprint Archive January 7, 2022 preprint
Study at a glance
AI-extracted from the abstract| Characteristics | Theoretical or philosophical paper |
|---|---|
| Topics | Microdosing |
| Keywords | Eess.iv Cs.cv |
| Key points | Proposes that knowledge distillation can reduce image decoder model size by a factor of 20 and decoding time by 50% without sacrificing capability, and that these benefits transfer to video compression. |
Abstract
Recently, significant progress has been made in learned image and video compression. In particular the usage of Generative Adversarial Networks has lead to impressive results in the low bit rate regime. However, the model size remains an important issue in current state-of-the-art proposals and existing solutions require significant computation effort on the decoding side. This limits their usage in realistic scenarios and the extension to video compression. In this paper, we demonstrate how to leverage knowledge distillation to obtain equally capable image decoders at a fraction of the original number of parameters. We investigate several aspects of our solution including sequence specialization with side information for image coding. Finally, we also show how to transfer the obtained benefits into the setting of video compression. Overall, this allows us to reduce the model size by a factor of 20 and to achieve 50% reduction in decoding time.
Comparable studies
Other theoretical and historical papers on microdosing, most cited first.
| Study | Year | Design | Participants |
|---|---|---|---|
| Positron emission tomography microdosing: a new concept with application in tracer and early clinical drug development | 2003 | Theoretical or philosophical paper | |
| Set and setting in microdosing: an oft-overlooked principle. | 2022 | Theoretical or philosophical paper | |
| Psilocybin Microdosing Schedules, 5-HT2A Adaptation, and the Age-Dependent Cortical State: From Intermittent to Continuous Exposure | 2026 | Theoretical or philosophical paper | |
| Daily Psilocybin Microdosing as a Receptor-Adaptation Paradigm: A Testable 5-HT2A Down-Regulation Hypothesis | 2026 | Theoretical or philosophical paper | |
| It’s all about the relationship: The caregiver experience of supporting a person with advanced cancer going through an LSD microdosing trial Caregivers of advanced cancer patients in an LSD microdosing trial | 2026 | Theoretical or philosophical paper |