Learning the Beauty in Songs: Neural Singing Voice Beautifier
Jinglin Liu, Chengxi Li, Yi Ren, Zhiying Zhu, Zhou Zhao
Zhejiang University
ACL 2022 Main conference
This repository is the official PyTorch implementation of our ACL-2022 paper. Now, we release the codes for SADTW algorithm in our paper. The current expected release time of the full version codes and data is at the ACL-2022 conference (before June. 2022). Please star us and stay tuned!
|--modules
|--voice_conversion
|--dtw
|--enhance_sadtw.py (Our algorithm)
|--tasks
|--singing
|--pitch_alignment_task.py (Usage example)
- Feb.24, 2022: Our new work, NeuralSVB was accepted by ACL-2022. Demo Page.
- Dec.01, 2021: Our recent work
DiffSingerwas accepted by AAAI-2022.|
.
- Sep.29, 2021: Our recent work
PortaSpeechwas accepted by NeurIPS-2021. - May.06, 2021: We submitted DiffSinger to Arxiv
.
Abstract
We are interested in a novel task, singing voice beautifying (SVB). Given the singing voice of an amateur singer, SVB aims to improve the intonation and vocal tone of the voice, while keeping the content and vocal timbre. Current automatic pitch correction techniques are immature, and most of them are restricted to intonation but ignore the overall aesthetic quality. Hence, we introduce Neural Singing Voice Beautifier (NSVB), the first generative model to solve the SVB task, which adopts a conditional variational autoencoder as the backbone and learns the latent representations of vocal tone. In NSVB, we propose a novel time-warping approach for pitch correction: Shape-Aware Dynamic Time Warping (SADTW), which ameliorates the robustness of existing time-warping approaches, to synchronize the amateur recording with the template pitch curve. Furthermore, we propose a latent-mapping algorithm in the latent space to convert the amateur vocal tone to the professional one. Extensive experiments on both Chinese and English songs demonstrate the effectiveness of our methods in terms of both objective and subjective metrics.
Issues
- Before raising a issue, please check our Readme and other issues for possible solutions.
- We will try to handle your problem in time but we could not guarantee a satisfying solution.
- Please be friendly.
21 May 18, 2022
![[CVPR 2022] Official code for the paper:](https://github.com/mdca-loss/MDCA-Calibration/raw/main/content/teaser.png)
21 Dec 22, 2022
64 Dec 17, 2022
107 Dec 2, 2022
45 Jan 9, 2023
17 Jun 19, 2022
39 Jan 1, 2023
4 Dec 22, 2022
25 Oct 30, 2022
1.3k Dec 29, 2022
6 Nov 14, 2022
850 Dec 24, 2022
74 Dec 12, 2022
2 Oct 05, 2022
6 Feb 08, 2022
2 Mar 26, 2022
11 Nov 27, 2022
42 Oct 29, 2022
33 Jan 04, 2023
149 Jan 07, 2023
10 Jan 05, 2023
54 Dec 08, 2022
6 Nov 29, 2022
54 Dec 12, 2022
360 Dec 28, 2022
4 Feb 22, 2022
1.5k Dec 17, 2022
150 Dec 12, 2022
539 Dec 28, 2022