I received Ph.D degree from Inner Mongolia University, China in 2020 and Bachelor degree in Taiyuan University of Technology, ShanXi, China in 2014. Currently, I am now with the Department of Electrical & Computer Engineering, the National University of Singapore for a research attachment under Prof. Haizhou Li.

My current research interests include Expressive Speech Synthesis. Revolving around this goal, I mainly study the following topics: (1) Machine Learning: End-to-End Text-to-Speech (TTS), Speech Emotion Recognition et. al. (2) Natural Language Processing: Prosody Structure Prediction, Grapheme-to-Phoneme Conversion (G2P), Syntax Parsing et. al.

Experiences

  1. Aug 2020-Present, National University of Singapore, Research Fellow, HLT Lab, Singapore.
  2. Aug 2019-Aug 2020, National University of Singapore, Visiting Ph.D. Student, HLT Lab, Singapore.
  3. Sep 2014-Aug 2020, Inner Mongolia University, Ph.D. Student, Hohhot, China. Research topic: Mongolian Text-to-Speech system [DEMO].


Recent News


  • ~ NEW ~ 2021/04 - Our journal paper titled 'Expressive TTS Training with Frame and Style Reconstruction Loss' is accepted for publication in IEEE/ACM Transactions on Audio, Speech, and Language Processing (JCR Q1, IF=3.398).

  • ~ NEW ~ 2021/04 - Our journal paper 'FastTalker: A Neural Text-to-Speech Architecture with Shallow and Group Autoregression' is accepted to be published in Neural Networks (JCR Q1, IF=5.535).

  • ~ NEW ~ 2021/01 - Our conference paper 'GraphSpeech: Syntax-aware Graph Attention Network for Neural Speech Synthesis' has been accepted by ICASSP2021 (CCF-B).

  • ~ NEW ~ 2021/01 - Our conference paper 'Seen and Unseen Emotional Style Transfer for Voice Conversion with a New Emotion Speech Dataset' has been accepted by ICASSP2021 (CCF-B).

  • ~ NEW ~ 2020/12 - Our journal paper titled 'Exploiting Morphological and Phonological Features to Improve Prosodic Phrasing for Mongolian Speech Synthesis' is accepted for publication in IEEE/ACM Transactions on Audio, Speech, and Language Processing (JCR Q1, IF=3.398).


Preprints

  1. Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability [PDF] [DEMO]
    Rui Liu, Berrak Sisman, Haizhou Li.
    Submitted to InterSpeech 2021.

Journal papers

  1. [TASLP] Expressive TTS Training with Frame and Style Reconstruction Loss [PDF] [DEMO]
    Rui Liu, Berrak Sisman, Guanglai Gao, Haizhou Li.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021
  2. [Neural Networks] FastTalker: A Neural Text-to-Speech Architecture with Shallow and Group Autoregression [PDF] [DEMO]
    Rui Liu, Berrak Sisman, Yixing Lin, Haizhou Li.
    Neural Networks. 2021.
  3. [TASLP] Exploiting Morphological and Phonological Features to Improve Prosodic Phrasing for Mongolian Speech Synthesis [PDF] [BIB]
    Rui Liu, Berrak Sisman, Feilong Bao, Jichen Yang, Guanglai Gao, Haizhou Li.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021.
  4. [SPL] Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS [PDF] [BIB] [DEMO]
    Rui Liu, Berrak Sisman, Feilong Bao, Guanglai Gao, Haizhou Li.
    IEEE Signal Processing Letters. 2020.


f

Conference papers

  1. [ICASSP'2021] GraphSpeech: Syntax-aware Graph Attention Network for Neural Speech Synthesis [PDF] [DEMO]
    Rui Liu, Berrak Sisman, Haizhou Li.
    To appear at ICASSP'2021 (Accepted).
  2. [ICASSP'2021] Seen and Unseen Emotional Style Transfer for Voice Conversion with a New Emotion Speech Dataset [PDF]
    Kun Zhou, Berrak Sisman, Rui Liu, Haizhou Li.
    To appear at ICASSP'2021 (Accepted).
  3. [ICASSP'2020] Teacher-Student Training For Robust Tacotron-based TTS [PDF] [BIB] [DEMO] [VIDEO]
    Rui Liu, Berrak Sisman, Jingdong Li, Feilong Bao, Guanglai Gao, Haizhou Li.
    2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2020), Oral.
  4. [Odyssey'2020] WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss [PDF] [BIB] [DEMO] [VIDEO]
    Rui Liu, Berrak Sisman, Feilong Bao, Guanglai Gao, Haizhou Li.
    The Speaker and Language Recognition Workshop 2020.
  5. [BC'2020] NUS-HLT System for Blizzard Challenge 2020 [PDF] [BIB]
    Yi Zhou, Xiaohai Tian, Xuehao Zhou, Mingyang Zhang, Grandee Lee, Rui Liu, Berrak Sisman, and Haizhou Li
    Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge 2020 (BC 2020).
  6. [BC'2019] The IMU Speech Synthesis Entry for Blizzard Challenge 2019 [PDF] [BIB]
    Rui Liu, Jingdong Li, Feilong Bao and Guanglai Gao.
    Blizzard Challenge Workshop 2019 (BC 2019).
  7. [INTERSPEECH'2018] Improving Mongolian Phrase Break Prediction by Using Syllable and Morphological Embeddings with BiLSTM Model [PDF] [BIB]
    Rui Liu, Feilong Bao, Guanglai Gao, Hui Zhang and Yonghe Wang.
    19th Annual Conference of the International Speech Communication Association (INTERSPEECH 2018), Oral
  8. [COLING'2018] A LSTM Approach with Sub-word Embeddings for Mongolian Phrase Break Prediction [PDF] [BIB]
    Rui Liu, Feilong Bao, Guanglai Gao, Hui Zhang and Yonghe Wang.
    27th International Conference on Computational Linguistics (COLING 2018).
  9. [ISCSLP'2018] End-to-End Mongolian Text-to-Speech System [PDF] [BIB]
    Jingdong Li, Hui Zhang, Rui Liu, Xueliang Zhang and Feilong Bao.
    11th International Symposium on Chinese Spoken Language Processing (ISCSLP 2018).
  10. [NCMMSC'2017] Mongolian Text-to-Speech System Based on Deep Neural Network [PDF] [BIB]
    Rui Liu, Feilong Bao, Guanglai Gao and Yonghe Wang.
    14th National Conference on Man-Machine Speech Communication (NCMMSC 2017), Oral.
  1. SUTD AI Grant, ’The Understanding and Synthesis of Expressive Speech by AI’
  1. Local Arrangement Co-chair, O-COCOSDA 2021, Singapore.
  2. Local Arrangement Co-chair, IWSDS 2021, Singapore
  3. Local Arrangement Co-chair, SIGDIAL 2021, Singapore
  4. Reviewer, InterSpeech 2021
  5. Reviewer, ICASSP 2021
  6. Reviewer, Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge 2020
  7. Student Volunteer, ASRU 2019 (IEEE Automatic Speech Recognition and Understanding Workshop), Singapore
  8. Student Volunteer, NLPCC 2018 (7th CCF International Conference on Natural Language Processing and Chinese Computing), Hohhot, China.
  1. July 2021, 2020 ACM China Doctoral Dissertation Award (Hohhot Chapter), ACM (Association for Computing Machinery) China Council
  2. Feb 2020, ICASSP IEEE SPS Travel Grant
  3. Aug 2019, Research Scholarship of China Scholarship Council (CSC)
  4. Oct 2018, National scholarship for Doctoral students (top 2% students), Ministry of Education of P.R.China
  5. Oct 2018, Academic scholarship of Inner Mongolia autonomous region
  6. Oct 2017, National scholarship for Doctoral students (top 2% students), Ministry of Education of P.R.China
  7. Oct 2017, Academic scholarship of Inner Mongolia autonomous region
  8. Oct 2016, Academic scholarship of Inner Mongolia autonomous region
  9. Oct 2011, National Encouragement scholarship