Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Controlled Comparison of Controlled Generators
Large language models trained on huge collections of text have achieved state-of-the-art performance across many NLP tasks. However, there is no systematic way to control the text that is generated by such models. The control aspect is specifically important when it comes to developing such models safely for real-world applications, e.g., to make sure that the generations are non-toxic and factual. For this reason, there has been a wide variety of methods proposed in the literature recently for controlled generation. The goal of this project is to do a systematic analysis of such methods across different tasks and metrics.
Keywords: Machine Learning, NLP, Large Language Models, Generation, Transformers, Decoding
**Your Role**
- Implementing controlled generators
- Defining a diverse set of tasks
- Doing a systematic and controlled evaluation of generators
**Requirements**
- Familiar with the fundamental concepts in Natural Language Processing
- Familiar with basic concepts in Machine Learning, Deep Learning, and different architectures, such as transformers
- Experience with Pytorch
**References**
Here are some pointers to papers that are relevant to this project:
- Kevin Yang and Dan Klein. FUDGE: Controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3511–3535, Online, June 2021. Association for Computational Linguistics.
- Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., and Liu, R.
(2020). Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations
- Kumar, S., Paria, B., and Tsvetkov, Y. (2022). Constrained sampling from language models via Langevin dynamics in embedding spaces.
**Your Role**
- Implementing controlled generators - Defining a diverse set of tasks - Doing a systematic and controlled evaluation of generators
**Requirements**
- Familiar with the fundamental concepts in Natural Language Processing - Familiar with basic concepts in Machine Learning, Deep Learning, and different architectures, such as transformers - Experience with Pytorch
**References**
Here are some pointers to papers that are relevant to this project:
- Kevin Yang and Dan Klein. FUDGE: Controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3511–3535, Online, June 2021. Association for Computational Linguistics. - Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., and Liu, R. (2020). Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations - Kumar, S., Paria, B., and Tsvetkov, Y. (2022). Constrained sampling from language models via Langevin dynamics in embedding spaces.
In this project, we aim to do a systematic analysis of controlled generators. We will explore the effectiveness of these generators across a diverse set of tasks and evaluation metrics.
In this project, we aim to do a systematic analysis of controlled generators. We will explore the effectiveness of these generators across a diverse set of tasks and evaluation metrics.
