The Taub Faculty of Computer Science Events and Talks

Between training and inference, lies a growing class of AI problems that involve fast optimization of a pre-trained model for a specific inference task. These are not pure “feed-forward” inference problems applied to a pre-trained model, because they involve some non-trivial inference-time optimization beyond what the model was trained for; neither are they training problems, because they focus on a specific input. These compute-heavy inference workflows raise new challenges in machine learning and open opportunities for new types of user experiences and use cases.

In this talk, I describe two main flavors of the new workflows in the context of text-to-image generative models: few-shot fine-tuning and inference-time optimization. I'll cover personalization of vision-language models using textual-inversion techniques, and techniques for model inversion, prompt-to-image alignment and consistent generation. I will also discuss the generation of rare classes, and future directions.

Short Bio: Gal Chechik is a Professor of computer science at Bar-Ilan University and a senior director of AI research at NVIDIA. His current research focuses on learning for reasoning and perception. In 2018, Gal joined NVIDIA to found and head nvidia's research in Israel. Prior to that, Gal was a staff research scientist at Google Brain and Google research developing large-scale algorithms for machine perception, used by millions daily. Gal earned his PhD in 2004 from the Hebrew University, and completed his postdoctoral training at Stanford CS department. Gal authored ~130 refereed publications, ~50 patents, including publications in Nature Biotechnology, Cell and PNAS. His work won awards at ICML and NeurIPS.