In the last two years, a new direction has all but monopolised research in artificial intelligence (AI). This direction involves training large neural networks - dubbed foundation models - on large amount of raw data, and without requiring a direct human supervision signal. Those models are trained once, on a static dataset through a process that requires access to a large cluster of computer processors specialised in matrix computation. From a certain perspective, reaching this state can be seen as the natural trend of the last 70 years of research in AI, which emphasises data-driven models as opposed to explicit inductive bias provided by the human designer. This new generation of models, which includes text generation model GPT3, image+text model CLIP and various text translation systems have impressed the community by winning benchmarks, as well as the general public by their capacity of generating content through what appears often as creativity or intuition.
In this talk, Matthias Gallé will first provide a general-audience overview of those models and some insights on how they are trained. Gallé will then move onwards the new challenges that this poses for policies, a territory which is mostly uncharted. This will involve copyright law, privacy policies and ethical conflicts.
Matthias Gallé leads various artificial intelligence (AI) research teams at Naver, a leading Asian tech company. He holds a PhD ('11) from INRIA in theoretical computer science, and has since worked in industrial research labs on topics in information retrieval, natural language processing and machine translation. He publishes regularly in top AI conferences, participates in outreach activities and co-organises community events, like BigScience - a large open-source initiative to train a multilingual language model. Matthias currently works at Naver Labs Europe, located in Grenoble (France) and has participated in several EC-funded projects.