🔬 The Coolest Diffusion Research Isn't in LLMs — Evan Feinberg & Sergey Edunov, Genesis Molecular AI

تفصیل

This episode has a fun personal twist: There’s a counterfactual world where I was employee #1 at Genesis Molecular AI, the company behind today’s episode. A certain introduction happened a few weeks too late and I had already happily signed at Atomwise, another ML-for-drug-discovery startup. Same problem, different company. I was certain ML was going to transform small molecule drug discovery. Early results were underwhelming. Useful at times, but nowhere near revolutionary. In the last year I’ve seen signs that ML is finally ready to deliver on my convictions from a decade ago. Genesis is one of the places that might have finally cracked this problem. I was super excited to come full circle and catch up with co-founder Evan Feinberg and CTO Sergey Edunov.If you are at all interested in small molecule drug discovery, we think you will find this fascinating!In our nearly two hour chat we cover:* What is small molecule drug discovery, and why is it hard* Structure prediction as a hotbed of innovation in AI algorithms* How advances in AI elsewhere have enabled stepwise improvements in predictive power* How the community benchmarks are essentially calling AI slop good enough* The Genesis flagship model (PEARL) can routinely hit a threshold that is necessary for real-world applications* New agentic workflows enabled by these highly accurate modelsRead on for more, and also some personal thoughts on the future at the end.The coolest diffusion research is happening at GenesisSergey Edunov came to Genesis from Meta where he led Llama 2 training and Llama 3 pretraining. Sergey was a former physicist who thought he was done with physics after many years of training LLMs. Then, he discovered Genesis, and was blown away with all the novel architecture work they’ve been developing.It probably surprises no one that modern LLM research has not resulted in fundamentally novel or exciting updates in architectures since almost the advent of the transformer — the entire field is using variants on the same idea that came out in the