Recent searches
Search options
Many types of generative AI use ideas related to maximum likelihood, expectation maximization (EM), variational inference, etc. So I did a deep dive, and summarized what I learned in the post below. 
Free Energy · A step-by-step explanation of how EM (expectation-maximization) is used for training a generative AI modelThese notes are based on CMU Prof. Bhiksha Raj’s lecture video on generative models and variational autoencoders and his lecture slides, specifically the section about expectation maximization and variational inference. Prof. Raj gives a great intuitive explanation for the principles behind EM (expectation maximization), so I highly recommend watching the lecture video. But the overall topic is complex and there are many moving pieces, so here I will try to condense down to the key principles behind how this works. I hope these notes will be useful for both myself and others in the future.
Thinking through generative models and conditional probabilities led to some new points of confusion for me. Here is a quick post on what I learned.
Free Energy · Confusion about conditional probabilitiesThe conditional probability of event A given event B is a very important concept that seems straightforward. “If I know that B has occured, what is the probability that A has occured?”