Score Matching and Langevin Dynamics for Machine Learning

Diffusion Models and Score Matching

1. The analysis in [3] reveals some interesting connections to score-based models. Indeed it is revealed that Diffusion Models and Score Matching are two sides of the same coin, much like the Heisenberg matrix and Schrodinger wave formulations of quantum mechanics. For details, see [3], but we will proceed with score matching because it is easier.
2. New connection between diffusion models and denoising score matching. Leads to simplified, weighted variational bound objective for diffusion models [3].

Score Based Models

1. [4] Rather than model pdf, represent the score - vector field pointing in direction where likelihood of data increases most rapdily.
2. [4] First, function learned via denoising score matching. Intuitively means NN (score network) trained to denoise images blurred with Gaussian noise. Key insight from [2] was to use multiple noise scales to capture both coarse and fine features.
3. [4] Then samples are generated with Langevin dynamics. Starts with white noise and denoises using score network. Could fail or take long time to converge when used in high D.
4. [4] Same authors as [2] propose set of techniques to scale score based models to high res images. Use simplified mixture model to propose method to analytically compute effective set of noise scales. Also propose efficient architecture to amortize the score estimation across large number (possibly inf) of noise scales with single NN.

Score Function

1. [4] or [2] for continuously sifferentiable pdf p(x), grad_x log p(x) is its score function.
2. Once known

inline: \( q(\textbf{x}{1:T}|\textbf{x}0) = \prod{i=1}^{T} q(\textbf{x}t|\textbf{x}{t-1}) = \prod{i=1}^{T} \mathcal{N}(\textbf{x}t; \sqrt{1-\beta_t}\textbf{x}{t-1}, \beta_t\textbf{I}) \)

\( \textbf{x}\beta_t \)

centered: \[\mathbb{R}\] sdas

asd

The Goal

1. The goal is quite simple - given dataset vectors x_1 => x_n iid from p(x), want to work "backwards" in order to determine the dsitrubtion p(x) from whcih they were generated. Once had, we can synthezie new datapoints by sampling [2]
2. One approach are likelihood based models. Seek to directly learn the pdf with approx MaxL [2]. But there is tradeoff between tractability and flexibility [1]. Can use variational methods to fit a predefined model family. Easy to sample from and fit, but assuming the form of the distribution restricts performance and is likely unable to capture nuances of complex datasets.
3. Can use a more flexible model - a distribution can be any nonnegative integral function. Problem is that function has to integrate to 1 to be a valid pdf, and computing this normalization constant is infeasible in most cases [1]. Also to train or draw samples, use expensive MC methods [1]
4. We could scrap MaxL methods entirely and use implicit generative models like GANs [2], but the problems of training GANs are well-documented

The Solution

1. The solution comes in the form of diffusion models
2. THey borrow idea from non-equil stat phys - use markov chain to gradually convert one distribution into another. Map from well known dist - Normal, and tranform into data.
3. In practice this comes as destroying image with noise and then learning to recover it, but really a mapping between distribution.
4. This allows for a highly dlexible model. But what about tractability? [2] instead of learning pdf directly, we learn the score function, or the gradient of the log PDF.
5. So important points - model with diffusion process and model the score function.

Why Score Function?

1. Let's say we have the form of a function exponential [2]. Train by maxxing log likelihood. To do this, require a normalized probability density function. I.e. we must compute Z, which is typically intractable. To circumvent this, practitioners will typically restrict the form of f to make tractable, or approximate Z which is expensive.
2. Instead of directly model the pdf, we model the score function. Such models are called score based models.
3. * SHOW HOW DOES NOT DEPEND ON Z *
4. So we use score function in order to avoid intractable normalization, therefore opening the door to more stuff. Don't need to approximate Z or restrict model structure to ensure normalized.

But how do we train?

1. That's all well and good, but how do we learn this function?
2. Minimize fischer divergence

What is Fischer Divergence

1. It minimizes the average distance between two distributions.
2. It makes sense to do this because we are training the model to approximate the true score function

How do we do this

1. Not possible. Would need access to score function which we are trying to approximate. We use score matching.
2. The score matching objective can be estimated on dataset and then minimized with gradient-based methods
3. Only requirement on score model is that it should be vector valued with same input and output dimensionality

What is Score Matching?

Stuff

How do We Use the Score Model?

1. Now that the we have learned score based model, how do we use it to sample?
2. MCMC method to sample from. It basically just looks like gradient ascent? Take point, add step size time gradient of log likelihood (gradient ascent) and then add some noise for stochasticity

Practical Guide to Score Matching:

1. Estimated score functions innacurate in low density regions where few data points available for computing score matching objective. Fischer divergence weighted by density. BUT initial randomy sample that we work from is very likely in low density regions when data reside in high D space.
2. Therefore use multiple noise peerturbations. ANNEALED langevin dynamics. Similar princple to simulated annealing
3. Choose as geometric progression. Use a U-net. Apply exponential moving average on the weights when used at test time.

Where do diffusion models come into play

1. We've been exactly describing diffusion model. Note that they don't reference a specific architecture, just a meta-approach. Like GANs. Contra CNN, diffusion models more akin to GANs in that they describe a general paradigm.

SO WE LEARN THE SCORE MODEL BY MINIMIZING FISCHER DIVERGENCE, AND THEN SAMPLE FROM IT USING LANGEVIN DYNAMICS?