The OrMachine
Tammo Rukat
Created: 2017-08-06 Sun 09:51
Single Cell Gene Expression
Boolean Matrix Factorisation
\[ \]
Observed Data
Factorisation
Example
Probabilistic Generative Model
\[ p(\underbrace{x_{nd}}_{\substack{\text{obser-} \\ \text{vation}}}|\overbrace{\mathbf{u}_d}^{\text{codes}},\underbrace{\mathbf{z}_n}_{\substack{\text{latent}\\ \text{rprsnt.}}},\overbrace{\lambda}^{\substack{\text{disper-}\\ \text{sion}}})= \begin{cases} \big(1+\exp[-\lambda]\big)^{-1};\;&\text{if}\;\color{darkgreen}{x_{nd}=\min(1,\mathbf{z}_n^T\mathbf{u}_d)}\;\; \\ \big(1+\exp[\lambda]\big)^{-1};\;&\text{else} \end{cases} \]
\[\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; = \sigma_{\substack{\text{logistic} \\ \text{sigmoid}}}\left[\lambda \underbrace{\tilde{x}_{nd}}_{\tilde{x} = 2x-1} \left(1-2\color{brown}{\prod\limits_{l}(1-z_{nl}u_{ld})}\right) \right]\]
Inference for the OrMachine
Gibbs Sampling
Full conditional
\[ p(z_{nl}|\text{rest}) = \sigma\bigg[\lambda \tilde{z}_{nl} \sum\limits_d \tilde{x}_{nd}\; \color{darkgreen}{u_{ld}} \color{brown}{\prod\limits_{l'\neq l} (1-z_{nl'}u_{l'd})}\bigg] \]
Computational shortcuts
- \(\color{darkgreen}{u_{ld} = 0}\)
→ \(z_{nl}\) and \(x_{nd}\) disconnected. - \(\color{brown}{z_{nl'}u_{l'd} = 1}\) for \(\color{brown}{l' \neq l}\)
→ \(x_{nd}\) is explained away.
Dispersion parameter set to maximise likelihood
\[ \lambda_{\text{MLE}} = \text{logit}\left[ \text{reconstruction accuracy} \right] \]
Modified Sampler – Always propose to change
Standard Gibbs sampler
Metropolised Gibbs sampler
Examples and Experiments
Synthetic Data Benchmarks
Random Matrix Factorisation
[Message Passing: Ravanbakshs et al., ICML 2016]
Problem Setting