Dec 4, 2008

Distributed Inference of LDA using Expectation-Propagation and GBR?

After learning expectation-propagation (EP) from PRML, I think it is possible to program EP algorithms designed for inference in factorized graphical models using GBR. However, the following issues are to be confirmed:
  • There has no proof that EP converges to a (local or global) optima. However, the EP algorithm developed for LDA [1] seems converges.
  • The EP for LDA [1] is in fact an extended EP algorithm, but not literally an EP algorithm. Need to make sure that this extended EP can be programmed using GBR.
  • Need to estimate the number and size of messages need to passed between workers. This is an approximate estimate of the communication cost of the GBR program.
References:
  1. Thomas Minka and John Lafferty, Expectation-propagation for the generative aspect model, UAI 2002

1 comment:

Kun Duan (Colin) said...

the link you give for that paper is wrong.....