PLSA中的EM算法

古城微笑少年丶 2022-05-21 09:36 142阅读 0赞

# [PLSA中的EM算法][PLSA_EM] #

主要记录下几个文章博客内容

[A Note on EM Algorithm for Probabilistic Latent SemanticAnalysis][]（翟成祥的NOTE）

[A Note on EM Algorithm and PLSA][]（一个中文比较好的总结 by Xinyan Lu）

注意这两个是一个思路

[Probabilistic Latent Semantic Analysis][] （原论文）

原论文是另一个思路

[Notes on Probabilistic Latent Semantic Analysis (PLSA)][Notes on Probabilistic Latent Semantic Analysis _PLSA]（这个里面对比了两种不同思路，原论文与翟成祥NOTE，Xinyan Lu中文总结 分布对应这两种思路）

# 先看第一种思路，这个更好理解一点 #

[![image][]][image 1]

[![image][image 2]][image_image 2]

典型的EM算法 hidden/latent variable 是主题Z，p(d)对于我们的计算可忽略,最后面那个博客的总结证明更完整。

[![image][image 3]][image_image 3] 类似前面的混合高斯模型，这里实际Estep要估算的就是对应d,w 情况下Z的概率

[![image][image 4]][image_image 4] Estep 对比前面高斯模型[![image][image 5]][image_image 5] 具体一个观察点情况下对应到隐藏分类的概率

解释下 sum\_z(p(z|d)p(w|z))= p(w|d)     p(d)p(w|d)p(z|d,w) = p(d,w,z)  => p(w|d)p(z|d,w) = p(d,w,z)/p(d) = p(w,z|d)=p(z|d)p(w|z)

=> p(z|w,d) = p(z|d)(pw|z) / p(w|d)

[![image][image 6]][image_image 6]

[![image][image 7]][image_image 7] Mstep

对比NG的课件

[![image][image 8]][image_image 8] [![image][image 9]][image_image 9] x对应这里的w

[![image][image 10]][image_image 10]

## 如果我们考虑背景噪音,翟成祥的NOTE更进一步给出了在这个基础上稍微复杂一点的MODLE和结果 ##

[![image][image 11]][image_image 11] 考虑下翟成祥那篇EM中的简单混合模型

[![image][image 12]][image_image 12] [![image][image 13]][image_image 13] topic Z

[![image][image 14]][image_image 14] 其实和上面公式一样[![image][image 15]][image_image 15]  p(z|d)

[![image][image 16]][image_image 16]

#  #

# 第二种思路 #

[![image][image 17]][image_image 17]

stanford的NLP课件是一个比较好的总结这里记录下。[http://www.stanford.edu/class/cs224n/handouts/fsnlp-em-slides.pdf][http_www.stanford.edu_class_cs224n_handouts_fsnlp-em-slides.pdf]

# MLE #

[![image][image 18]][image_image 18]

[![image][image 19]][image_image 19]

[![image][image 20]][image_image 20] 又一个不同的应用场景，但是可以看出基本都是mixture…

[![image][image 21]][image_image 21]

[![image][image 22]][image_image 22] 引入hidden variable让计算变的容易,因为确定了具体哪个part model产生

[![image][image 23]][image_image 23]

[![image][image 24]][image_image 24]

[![image][image 25]][image_image 25]   这个推导看前面的总结

[![image][image 26]][image_image 26]  关键词[![image][image 27]][image_image 27] [![image][image 28]][image_image 28]

## 下面是这个课件独有的，EM made easy ##

其实作者也是想证明Qfunction怎么获得的，思路和总结2中殊途同归，不过没有用log直接用概率\*，利用artithmetic means >= geometric means

[![image][image 29]][image_image 29]

[![image][image 30]][image_image 30]  如果再取log形式就一样了

[PLSA_EM]: http://www.cnblogs.com/rocketfan/archive/2011/07/03/2096953.html
[A Note on EM Algorithm for Probabilistic Latent SemanticAnalysis]: http://sifaka.cs.uiuc.edu/course/410s11/plsa-note.pdf
[A Note on EM Algorithm and PLSA]: http://luxinxin.is-programmer.com/user_files/luxinxin/File/plsanote.pdf
[Probabilistic Latent Semantic Analysis]: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.1137&rep=rep1&type=pdf
[Notes on Probabilistic Latent Semantic Analysis _PLSA]: http://www.hongliangjie.com/2010/01/04/notes-on-probabilistic-latent-semantic-analysis-plsa/
[image]: /images/20220521/0e86fcd1d5cf4547af5fa8eb1e1c4b26.png
[image 1]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032233581849.png
[image 2]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234026357.png
[image_image 2]: /images/20220521/c50b7a2fe34c482c9d2d8110614ea60b.png
[image 3]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234025310.png
[image_image 3]: /images/20220521/097470a41c7446d7903c642fff268041.png
[image 4]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234035900.png
[image_image 4]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234036739.png
[image 5]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234036805.png
[image_image 5]: /images/20220521/6a4cb8af9eaa4529b0b1a3c7d3bedfff.png
[image 6]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107040835487373.png
[image_image 6]: /images/20220521/2cd12f9fd8604743adf6d633e4ec6d92.png
[image 7]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234043491.png
[image_image 7]: /images/20220521/6bd21b1fddd746f2b56900a34d1aad45.png
[image 8]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107040843235526.png
[image_image 8]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107040843223923.png
[image 9]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107040843239364.png
[image_image 9]: /images/20220521/504d27d58b69425f9a7002ff2ca5c21f.png
[image 10]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234046904.png
[image_image 10]: /images/20220521/e417cce0f2a44251a8fb7ae2bf2fbacb.png
[image 11]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234043033.png
[image_image 11]: /images/20220521/a9a614039fdb4b62bb2a7582baa81c98.png
[image 12]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234058083.png
[image_image 12]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/20110703223405558.png
[image 13]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234055607.png
[image_image 13]: /images/20220521/9dc7943ac112416bb9f1458244aab1cd.png
[image 14]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234069577.png
[image_image 14]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234059544.png
[image 15]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234067102.png
[image_image 15]: /images/20220521/1e84767b83ff46beb36f169e2127d739.png
[image 16]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234072708.png
[image_image 16]: /images/20220521/033ba081a9c64ca48aae3622b0ab6abd.png
[image 17]: http://images.cnblogs.com/cnblogs_com/rocketfan/201107/201107032234074410.png
[image_image 17]: /images/20220521/84d79da3c1da456790022f2407d8d9cd.png
[http_www.stanford.edu_class_cs224n_handouts_fsnlp-em-slides.pdf]: http://www.stanford.edu/class/cs224n/handouts/fsnlp-em-slides.pdf
[image 18]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604081369.png
[image_image 18]: /images/20220521/74cf588cef3140268c9762291e15b54c.png
[image 19]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604082515.png
[image_image 19]: /images/20220521/f7c63930f65b45ba9bb1b6c9aed78bf3.png
[image 20]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604092581.png
[image_image 20]: /images/20220521/60dd470254284a5cb7f76341a165b8af.png
[image 21]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604109059.png
[image_image 21]: /images/20220521/05f8b64a79734fc4b4307182b65594ea.png
[image 22]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604146108.png
[image_image 22]: /images/20220521/eb54403b6b0c4978bc4d53bc34a1bd89.png
[image 23]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604142586.png
[image_image 23]: /images/20220521/01d62c6344054e58b098640959e274c7.png
[image 24]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/20110618160415700.png
[image_image 24]: /images/20220521/fdf6531896df42cbbceeb3e78874eb94.png
[image 25]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/20110618160415766.png
[image_image 25]: /images/20220521/253ea92ced6e46a8a3d9065ad6cb6312.png
[image 26]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604168880.png
[image_image 26]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/20110618160416799.png
[image 27]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604161421.png
[image_image 27]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181604163897.png
[image 28]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106181610074532.png
[image_image 28]: /images/20220521/bc0e4420fc44482883a0959d67862e01.png
[image 29]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106191256096188.png
[image_image 29]: /images/20220521/417102f30ae945cc90803f8ca4da8fc8.png
[image 30]: http://images.cnblogs.com/cnblogs_com/rocketfan/201106/201106191256097093.png
[image_image 30]: /images/20220521/a5c3324c6eb5405cb6288b99072a9a4b.png