admin管理员组

文章数量:1627749

  • 首页
                                </a>
                                
                                
                              </li><li class="active " title="阅读深度、前沿文章">
                                <a data-report-click="{&quot;mod&quot;:&quot;popu_336&quot;,&quot;dest&quot;:&quot;https://blog.csdn/&quot;,&quot;spm&quot;:&quot;3001.4477&quot;}" data-report-query="spm=3001.4477" href="https://blog.csdn/">
                                  博客
                                  
                                  
                                </a>
                                
                                
                              </li><li class="" title="马上开始系统学习">
                                <a data-report-click="{&quot;mod&quot;:&quot;popu_336&quot;,&quot;dest&quot;:&quot;https://edu.csdn/&quot;,&quot;spm&quot;:&quot;3001.4482&quot;}" data-report-query="spm=3001.4482" href="https://edu.csdn/">
                                  专栏课程
                                  
                                  
                                </a>
                                
                                
                              </li><li class="" title="获取源码、文档、学习资源">
                                <a data-report-click="{&quot;mod&quot;:&quot;popu_336&quot;,&quot;dest&quot;:&quot;https://download.csdn/&quot;,&quot;spm&quot;:&quot;3001.4483&quot;}" data-report-query="spm=3001.4483" href="https://download.csdn/">
                                  下载
                                  
                                  
                                </a>
                                
                                
                              </li><li class="" title="技术问题,有问必答">
                                <a data-report-click="{&quot;mod&quot;:&quot;popu_336&quot;,&quot;dest&quot;:&quot;https://ask.csdn/&quot;,&quot;spm&quot;:&quot;3001.4492&quot;}" data-report-query="spm=3001.4492" href="https://ask.csdn/">
                                  问答
                                  
                                  
                                </a>
                                
                                
                              </li><li class="" title="找到志同道合的伙伴">
                                <a data-report-click="{&quot;mod&quot;:&quot;popu_336&quot;,&quot;dest&quot;:&quot;https://www.csdn/c/&quot;,&quot;spm&quot;:&quot;3001.6068&quot;}" data-report-query="spm=3001.6068" href="https://www.csdn/c/">
                                  社区
                                  
                                  
                                </a>
                                
                                
                              </li><li class="" title="安装你的浏览器助手">
                                <a data-report-click="{&quot;spm&quot;:&quot;3001.6135&quot;,&quot;dist&quot;:&quot;https://t.csdnimg/OSqE&quot;}" data-report-query="spm=3001.6135" href="https://t.csdnimg/OSqE">
                                  插件
                                  
                                  
                                </a>
                                
                                
                              </li><li class="" title="认证你的技术能力">
                                <a data-report-click="{&quot;mod&quot;:&quot;popu_336&quot;,&quot;dest&quot;:&quot;https://ac.csdn/&quot;,&quot;spm&quot;:&quot;3001.5716&quot;}" data-report-query="spm=3001.5716" href="https://ac.csdn/">
                                  认证
                                  
                                  
                                </a>
                                
                                
                              </li></ul>
                        </div>
                        <div class="toolbar-container-middle">
                          <div class="toolbar-search onlySearch"><div class="toolbar-search-container">
                    <input id="toolbar-search-input" autocomplete="off" type="text" value="" placeholder="信息论">
                    <button id="toolbar-search-button"></button>
                    <input type="password" autocomplete="new-password" readonly="" disabled="true" style="display: none; position:absolute;left:-9999999px;width:0;height:0;">
                  </div></div>
                        </div>
                        <div class="toolbar-container-right">
                          <div class="toolbar-btns onlyUser"><div class="toolbar-btn toolbar-btn-login csdn-toolbar-fl toolbar-subMenu-box">
                      <a class="hasAvatar" data-report-click="{&quot;spm&quot;: &quot;3001.5343&quot;}" data-report-query="spm=3001.5343" href="https://blog.csdn/weixin_42612518"><img src="https://profile.csdnimg/E/B/8/2_weixin_42612518"></a>
                      <div id="csdn-toolbar-profile" class="csdn-toolbar-plugin">
            <div class="csdn-profile-top">
              <a class="csdn-profile-avatar" data-report-click="{&quot;spm&quot;: &quot;3001.5343&quot;}" data-report-query="spm=3001.5343" href="https://blog.csdn/weixin_42612518"><img src="https://profile.csdnimg/E/B/8/0_weixin_42612518"></a>
              <p class="csdn-profile-nickName">Alex龙雅</p>
              <a data-report-click="{&quot;spm&quot;: &quot;3001.5344&quot;}" data-report-query="spm=3001.5344" href="https://mall.csdn/vip" class="csdn-profile-no-vip"></a>
            </div>
            <div class="csdn-profile-mid">
              <a data-report-click="{&quot;spm&quot;: &quot;3001.5347&quot;}" data-report-query="spm=3001.5347" href="https://blog.csdn/weixin_42612518?type=sub&amp;subType=fans"><i class="csdn-profile-fansCount">42</i>粉丝</a>
              <a data-report-click="{&quot;spm&quot;: &quot;3001.5348&quot;}" data-report-query="spm=3001.5348" href="https://blog.csdn/weixin_42612518?type=sub"><i class="csdn-profile-followCount">5</i>关注</a>
              <a data-report-click="{&quot;spm&quot;: &quot;3001.5349&quot;}" data-report-query="spm=3001.5349" href="https://blog.csdn/weixin_42612518"><i class="csdn-profile-likeCount">39</i>获赞</a>
            </div>
            <div class="csdn-profile-bottom">
              <ul class="csdn-border-bottom">
                <li class=""><a href="https://i.csdn/#/user-center/profile" data-report-click="{&quot;dest&quot;:&quot;https://i.csdn/#/user-center/profile&quot;,&quot;spm&quot;:&quot;3001.5111&quot;}" data-report-query="spm=3001.5111"><i class="csdn-profile-icon csdn-profile-icon-person"></i>个人中心</a></li><li class=""><a href="https://mp.csdn/mp_blog/manage/article?spm=1011.2124.3001.5298" data-report-click="{&quot;dest&quot;:&quot;https://mp.csdn/mp_blog/manage/article?spm=1011.2124.3001.5298&quot;,&quot;spm&quot;:&quot;3001.5448&quot;}" data-report-query="spm=3001.5448"><i class="csdn-profile-icon csdn-profile-icon-pages"></i>内容管理</a></li><li class=""><a href="https://edu.csdn/" data-report-click="{&quot;dest&quot;:&quot;https://edu.csdn/&quot;,&quot;spm&quot;:&quot;3001.5350&quot;}" data-report-query="spm=3001.5350"><i class="csdn-profile-icon csdn-profile-icon-study"></i>学习平台</a></li><li class=""><a href="https://mall.csdn/myorder" data-report-click="{&quot;dest&quot;:&quot;https://mall.csdn/myorder&quot;,&quot;spm&quot;:&quot;3001.5137&quot;}" data-report-query="spm=3001.5137"><i class="csdn-profile-icon csdn-profile-icon-order"></i>我的订单</a></li><li class="pb-8 csdn-border-bottom"><a href="https://i.csdn/#/wallet/index" data-report-click="{&quot;dest&quot;:&quot;https://i.csdn/#/wallet/index&quot;,&quot;spm&quot;:&quot;3001.5136&quot;}" data-report-query="spm=3001.5136"><i class="csdn-profile-icon csdn-profile-icon-wallet"></i>我的钱包</a></li><li class="pt-8 pb-8 csdn-border-bottom"><a href="https://i.csdn/#/uc/reward" data-report-click="{&quot;dest&quot;:&quot;https://i.csdn/#/uc/reward&quot;,&quot;spm&quot;:&quot;3001.5351&quot;}" data-report-query="spm=3001.5351"><i class="csdn-profile-icon csdn-profile-icon-draw"></i>签到抽奖</a></li><li class="pt-8 csdn-profile-logout"><a href="javascript:;" data-report-click="{&quot;spm&quot;:&quot;3001.5139&quot;}" data-report-query="spm=3001.5139"><i class="csdn-profile-icon csdn-profile-icon-logout"></i>退出</a></li>
              </ul>
            </div>
          </div></div>
                    <div class="toolbar-btn toolbar-btn-vip csdn-toolbar-fl">
                      <a data-report-click="{&quot;mod&quot;:&quot;popu_336&quot;,&quot;dest&quot;:&quot;https://mall.csdn/vip&quot;,&quot;spm&quot;:&quot;3001.4496&quot;}" data-report-query="spm=3001.4496" href="https://mall.csdn/vip">会员中心 </a>
                    <div id="csdn-plugin-vip" style="display: none;">
                        <div class="csdn-plugin-vip-header">
                            会员特权
                        </div>
                        <div class="csdn-plugin-vip-body">
                            <a href="https://download.csdn/?utm_source=vip_xz_862637798_yy"><i class="csdn-plugin-vip-icon csdn-plugin-vip-icon-download"></i><br>资源下载</a>
                            <a href="https://edu.csdn/course?channelType=3&amp;utm_source=vip_jpk_862637798_yy"><i class="csdn-plugin-vip-icon csdn-plugin-vip-icon-class"></i><br>千节精品课</a>
                            <a href="https://book.csdn/?utm_source=vip_dzs_862637798_yy"><i class="csdn-plugin-vip-icon csdn-plugin-vip-icon-book"></i><br>千本电子书</a>
                            <a href="https://so.csdn/so/search?q=&amp;t=vip&amp;s=0&amp;pnt=-1&amp;ct=100&amp;utm_source=vip_jswz_862637798_yy"><i class="csdn-plugin-vip-icon csdn-plugin-vip-icon-pages"></i><br>VIP技术文章</a>
                        </div>
                        <div class="csdn-plugin-vip-footer">
                            <a data-report-click="{&quot;spm&quot;: &quot;3001.5400&quot;}" data-report-query="spm=3001.5400" class="csdn-plugin-vip-footer-btn" href="https://mall.csdn/vip">
                              开通会员
                            </a>
                            <a data-report-click="{&quot;spm&quot;: &quot;3001.5401&quot;}" data-report-query="spm=3001.5401" class="csdn-plugin-vip-footer-link" href="https://www.csdn/vip">
                              查看完整会员权益<i></i>
                            </a>
                        </div>
                    </div></div>
                    <div class="toolbar-btn toolbar-btn-collect csdn-toolbar-fl">
                      <a data-report-click="{&quot;spm&quot;:&quot;3001.4506&quot;}" data-report-query="spm=3001.4506" href="https://i.csdn/#/user-center/collection-list?type=1">收藏</a>
                    
          <div id="csdn-toolbar-collection" class="csdn-toolbar-plugin" style="display: none;">
            <div class="toolbar-collection-left csdn-toolbar-scroll-box">
              <ul>
          <li class="collection-folder-active">
            <div class="toolbar-collection-folder-name">默认收藏夹</div>
            <div class="toolbar-collection-folder-count">9</div>
          </li>
        </ul>
            </div>
            <div class="toolbar-collection-right">
              <ul class="csdn-toolbar-scroll-box">
          <li>
            <a href="https://fff2zrx.blog.csdn/article/details/79303083" target="_blank">
              <span class="toolbar-collection-type">博客</span>
              <span class="toolbar-collection-title">matlab中cell数组用法</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/Tonywu2018/article/details/91490158" target="_blank">
              <span class="toolbar-collection-type">博客</span>
              <span class="toolbar-collection-title">详解独立成分分析</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/Cratial/article/details/88918132" target="_blank">
              <span class="toolbar-collection-type">博客</span>
              <span class="toolbar-collection-title">多尺度熵---Understanding Multiscale Entropy</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/xiwan0902/article/details/72280352" target="_blank">
              <span class="toolbar-collection-type">博客</span>
              <span class="toolbar-collection-title">5种常用的相关分析方法</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/qq_37885346/article/details/80542740" target="_blank">
              <span class="toolbar-collection-type toolbar-collection-otherType">其他</span>
              <span class="toolbar-collection-title">matlab_格兰杰因果关系检验 - qq_37885346的博客</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/zengxiantao1994/article/details/72787849" target="_blank">
              <span class="toolbar-collection-type toolbar-collection-otherType">其他</span>
              <span class="toolbar-collection-title">极大似然估计详解 - 知行流浪</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/yuhq3/article/details/78585774" target="_blank">
              <span class="toolbar-collection-type toolbar-collection-otherType">其他</span>
              <span class="toolbar-collection-title">Matlab多目标跟踪示例(一):Motion-Based Multiple Object Tracking</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/qq_25819827/article/details/53445567" target="_blank">
              <span class="toolbar-collection-type toolbar-collection-otherType">其他</span>
              <span class="toolbar-collection-title">用MATLAB实现对运动物体识别与跟踪</span>
            </a>
          </li>
        
          <li>
            <a href="https://blog.csdn/shmilymdt/article/details/48245519" target="_blank">
              <span class="toolbar-collection-type toolbar-collection-otherType">其他</span>
              <span class="toolbar-collection-title">matlab读取视频VideoReader类</span>
            </a>
          </li>
        </ul>
            </div>
            <div class="csdn-toolbar-plugin-triangle"></div>
          </div>
        </div>
                     <div class="toolbar-btn csdn-toolbar-fl"><a data-report-click="{&quot;spm&quot;:&quot;3001.4507&quot;}" data-report-query="spm=3001.4507" href="https://blog.csdn/nav/watchers">动态</a></div>
                    <div class="toolbar-btn toolbar-btn-msg csdn-toolbar-fl">
                      <div class="toolbar-subMenu-box">
                        <a data-report-click="{&quot;spm&quot;:&quot;3001.4508&quot;}" data-report-query="spm=3001.4508" id="toolbar-remind" href="https://i.csdn/#/msg/index"><span class="pos-rel">消息</span></a>
                      <div class="toolbar-subMenu">
                          <a data-type="comment" href="https://i.csdn/#/msg/index"><span class="pos-rel">评论</span></a>
                          <a data-type="attention" href="https://i.csdn/#/msg/attention"><span class="pos-rel">关注</span></a>         
                          <a data-type="like" href="https://i.csdn/#/msg/like"><span class="pos-rel">点赞</span></a>
                          <a data-type="chat" href="https://im.csdn/im/main.html"><span class="pos-rel">私信</span></a>
                          <a data-type="notice" href="https://i.csdn/#/msg/notice"><span class="pos-rel">系统通知</span></a>
                          <a href="https://i.csdn/#/msg/setting">消息设置</a>
                     </div></div>
                      <div class="toolbar-msg-box"></div>
                    </div>
                    <div class="toolbar-btn toolbar-btn-write csdn-toolbar-fl toolbar-subMenu-box">
                      <a data-report-click="{&quot;spm&quot;:&quot;3001.4503&quot;}" data-report-query="spm=3001.4503" href="https://mp.csdn"><i></i>创作中心</a>
                    </div></div>
                        </div>
                      </div>
                    </div>
                  </div>
        <script>
            var toolbarSearchExt = '{"landingWord":[],"queryWord":"信息熵","tag":["信息论","信息熵","人工智能"],"title":"一文看懂信息熵的本质&mdash;&mdash;谈谈自己对信息熵的理解"}';
        </script>
    <script src="https://g.csdnimg/common/csdn-toolbar/csdn-toolbar.js" type="text/javascript"></script>
    <script>
    (function(){
        var bp = document.createElement('script');
        var curProtocol = window.location.protocol.split(':')[0];
        if (curProtocol === 'https') {
            bp.src = 'https://zz.bdstatic/linksubmit/push.js';
        }
        else {
            bp.src = 'http://push.zhanzhang.baidu/push.js';
        }
        var s = document.getElementsByTagName("script")[0];
        s.parentNode.insertBefore(bp, s);
    })();
    </script>
    

一文看懂信息熵的本质——谈谈自己对信息熵的理解

置顶 StudentWang_ 2020-04-24 10:33:39 3152 收藏 57 原力计划 分类专栏: 信息融合 文章标签: 信息论 信息熵 人工智能 版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。 本文链接: https://blog.csdn/qq_35624030/article/details/105704098 版权 信息融合 专栏收录该内容 2 篇文章 0 订阅 订阅专栏

一、序言

    初次看到信息熵的公式有很多不理解的地方,只知道信息熵如何进行计算,却不懂得公式背后的原理,我通过查阅了一些资料,加深了对信息熵的理解,现在将这些理解分享给大家。如有疑问欢迎评论,若对你有帮助,麻烦点个赞。未经允许、请勿转载。(本文适合只知道信息熵的公式,但是不明白其中原理的人进行阅读)

二、什么是信息熵

    正如我们想要衡量某个物体的质量引入了克这个单位、我们想衡量时间,我们设计一秒钟这么长。香农老人家想要量化一条消息中带有的“信息量”的大小,提出了信息熵。
那么,首先明确一个问题,什么样的消息算作“信息量大”呢?什么样的消息又算作“信息量小呢”?举个例子昨天小明和我说:“今天罗志祥又和周扬青秀恩爱了!”,我就觉得这有啥的,他们天天秀恩爱。也就是说小明的这条消息并不能给我带来很大的信息量。
BUT今天小明和我说:”周扬青怒锤罗志祥!!!罗志祥人设崩塌!!!“,我就会很惊讶,因为这条消息给我的信息量很大。(类似的信息量很大的消息还有:小明告诉我今天太阳会从西边升起)
我们用信息熵来描述一个事件混乱程度的大小(一个事件我们一定知道结果,那么这个事件的混乱程度就是0;一个时间充满随机性,我们猜不到或者很难猜到结果,那么他的混乱度就很大)
引用下面一个在箱子里面摸球的例子,我们来更具体的了解信息熵。
(此例引自:Youtube的一个视频)

图左侧中有一个装有四个球的封闭箱子(这个箱子里面有三个红球、一个蓝球),现在我们从箱子中随机取出一个球,记录它的颜色后再放回箱子中,重复四次操作。如果你依次取出的序列为右上角所示(第一次取到红色、第二次取到红色、第三次取到红色、第四次取到蓝色),则你可以获得奖金;否则你就输了。大家用初中数学来算一算,我赢得奖金的概率是

     P
    
    
     (
    
    
     x
    
    
     )
    
    
     =
    
    
     
      3
     
     
      4
     
    
    
     ∗
    
    
     
      3
     
     
      4
     
    
    
     ∗
    
    
     
      3
     
     
      4
     
    
    
     ∗
    
    
     
      1
     
     
      4
     
    
    
     =
    
    
     
      27
     
     
      256
     
    
   
   
     P(x)=\frac{3}{4}*\frac{3}{4}*\frac{3}{4}*\frac{1}{4}=\frac{27}{256} 
   
  
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.68em; vertical-align: 0em;"></span><span class="mord mathdefault" style="margin-right: 0.13em;">P</span><span class="mord cjk_fallback">(</span><span class="mord mathdefault">x</span><span class="mord cjk_fallback">)</span><span class="mspace" style="margin-right: 0.27em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.27em;"></span></span><span class="base"><span class="strut" style="height: 2em; vertical-align: -0.68em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32em;"><span style="top: -2.31em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">4</span></span></span><span style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span style="top: -3.67em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.68em;"></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right: 0.22em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right: 0.22em;"></span></span><span class="base"><span class="strut" style="height: 2em; vertical-align: -0.68em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32em;"><span style="top: -2.31em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">4</span></span></span><span style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span style="top: -3.67em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.68em;"></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right: 0.22em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right: 0.22em;"></span></span><span class="base"><span class="strut" style="height: 2em; vertical-align: -0.68em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32em;"><span style="top: -2.31em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">4</span></span></span><span style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span style="top: -3.67em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.68em;"></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right: 0.22em;"></span><span class="mbin">∗</span><span class="mspace" style="margin-right: 0.22em;"></span></span><span class="base"><span class="strut" style="height: 2em; vertical-align: -0.68em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32em;"><span style="top: -2.31em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">4</span></span></span><span style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span style="top: -3.67em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.68em;"></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right: 0.27em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.27em;"></span></span><span class="base"><span class="strut" style="height: 2em; vertical-align: -0.68em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32em;"><span style="top: -2.31em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span><span class="mord">5</span><span class="mord">6</span></span></span><span style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span style="top: -3.67em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span><span class="mord">7</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.68em;"></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></span></span><br>   现在明确了这个游戏的规则,那么我们分析一下如下几个箱子与赢得奖金的情况,分别计算一下获胜概率<br> <img alt="在这里插入图片描述" src="https://img-blog.csdnimg/20200423165753935.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1NjI0MDMw,size_16,color_FFFFFF,t_70"><br> <img alt="在这里插入图片描述" src="https://img-blog.csdnimg/20200423165836890.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1NjI0MDMw,size_16,color_FFFFFF,t_70"><br> <img alt="在这里插入图片描述" src="https://img-blog.csdnimg/20200423165853642.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1NjI0MDMw,size_16,color_FFFFFF,t_70"><br> (每个图的左边是这个箱子的红球与蓝球初始条件,右侧是获胜所要求的序列,大家自己算算这个概率和图上写的一样吗?)<br> <img alt="在这里插入图片描述" src="https://img-blog.csdnimg/20200423170009974.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1NjI0MDMw,size_16,color_FFFFFF,t_70"><br> 我们来分析一下这三种状态的游戏,第一次由于箱子内全是红球、该箱子的随机性很弱,也就是说还没摸我就知道结果了。这种状态带来的信息量很小;第三个箱子随机性很强,也就是说,对于结果我们充满未知。这个状态的信息量很大。我们算出的这个概率越接近1时候,这个信息熵应该越接近0;算出的这个概率越小的时候,信息熵反而应该越大。我们需要找到这么一个公式来满足这一点。(暂停思考一下,有哪些公式可以满足)<br>  <strong>该可以很轻松的想到,用刚才P(winning)的概率取倒数可以吗?</strong> 答案是,只考虑刚才的问题,是可以的。但取倒数这个方式还是存在一些问题,比如第三个箱子取0.0625的倒数,算出来的这个信息量为16,如果这个P(winning)概率算出结果很小时,取倒数后就会变得非常大,所以我们认为这个乘法的规则并不是很好(换言之,我们希望曲线平滑一些)。香农发现log函数可以很好的解决这个问题。 因为log(a*b) = log(a)+log(b);而恰巧像抽箱子的独立事件P(AB) = P(A)*P(B)。【原因不止于此,后面我们还会再详细讨论】。<br> 所以香农取了一个-log(x)这么一个函数来表示某一状态的信息量大小,因为x是概率事件取[0,1],所以log(x)是一个递增的恒负的值,取一个负号,-log(x)是一个恒正、递减的函数,正好符合我们的预期。<br> <img alt="在这里插入图片描述" src="https://img-blog.csdnimg/2020042410083693.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1NjI0MDMw,size_16,color_FFFFFF,t_70"></p> 

本文标签: