每层应该有一个偏置，还是每个节点一个偏置?

本文介绍了每层应该有一个偏置，还是每个节点一个偏置?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我正在寻求实现一个通用的神经网络，其中1个输入层包括输入节点，1个输出层包括输出节点，N个隐藏层包括隐藏节点.节点按层次进行组织，规则是同一层中的节点不能连接.

I am looking to implement a generic neural network, with 1 input layer consisting of input nodes, 1 output layer consisting of output nodes, and N hidden layers consisting of hidden nodes. Nodes are organized into layers, with the rule that nodes in the same layer cannot be connected.

我最了解偏见的概念，但是我有一个问题.

I mostly understand the concept of the bias, but I have a question.

每层应该有一个偏差值(由该层中的所有节点共享)还是每个节点(输入层中的节点除外)都有自己的偏差值?

Should there be one bias value per layer (shared by all nodes in that layer) or should each node (except nodes in the input layer) have their own bias value?

我觉得可以同时实现两种方法，并且想了解每种方法的取舍，并且还知道最常用的实现方式.

I have a feeling it could be done both ways, and would like to understand the trade-offs of each approach, and also know what implementation is most commonly used.

推荐答案

直观视图

要正确回答这个问题，我们首先应该像问题中所说的那样完全确定我们所说的偏见价值"的含义.通常将神经网络直观地视为节点(神经元)和节点之间的加权定向连接网络(并向初学者进行解释).在此视图中，经常将Bias绘制为其他输入"节点，这些节点的激活级别始终精确地为1.0. 1.0的值可能是某些人听到"Bias值"时所想到的.这样的偏置节点将具有可训练的权重与其他节点的连接.其他人可能会将这些权重视为偏见值".由于问题是用bias-neuron标签标记的，因此我将在假设我们使用第一个定义的情况下回答问题，例如某些Bias节点/神经元的Bias值= 1.0.

Intuitive View

To answer this question properly, we should first establish exactly what we mean when we say "Bias value" as done in the question. Neural Networks are typically intuitively viewed (and explained to beginners) as a network of nodes (neurons) and weighted, directed connections between nodes. In this view, Biases are very frequently drawn as additional ''input'' nodes, which always have an activation level of exactly 1.0. This value of 1.0 may be what some people think of when they hear "Bias Value". Such a Bias Node would have connections to other nodes, with trainable weights. Other people may think of those weights as "Bias Values". Since the question was tagged with the bias-neuron tag, I'll answer the question under the assumption that we use the first definition, e.g. Bias Value = 1.0 for some Bias Node / neuron.

从这个角度来看，只要确保将它们连接到正确的节点，从数学上讲，我们在网络中放置多少个Bias节点/值绝对不重要.您可以直观地将整个网络视为只有一个偏置节点，其值1.0不属于任何特定层，并且与除输入节点之外的所有节点都具有连接.但是，这可能很难绘制，如果要绘制神经网络，则在输出层以外的每一层中放置一个单独的偏置节点(每个偏置节点的值为1.0)可能更方便，并且将这些偏置节点中的每一个直接连接到该层之后的该层中的所有节点.从数学上讲，这两种解释是等效的，因为在两种情况下，每个非输入节点都具有始终激活级别为1.0的节点的传入加权连接.

From this point of view... it absolutely does not matter at all mathematically how many Bias nodes/values we put in our network, as long as we make sure to connect them to the correct nodes. You could intuitively think of the entire network as having only a single bias node with a value of 1.0 that does not belong to any particular layer, and has connections to all nodes other than the input nodes. This may be difficult to draw though, if you want to make a drawing of your neural network it may be more convenient to place a separate bias node (each with a value of 1.0) in every layer except for the output layer, and connect each of those bias nodes to all the nodes in the layer directly after it. Mathematically, these two interpretations are equivalent, since in both cases every non-input node has an incoming weighted connection from a node that always has an activation level of 1.0.

对神经网络进行编程时，通常根本没有任何显式节点对象"(至少在有效实现中).通常只有权重矩阵.从这个角度来看，不再有任何选择.我们(几乎)总是想要一个偏权重"(权重乘以恒定激活水平1.0)进入每个非输入节点，并且我们必须确保所有这些权重出现在我们体重矩阵的正确位置.

When Neural Networks are programmed, there typically aren't any explicit node ''objects'' at all (at least in efficient implementations). There will generally just be matrices for the weights. From this point of view, there is no longer any choice. We'll (almost) always want one ''bias-weight'' (a weight being multiplied by a constant activation level of 1.0) going to every non-input node, and we'll have to make sure all those weights appear in the correct spots in our weight matrices.