Theano神经网络的所有输出汇集到所有输入的相同值(Theano Neural Net All Outputs Converge to Same Value For All Inputs)

我一直在努力实现一个神经网络来融合有意义的价值。我有黑白图像。每幅图像都是40％黑和60％白或60％白和40％黑。分类为更多黑色或白色。

我将图像分解为像素值数组并通过网络馈送它们。问题是它会聚合到所有图像的恒定值。我正在使用1000张图像进行训练。输入为25 * 25像素，隐藏层为20。

码：

def layer(x, w): ##bias node b = np.array([1], dtype=theano.config.floatX) ##concate bias node new_x = T.concatenate([x, b]) ##evalu. matrix mult m = T.dot(w.T, new_x) ##run through sigmoid h = nnet.sigmoid(m) return h ##for gradient descient, calc cost function to mininize def grad_desc(cost, theta): return theta - (.01 * T.grad(cost, wrt=theta)) ##input x x = T.dvector() ##y target y = T.dscalar() alpha = .1 #learning rate ###first layer weights theta1 = theano.shared(np.array(np.random.rand((25*25)+1,20), dtype=theano.config.floatX)) # randomly initialize ###output layer weights theta3 = theano.shared(np.array(np.random.rand(21,1), dtype=theano.config.floatX)) hid1 = layer(x, theta1) #hidden layer out1 = T.sum(layer(hid1, theta3)) #output layer fc = (out1 - y)**2 #cost expression to minimize cost = theano.function(inputs=[x, y], outputs=fc, updates=[ ##updates gradient weights (theta1, grad_desc(fc, theta1)), (theta3, grad_desc(fc, theta3))]) run_forward = theano.function(inputs=[x], outputs=out1) inputs = np.array(inputs).reshape(1000,25*25) #training data X exp_y = np.array(exp_y) #training data Y cur_cost = 0 for i in range(10000): for k in range(len(inputs)): cur_cost = cost(inputs[k], exp_y[k]) if i % 10 == 0: print('Cost: %s' % (cur_cost,))

成本覆盖单个值以及具有相同输出的任何输入：

.... Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066

I've been struggling to get the implementation of a neural net to converge to meaningful values. I have black and white images. Each image is either 40% black and 60% white or 60% white and 40% black. Classifying for more black or white.

I break the images into arrays of pixel values and feed them through the network. The issues is that it converges to the same constant value for all images. I am using 1000 images to train. 25*25 pixels for input and a hidden layer of 20.

CODE:

def layer(x, w): ##bias node b = np.array([1], dtype=theano.config.floatX) ##concate bias node new_x = T.concatenate([x, b]) ##evalu. matrix mult m = T.dot(w.T, new_x) ##run through sigmoid h = nnet.sigmoid(m) return h ##for gradient descient, calc cost function to mininize def grad_desc(cost, theta): return theta - (.01 * T.grad(cost, wrt=theta)) ##input x x = T.dvector() ##y target y = T.dscalar() alpha = .1 #learning rate ###first layer weights theta1 = theano.shared(np.array(np.random.rand((25*25)+1,20), dtype=theano.config.floatX)) # randomly initialize ###output layer weights theta3 = theano.shared(np.array(np.random.rand(21,1), dtype=theano.config.floatX)) hid1 = layer(x, theta1) #hidden layer out1 = T.sum(layer(hid1, theta3)) #output layer fc = (out1 - y)**2 #cost expression to minimize cost = theano.function(inputs=[x, y], outputs=fc, updates=[ ##updates gradient weights (theta1, grad_desc(fc, theta1)), (theta3, grad_desc(fc, theta3))]) run_forward = theano.function(inputs=[x], outputs=out1) inputs = np.array(inputs).reshape(1000,25*25) #training data X exp_y = np.array(exp_y) #training data Y cur_cost = 0 for i in range(10000): for k in range(len(inputs)): cur_cost = cost(inputs[k], exp_y[k]) if i % 10 == 0: print('Cost: %s' % (cur_cost,))

Cost Coverages to Single value as well as any inputs having same output:

.... Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066 Cost: 0.160380273066

最满意答案

只是一个想法：我已经看到了一些例子，其中整个图像以相同的方式呈现给NN。然而，这些网络是为字符识别和类似的图像处理而设计的。因此，如果您将整个图像提供给网络，它会尝试找到相似的图像。我了解你的图像是随机的，这可能是它未能训练的原因。实际上，训练图像之间可能没有相似之处，没有什么可学的。如果我想区分圆形和正方形的图像，我会以这种方式将图片呈现给程序。然而，为了决定一张图片是相当黑还是浅，我只需要给网络提供黑色像素和白色像素的数量。一些线性预处理可能非常有益。

Just an idea: I have seen examples where the entire image was presented to the NN same way as you do it. However those networks were designed for character recognition and similar image processing. So if you feed the entire image to the network it will try to find similar images. I understood that your images are random and that can be the reason why it fails to train. Actually there may be no similarities between training images and there is nothing to learn. I would present the picture to the program this way if I want to distinguish between images of circles and squares. However for deciding if a picture is rather dark or light I would simply feed the network the count of black pixels and white pixels. Some linear pre-processing can be very beneficial.

更多推荐