TensfoFlow:线性回归损失随着连续 epoch 增加(而不是减少)

本文介绍了TensfoFlow:线性回归损失随着连续 epoch 增加(而不是减少)的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

限时送ChatGPT账号..

我正在学习 TensorFlow 并尝试将其应用于一个简单的线性回归问题.data 是形状为 [42x2] 的 numpy.ndarray.

I'm learning TensorFlow and trying to apply it on a simple linear regression problem. data is numpy.ndarray of shape [42x2].

我有点困惑，为什么在每个连续的 epoch 之后损失都在增加.不是每个连续的 epoch 都会减少损失吗！

I'm a bit puzzled why after each succesive epoch the loss is increasing. Isn't the loss expected to to go down with each successive epoch!

这是我的代码(如果您希望我也分享输出，请告诉我！):(非常感谢您花时间回答它.)

Here is my code (let me know, if you'd like me to share the output as well!): (Thanks a lot for taking your time to answer to it.)

1) 为因变量/自变量创建占位符

1) created the placeholders for dependent / independent variables

X = tf.placeholder(tf.float32, name='X')
Y = tf.placeholder(tf.float32,name='Y')

2) 为权重、偏差、total_loss 创建变量(在每个 epoch 之后)

2) created vars for weight, bias, total_loss (after each epoch)

w = tf.Variable(0.0,name='weights')
b = tf.Variable(0.0,name='bias')

3) 定义损失函数 &优化器

3) defined loss function & optimizer

Y_pred = X * w + b
loss = tf.reduce_sum(tf.square(Y - Y_pred), name = 'loss')
optimizer = tf.train.GradientDescentOptimizer(learning_rate = 0.001).minimize(loss)

4) 创建摘要事件 &事件文件编写器

4) created summary events & event file writer

tf.summary.scalar(name = 'weight', tensor = w)
tf.summary.scalar(name = 'bias', tensor = b)
tf.summary.scalar(name = 'loss', tensor = loss)
merged = tf.summary.merge_all()

evt_file = tf.summary.FileWriter('def_g')
evt_file.add_graph(tf.get_default_graph())

5) 并在一个会话中执行所有

5) and execute all in a session

with tf.Session() as sess1:
sess1.run(tf.variables_initializer(tf.global_variables()))
for epoch in range(10):
    summary, _,l = sess1.run([merged,optimizer,loss],feed_dict={X:data[:,0],Y:data[:,1]})
    evt_file.add_summary(summary,epoch+1)
    evt_file.flush()
    print("  new_loss: {}".format(sess1.run(loss,feed_dict={X:data[:,0],Y:data[:,1]})))            

干杯！

推荐答案

简短的回答是你的学习率太大了.通过将其从 0.001 更改为 0.0001，我能够获得合理的结果，但是我只使用了您倒数第二条评论中的 23 分(我最初没有注意到您的最后一条评论)，因此使用所有数据可能需要更低的号.

The short answer is that your learning rate is too big. I was able to get reasonable results by changing it from 0.001 to 0.0001, but I only used the 23 points from your second-last comment (I initially didn't notice your last comment), so using all the data might require an even lower number.

0.001 似乎是一个非常低的学习率.然而，真正的问题是您的损失函数使用的是 reduce_sum 而不是 reduce_mean.这会导致您的损失变得很大，这会向 GradientDescentOptimizer 发送非常强的信号，因此尽管学习率很低，但它还是会过冲.如果您在训练数据中添加更多点，问题只会变得更糟.所以使用 reduce_mean 来获得平均平方误差，你的算法会表现得更好.

0.001 seems like a really low learning rate. However, the real problem is that your loss function is using reduce_sum instead of reduce_mean. This causes your loss to be a large number, which sends a very strong signal to the GradientDescentOptimizer, so it's overshooting despite the low learning rate. The problem would only get worse if you added more points to your training data. So use reduce_mean to get the average squared error and your algorithms will be much better behaved.

这篇关于TensfoFlow:线性回归损失随着连续 epoch 增加(而不是减少)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！