TensorFlow 摘要 标量在示例中作为张量写入事件日志

本文介绍了TensorFlow 摘要 标量在示例中作为张量写入事件日志的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

限时送ChatGPT账号..

TensorFlow 版本 = 2.0.0

TensorFlow version = 2.0.0

我正在按照 https://www.tensorflow/api_docs/python/tf/summary" rel="nofollow noreferrer">https://www.tensorflow/api_docs/python/tf/summary;页面上的第一个，为了完整起见，我将粘贴在下面:

I am following the example of how to use the TensorFlow summary module at https://www.tensorflow/api_docs/python/tf/summary; the first one on the page, which for completeness I will paste below:

writer = tf.summary.create_file_writer("/tmp/mylogs")
with writer.as_default():
  for step in range(100):
    # other model code would go here
    tf.summary.scalar("my_metric", 0.5, step=step)
    writer.flush()

运行这个很好，我得到了可以在 TensorBoard 中查看的事件日志.伟大的！但是，当我使用以下方法查看事件日志时:

Running this is fine, and I get event logs that I can view in TensorBoard. Great! However when I look in the event log using:

tensorboard --inspect --logdir=tmp/mylogs

它告诉我由于某种原因，我的摘要变量已作为张量而不是标量写入日志:

it tells me that my summary variable has been written to the log as a Tensor for some reason, not a Scalar:

Event statistics for tmp/mylogs:
audio -
graph -
histograms -
images -
scalars -
sessionlog:checkpoint -
sessionlog:start -
sessionlog:stop -
tensor
   first_step           0
   last_step            99
   max_step             99
   min_step             0
   num_steps            100
   outoforder_steps     [(99, 0)]

我想这可能不是问题，除了当我尝试按照例如中的方法从事件日志中读取时https://stackoverflow/a/45899735/1447953:

I guess that might not be a problem, except that when I try to read from the event log following the method in e.g. https://stackoverflow/a/45899735/1447953:

from tensorboard.backend.event_processing.event_accumulator import EventAccumulator
x = EventAccumulator(path="tmp/mylogs")
x.Reload()
print(x.Tags())

然后它再次告诉我 my_metric 是一个张量:

then it again tells me that my_metric is a Tensor:

{'images': [], 'audio': [], 'histograms': [], 'scalars': [], 'distributions': [], 'tensors': ['my_metric'], 'graph': False, 'meta_graph': False, 'run_metadata': []}

当我试图查看数据时，它是胡言乱语

and when I try to look at the data it is gibberish

w_times, step_nums, vals = zip(*x.Tensors('my_metric'))
print("vals:", vals)

vals: (dtype: DT_FLOAT
tensor_shape {
}
tensor_content: "\000\000\000?"
, dtype: DT_FLOAT
tensor_shape {
}
tensor_content: "\000\000\000?"
, dtype: DT_FLOAT
tensor_shape {
}
...
etc.            

我在这里做错了吗?这个例子看起来很简单，所以我不确定可能是什么问题.我只是复制/粘贴它.或者，也许他们决定始终将数据粘贴在张量"标签下，并且有某种方法可以将这些值转换回标准绘图工具中可用的值?

Am I doing something wrong here? The example seemed pretty simple so I'm not sure what the problem could be. I just copy/pasted it. Or maybe they decided to always stick data under the 'Tensor' tags and there is some way to convert the values back to something usable in standard plotting tools?

就在迁移文档的底部 https://www.tensorflow/张量板/迁移它说:

Ok right at the bottom of the migration doc https://www.tensorflow/tensorboard/migrate it says:

事件文件二进制表示已更改:

The event file binary representation has changed:

TensorBoard 1.x 已经支持新格式；此差异仅影响手动解析事件文件中的摘要数据的用户

TensorBoard 1.x already supports the new format; this difference only affects users who are manually parsing summary data from event files

摘要数据现在存储为张量字节；您可以使用 tf.make_ndarray(event.summary.value[0].tensor) 将其转换为 numpy

Summary data is now stored as tensor bytes; you can use tf.make_ndarray(event.summary.value[0].tensor) to convert it to numpy

所以我猜这意味着存储为张量"是正常的.不过，转换对我来说仍然很神秘，它们似乎指的是与我发现的 EventAccumulator 不同的接口.而且似乎由于某种原因，我只记录了 100 个事件中的 10 个，我也觉得这很神秘.

So I guess that means the storage as 'tensor' is normal. The conversion is still mysterious to me though, they seem to be referring to a different interface than the EventAccumulator one I found. And it also seems that I only get 10 out 100 events recorded for some reason, which I also find mysterious.

推荐答案

我遇到了同样的问题，我能够使用 tfpat.v1.train.summary_iterator() 加载所有数据.

I had the same issue and I was able to load all the data using tfpat.v1.train.summary_iterator().

import os
import matplotlib.pyplot as plt
import tensorflow as tf
import pandas as pd

path = "logs"
listOutput = os.listdir(path)

listDF = []
key = "loss". # tag

for tb_output_folder in listOutput:
    print(tb_output_folder)
    folder_path = os.path.join(path, tb_output_folder)
    file = os.listdir(folder_path)[0]

    tensors = []
    steps = []
    for e in tfpat.v1.train.summary_iterator(os.path.join(folder_path, file)):
        for v in e.summary.value:
            if v.tag == key:
                tensors.append(v.tensor)
                steps.append(e.step)

    values = [tf.make_ndarray(t) for t in tensors]

    plt.plot(steps, values)

    df = pd.DataFrame(data=values)
    df.to_csv("{}.csv".format(tb_output_folder))

plt.show()

这篇关于TensorFlow 摘要 标量在示例中作为张量写入事件日志的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！