Keras聚合目标函数

本文介绍了Keras聚合目标函数的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

如何向keras模型添加汇总错误? 有桌子:

How to add aggregated error to keras model? Having table:

g x y 0 1 1 1 1 1 2 2 2 1 3 3 3 2 1 2 4 2 2 1

我希望能够将sum((y - y_pred) ** 2)错误以及 每组sum((sum(y) - sum(y_pred)) ** 2). 可以有更大的单个样本错误，但对我而言，拥有正确的总数至关重要.

I would like to be able to minimize sum((y - y_pred) ** 2) error along with sum((sum(y) - sum(y_pred)) ** 2) per group. I'm fine to have bigger individual sample errors, but it is crucial for me to have right totals.

SciPy示例:

import pandas as pd from scipy.optimize import differential_evolution df = pd.DataFrame({'g': [1, 1, 1, 2, 2], 'x': [1, 2, 3, 1, 2], 'y': [1, 2, 3, 2, 1]}) g = df.groupby('g') def linear(pars, fit=False): a, b = pars df['y_pred'] = a + b * df['x'] if fit: sample_errors = sum((df['y'] - df['y_pred']) ** 2) group_errors = sum((g['y'].sum() - g['y_pred'].sum()) ** 2) total_error = sum(df['y'] - df['y_pred']) ** 2 return sample_errors + group_errors + total_error else: return df['y_pred'] pars = differential_evolution(linear, [[0, 10]] * 2, args=[('fit', True)])['x'] print('SAMPLES:\n', df, '\nGROUPS:\n', g.sum(), '\nTOTALS:\n', df.sum())

输出:

SAMPLES: g x y y_pred 0 1 1 1 1.232 1 1 2 2 1.947 2 1 3 3 2.662 3 2 1 2 1.232 4 2 2 1 1.947 GROUPS: x y y_pred g 1 6 6 5.841 2 3 3 3.179 TOTALS: g 7.000 x 9.000 y 9.000 y_pred 9.020

推荐答案

对于分组，只要在整个训练过程中保持相同的组，损失函数就不会出现不可微分的问题.

For grouping, as long as you keep the same groups throughout training, your loss function will not have problems about being not differentiable.

作为一种简单的分组方式，您可以简单地将批次分开.

As a naive form of grouping, you can simply separate the batches.

我为此建议一个生成器.

I suggest a generator for that.

#suppose you have these three numpy arrays: gTrain xTrain yTrain #create this generator def grouper(g,x,y): while True: for gr in range(1,g.max()+1): indices = g == gr yield (x[indices],y[indices])

对于损失功能，您可以自己制作:

For the loss function, you can make your own:

import keras.backend as K def customLoss(yTrue,yPred): return K.sum(K.square(yTrue-yPred)) + K.sum(K.sum(yTrue) - K.sum(yPred)) modelpile(loss=customLoss, ....)

如果您使用负值，请小心第二项.

Just be careful with the second term if you have negative values.

现在，您使用方法fit_generator进行训练:

Now you train using the method fit_generator:

model.fit_generator(grouper(gTrain,xTrain, yTrain), steps_per_epoch=gTrain.max(), epochs=...)