使用numpy和maplotlib求和的直方图而不是计数(Histogram of sum instead of count using numpy and matplotlib)

使用numpy和maplotlib求和的直方图而不是计数(Histogram of sum instead of count using numpy and matplotlib)

我有一些数据，每行有两列。 在我的情况下，工作提交时间和地区。

我已经使用了matplotlib的hist函数来生成一个图表，该图表在x轴上按天分隔时间，在y轴上每天计数：

import numpy as np import matplotlib.pyplot as plt import matplotlib as mpl import datetime as dt def timestamp_to_mpl(timestamp): return mpl.dates.date2num(dt.datetime.fromtimestamp(timestamp)) nci_file_name = 'out/nci.csv' jobs = np.genfromtxt(nci_file_name, dtype=int, delimiter=',', names=True, usecols(1,2,3,4,5)) fig, ax = plt.subplots(2, 1, sharex=True) vect_timestamp_to_mpl = np.vectorize(timestamp_to_mpl) qtime = vect_timestamp_to_mpl(jobs['queued_time']) start_date = dt.datetime(2013, 1, 1) end_date = dt.datetime(2013, 4, 1) bins = mpl.dates.drange(start_date, end_date, dt.timedelta(days=1)) ax[0].hist(qtime[jobs['charge_rate']==1], bins=bins, label='Normal', color='b') ax[1].hist(qtime[jobs['charge_rate']==3], bins=bins, label='Express', color='g') ax[0].grid(True) ax[1].grid(True) fig.suptitle('NCI Workload Submission Daily Rate') ax[0].set_title('Normal Queue') ax[1].set_title('Express Queue') ax[1].xaxis.set_major_locator(mpl.dates.AutoDateLocator()) ax[1].xaxis.set_major_formatter(mpl.dates.AutoDateFormatter(ax[1].xaxis.get_major_locator())) ax[1].set_xlim(mpl.dates.date2num(start_date), mpl.dates.date2num(end_date)) plt.setp(ax[1].xaxis.get_majorticklabels(), rotation=25, ha='right') ax[1].set_xlabel('Date') ax[0].set_ylabel('Jobs per Day') ax[1].set_ylabel('Jobs per Day') fig.savefig('out/figs/nci_sub_rate_day_sub.png') plt.show()

我现在想要一个图表，其中x轴上的日期分为时间，y轴上的bin区域的总和。

到目前为止，我已经使用列表理解得出了这个：

import numpy as np import matplotlib.pyplot as plt import matplotlib as mpl import datetime as dt def timestamp_to_mpl(timestamp): return mpl.dates.date2num(dt.datetime.fromtimestamp(timestamp)) def binsum(bin_by, sum_by, bins): bin_index = np.digitize(bin_by, bins) sums = [np.sum(sum_by[bin_index==i]) for i in range(len(bins))] return sums fig, ax = plt.subplots(2, 1, sharex=True) vect_timestamp_to_mpl = np.vectorize(timestamp_to_mpl) qtime = vect_timestamp_to_mpl(jobs['queued_time']) area = jobs['run_time'] * jobs['req_procs'] start_date = dt.datetime(2013, 1, 1) end_date = dt.datetime(2013, 4, 1) delta = dt.timedelta(days=1) bins = mpl.dates.drange(start_date, end_date, delta) sums_norm = binsum(qtime[jobs['charge_rate']==1], area[jobs['charge_rate']==1], bins) sums_expr = binsum(qtime[jobs['charge_rate']==3], area[jobs['charge_rate']==3], bins) ax[0].bar(bins, sums_norm, width=1.0, label='Normal', color='b') ax[1].bar(bins, sums_expr, width=1.0, label='Express', color='g') ax[0].grid(True) ax[1].grid(True) fig.suptitle('NCI Workload Area Daily Rate') ax[0].set_title('Normal Queue') ax[1].set_title('Express Queue') ax[1].xaxis.set_major_locator(mpl.dates.AutoDateLocator()) ax[1].xaxis.set_major_formatter(mpl.dates.AutoDateFormatter(ax[1].xaxis.get_major_locator())) ax[1].set_xlim(mpl.dates.date2num(start_date), mpl.dates.date2num(end_date)) plt.setp(ax[1].xaxis.get_majorticklabels(), rotation=25, ha='right') ax[1].set_xlabel('Date') ax[0].set_ylabel('Area per Day') ax[1].set_ylabel('Area per Day') fig.savefig('out/figs/nci_area_day_sub.png') plt.show()

我还是NumPy的新手，想知道我是否可以改进：

def binsum(bin_by, sum_by, bins): bin_index = np.digitize(bin_by, bins) sums = [np.sum(sum_by[bin_index==i]) for i in range(len(bins))] return sums

所以它不使用Python列表。

有可能以某种方式爆炸sum_by[bin_index==i]所以我得到一个数组数组，长度为len(bins) ？ 然后np.sum()将返回一个numpy数组。

I have some data with two columns per row. In my case job submission time and area.

I have used matplotlib's hist function to produce a graph with time binned by day on the x axis, and count per day on the y axis:

import numpy as np import matplotlib.pyplot as plt import matplotlib as mpl import datetime as dt def timestamp_to_mpl(timestamp): return mpl.dates.date2num(dt.datetime.fromtimestamp(timestamp)) nci_file_name = 'out/nci.csv' jobs = np.genfromtxt(nci_file_name, dtype=int, delimiter=',', names=True, usecols(1,2,3,4,5)) fig, ax = plt.subplots(2, 1, sharex=True) vect_timestamp_to_mpl = np.vectorize(timestamp_to_mpl) qtime = vect_timestamp_to_mpl(jobs['queued_time']) start_date = dt.datetime(2013, 1, 1) end_date = dt.datetime(2013, 4, 1) bins = mpl.dates.drange(start_date, end_date, dt.timedelta(days=1)) ax[0].hist(qtime[jobs['charge_rate']==1], bins=bins, label='Normal', color='b') ax[1].hist(qtime[jobs['charge_rate']==3], bins=bins, label='Express', color='g') ax[0].grid(True) ax[1].grid(True) fig.suptitle('NCI Workload Submission Daily Rate') ax[0].set_title('Normal Queue') ax[1].set_title('Express Queue') ax[1].xaxis.set_major_locator(mpl.dates.AutoDateLocator()) ax[1].xaxis.set_major_formatter(mpl.dates.AutoDateFormatter(ax[1].xaxis.get_major_locator())) ax[1].set_xlim(mpl.dates.date2num(start_date), mpl.dates.date2num(end_date)) plt.setp(ax[1].xaxis.get_majorticklabels(), rotation=25, ha='right') ax[1].set_xlabel('Date') ax[0].set_ylabel('Jobs per Day') ax[1].set_ylabel('Jobs per Day') fig.savefig('out/figs/nci_sub_rate_day_sub.png') plt.show()

I now want a graph with time binned by day on the x axis and the summed by bin area on the y axis.

So far I have come up with this using a list comprehension:

import numpy as np import matplotlib.pyplot as plt import matplotlib as mpl import datetime as dt def timestamp_to_mpl(timestamp): return mpl.dates.date2num(dt.datetime.fromtimestamp(timestamp)) def binsum(bin_by, sum_by, bins): bin_index = np.digitize(bin_by, bins) sums = [np.sum(sum_by[bin_index==i]) for i in range(len(bins))] return sums fig, ax = plt.subplots(2, 1, sharex=True) vect_timestamp_to_mpl = np.vectorize(timestamp_to_mpl) qtime = vect_timestamp_to_mpl(jobs['queued_time']) area = jobs['run_time'] * jobs['req_procs'] start_date = dt.datetime(2013, 1, 1) end_date = dt.datetime(2013, 4, 1) delta = dt.timedelta(days=1) bins = mpl.dates.drange(start_date, end_date, delta) sums_norm = binsum(qtime[jobs['charge_rate']==1], area[jobs['charge_rate']==1], bins) sums_expr = binsum(qtime[jobs['charge_rate']==3], area[jobs['charge_rate']==3], bins) ax[0].bar(bins, sums_norm, width=1.0, label='Normal', color='b') ax[1].bar(bins, sums_expr, width=1.0, label='Express', color='g') ax[0].grid(True) ax[1].grid(True) fig.suptitle('NCI Workload Area Daily Rate') ax[0].set_title('Normal Queue') ax[1].set_title('Express Queue') ax[1].xaxis.set_major_locator(mpl.dates.AutoDateLocator()) ax[1].xaxis.set_major_formatter(mpl.dates.AutoDateFormatter(ax[1].xaxis.get_major_locator())) ax[1].set_xlim(mpl.dates.date2num(start_date), mpl.dates.date2num(end_date)) plt.setp(ax[1].xaxis.get_majorticklabels(), rotation=25, ha='right') ax[1].set_xlabel('Date') ax[0].set_ylabel('Area per Day') ax[1].set_ylabel('Area per Day') fig.savefig('out/figs/nci_area_day_sub.png') plt.show()

I am still new to NumPy and would like to know if I can improve:

def binsum(bin_by, sum_by, bins): bin_index = np.digitize(bin_by, bins) sums = [np.sum(sum_by[bin_index==i]) for i in range(len(bins))] return sums

So it doesn't use Python lists.

Is it possible to somehow explode out sum_by[bin_index==i] so I get an array of arrays, with length len(bins)? Then np.sum() would return a numpy array.

最满意答案

Matplotlib的hist函数和NumPy的histogram函数都有一个weights可选的关键字参数。 我认为在您的第一个代码中唯一相关的更改行应该看起来像：

ax[0].hist(qtime[jobs['charge_rate']==1], weights=area[jobs['charge_rate']==1], bins=bins, label='Normal', color='b') ax[1].hist(qtime[jobs['charge_rate']==3], weights=area[jobs['charge_rate']==3], bins=bins, label='Express', color='g')

Both Matplotlib's hist function and NumPy's histogram function have a weights optional keyword argument. I think the only relevant lines to change in your first code should end up looking like:

ax[0].hist(qtime[jobs['charge_rate']==1], weights=area[jobs['charge_rate']==1], bins=bins, label='Normal', color='b') ax[1].hist(qtime[jobs['charge_rate']==3], weights=area[jobs['charge_rate']==3], bins=bins, label='Express', color='g')