如何按一段时间将DataFrame分组?

本文介绍了如何按一段时间将DataFrame分组?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！问题描述

我从日志文件中获取了一些数据，想按分钟对条目进行分组:

def gen(date, count=10): while count > 0: yield date, "event{}".format(randint(1,9)), "source{}".format(randint(1,3)) count -= 1 date += DateOffset(seconds=randint(40)) df = DataFrame.from_records(list(gen(datetime(2012,1,1,12, 30))), index='Time', columns=['Time', 'Event', 'Source'])

df:

Event Source 2012-01-01 12:30:00 event3 source1 2012-01-01 12:30:12 event2 source2 2012-01-01 12:30:12 event2 source2 2012-01-01 12:30:29 event6 source1 2012-01-01 12:30:38 event1 source1 2012-01-01 12:31:05 event4 source2 2012-01-01 12:31:38 event4 source1 2012-01-01 12:31:44 event5 source1 2012-01-01 12:31:48 event5 source2 2012-01-01 12:32:23 event6 source1

我尝试了以下选项:

df.resample('Min')级别太高，想要汇总.

df.groupby(date_range(datetime(2012,1,1,12, 30), freq='Min', periods=4))失败，发生异常.

df.groupby(TimeGrouper(freq='Min'))正常工作，并返回DataFrameGroupBy对象以进行进一步处理，例如:

grouped = df.groupby(TimeGrouper(freq='Min')) grouped.Source.value_counts() 2012-01-01 12:30:00 source1 1 2012-01-01 12:31:00 source2 2 source1 2 2012-01-01 12:32:00 source2 2 source1 2 2012-01-01 12:33:00 source1 1

但是，没有记录TimeGrouper类.

按时间段分组的正确方法是什么?如何按分钟并按源"列对数据进行分组，例如groupby([TimeGrouper(freq='Min'), df.Source])?

解决方案

您可以对与DataFrame长度相同的任何数组/系列进行分组-甚至是实际上不是DataFrame列的计算因子.因此，您可以按分钟分组:

df.groupby(df.index.map(lambda t: t.minute))

如果要按分钟分组，则可以将上面的内容与要使用的列混合使用:

df.groupby([df.index.map(lambda t: t.minute), 'Source'])

我个人认为，如果我想经常对它们进行分组，那么只需将列添加到DataFrame来存储其中一些计算出的内容(例如，"Minute"列)会很有用，因为这使分组代码不太冗长. /p>

或者您可以尝试以下操作:

df.groupby([df['Source'],pd.TimeGrouper(freq='Min')])

I have some data from log files and would like to group entries by a minute:

def gen(date, count=10): while count > 0: yield date, "event{}".format(randint(1,9)), "source{}".format(randint(1,3)) count -= 1 date += DateOffset(seconds=randint(40)) df = DataFrame.from_records(list(gen(datetime(2012,1,1,12, 30))), index='Time', columns=['Time', 'Event', 'Source'])

df:

Event Source 2012-01-01 12:30:00 event3 source1 2012-01-01 12:30:12 event2 source2 2012-01-01 12:30:12 event2 source2 2012-01-01 12:30:29 event6 source1 2012-01-01 12:30:38 event1 source1 2012-01-01 12:31:05 event4 source2 2012-01-01 12:31:38 event4 source1 2012-01-01 12:31:44 event5 source1 2012-01-01 12:31:48 event5 source2 2012-01-01 12:32:23 event6 source1

I tried these options:

df.resample('Min') is too high level and wants to aggregate.

df.groupby(date_range(datetime(2012,1,1,12, 30), freq='Min', periods=4)) fails with exception.

df.groupby(TimeGrouper(freq='Min')) works fine and returns a DataFrameGroupBy object for further processing, e.g.:

grouped = df.groupby(TimeGrouper(freq='Min')) grouped.Source.value_counts() 2012-01-01 12:30:00 source1 1 2012-01-01 12:31:00 source2 2 source1 2 2012-01-01 12:32:00 source2 2 source1 2 2012-01-01 12:33:00 source1 1

However, the TimeGrouper class is not documented.

What is the correct way to group by a period of time? How can I group the data by a minute AND by the Source column, e.g. groupby([TimeGrouper(freq='Min'), df.Source])?

解决方案

You can group on any array/Series of the same length as your DataFrame --- even a computed factor that's not actually a column of the DataFrame. So to group by minute you can do:

df.groupby(df.index.map(lambda t: t.minute))

If you want to group by minute and something else, just mix the above with the column you want to use:

df.groupby([df.index.map(lambda t: t.minute), 'Source'])

Personally I find it useful to just add columns to the DataFrame to store some of these computed things (e.g., a "Minute" column) if I want to group by them often, since it makes the grouping code less verbose.

Or you could try something like this:

df.groupby([df['Source'],pd.TimeGrouper(freq='Min')])

更多推荐

如何按一段时间将DataFrame分组?