python 时间序列突变检测

突变检测"/>

python 时间序列突变检测

也许尝试区分时间索引并使用模式(或最小差异)作为频率.

import pandas as pd

import numpy as np

# simulate some data

# ===================================

np.random.seed(0)

dt_rng = pd.date_range('2015-03-02 00:00:00', '2015-07-19 23:00:00', freq='H')

dt_idx = pd.DatetimeIndex(np.random.choice(dt_rng, size=2000, replace=False))

df = pd.DataFrame(np.random.randn(2000), index=dt_idx, columns=['col']).sort_index()

df

col

2015-03-02 01:00:00 2.0261

2015-03-02 04:00:00 1.3325

2015-03-02 05:00:00 -0.9867

2015-03-02 06:00:00 -0.0671

2015-03-02 08:00:00 -1.1131

2015-03-02 09:00:00 0.0494

2015-03-02 10:00:00 -0.8130

2015-03-02 11:00:00 1.8453

... ...

2015-07-19 13:00:00 -0.4228

2015-07-19 14:00:00 1.1962

2015-07-19 15:00:00 1.1430

2015-07-19 16:00:00 -1.0080

2015-07-19 18:00:00 0.4009

2015-07-19 19:00:00 -1.8434

2015-07-19 20:00:00 0.5049

2015-07-19 23:00:00 -0.5349

[2000 rows x 1 columns]

# processing

# ==================================

# the gap distribution

res = (pd.Series(df.index[1:]) - pd.Series(df.index[:-1])).value_counts()

01:00:00 1181

02:00:00 499

03:00:00 180

04:00:00 93

05:00:00 24

06:00:00 10

07:00:00 9

08:00:00 3

dtype: int64

# the mode can be considered as frequency

res.index[0] # output: Timedelta('0 days 01:00:00')

# or maybe the smallest difference

res.index.min() # output: Timedelta('0 days 01:00:00')

# get full datetime rng

full_rng = pd.date_range(df.index[0], df.index[-1], freq=res.index[0])

full_rng

DatetimeIndex(['2015-03-02 01:00:00', '2015-03-02 02:00:00',

'2015-03-02 03:00:00', '2015-03-02 04:00:00',

'2015-03-02 05:00:00', '2015-03-02 06:00:00',

'2015-03-02 07:00:00', '2015-03-02 08:00:00',

'2015-03-02 09:00:00', '2015-03-02 10:00:00',

...

'2015-07-19 14:00:00', '2015-07-19 15:00:00',

'2015-07-19 16:00:00', '2015-07-19 17:00:00',

'2015-07-19 18:00:00', '2015-07-19 19:00:00',

'2015-07-19 20:00:00', '2015-07-19 21:00:00',

'2015-07-19 22:00:00', '2015-07-19 23:00:00'],

dtype='datetime64[ns]', length=3359, freq='H', tz=None)