Decompose a time series in preparation for anomaly detection

time_decompose(data, target, method = c("stl", "twitter"),
  frequency = "auto", trend = "auto", ..., merge = FALSE,
  message = TRUE)

Arguments

data

A tibble or tbl_time object.

target

A column to apply the function to

method

The time series decomposition method. One of "stl" or "twitter". The STL method uses seasonal decomposition (see decompose_stl()). The Twitter method uses trend to remove the trend (see decompose_twitter()).

frequency

Controls the seasonal adjustment (removal of seasonality). Input can be either "auto", a time-based definition (e.g. "2 weeks"), or a numeric number of observations per frequency (e.g. 10). Refer to time_frequency().

trend

Controls the trend component For stl, the trend controls the sensitivity of the lowess smoother, which is used to remove the remainder. For twitter, the trend controls the period width of the median, which are used to remove the trend and center the remainder.

...

Additional parameters passed to the underlying method functions.

merge

A boolean. FALSE by default. If TRUE, will append results to the original data.

message

A boolean. If TRUE, will output information related to tbl_time conversions, frequencies, and trend / median spans (if applicable).

Value

Returns a tbl_time object.

Details

The time_decompose() function generates a time series decomposition on tbl_time objects. The function is "tidy" in the sense that it works on data frames. It is designed to work with time-based data, and as such must have a column that contains date or datetime information. The function also works with grouped data. The function implements several methods of time series decomposition, each with benefits.

STL:

The STL method (method = "stl") implements time series decomposition using the underlying decompose_stl() function. If you are familiar with stats::stl(), the function is a "tidy" version that is designed to work with tbl_time objects. The decomposition separates the "season" and "trend" components from the "observed" values leaving the "remainder" for anomaly detection. The user can control two parameters: frequency and trend. The frequency parameter adjusts the "season" component that is removed from the "observed" values. The trend parameter adjusts the trend window (t.window parameter from stl()) that is used. The user may supply both frequency and trend as time-based durations (e.g. "6 weeks") or numeric values (e.g. 180) or "auto", which predetermines the frequency and/or trend based on the scale of the time series.

Twitter:

The Twitter method (method = "twitter") implements time series decomposition using the methodology from the Twitter AnomalyDetection package. The decomposition separates the "seasonal" component and then removes the median data, which is a different approach than the STL method for removing the trend. This approach works very well for low-growth + high seasonality data. STL may be a better approach when trend is a large factor. The user can control two parameters: frequency and trend. The frequency parameter adjusts the "season" component that is removed from the "observed" values. The trend parameter adjusts the period width of the median spans that are used. The user may supply both frequency and trend as time-based durations (e.g. "6 weeks") or numeric values (e.g. 180) or "auto", which predetermines the frequency and/or median spans based on the scale of the time series.

References

  1. CLEVELAND, R. B., CLEVELAND, W. S., MCRAE, J. E., AND TERPENNING, I. STL: A Seasonal-Trend Decomposition Procedure Based on Loess. Journal of Official Statistics, Vol. 6, No. 1 (1990), pp. 3-73.

  2. Owen S. Vallis, Jordan Hochenbaum and Arun Kejariwal (2014).A Novel Technique for Long-Term Anomaly Detection in the Cloud. Twitter Inc.

  3. Owen S. Vallis, Jordan Hochenbaum and Arun Kejariwal (2014). AnomalyDetection: Anomaly Detection UsingSeasonal Hybrid Extreme Studentized Deviate Test. R package version 1.0.

See also

Decomposition Methods (Powers time_decompose)

Time Series Anomaly Detection Functions (anomaly detection workflow):

Examples

library(dplyr) data(tidyverse_cran_downloads) # Basic Usage tidyverse_cran_downloads %>% time_decompose(count, method = "stl")
#> # A time tibble: 6,375 x 6 #> # Index: date #> # Groups: package [15] #> package date observed season trend remainder #> <chr> <date> <dbl> <dbl> <dbl> <dbl> #> 1 tidyr 2017-01-01 873. -2761. 5053. -1418. #> 2 tidyr 2017-01-02 1840. 901. 5047. -4108. #> 3 tidyr 2017-01-03 2495. 1460. 5041. -4006. #> 4 tidyr 2017-01-04 2906. 1430. 5035. -3559. #> 5 tidyr 2017-01-05 2847. 1239. 5029. -3421. #> 6 tidyr 2017-01-06 2756. 367. 5024. -2635. #> 7 tidyr 2017-01-07 1439. -2635. 5018. -944. #> 8 tidyr 2017-01-08 1556. -2761. 5012. -695. #> 9 tidyr 2017-01-09 3678. 901. 5006. -2229. #> 10 tidyr 2017-01-10 7086. 1460. 5000. 626. #> # ... with 6,365 more rows
# twitter tidyverse_cran_downloads %>% time_decompose(count, method = "twitter", frequency = "1 week", trend = "2 months", merge = TRUE, message = FALSE)
#> # A time tibble: 6,375 x 7 #> # Index: date #> # Groups: package [15] #> package date count observed season median_spans remainder #> <chr> <date> <dbl> <dbl> <dbl> <dbl> <dbl> #> 1 tidyr 2017-01-01 873. 873. -2619. 5297. -1805. #> 2 tidyr 2017-01-02 1840. 1840. 891. 5297. -4348. #> 3 tidyr 2017-01-03 2495. 2495. 1409. 5297. -4211. #> 4 tidyr 2017-01-04 2906. 2906. 1346. 5297. -3737. #> 5 tidyr 2017-01-05 2847. 2847. 1199. 5297. -3649. #> 6 tidyr 2017-01-06 2756. 2756. 247. 5297. -2788. #> 7 tidyr 2017-01-07 1439. 1439. -2472. 5297. -1386. #> 8 tidyr 2017-01-08 1556. 1556. -2619. 5297. -1122. #> 9 tidyr 2017-01-09 3678. 3678. 891. 5297. -2510. #> 10 tidyr 2017-01-10 7086. 7086. 1409. 5297. 380. #> # ... with 6,365 more rows