nenmily
Subject: Using rolling average for data cleaning / remove outliers
Hi Guys,
I have a set of data which has daily and seasonal cycle and expands over ten years. I need to remove erroneous values using the rolling average of 5 pints at a time +/- 3 standard deviations. I might end up using more points for the rolling average (maximum of 24 values which represent the whole day) till I get something that just eliminates the wrong values. Also the +/- 3 standard deviations might be replaced by +/- 2 standard deviations as it suites my work.
Essentially I want to remove outliers based on a rolling average principle (averaging the whole sample would dampen the cycles I have)….
Can anyone suggest how to do this in SPSS?
Ideally, the method should show me the points it identifies as outliers before just deleting them
Thank you,
Regards,
Abhilash
Hi Guys,
I have a set of data which has daily and seasonal cycle and expands over ten years. I need to remove erroneous values using the rolling average of 5 pints at a time +/- 3 standard deviations. I might end up using more points for the rolling average (maximum of 24 values which represent the whole day) till I get something that just eliminates the wrong values. Also the +/- 3 standard deviations might be replaced by +/- 2 standard deviations as it suites my work.
Essentially I want to remove outliers based on a rolling average principle (averaging the whole sample would dampen the cycles I have)….
Can anyone suggest how to do this in SPSS?
Ideally, the method should show me the points it identifies as outliers before just deleting them
Thank you,
Regards,
Abhilash