Bug #15
Data transfer warnings often flip/flop at the start of the period
| Status: | Closed | Start: | 2010-06-21 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assigned to: | - | % Done: | 0% |
|
| Category: | - | |||
| Target version: | - | |||
| Votes: | 3 |
Description
It's common for the data transfer warning feature to repeatedly flip/flop between "WARNING" and "RECOVERY" states near the start of the period, sending out lots of emails. This is because it's extrapolating a 30-day usage out of sometimes only minutes worth of real data.
While we do have the full set of data transfer readings in one MySQL table, this is quite large: 35.3 million rows. It long ago became too unwieldy to calculate the past 30 days usage or estimate the current 30 days usage from this large amount of data. The system was changed so that at the point where some data is recorded, it's also added to a running total for the month. These running totals are reset when the cycle starts over.
As a workaround we could just not send any warnings for the first N hours of the period. It is unlikely that someone will use an appreciable chunk of their month's quota in 6 hours, though not impossible. 50GB in 6 hours is ~18.5Mbit/sec continuously. Suppressing warnings for the first 6 hours would almost completely avoid the flip/flop warnings problem.
History
Updated by admin over 14 years ago
I believe I have fixed this. I've made it use the inexpensive check to see if it's likely there will be overage and then if this fires then the more expensive check using up to 15 days of historical data transfer records will be used.
The inexpensive check just extrapolates from the running total in the current cycle, which is a problem if we're only a short way into the cycle. For example if we're only a few hours in and you do a spike of traffic then it extrapolates the entire rest of the 30 days from this few hours, and triggers a warning even though this few hours may be an aberration. Worse, if it's an aberration then you'll get a recovery notice and then another warnings as soon as you transfer a decent amount again. This has led to some higher usage customers getting many emails flip-flopping between warning and recovery until there's enough records for the inexpensive check to settle down.
Now whenever it wants to trigger a warning I have made it use at least 15 days of actual readings and only extrapolate the next 15 days from this, instead of potentially nearly 30 days.
I'm still testing if this has the desired effect.
Updated by admin over 14 years ago
- Status changed from New to Closed
I'm satisfied that this is fixed now.