
Head over to your escalation policies and set a Wait time.
This will enable you to wait for 'x' minutes before alerts begin. Wait time is especially catered towards users who see incidents automatically resolving on it's own every often.
Why get an alert if an incident you know will auto-resolve in 5 mins? That's where wait time comes in :)
You maybe noticed we were working on this - https://hub.spike.sh/10