Upon feature request, we promptly looked in and released a new feature on our Slack app.
If any number of Slack alert(s) been sent for a new incident and you managed to resolve it from either Phone, SMS, Email or Dashboard or if the incident was automatically resolved, then the Slack message with automatically reflect the resolved state killing ambiguity on the status of an incident.
No actions are needed from your end and this will be available to all the existing and new Slack app installations.