This is not currently possible but it's something that we're looking into implementing in the future. At the moment, it's only possible to have email alerts by themselves or both email and push notification alerts.