You can use a good set of tools to test audio/voice transactions on mobile devices (as described here: https://www.techwell.com/techwell-insights/2018/02/testing-next-generation-digital-interfaces).
Complete audio/voice test solution
Let's split this into three levels of testing:
- basic functional testing: you can use text-in to convert text into audio and inject the resulting audio into the chatbot. You can then validate the response (both functionally and responsiveness by any of the three methods:
- Analyzing the text that shows on the screen (most voice agents respond with text as well)
- Record the audio and convert it into text, validate the text
- Record the audio and listen to it later
- advanced functional testing: you can use the audio inject API to diversify audio inputs (representing background noise, stuttering, accents etc.).
- Audio/voice quality testing: you can further measure the quality of the outbound voice based on Mean Opinion Score algorithm.
Note that this mechanism can work for Siri, Google Assistant, Alexa, In-app conversational agents (like BOFA's Erica), testing of streaming audio, testing of IVR call center, as well as testing of voice calls in different networks for telcos.
Just for example, we've created a script that measures the clarity of DTMF over a telco network: https://digitalqualityblog.wordpress.com/2017/12/04/validating-audio-and-dtmf-on-voice-calls/
- Create a voice call from one device to another
- Inject a known audio file, representing the DTMF sequence 12345#*
- Record the audio on the other device
- Hang the call
- Use a mobile app (the likes of “DTMF Transceiver”)
- Inject the recorded audio and collect the detected sequence from the app