Hello, I have had the problem for some time and I have not found a solution ... It happens in my Nexus 6P as well as in my wife's Nexus 5X.
The problem is when listening to audio messages on WhatsApp, when listening to them through the internal headset (holding the cell phone to your ear) instead activates the speaker and at a high volume, without allowing to download it with the volume buttons.
The problem is long-standing, I would say that since they were updated to Android 8.1. Since I have another cell phone that I use more frequently, it did not affect me much, but lately they are sending voice messages to me and it is starting to bother me not being able to listen in private.