My Nexus 6 started having intermittent volume during videos (some kind of safeguard to protect the stereo speakers if one was out,) and phone calls became impossible without a headset.
I removed the back cover, undid the T3 screws, and the speaker just clips in. Very easy to remove and replace. I put a few screws in and left the back cover off to test.
1) Removal of the back cover: There is a hole inside the SIM slot to give you leverage to push up on the cover. I didn't know this and just pried it off. I took the back cover off enough that I partially ruined the glue on the back cover, but new ones are under $10 if you're like me and need/want a replacement.
2) Removal of the hex screws: You don't need to remove the silver one! All it does is hold in the volume rocker.
3) Disconnecting the battery and other connector: One has a super soft rubber cover and the other a very stiff one. Not sure why. Pry both of those little doors off, and you can disconnect the connectors just by pulling straight up gently.
4) Separation: Make sure you don't forget to remove the SIM tray like me because it will be very confusing as to why the phone won't come apart!
5) Speaker replacement: Pry upwards on the plastic speaker housing. Make sure you line up the new speaker with the contacts