Several things come to mind....
Is there a valid reason for his correspondence with his ex? Co-parenting maybe? If not, did you explain that it bothered you that he was corresponding with his ex?
For me, if someone deletes messages from anyone, I think it's a red flag. I don't go through my significant other's phone, but I also trust them. Should I pick his phone up because I needed to use it for whatever reason, my phone was dead or something, it shouldn't be a problem. That's just me though.
Has he ever given you a reason mistrust him, thus giving a reason to go through his phone? I believe you are both wrong. I would explain why I wasn't okay with the communication, and if it was a valid reason, I would trust him to make a compromise on the situation to satisfy both of us.
I'm thinking 'bout how people fall in love in mysterious ways, Maybe it's all part of a plan, Well, I'll just keep on making the same mistakes, Hoping that you'll understand--Ed Sheeran