Just tell your EX that he is no longer included in any activities that involve you or your family.
It's your family and YOU don not need to leave the party for anyone, especially him. Unfortunately as for the graduation you won't be able to keep him from that as anyone can come to that. Just as long as he doesn't sit near you.
AND WHATEVER YOU DO DON'T LET HIM SEE YOU CRY, IT ONLY FEEDS HIS EGO.
I know I've been there and these kind of men love it when they know they can get to you. Just remember all those rotten things he did and get MAD.
Good luck and CONGRADULATIONS TO YOUR SISTER!