You are a big man.
If you eat healthy, excersize regularly, and are still big, then you are big. Call it God's design, genetics, fate, whatever you want.
Bottom line is, you can't change who you are. You can waste time trying, or you can love yourself and say "fu ck off" to anyone who says otherwise. You don't seem to be doing the former, and won't have any trouble doing the latter.
As for the department fruitloop, if you'll get in trouble for ending his ass, you'll always have plenty of friends who'll take care of it for you.