Yup, calling a person of japanese ancestry (or any asian for that matter) a jap is very derogatory. I think this comes from WW2 during the japanese internment here in the US. My dad absolutely hates the word but me being of a much younger generation don't have the emotional attachment to the word. Likewise my wife, who is from Japan, also doesn't think it's that bad, but recognizes it's not good.
Having said that, I cut people slack because it's very easy to think it's just a shortcut to say japanese. An honest mistake.