I was born in Washington, D.C. and we moved to upstate NY (outside of Albany) when I was 9. Man, did the other kids give me a hard time. Lost that accent fast. Then we moved to Westchester, north of the city and I was expecting a Bronx accent. But no real accent. I don't think that I have much of an accent, but there are some words that I just can't pronounce. Also, I've spent a lot of years overseas and speak slowly. Not as slow as a Southerner though.
My wife is Japanese and her English is good - we only argue in English and she still wins every time. She works hard on the Ls and Rs and does pretty good. But "oh really" comes out like an Irish name - "O'Leary." I never say anything.