He's a bit... odd, to put it lightly. I mean, he's incredible, but he has his own way of doing things. I suspect he didn't want to sound creepy or like he was hitting on the girl who just walked into his shop.
Technically, he's our godfather. Monty just calls him our father around other people because it's easier to explain and he's also afraid people will find it suspicious that two young men are living with a 50-year-old eccentric recluse. Well, technically three young men, but Charlie stays with Oscar's other daughter more often than he stays with us.