This is a very valuable advise. I have a friend from a neighboring country. She moved to another rented place where the landlord knew me too. She always told me to go over to her place during her off days. But I did not want to keep going over as if I am out of place or having no other places to go.
Your grandpa made a good point. Keep visiting people and you are the one indebted to them. There is a Muslim advise from the prophet that we should not frequent visitation to other people's house, although it is encouraged to keep in touch with one another.