It's called "tacit knowledge" and I think we generally overindex on explicit, formal knowledge and ignore tacit knowledge. You can see that with language learning, we treat languages like something you "learn", but in my experience it's closer to a motor skill like playing tennis.
The most recent phone numbers I actually remember are those I learned and used just before getting a smartphone. I guess tapping on a screen doesn't quite give you that same effect!
