love problems in the world

I am dating this guy. We have known each other since we were little. We became really close like three years ago, and started dating just five months ago. We live in different cities (like 50 minutes apart from each other), but we still see each other almost every weekend. And we do a lot of Skype.
We are both in college so we are pretty busy. I also have a job and work a few hours a week.
Now to the point. Last weekend, I asked him not to come see me. I just kind of wanted a break. Time for myself. Time with my friends (whom I hadn't hung out with more than twice since we started dating). And it really has nothing to do with him. I just needed some "me time." Yesterday we were talking and he said that sometimes he felt like he misses me more than I miss him. He even said that he feels my attitude is indifferent about seeing him over the weekend. It made me think, how bad is that?
I know I love him. There's no doubt. I feel great when we are together. But it's true that I don't miss him all the time. And also, spending too much time with him (an entire spring break) can get annoying by the end. I feel like I am busier than he is. I care a lot about school. I have priorities and so does he, but it's like I am his priority No. 1. He's told me that he wishes to marry me some day in the future. And I do like the idea as well. But we are talking about four to five years in the future. I mean, we are 22 and 23. We are young and barely starting life. Is he getting too serious and dependent, or am I being too detached? I need some opinions.