Think of it as a corollary of the Presentment Clause. In Article I, the Framers explicitly considered what should happen if the President simply chose not to act on a bill presented to him. That is, if he didn't sign, but also didn't veto it. And they decided that, except in cases where he couldn't return it to Congress because they had adjourned, his failure to act would be treated as consent.
Now, in Article II, they did not explicitly consider what should be done if the Senate behaved in the same way. I.e., they seem to have simply assumed that the Senate would vote on any Presidential appointment, rather than try to engage in a sort of legislative pocket veto. What some have argued in this thread is the idea that, absent explicit instructions in the Article, we should look to the common law understanding of consent, and in that context, it's reasonable to cast the Senate's refusal to act as de facto consent. Furthermore, the Framers did explicitly consider what to do when the Senate couldn't act: recess appointments.
Failure of imagination, I guess, on the part of the Framers, not to figure that the Senate might one day simply refuse to do its job, but when you look at the big picture, the idea that silence qualifies as consent isn't out of left field. That said, I don't think a President should try it, and I'm glad Obama decided against it.