The majority of home wireless routers have four wired ports. These should be used for computers close enough to plug in directly (they will be somewhat faster than the wireless). Use only one of the outputs on the modem to go to the wireless router -- it doesn't really make any difference which one.
The reason to run all of your computers (and phones, iPads, etc.) from the router is that it will take care of assigning a unique IP address to each of these devices. (If you don't understand what this means, trust me, this is a good thing.)