Need a bit more info than you've given us to be able to help. How exactly is your setup hooked up? Do you just have a DSL/Cable modem hooked to a switch then the two computers hooked to that? Do you have your ISPs modem hooked to a 3rd party router than the two computers hooked to that? What kind of router are we talking here?
Most DSL modems will only allow one device to be connected at a time. You need to get a 3rd party router to sit between the computers and the modem in order to have more than one online at the same time.
Comments
If you only need wired connections, get something like this
http://www.newegg.com/Product/Product.aspx?Item=N82E16833127091
If you want wireless, I'd suggest something along this line
http://www.newegg.com/Product/Product.aspx?Item=N82E16833162026
If you buy from NewEgg, please use the Icrontic affiliate link, it doesn't cost you anything but it helps Icrontic stay in business.