I would get a domain at
http://godaddy.com, they've got good prices and you can find a lot of discount coupons online:
http://www.retailmenot.com/view/godaddy.comYou have to purchase a server separately. The domain is payed on a yearly basis at least, will be around $7-$10 for a .com domain, you can get a .info for about $1 at godaddy (note that the renewal of those .info domains costs more). The server is payed on at least a monthly basis, the price depends on the kind of server you pick. If you get shared hosting you can basically host a website, run php scripts, use databases, ... Reseller hosting allows you to do the same with the added ability that you can creates subhosting accounts which you could for example sell to other people. Then there's vps hosting, which gives you root access to a virtual server which you can install any operating system onto (linux unless you have a windows server), and you can install any piece of software onto it you like. If you for example want subversion repositories then you will probably need to get a vps as most if not all hosts don't allow it on shared hosting, where as you can install it perfectly fine on a vps. Finally there's dedicated hosting, which you probably shouldn't get, it's the most expensive, basically gives you an entire server, not visualized. this site is hosted on a vps server.
If you get vps or dedicated hosting you should consider if you want it managed or unmanaged, unmanaged is the cheapest, but you have to install all software yourself and if it doesn't work you have to fix it (unless there's a problem with the server itself of course). Managed hosting is a lot more costy, but you can ask the hosting company to basically do everything for you, install software and stuff... Basically you pay extra for the service.
I only have experience with 2 hosting companies, I can recommend them though, they're pretty good, certainly Knownhost, they're both not the cheapest, but cheap hosts are often overselling which results in bad servers.
At knownhost you can get shared hosting, and managed vps/hybrid/dedi hosting.
Shared:
http://knownhost.com/shared-hosting.htmlVPS:
http://knownhost.com/vps_packages.htmlHybrid:
http://knownhost.com/hybrid_packages.htmlDedicated:
http://knownhost.com/dedicated-servers.htmlThen there's FsckVPS' unmanaged VPS hosting:
http://fsckvps.com/hosting/I would started with simple shared hosting and move up as you need more processing power/memory/...