Depending your needs and your budgets there are a lot of different options. "Affordable" means different things to different people. Many consumer GPS hand held devices like you can get at Best Buy or REI have OK Lat and Long resolution but the elevation accuracy isn't great (~$300-500). If your GCP work doesn't require accurate elevations then that might be fine. Beyond that, the sky is the limit. You can get into a Trimble handheld for $1500 or one of their larger systems for $8k. The more you spend the better the accuracy will be. The question is, how much accuracy do you NEED?
We don't use a GPS at all to create our GCPs we test with. You can right click on the maps in your Map Detail page to get the coordinates on the screen. If you hide the layer you created and look at the basemap you can choose features to use as GCPs and get elevation values from Google Earth. This makes your created maps line up perfectly with the basemap.
Hopefully that helps.
