For the Manual Rereferencing process to work you need to have the GCPs spread throughout the area, away from the edges, be taken with a high accuracy coordinate source and have at least 5 but not too many (20 or so).
If you just want things to line up nicely, you can do what we do most of the time. We don't use a GPS at all to create our GCPs we test with. You can right click on the maps in your Map Detail page to get the coordinates on the screen. If you hide the layer you created and look at the basemap you can choose features to use as GCPs and get elevation values from Google Earth. This makes your created maps line up perfectly with the basemap.