Small offsets are not uncommon but how are you verifying this? Against outputs from another software or surveyed GCP check points?
Regardless, there looks to be a few issues with your data collection that could be affecting your accuracy. Please review the Data Collection guidelines for tips and use Map Pilot Pro for your data collection if possible.