Official comment
It sounds like you are doing everything properly. We are using SRTM data to get an above sea level offset for the takeoff point. Then all of the above takeoff point altitudes that get measured with the accurate barometric altimeter get that offset added to them to give a more accurate final product.
The source of your error could be a discrepancy between your surveyed points and the SRTM data (which is known to be noisy to a few meters) or the SRTM data could not be an accurate reflection of the takeoff location due to dirt being moved. We always recommend taking off from undisturbed earth to make sure the SRTM value will be as accurate as possible. If the location has had dirt moved around since 2000 the SRTM data will likely be off.
Also, getting two GPSs to agree on a height is like the old joke about how to get two drummers to play together: shoot one. It is likely that if you want back to that exact same location again with the same GPS, the elevation reading would be a few meters different every time.