As Kayleigh mentioned, Adam, camera shake can be a real problem and I suspect this could be one of your chief difficulties.
With that lens I wouldn't really like to drop below 1/200 shutter speed when handheld. And, of course, a sufficient speed is needed where there is movement from your subject. For instance, I would use at least 1/1000 for a slowly flying bird.
Also, although you are just using the centre focusing point auto focus can still be a problem in some circumstances and manual focusing might give better results. Subject to having sufficient time to focus.
Using Evaluative metering can cause problems with some mixed lighting conditions, so occasionally you will have to also use a bit of Exposure Compensation or Spot Metering.
However, showing us a shot straight from the camera with full shooting details, may help to diagnose your exact problems.