Method 1 Fix your Fence Post with Concrete
First off, dig up the soil around the fence post to expose the existing concrete base. Mix up some concrete and water in a wheelbarrow and have it ready to go, it should be the consistency of oatmeal. Re-level the fence post and add a support beam to hold it in place.
With the post supported, fill the hole with the concrete compound to about an inch below ground level. Work the air out of it with your shovel to ensure that it fills the hole completely. Once the concrete has hardened, you can add soil and grass seed over top to cover it.