Would this depend on the axle loading the line had to handle? The type of terrain it ran over?

Yes
Heavier loads, higher speeds would normally require deeper ballast depth below the sleepers.
Typically on the ECML in the UK ballast depth is approx 350mm not including any allowance for cant or crossfall. That would be for speeds up to 140mph and the highest axle loads.
Lightly used branch lines running Pacer / Sprinter type units would probably have 150mm of ballast.
An additional factor would be the amount of ballast shoulder provided.
For jointed track this would be approx 200mm from end of sleeper and level with sleeper tops.
For CWR on the straight the profile would be approx 450 from end of sleeper rising to approx 150mm above sleeper before falling back to just clear of the fastenings.
Shoulder wider on curves as additional protection against buckling in heat.

Once did a ballast train wth 10 x75 ton ballast trucks on from Huddersfield Station on the down line dropping towards Mirfield.
We ran at 5 mph dropping stone, and every 5 mins we stopped as the Per Way men had emptied a truck.
The train had to stop to let the PW open the doors on the next full truck to start dropping again.
Within less than an hour we had just one full truck left, this was for Heaton Lodge Junction.
When the last truck was dropped, we sat at Mirfield Station, till the drivers releif came out.
After the train was completely discharged of stone, the guard had to examine the train for any loose stone on the trucks, and make sure it was in order to run.
The PIC of the PW, gave you a certificate of readiness, to say all doors were shut and the train was in good order to proceed.
The guard had to ring tops and get details from them of the new train status, from which he made out a drivers slip, which he gave to the driver.
After this was completed the guard would get into his EWS van, go back to the Mill, put his ticket in a go home. Another Saturday Night At The Mill completed.

We did the same once between Gilberdyke and Broomfleet and return after crossing over at Broomfleet on a renewal site. We had 25 - 30 Seacows.
The only difference was we did not stopwhile tipping just jump off and leap on the next wagon, naughty but saved big heaps of stone all over the place.