I suspect that it is slighlty more complicated. In general:- yes NR will be responsible, but there with be structures where ownership has transferred to others such as the Highways Agency, British Waterways etc. Some private owners will have even less resorces then NR to look after the structures.
Problem is that if the strcutures weren't the responsibility of someone, they would all be destroyed very quickly for hardcore, or just to improve the view.
I rather like the landscape returned to rolling green fileds, and only a railway bridge or tunnel to remind me of old ghosts. Saw a massive overgrown embankment the other day way out in the country.
Took me totally by surprise, but I worked out later that it must have been the Meon Valley line.