I've heard a lot of 'blah blah' on this idea for years.
Of the vehicles I've had, downshifting to slow down didn't give me any unreasonable issues with durability vs not. Here's my experience to support why.
First vehicle to account for was a Plymouth Champ. That's the same as the Dodge Colt, made by Mitsubishi and sold as the Mirage. It was a front wheel drive compact. When I got it (used from in the family), it had the stock 3 speed automatic behind the 1.6L I4. I didn't bother downshifting it manually, and being a non-computer controlled automatic, it would downshift itself on longer, more drawn out decelerations. That transmission had gone out and been replaced before I got the car, and was in mega-slip mode when I rebuilt the engine @ ~100,000 miles. I looked at the other versions of the car, and found there was a 4 speed manual 'twin stick' (dual input ranges), and a plain 4 speed. The holes for mounting the hardware for a stick were all in my chassis, and the auto and stick shifters mounted in the same hole, the same way. I swapped for the plain 4 speed. I proceeded to flog it for over 125k miles and had loads of fun. I finally broke the clutch after burnouts and hard launches, and downshifting EVERY time, at that 125k mile mark. The friction material was still there, but one of the recoil springs on the friction disc finally gave out. Most people don't get that many miles out of driving a clutch 'normally', which usually means way too much slipping on 'normal' takeoffs and never downshifting. The car had no ABS, so without downshifting I had a choice of only so much braking or lockup, or the much more difficult mastery of modulating the braking myself, or adding in engine braking to compliment the regular brakes. My brakes lasted longer than normal, and make no mistake - I was (and really still am) a fairly aggressive driver.
I followed that car with an Astro van. Auto trans. Handled like a barn on wheels. Went through a trans mount, trans, U-joint, and a couple of sets of brakes in the same 125k mile span. Only engine braking used there was for long or exceptionally steep downhill grades, which was certainly less than 1k miles of that span.
ABS doesn't make your vehicle stop faster, it only maintains your ability to steer and control the vehicle. It also doesn't magically make your brakes bigger, which is the only way to make the brakes themselves really stop you faster. You're converting velocity (inertia) into heat, and it takes mass to absorb the heat, and surface area to apply friction to convert the inertia into heat. Engine braking compliments what your normal braking system does by basically bypassing it and directing some of that friction to the tires vs the road, and the rotating masses against the air being compressed in the engine. Oh, and if you think ABS wasn't designed and tested in situations where the engine was braking because of the gear it is in, you assume poorly. What do you think happens when people are still accelerating and need to perform a panic stop? Many won't remember to press in the clutch at such a panic, they'll just punch the brakes. Ever noticed how many people stall after a REAL panic stop? You can bet automakers considered this situation, and ABS will still do its best even when such is the case.