I believe this is correct: https://www.gamefaqs.com/boards/164594-fallout-4/72860151
You can check by looking at special stat bonuses. Like wear a military cap +1 PER and then get in PA and watch it dissapear.
It's actually one of the reasons I tend to run less PA than other types of builds
I agree with Motty and Amornar. Even though you can still visibly see a head piece with an unequipped PA helmet, it offers no benefit to stats or special value. Which I believe is fair since PA is already ridiculously protective but it is a bummer that you don't retain any SPECIAL bonuses