There are keyed pinheaders and unkeyed pinheaders. Keyed headers have a (usually black) plastic part around them to prevent you from plugging in the cable the wrong way. Of course this takes more space and doesn't really help if the cable is wired the wrong way.
Every module uses some sort of Bypass Capacitors. These are charged on power on and act as short term buffers for the module circuit. This way the load on the Power Supply in the case is more uniform over time and the rails are so too. Both sides win.
But what kind of capacitor should be used here? Typically you will see a combination of electrolitic "big" caps with values between 10μF (10.000nF) and 150μF (150.000 nF) depending on the power draw and a cluster of smaller ceramic caps of 100nF. This combination is superiour because you get the high frequency response of the small 100nF caps, while still profiting from the bigger capacitance of the electrolytics. The electrolytics are typically placed quite close to the 2x5 power connector, while the small ceramics (100nF) are placed directly where they are needed: at the Vcc and Vee Pins of the ICs.
Sometimes you see Ferrite Beads in Series with the ±12V coming from the 2x5 Power Connector. These are there to Reject Noise and stray signals in the Megahertz Range. Inaudible Frequencies that can still have an (audible) impact on the circuit itself.
Reverse Polarity Protection
There are multiple ways to achieve the same goal. Typicially some sort of Reverse Polarity Protection is good to have, to avoid breaking modules, when you accidentally plug them in the wrong way