Just a couple of comments to clarify some things:
a) In a sense, all ports are in paralell: most lines like VCC, VBat, GND, Clock, Data, Instructions and Address (ISA), and Flag Input (FI) go to all ports. There are a couple of contacts on each port which define the port address (each contact may or may not be connected , allowing for the four combinations 00, 01, 10, 11).
b) The backplane or Input/Output assembly, which is a plastic piece wrapped with a flexible printed circuit going to the ports and also to the battery contacts is a usual suspect in HP41 problems. Both dirty contacts, corrossion or "simply" lack of the right pressure may affect the quality of contacts between this assembly and the main circuit board. There is a small piece of plastic foam below the contacts which are supposed to get pressed against the main board, the piece of foam tend to become dried and less elastic over time. Also any attempt to over-adjust the screws which close the case halves together, or a knock against the floor, etc. may break the plastic posts which holds the screws, making the pressure (and hence the contact) weaker.
The modules that are worrying you need a good deal of power to work, while ROM or RAM did not. Also they probably use the Flag Input (FI) line, since they are peripherals, I think (not 100% sure) that ROM-RAM don't use FI. So a good idea may be to check VCC and FI contacts first.
A careful disassembly and cleaning may be the first step, you also will obtain a good insight on how the HP 41 is built; without such knowledge is difficult even to explain possible causes.
Please look at previous and archived postings here at the MoHPC, and also in the Articles forum, I suggest you should give a good look at postings by Tony Duell, among others, to obtain extra information on what to do and how.
I hope this helps, and please disregard any idiomatic mistake...