OK, mystery solved:
The problem is with the big plastic wire connector at the back of the rear offside light cluster.
It seems to be loose so if you tug at the wires it somehow connects better and the brake lights work. When you let go, the must come loose again and it breaks the connection to the brake lights. As the nearside light cluster is fed with wires running from the offside cluster, the bad connection affects both rear brake lights
Just managed to stumble across this by chance. Next I need to take the connector off and clean up all the contacts (maybe corroded?) and fit it back so that there is no movement. May need to put in some sort of packing material so the male fits more tightly into the female.
Hope this helps somebody else some days.