I played around a bit with chaocipher and came to a realization: You could use a simplified alphabet and set aside a few cards to use them as a flags for numbers, case, punctuation and whitespace.
Arranging them in a separate stack minimizes mental overhead, when adding the values and allows for another simple permutation.
I decided to use 24 cards in my large stacks, because it's a nice base and with 1 global flag, you get just enough symbol space for: lowercase (without c and j), uppercase and " [email protected]
My deck had a third joker, so I used it as a global flag, that is toggled, when the top card of the small stack is a joker. Thereby you could use 3 additional spaces for base 24 numbers and maybe regional symbols or mathematic/scientific notation.>>10415
Secure RSA keys should at least be 128bits long (39 decimal digits). I don't think any human can do multiplication with those https://www.recordholders.org/en/events/worldcup/2018/results.html
By using a higher base, you could try to make the numbers more memorizable (26 digits with base 24, 24 digits with base 32, 23 digits with base 64, 11 digits with base 5040).