Marshalling text portably in Common Lisp
2023-02-16
----------------------------------------

My need was to create textual messages in a simple network protocol
I'd developed.

http://www.lispworks.com/documentation/HyperSpec/Body/f_wr_pr.htm
Common Lisp HyperSpec (CLHS) section 22.4
says,
"The general rule is that output from princ is intended to look good to
people, while output from prin1 is intended to be acceptable to read."

Nothing here says that princ and prin1 aren't allowed to introduce
whitespace/newlines at will. Looking good to a human is subjective and
the Lisp reader is flexible about whitespace.

format's ~A and ~S are princ/prin1 underneath, so what I wrote is also
true for format.

So the only portably correct way for me to create my protocol messages
was to use write-char/write-sequence e.g.:
(with-output-to-string (o)
  (write-char #\a o)
  (write-char #\Space o)
  (write-sequence "sometext" o))
For example in CLISP I didn't originally expect a newline after A below,
but then I realised it was legal for CLISP to put one there:
(with-output-to-string (o)
  (princ "A" o)
  (princ "B
C" o))
"A
B
C"

What all this also taught me was that any Common Lisp (library) code
that uses princ/prin1 or their derivatives for marshalling data into a
precise structure is probably broken (in the sense of not being _Common_
Lisp but implementation specific Lisp).

----
An unrelated footnote about the "general rule" from the HyperSpec that
I quoted above:
prin1 needn't _necessarily_ produce output suitable for the reader,
even when you'd imagine it could. Setting/binding *print-readably*
explicitly will solve that.
As the same HyperSpec page says,
"The functions prin1 and print do not bind *print-readably*."