The key innovation of Scryer Prolog in this respect is that the system internally, i.e., within Rust, represents each string as a sequence of raw bytes (UTF-8 e

The key innovation of Scryer Prolog in this respect is that the system internally, i.e., within Rust, represents each string as a sequence of raw bytes (UTF-8 encoded, 0-terminated), and to Prolog programs, the string appears as if it were a list of characters (i.e., atoms of length 1). So, no conversion is necessary at all: The compact internal representation is used whenever possible, and Prolog programs only see lists of characters all of the time, making common predicates such as append/3 and length/2, and most importantly DCGs, automatically usable for reasoning about strings. For example, the string "abc" takes only 4 bytes in this representation, and the unification "abc" = [a,b,c] succeeds. Indeed, we have: ?- write_canonical("abc"). '.'(a,'.'(b,'.'(c,[]))) true. On 64-bit systems, a conventional internal representation of "abc" as the compound term .(a, .(b, .(c, []))) takes 8 bytes per functor (each '.'/2 takes 8 bytes, i.e., 1 cell in the WAM), 8 bytes per character (as a pointer to the atom table, again 1 cell in the WAM), and 8 bytes for each tail. All Prolog systems before Scryer Prolog use 16 to 24 times as much memory to represent a list of characters. As of recent, Trealla Prolog also uses the efficient representation, at least for fully instantiated strings. Scryer Prolog can also represent partial strings such as [a,b,c|Ls] with the efficient encoding.