Telegram Open Network Virtual Machine
Whitepaper by Dr. Nikolai Durov
Author: Nikolai Durov
Date: March 23, 2020
: Original whitepaper, PDF
Abstract
The aim of this text is to provide a description of the Telegram Open Network Virtual Machine (TON VM or TVM), used to execute smart contracts in the TON Blockchain.
Introduction
The primary purpose of the Telegram Open Network Virtual Machine (TON VM or TVM) is to execute smart-contract code in the TON Blockchain. TVM must support all operations required to parse incoming messages and persistent data, and to create new messages and modify persistent data.
Additionally, TVM must meet the following requirements:
- It must provide for possible future extensions and improvements while retaining backward compatibility and interoperability, because the code of a smart contract, once committed into the blockchain, must continue working in a predictable manner regardless of any future modifications to the VM.
- It must strive to attain high "(virtual) machine code" density, so that the code of a typical smart contract occupies as little persistent blockchain storage as possible.
- It must be completely deterministic. In other words, each run of the same code with the same input data must produce the same result, regardless of specific software and hardware used.1
The design of TVM is guided by these requirements. While this document describes a preliminary and experimental version of TVM,2 the backward compatibility mechanisms built into the system allow us to be relatively unconcerned with the efficiency of the operation encoding used for TVM code in this preliminary version.
TVM is not intended to be implemented in hardware (e.g., in a specialized microprocessor chip); rather, it should be implemented in software running on conventional hardware. This consideration lets us incorporate some high-level concepts and operations in TVM that would require convoluted microcode in a hardware implementation but pose no significant problems for a software implementation. Such operations are useful for achieving high code density and minimizing the byte (or storage cell) profile of smart-contract code when deployed in the TON Blockchain.
1 Overview
This chapter provides an overview of the main features and design principles of TVM. More detail on each topic is provided in subsequent chapters.
1.0 Notation for bitstrings
The following notation is used for bit strings (or bitstrings)—i.e., finite strings consisting of binary digits (bits), \texttt{0} and \texttt{1}—throughout this document.
1.0.1. Hexadecimal notation for bitstrings
When the length of a bitstring is a multiple of four, we subdivide it into groups of four bits and represent each group by one of sixteen hexadecimal digits \texttt{0}--\texttt{9}, \texttt{A}--\texttt{F} in the usual manner: \texttt{0}_{16}\leftrightarrow\texttt{0000}, \texttt{1}_{16}\leftrightarrow\texttt{0001}, \ldots, \texttt{F}_{16}\leftrightarrow\texttt{1111}. The resulting hexadecimal string is our equivalent representation for the original binary string.
1.0.2. Bitstrings of lengths not divisible by four
If the length of a binary string is not divisible by four, we augment it by one \texttt{1} and several (maybe zero) \texttt{0}s at the end, so that its length becomes divisible by four, and then transform it into a string of hexadecimal digits as described above. To indicate that such a transformation has taken place, a special "completion tag" \texttt{\_} is added to the end of the hexadecimal string. The reverse transformation (applied if the completion tag is present) consists in first replacing each hexadecimal digit by four corresponding bits, and then removing all trailing zeroes (if any) and the last \texttt{1} immediately preceding them (if the resulting bitstring is non-empty at this point).
Notice that there are several admissible hexadecimal representations for the same bitstring. Among them, the shortest one is "canonical". It can be deterministically obtained by the above procedure.
For example, \texttt{8A} corresponds to binary string \texttt{10001010}, while \texttt{8A\_} and \texttt{8A0\_} both correspond to \texttt{100010}. An empty bitstring may be represented by either ' ', '\texttt{8\_}', '\texttt{0\_}', '\texttt{\_}', or '\texttt{00\_}'.
1.0.3. Emphasizing that a string is a hexadecimal representation of a bitstring
Sometimes we need to emphasize that a string of hexadecimal digits (with or without a \texttt{\_} at the end) is the hexadecimal representation of a bitstring. In such cases, we either prepend \texttt{x} to the resulting string (e.g., \texttt{x8A}), or prepend \texttt{x\{} and append \texttt{\}} (e.g., \texttt{x\{2D9\_\}}, which is \texttt{00101101100}). This should not be confused with hexadecimal numbers, usually prepended by \texttt{0x} (e.g., \texttt{0x2D9} or \texttt{0x2d9}, which is the integer 729).
1.0.4. Serializing a bitstring into a sequence of octets
When a bitstring needs to be represented as a sequence of 8-bit bytes (octets), which take values in integers 0\ldots255, this is achieved essentially in the same fashion as above: we split the bitstring into groups of eight bits and interpret each group as the binary representation of an integer 0\ldots255. If the length of the bitstring is not a multiple of eight, the bitstring is augmented by a binary \texttt{1} and up to seven binary \texttt{0}s before being split into groups. The fact that such a completion has been applied is usually reflected by a "completion tag" bit.
For instance, \texttt{00101101100} corresponds to the sequence of two octets (\texttt{0x2d}, \texttt{0x90}) (hexadecimal), or (45,144) (decimal), along with a completion tag bit equal to \texttt{1} (meaning that the completion has been applied), which must be stored separately.
In some cases, it is more convenient to assume the completion is enabled by default rather than store an additional completion tag bit separately. Under such conventions, 8n-bit strings are represented by n+1 octets, with the last octet always equal to \texttt{0x80}=128.
1.1 TVM is a stack machine
First of all, TVM is a stack machine. This means that, instead of keeping values in some "variables" or "general-purpose registers", they are kept in a (LIFO) stack, at least from the "low-level" (TVM) perspective.3
Most operations and user-defined functions take their arguments from the top of the stack, and replace them with their result. For example, the integer addition primitive (built-in operation) \texttt{ADD} does not take any arguments describing which registers or immediate values should be added together and where the result should be stored. Instead, the two top values are taken from the stack, they are added together, and their sum is pushed into the stack in their place.
1.1.1. TVM values
The entities that can be stored in the TVM stack will be called TVM values, or simply values for brevity. They belong to one of several predefined value types. Each value belongs to exactly one value type. The values are always kept on the stack along with tags uniquely determining their types, and all built-in TVM operations (or primitives) only accept values of predefined types.
For example, the integer addition primitive \texttt{ADD} accepts only two integer values, and returns one integer value as a result. One cannot supply \texttt{ADD} with two strings instead of two integers expecting it to concatenate these strings or to implicitly transform the strings into their decimal integer values; any attempt to do so will result in a run-time type-checking exception.
1.1.2. Static typing, dynamic typing, and run-time type checking
In some respects TVM performs a kind of dynamic typing using run-time type checking. However, this does not make the TVM code a "dynamically typed language" like PHP or Javascript, because all primitives accept values and return results of predefined (value) types, each value belongs to strictly one type, and values are never implicitly converted from one type to another. If, on the other hand, one compares the TVM code to the conventional microprocessor machine code, one sees that the TVM mechanism of value tagging prevents, for example, using the address of a string as a number—or, potentially even more disastrously, using a number as the address of a string—thus eliminating the possibility of all sorts of bugs and security vulnerabilities related to invalid memory accesses, usually leading to memory corruption and segmentation faults. This property is highly desirable for a VM used to execute smart contracts in a blockchain. In this respect, TVM's insistence on tagging all values with their appropriate types, instead of reinterpreting the bit sequence in a register depending on the needs of the operation it is used in, is just an additional run-time type-safety mechanism.
An alternative would be to somehow analyze the smart-contract code for type correctness and type safety before allowing its execution in the VM, or even before allowing it to be uploaded into the blockchain as the code of a smart contract. Such a static analysis of code for a Turing-complete machine appears to be a time-consuming and non-trivial problem (likely to be equivalent to the stopping problem for Turing machines), something we would rather avoid in a blockchain smart-contract context.
One should bear in mind that one always can implement compilers from statically typed high-level smart-contract languages into the TVM code (and we do expect that most smart contracts for TON will be written in such languages), just as one can compile statically typed languages into conventional machine code (e.g., x86 architecture). If the compiler works correctly, the resulting machine code will never generate any run-time type-checking exceptions. All type tags attached to values processed by TVM will always have expected values and may be safely ignored during the analysis of the resulting TVM code, apart from the fact that the run-time generation and verification of these type tags by TVM will slightly slow down the execution of the TVM code.
1.1.3. Preliminary list of value types
A preliminary list of value types supported by TVM is as follows:
- Integer — Signed 257-bit integers, representing integer numbers in the range
-2^{256}\ldots2^{256}-1, as well as a special "not-a-number" value\texttt{NaN}. - Cell — A TVM cell consists of at most 1023 bits of data, and of at most four references to other cells. All persistent data (including TVM code) in the TON Blockchain is represented as a collection of TVM cells.
- Tuple — An ordered collection of up to 255 components, having arbitrary value types, possibly distinct. May be used to represent non-persistent values of arbitrary algebraic data types.
- Null — A type with exactly one value
\bot, used for representing empty lists, empty branches of binary trees, absence of return value in some situations, and so on. - Slice — A TVM cell slice, or slice for short, is a contiguous "sub-cell" of an existing cell, containing some of its bits of data and some of its references. Essentially, a slice is a read-only view for a subcell of a cell. Slices are used for unpacking data previously stored (or serialized) in a cell or a tree of cells.
- Builder — A TVM cell builder, or builder for short, is an "incomplete" cell that supports fast operations of appending bitstrings and cell references at its end. Builders are used for packing (or serializing) data from the top of the stack into new cells (e.g., before transferring them to persistent storage).
- Continuation — Represents an "execution token" for TVM, which may be invoked (executed) later. As such, it generalizes function addresses (i.e., function pointers and references), subroutine return addresses, instruction pointer addresses, exception handler addresses, closures, partial applications, anonymous functions, and so on.
This list of value types is incomplete and may be extended in future revisions of TVM without breaking the old TVM code, due mostly to the fact that all originally defined primitives accept only values of types known to them and will fail (generate a type-checking exception) if invoked on values of new types. Furthermore, existing value types themselves can also be extended in the future: for example, 257-bit Integer might become 513-bit LongInteger, with originally defined arithmetic primitives failing if either of the arguments or the result does not fit into the original subtype Integer. Backward compatibility with respect to the introduction of new value types and extension of existing value types will be discussed later.
1.2 Categories of TVM instructions
TVM instructions, also called primitives and sometimes (built-in) operations, are the smallest operations atomically performed by TVM that can be present in the TVM code. They fall into several categories, depending on the types of values they work on. The most important of these categories are:
- Stack (manipulation) primitives — Rearrange data in the TVM stack, so that the other primitives and user-defined functions can later be called with correct arguments. Unlike most other primitives, they are polymorphic, i.e., work with values of arbitrary types.
- Tuple (manipulation) primitives — Construct, modify, and decompose Tuples. Similarly to the stack primitives, they are polymorphic.
- Constant or literal primitives — Push into the stack some "constant" or "literal" values embedded into the TVM code itself, thus providing arguments to the other primitives. They are somewhat similar to stack primitives, but are less generic because they work with values of specific types.
- Arithmetic primitives — Perform the usual integer arithmetic operations on values of type Integer.
- Cell (manipulation) primitives — Create new cells and store data in them (cell creation primitives) or read data from previously created cells (cell parsing primitives). Because all memory and persistent storage of TVM consists of cells, these cell manipulation primitives actually correspond to "memory access instructions" of other architectures. Cell creation primitives usually work with values of type Builder, while cell parsing primitives work with Slices.
- Continuation and control flow primitives — Create and modify Continuations, as well as execute existing Continuations in different ways, including conditional and repeated execution.
- Custom or application-specific primitives — Efficiently perform specific high-level actions required by the application (in our case, the TON Blockchain), such as computing hash functions, performing elliptic curve cryptography, sending new blockchain messages, creating new smart contracts, and so on. These primitives correspond to standard library functions rather than microprocessor instructions.
1.3 Control registers
While TVM is a stack machine, some rarely changed values needed in almost all functions are better passed in certain special registers, and not near the top of the stack. Otherwise, a prohibitive number of stack reordering operations would be required to manage all these values.
To this end, the TVM model includes, apart from the stack, up to 16 special control registers, denoted by \texttt{c0} to \texttt{c15}, or \texttt{c}(0) to \texttt{c}(15). The original version of TVM makes use of only some of these registers; the rest may be supported later.
1.3.1. Values kept in control registers
The values kept in control registers are of the same types as those kept on the stack. However, some control registers accept only values of specific types, and any attempt to load a value of a different type will lead to an exception.
1.3.2. List of control registers
The original version of TVM defines and uses the following control registers:
\texttt{c0}— Contains the next continuation or return continuation (similar to the subroutine return address in conventional designs). This value must be a Continuation.\texttt{c1}— Contains the alternative (return) continuation; this value must be a Continuation. It is used in some (experimental) control flow primitives, allowing TVM to define and call "subroutines with two exit points".\texttt{c2}— Contains the exception handler. This value is a Continuation, invoked whenever an exception is triggered.\texttt{c3}— Contains the current dictionary, essentially a hashmap containing the code of all functions used in the program. This value is also a Continuation, not a Cell as one might expect.\texttt{c4}— Contains the root of persistent data, or simply the data. This value is a Cell. When the code of a smart contract is invoked,\texttt{c4}points to the root cell of its persistent data kept in the blockchain state. If the smart contract needs to modify this data, it changes\texttt{c4}before returning.\texttt{c5}— Contains the output actions. It is also a Cell initialized by a reference to an empty cell, but its final value is considered one of the smart contract outputs. For instance, the\texttt{SENDMSG}primitive, specific for the TON Blockchain, simply inserts the message into a list stored in the output actions.\texttt{c7}— Contains the root of temporary data. It is a Tuple, initialized by a reference to an empty Tuple before invoking the smart contract and discarded after its termination.4
More control registers may be defined in the future for specific TON Blockchain or high-level programming language purposes, if necessary.
1.4 Total state of TVM (SCCCG)
The total state of TVM consists of the following components:
- Stack — Contains zero or more values, each belonging to one of value types.
- Control registers
\mathit{c0}–\mathit{c15}— Contain some specific values. (Only seven control registers are used in the current version.) - Current continuation
\mathit{cc}— Contains the current continuation (i.e., the code that would be normally executed after the current primitive is completed). This component is similar to the instruction pointer register (\texttt{ip}) in other architectures. - Current codepage
\mathit{cp}— A special signed 16-bit integer value that selects the way the next TVM opcode will be decoded. For example, future versions of TVM might use different codepages to add new opcodes while preserving backward compatibility. - Gas limits
\mathit{gas}— Contains four signed 64-bit integers: the current gas limitg_l, the maximal gas limitg_m, the remaining gasg_r, and the gas creditg_c. Always0\leq g_l\leq g_m,g_c\geq0, andg_r\leq g_l+g_c;g_cis usually initialized by zero,g_ris initialized byg_l+g_cand gradually decreases as the TVM runs. Wheng_rbecomes negative or if the final value ofg_ris less thang_c, an out of gas exception is triggered.
Notice that there is no "return stack" containing the return addresses of all previously called but unfinished functions. Instead, only control register \texttt{c0} is used.
Also notice that there are no general-purpose registers, because TVM is a stack machine. So the above list, which can be summarized as "stack, control, continuation, codepage, and gas" (SCCCG), similarly to the classical SECD machine state ("stack, environment, control, dump"), is indeed the total state of TVM.5
1.5 Integer arithmetic
All arithmetic primitives of TVM operate on several arguments of type Integer, taken from the top of the stack, and return their results, of the same type, into the stack. Recall that Integer represents all integer values in the range -2^{256}\leq x<2^{256}, and additionally contains a special value \texttt{NaN} ("not-a-number").
If one of the results does not fit into the supported range of integers—or if one of the arguments is a \texttt{NaN}—then this result or all of the results are replaced by a \texttt{NaN}, and (by default) an integer overflow exception is generated. However, special "quiet" versions of arithmetic operations will simply produce \texttt{NaN}s and keep going. If these \texttt{NaN}s end up being used in a "non-quiet" arithmetic operation, or in a non-arithmetic operation, an integer overflow exception will occur.
1.5.1. Absence of automatic conversion of integers
Notice that TVM Integers are "mathematical" integers, and not 257-bit strings interpreted differently depending on the primitive used, as is common for other machine code designs. For example, TVM has only one multiplication primitive \texttt{MUL}, rather than two (\texttt{MUL} for unsigned multiplication and \texttt{IMUL} for signed multiplication) as occurs, for example, in the popular x86 architecture.
1.5.2. Automatic overflow checks
Notice that all TVM arithmetic primitives perform overflow checks of the results. If a result does not fit into the Integer type, it is replaced by a \texttt{NaN}, and (usually) an exception occurs. In particular, the result is not automatically reduced modulo 2^{256} or 2^{257}, as is common for most hardware machine code architectures.
1.5.3. Custom overflow checks
In addition to automatic overflow checks, TVM includes custom overflow checks, performed by primitives \texttt{FITS} n and \texttt{UFITS} n, where 1\leq n\leq256. These primitives check whether the value on (the top of) the stack is an integer x in the range -2^{n-1}\leq x<2^{n-1} or 0\leq x<2^n, respectively, and replace the value with a \texttt{NaN} and (optionally) generate an integer overflow exception if this is not the case. This greatly simplifies the implementation of arbitrary n-bit integer types, signed or unsigned: the programmer or the compiler must insert appropriate \texttt{FITS} or \texttt{UFITS} primitives either after each arithmetic operation (which is more reasonable, but requires more checks) or before storing computed values and returning them from functions. This is important for smart contracts, where unexpected integer overflows happen to be among the most common source of bugs.
1.5.4. Reduction modulo 2^n
TVM also has a primitive \texttt{MODPOW2} n, which reduces the integer at the top of the stack modulo 2^n, with the result ranging from 0 to 2^n-1.
1.5.5. Integer is 257-bit, not 256-bit
One can understand now why TVM's Integer is (signed) 257-bit, not 256-bit. The reason is that it is the smallest integer type containing both signed 256-bit integers and unsigned 256-bit integers, which does not require automatic reinterpreting of the same 256-bit string depending on the operation used.
1.5.6. Division and rounding
The most important division primitives are \texttt{DIV}, \texttt{MOD}, and \texttt{DIVMOD}. All of them take two numbers from the stack, x and y (y is taken from the top of the stack, and x is originally under it), compute the quotient q and remainder r of the division of x by y (i.e., two integers such that x=yq+r and |r|<|y|), and return either q, r, or both of them. If y is zero, then all of the expected results are replaced by \texttt{NaN}s, and (usually) an integer overflow exception is generated.
The implementation of division in TVM somewhat differs from most other implementations with regards to rounding. By default, these primitives round to -\infty, meaning that q=\lfloor x/y\rfloor, and r has the same sign as y. (Most conventional implementations of division use "rounding to zero" instead, meaning that r has the same sign as x.) Apart from this "floor rounding", two other rounding modes are available, called "ceiling rounding" (with q=\lceil x/y\rceil, and r and y having opposite signs) and "nearest rounding" (with q=\lfloor x/y+1/2\rfloor and |r|\leq|y|/2). These rounding modes are selected by using other division primitives, with letters \texttt{C} and \texttt{R} appended to their mnemonics. For example, \texttt{DIVMODR} computes both the quotient and the remainder using rounding to the nearest integer.
1.5.7. Combined multiply-divide, multiply-shift, and shift-divide operations
To simplify implementation of fixed-point arithmetic, TVM supports combined multiply-divide, multiply-shift, and shift-divide operations with double-length (i.e., 514-bit) intermediate product. For example, \texttt{MULDIVMODR} takes three integer arguments from the stack, a, b, and c, first computes ab using a 514-bit intermediate result, and then divides ab by c using rounding to the nearest integer. If c is zero or if the quotient does not fit into Integer, either two \texttt{NaN}s are returned, or an integer overflow exception is generated, depending on whether a quiet version of the operation has been used. Otherwise, both the quotient and the remainder are pushed into the stack.
2 The stack
This chapter contains a general discussion and comparison of register and stack machines, expanded further in Appendix C, and describes the two main classes of stack manipulation primitives employed by TVM: the basic and the compound stack manipulation primitives. An informal explanation of their sufficiency for all stack reordering required for correctly invoking other primitives and user-defined functions is also provided. Finally, the problem of efficiently implementing TVM stack manipulation primitives will be discussed.
2.1 Stack calling conventions
A stack machine, such as TVM, uses the stack—and especially the values near the top of the stack—to pass arguments to called functions and primitives (such as built-in arithmetic operations) and receive their results. This section discusses the TVM stack calling conventions, introduces some notation, and compares TVM stack calling conventions with those of certain register machines.
2.1.1. Notation for "stack registers"
Recall that a stack machine, as opposed to a more conventional register machine, lacks general-purpose registers. However, one can treat the values near the top of the stack as a kind of "stack registers".
We denote by \texttt{s0} or \texttt{s}(0) the value at the top of the stack, by \texttt{s1} or \texttt{s}(1) the value immediately under it, and so on. The total number of values in the stack is called its depth. If the depth of the stack is n, then \texttt{s}(0), \texttt{s}(1), \ldots, \texttt{s}(n-1) are well-defined, while \texttt{s}(n) and all subsequent \texttt{s}(i) with i>n are not. Any attempt to use \texttt{s}(i) with i\geq n should produce a stack underflow exception.
A compiler, or a human programmer in "TVM code", would use these "stack registers" to hold all declared variables and intermediate values, similarly to the way general-purpose registers are used on a register machine.
2.1.2. Pushing and popping values
When a value x is pushed into a stack of depth n, it becomes the new \texttt{s0}; at the same time, the old \texttt{s0} becomes the new \texttt{s1}, the old \texttt{s1}—the new \texttt{s2}, and so on. The depth of the resulting stack is n+1.
Similarly, when a value x is popped from a stack of depth n\geq1, it is the old value of \texttt{s0} (i.e., the old value at the top of the stack). After this, it is removed from the stack, and the old \texttt{s1} becomes the new \texttt{s0} (the new value at the top of the stack), the old \texttt{s2} becomes the new \texttt{s1}, and so on. The depth of the resulting stack is n-1.
If originally n=0, then the stack is empty, and a value cannot be popped from it. If a primitive attempts to pop a value from an empty stack, a stack underflow exception occurs.
2.1.3. Notation for hypothetical general-purpose registers
In order to compare stack machines with sufficiently general register machines, we will denote the general-purpose registers of a register machine by \texttt{r0}, \texttt{r1}, and so on, or by \texttt{r}(0), \texttt{r}(1), \ldots, \texttt{r}(n-1), where n is the total number of registers. When we need a specific value of n, we will use n=16, corresponding to the very popular x86-64 architecture.
2.1.4. The top-of-stack register \texttt{s0} vs. the accumulator register \texttt{r0}
Some register machine architectures require one of the arguments for most arithmetic and logical operations to reside in a special register called the accumulator. In our comparison, we will assume that the accumulator is the general-purpose register \texttt{r0}; otherwise we could simply renumber the registers. In this respect, the accumulator is somewhat similar to the top-of-stack "register" \texttt{s0} of a stack machine, because virtually all operations of a stack machine both use \texttt{s0} as one of their arguments and return their result as \texttt{s0}.
2.1.5. Register calling conventions
When compiled for a register machine, high-level language functions usually receive their arguments in certain registers in a predefined order. If there are too many arguments, these functions take the remainder from the stack (yes, a register machine usually has a stack, too!). Some register calling conventions pass no arguments in registers at all, however, and only use the stack (for example, the original calling conventions used in implementations of Pascal and C, although modern implementations of C use some registers as well).
For simplicity, we will assume that up to m\leq n function arguments are passed in registers, and that these registers are \texttt{r0}, \texttt{r1}, \ldots, \texttt{r}(m-1), in that order (if some other registers are used, we can simply renumber them).6
2.1.6. Order of function arguments
If a function or primitive requires m arguments x_1, \ldots, x_m, they are pushed by the caller into the stack in the same order, starting from x_1. Therefore, when the function or primitive is invoked, its first argument x_1 is in \texttt{s}(m-1), its second argument x_2 is in \texttt{s}(m-2), and so on. The last argument x_m is in \texttt{s0} (i.e., at the top of the stack). It is the called function or primitive's responsibility to remove its arguments from the stack.
In this respect the TVM stack calling conventions—obeyed, at least, by TVM primitives—match those of Pascal and Forth, and are the opposite of those of C (in which the arguments are pushed into the stack in the reverse order, and are removed by the caller after it regains control, not the callee).
Of course, an implementation of a high-level language for TVM might choose some other calling conventions for its functions, different from the default ones. This might be useful for certain functions—for instance, if the total number of arguments depends on the value of the first argument, as happens for "variadic functions" such as \texttt{scanf} and \texttt{printf}. In such cases, the first one or several arguments are better passed near the top of the stack, not somewhere at some unknown location deep in the stack.
2.1.7. Arguments to arithmetic primitives on register machines
On a stack machine, built-in arithmetic primitives (such as \texttt{ADD} or \texttt{DIVMOD}) follow the same calling conventions as user-defined functions. In this respect, user-defined functions (for example, a function computing the square root of a number) might be considered as "extensions" or "custom upgrades" of the stack machine. This is one of the clearest advantages of stack machines (and of stack programming languages such as Forth) compared to register machines.
In contrast, arithmetic instructions (built-in operations) on register machines usually get their parameters from general-purpose registers encoded in the full opcode. A binary operation, such as \texttt{SUB}, thus requires two arguments, \texttt{r}(i) and \texttt{r}(j), with i and j specified by the instruction. A register \texttt{r}(k) for storing the result also must be specified. Arithmetic operations can take several possible forms, depending on whether i, j, and k are allowed to take arbitrary values:
-
Three-address form — Allows the programmer to arbitrarily choose not only the two source registers
\texttt{r}(i)and\texttt{r}(j), but also a separate destination register\texttt{r}(k). This form is common for most RISC processors, and for the XMM and AVX SIMD instruction sets in the x86-64 architecture. -
Two-address form — Uses one of the two operand registers (usually
\texttt{r}(i)) to store the result of an operation, so thatk=iis never indicated explicitly. Onlyiandjare encoded inside the instruction. This is the most common form of arithmetic operations on register machines, and is quite popular on microprocessors (including the x86 family). -
One-address form — Always takes one of the arguments from the accumulator
\texttt{r0}, and stores the result in\texttt{r0}as well; theni=k=0, and onlyjneeds to be specified by the instruction. This form is used by some simpler microprocessors (such as Intel 8080).
Note that this flexibility is available only for built-in operations, but not for user-defined functions. In this respect, register machines are not as easily "upgradable" as stack machines.7
2.1.8. Return values of functions
In stack machines such as TVM, when a function or primitive needs to return a result value, it simply pushes it into the stack (from which all arguments to the function have already been removed). Therefore, the caller will be able to access the result value through the top-of-stack "register" \texttt{s0}.
This is in complete accordance with Forth calling conventions, but differs slightly from Pascal and C calling conventions, where the accumulator register \texttt{r0} is normally used for the return value.
2.1.9. Returning several values
Some functions might want to return several values y_1, \ldots, y_k, with k not necessarily equal to one. In these cases, the k return values are pushed into the stack in their natural order, starting from y_1.
For example, the "divide with remainder" primitive \texttt{DIVMOD} needs to return two values, the quotient q and the remainder r. Therefore, \texttt{DIVMOD} pushes q and r into the stack, in that order, so that the quotient is available thereafter at \texttt{s1} and the remainder at \texttt{s0}. The net effect of \texttt{DIVMOD} is to divide the original value of \texttt{s1} by the original value of \texttt{s0}, and return the quotient in \texttt{s1} and the remainder in \texttt{s0}. In this particular case the depth of the stack and the values of all other "stack registers" remain unchanged, because \texttt{DIVMOD} takes two arguments and returns two results. In general, the values of other "stack registers" that lie in the stack below the arguments passed and the values returned are shifted according to the change of the depth of the stack.
In principle, some primitives and user-defined functions might return a variable number of result values. In this respect, the remarks above about variadic functions apply: the total number of result values and their types should be determined by the values near the top of the stack. (For example, one might push the return values y_1, \ldots, y_k, and then push their total number k as an integer. The caller would then determine the total number of returned values by inspecting \texttt{s0}.)
In this respect TVM, again, faithfully observes Forth calling conventions.
2.1.10. Stack notation
When a stack of depth n contains values z_1, \ldots, z_n, in that order, with z_1 the deepest element and z_n the top of the stack, the contents of the stack are often represented by a list z_1 z_2 \ldots z_n, in that order. When a primitive transforms the original stack state S' into a new state S'', this is often written as S' -- S''; this is the so-called stack notation. For example, the action of the division primitive \texttt{DIV} can be described by S x y -- S \lfloor x/y\rfloor, where S is any list of values. This is usually abbreviated as x y -- \lfloor x/y\rfloor, tacitly assuming that all other values deeper in the stack remain intact.
Alternatively, one can describe \texttt{DIV} as a primitive that runs on a stack S' of depth n\geq2, divides \texttt{s1} by \texttt{s0}, and returns the floor-rounded quotient as \texttt{s0} of the new stack S'' of depth n-1. The new value of \texttt{s}(i) equals the old value of \texttt{s}(i+1) for 1\leq i<n-1. These descriptions are equivalent, but saying that \texttt{DIV} transforms x y into \lfloor x/y\rfloor, or \ldots x y into \ldots \lfloor x/y\rfloor, is more concise.
The stack notation is extensively used throughout Appendix A, where all currently defined TVM primitives are listed.
2.1.11. Explicitly defining the number of arguments to a function
Stack machines usually pass the current stack in its entirety to the invoked primitive or function. That primitive or function accesses only the several values near the top of the stack that represent its arguments, and pushes the return values in their place, by convention leaving all deeper values intact. Then the resulting stack, again in its entirety, is returned to the caller.
Most TVM primitives behave in this way, and we expect most user-defined functions to be implemented under such conventions. However, TVM provides mechanisms to specify how many arguments must be passed to a called function. When these mechanisms are employed, the specified number of values are moved from the caller's stack into the (usually initially empty) stack of the called function, while deeper values remain in the caller's stack and are inaccessible to the callee. The caller can also specify how many return values it expects from the called function.
Such argument-checking mechanisms might be useful, for example, for a library function that calls user-provided functions passed as arguments to it.
2.2 Stack manipulation primitives
A stack machine, such as TVM, employs a lot of stack manipulation primitives to rearrange arguments to other primitives and user-defined functions, so that they become located near the top of the stack in correct order. This section discusses which stack manipulation primitives are necessary and sufficient for achieving this goal, and which of them are used by TVM. Some examples of code using these primitives can be found in Appendix C.
2.2.1. Basic stack manipulation primitives
The most important stack manipulation primitives used by TVM are the following:
-
Top-of-stack exchange operation:
\texttt{XCHG s0,s}(i)or\texttt{XCHG s}(i)— Exchanges values of\texttt{s0}and\texttt{s}(i). Wheni=1, operation\texttt{XCHG s1}is traditionally denoted by\texttt{SWAP}. Wheni=0, this is a\texttt{NOP}(an operation that does nothing, at least if the stack is non-empty). -
Arbitrary exchange operation:
\texttt{XCHG s}(i)\texttt{,s}(j)— Exchanges values of\texttt{s}(i)and\texttt{s}(j). Notice that this operation is not strictly necessary, because it can be simulated by three top-of-stack exchanges:\texttt{XCHG s}(i);\texttt{XCHG s}(j);\texttt{XCHG s}(i). However, it is useful to have arbitrary exchanges as primitives, because they are required quite often. -
Push operation:
\texttt{PUSH s}(i)— Pushes a copy of the (old) value of\texttt{s}(i)into the stack. Traditionally,\texttt{PUSH s0}is also denoted by\texttt{DUP}(it duplicates the value at the top of the stack), and\texttt{PUSH s1}by\texttt{OVER}. -
Pop operation:
\texttt{POP s}(i)— Removes the top-of-stack value and puts it into the (new)\texttt{s}(i-1), or the old\texttt{s}(i). Traditionally,\texttt{POP s0}is also denoted by\texttt{DROP}(it simply drops the top-of-stack value), and\texttt{POP s1}by\texttt{NIP}.
Some other "unsystematic" stack manipulation operations might be also defined (e.g., \texttt{ROT}, with stack notation a b c -- b c a). While such operations are defined in stack languages like Forth (where \texttt{DUP}, \texttt{DROP}, \texttt{OVER}, \texttt{NIP} and \texttt{SWAP} are also present), they are not strictly necessary because the basic stack manipulation primitives listed above suffice to rearrange stack registers to allow any arithmetic primitives and user-defined functions to be invoked correctly.
2.2.2. Basic stack manipulation primitives suffice
A compiler or a human TVM-code programmer might use the basic stack primitives as follows.
Suppose that the function or primitive to be invoked is to be passed, say, three arguments x, y, and z, currently located in stack registers \texttt{s}(i), \texttt{s}(j), and \texttt{s}(k). In this circumstance, the compiler (or programmer) might issue operation \texttt{PUSH s}(i) (if a copy of x is needed after the call to this primitive) or \texttt{XCHG s}(i) (if it will not be needed afterwards) to put the first argument x into the top of the stack. Then, the compiler (or programmer) could use either \texttt{PUSH s}(j') or \texttt{XCHG s}(j'), where j'=j or j+1, to put y into the new top of the stack.8
Proceeding in this manner, we see that we can put the original values of x, y, and z—or their copies, if needed—into locations \texttt{s2}, \texttt{s1}, and \texttt{s0}, using a sequence of push and exchange operations (2.2.4 and 2.2.5 for a more detailed explanation). In order to generate this sequence, the compiler will need to know only the three values i, j and k, describing the old locations of variables or temporary values in question, and some flags describing whether each value will be needed thereafter or is needed only for this primitive or function call. The locations of other variables and temporary values will be affected in the process, but a compiler (or a human programmer) can easily track their new locations.
Similarly, if the results returned from a function need to be discarded or moved to other stack registers, a suitable sequence of exchange and pop operations will do the job. In the typical case of one return value in \texttt{s0}, this is achieved either by an \texttt{XCHG s}(i) or a \texttt{POP s}(i) (in most cases, a \texttt{DROP}) operation.9
Rearranging the result value or values before returning from a function is essentially the same problem as arranging arguments for a function call, and is achieved similarly.
2.2.3. Compound stack manipulation primitives
In order to improve the density of the TVM code and simplify development of compilers, compound stack manipulation primitives may be defined, each combining up to four exchange and push or exchange and pop basic primitives. Such compound stack operations might include, for example:
\texttt{XCHG2 s}(i)\texttt{,s}(j)— Equivalent to\texttt{XCHG s1,s}(i);\texttt{XCHG s}(j).\texttt{PUSH2 s}(i)\texttt{,s}(j)— Equivalent to\texttt{PUSH s}(i);\texttt{PUSH s}(j+1).\texttt{XCPU s}(i)\texttt{,s}(j)— Equivalent to\texttt{XCHG s}(i);\texttt{PUSH s}(j).\texttt{PUXC s}(i)\texttt{,s}(j)— Equivalent to\texttt{PUSH s}(i);\texttt{SWAP};\texttt{XCHG s}(j+1). Whenj\neq iandj\neq0, it is also equivalent to\texttt{XCHG s}(j);\texttt{PUSH s}(i);\texttt{SWAP}.\texttt{XCHG3 s}(i)\texttt{,s}(j)\texttt{,s}(k)— Equivalent to\texttt{XCHG s2,s}(i);\texttt{XCHG s1,s}(j);\texttt{XCHG s}(k).\texttt{PUSH3 s}(i)\texttt{,s}(j)\texttt{,s}(k)— Equivalent to\texttt{PUSH s}(i);\texttt{PUSH s}(j+1);\texttt{PUSH s}(k+2).
Of course, such operations make sense only if they admit a more compact encoding than the equivalent sequence of basic operations. For example, if all top-of-stack exchanges, \texttt{XCHG s1,s}(i) exchanges, and push and pop operations admit one-byte encodings, the only compound stack operations suggested above that might merit inclusion in the set of stack manipulation primitives are \texttt{PUXC}, \texttt{XCHG3}, and \texttt{PUSH3}.
These compound stack operations essentially augment other primitives (instructions) in the code with the "true" locations of their operands, somewhat similarly to what happens with two-address or three-address register machine code. However, instead of encoding these locations inside the opcode of the arithmetic or another instruction, as is customary for register machines, we indicate these locations in a preceding compound stack manipulation operation. The advantage of such an approach is that user-defined functions (or rarely used specific primitives added in a future version of TVM) can benefit from it as well.
2.2.4. Mnemonics of compound stack operations
The mnemonics of compound stack operations, some examples of which have been provided are created as follows.
The \gamma\geq2 formal arguments \texttt{s}(i_1), \ldots, \texttt{s}(i_\gamma) to such an operation O represent the values in the original stack that will end up in \texttt{s}(\gamma-1), \ldots, \texttt{s0} after the execution of this compound operation, at least if all i_\nu, 1\leq\nu\leq\gamma, are distinct and at least \gamma. The mnemonic itself of the operation O is a sequence of \gamma two-letter strings \texttt{PU} and \texttt{XC}, with \texttt{PU} meaning that the corresponding argument is to be PUshed (i.e., a copy is to be created), and \texttt{XC} meaning that the value is to be eXChanged (i.e., no other copy of the original value is created). Sequences of several \texttt{PU} or \texttt{XC} strings may be abbreviated to one \texttt{PU} or \texttt{XC} followed by the number of copies. (For instance, we write \texttt{PUXC2PU} instead of \texttt{PUXCXCPU}.)
As an exception, if a mnemonic would consist of only \texttt{PU} or only \texttt{XC} strings, so that the compound operation is equivalent to a sequence of m PUSHes or eXCHanGes, the notation \texttt{PUSH}m or \texttt{XCHG}m is used instead of \texttt{PU}m or \texttt{XC}m.
2.2.5. Semantics of compound stack operations
Each compound \gamma-ary operation O \texttt{s}(i_1),\ldots,\texttt{s}(i_\gamma) is translated into an equivalent sequence of basic stack operations by induction in \gamma as follows:
-
As a base of induction, if
\gamma=0, the only nullary compound stack operation corresponds to an empty sequence of basic stack operations. -
Equivalently, we might begin the induction from
\gamma=1. Then\texttt{PU s}(i)corresponds to the sequence consisting of one basic operation\texttt{PUSH s}(i), and\texttt{XC s}(i)corresponds to the one-element sequence consisting of\texttt{XCHG s}(i). -
Otherwise, if
\gamma\geq2, compound operationOhas one of two forms:-
O\texttt{s}(i_1),\ldots,\texttt{s}(i_\gamma), withO=\texttt{XC}O', whereO'is a compound operation of arity\gamma-1(i.e., the mnemonic ofO'consists of\gamma-1strings\texttt{XC}and\texttt{PU}). Let\alphabe the total quantity of\texttt{PU}shes inO, and\betabe that of e\texttt{XC}hanges, so that\alpha+\beta=\gamma. Then the original operation is translated into\texttt{XCHG s}(\beta-1)\texttt{,s}(i_1), followed by the translation ofO'\texttt{s}(i_2),\ldots,\texttt{s}(i_\gamma), defined by the induction hypothesis. -
O\texttt{s}(i_1),\ldots,\texttt{s}(i_\gamma), withO=\texttt{PU}O', whereO'is a compound operation of arity\gamma-1. Then the original operation is translated into\texttt{PUSH s}(i_1);\texttt{XCHG s}(\beta), followed by the translation ofO'\texttt{s}(i_2+1),\ldots,\texttt{s}(i_\gamma+1), defined by the induction hypothesis.10
-
2.2.6. Stack manipulation instructions are polymorphic
Notice that the stack manipulation instructions are almost the only "polymorphic" primitives in TVM—i.e., they work with values of arbitrary types (including the value types that will appear only in future revisions of TVM). For example, \texttt{SWAP} always interchanges the two top values of the stack, even if one of them is an integer and the other is a cell. Almost all other instructions, especially the data processing instructions (including arithmetic instructions), require each of their arguments to be of some fixed type (possibly different for different arguments).
2.3 Efficiency of stack manipulation primitives
Stack manipulation primitives employed by a stack machine, such as TVM, have to be implemented very efficiently, because they constitute more than half of all the instructions used in a typical program. In fact, TVM performs all these instructions in a (small) constant time, regardless of the values involved (even if they represent very large integers or very large trees of cells).
2.3.1. Implementation of stack manipulation primitives: using references for operations instead of objects
The efficiency of TVM's implementation of stack manipulation primitives results from the fact that a typical TVM implementation keeps in the stack not the value objects themselves, but only the references (pointers) to such objects. Therefore, a \texttt{SWAP} instruction only needs to interchange the references at \texttt{s0} and \texttt{s1}, not the actual objects they refer to.
2.3.2. Efficient implementation of \texttt{DUP} and \texttt{PUSH} instructions using copy-on-write
Furthermore, a \texttt{DUP} (or, more generally, \texttt{PUSH s}(i)) instruction, which appears to make a copy of a potentially large object, also works in small constant time, because it uses a copy-on-write technique of delayed copying: it copies only the reference instead of the object itself, but increases the "reference counter" inside the object, thus sharing the object between the two references. If an attempt to modify an object with a reference counter greater than one is detected, a separate copy of the object in question is made first (incurring a certain "non-uniqueness penalty" or "copying penalty" for the data manipulation instruction that triggered the creation of a new copy).
2.3.3. Garbage collecting and reference counting
When the reference counter of a TVM object becomes zero (for example, because the last reference to such an object has been consumed by a \texttt{DROP} operation or an arithmetic instruction), it is immediately freed. Because cyclic references are impossible in TVM data structures, this method of reference counting provides a fast and convenient way of freeing unused objects, replacing slow and unpredictable garbage collectors.
2.3.4. Transparency of the implementation: Stack values are "values", not "references"
Regardless of the implementation details just discussed, all stack values are really "values", not "references", from the perspective of the TVM programmer, similarly to the values of all types in functional programming languages. Any attempt to modify an existing object referred to from any other objects or stack locations will result in a transparent replacement of this object by its perfect copy before the modification is actually performed.
In other words, the programmer should always act as if the objects themselves were directly manipulated by stack, arithmetic, and other data transformation primitives, and treat the previous discussion only as an explanation of the high efficiency of the stack manipulation primitives.
2.3.5. Absence of circular references
One might attempt to create a circular reference between two cells, A and B, as follows: first create A and write some data into it; then create B and write some data into it, along with a reference to previously constructed cell A; finally, add a reference to B into A. While it may seem that after this sequence of operations we obtain a cell A, which refers to B, which in turn refers to A, this is not the case. In fact, we obtain a new cell A', which contains a copy of the data originally stored into cell A along with a reference to cell B, which contains a reference to (the original) cell A.
In this way the transparent copy-on-write mechanism and the "everything is a value" paradigm enable us to create new cells using only previously constructed cells, thus forbidding the appearance of circular references. This property also applies to all other data structures: for instance, the absence of circular references enables TVM to use reference counting to immediately free unused memory instead of relying on garbage collectors. Similarly, this property is crucial for storing data in the TON Blockchain.
3 Cells, memory, and persistent storage
This chapter briefly describes TVM cells, used to represent all data structures inside the TVM memory and its persistent storage, and the basic operations used to create cells, write (or serialize) data into them, and read (or deserialize) data from them.
3.1 Generalities on cells
This section presents a classification and general descriptions of cell types.
3.1.1. TVM memory and persistent storage consist of cells
Recall that the TVM memory and persistent storage consist of (TVM) cells. Each cell contains up to 1023 bits of data and up to four references to other cells.11 Circular references are forbidden and cannot be created by means of TVM. In this way, all cells kept in TVM memory and persistent storage constitute a directed acyclic graph (DAG).
3.1.2. Ordinary and exotic cells
Apart from the data and references, a cell has a cell type, encoded by an integer -1\ldots255. A cell of type -1 is called ordinary; such cells do not require any special processing. Cells of other types are called exotic, and may be loaded—automatically replaced by other cells when an attempt to deserialize them (i.e., to convert them into a Slice by a \texttt{CTOS} instruction) is made. They may also exhibit a non-trivial behavior when their hashes are computed.
The most common use for exotic cells is to represent some other cells—for instance, cells present in an external library, or pruned from the original tree of cells when a Merkle proof has been created.
The type of an exotic cell is stored as the first eight data bits of its data. If an exotic cell has less than eight data bits, it is invalid.
3.1.3. The level of a cell
Every cell c has another attribute \text{Lvl}(c) called its (de Brujn) level, which currently takes integer values in the range 0\ldots3. The level of an ordinary cell is always equal to the maximum of the levels of all its children c_i:
\text{Lvl}(c)=\max_{1\leq i\leq r}\text{Lvl}(c_i) \tag{1}for an ordinary cell c containing r references to cells c_1, \ldots, c_r. If r=0, \text{Lvl}(c)=0. Exotic cells may have different rules for setting their level.
A cell's level affects the number of higher hashes it has. More precisely, a level l cell has l higher hashes \text{Hash}_1(c), \ldots, \text{Hash}_l(c) in addition to its representation hash \text{Hash}(c)=\text{Hash}_\infty(c). Cells of non-zero level appear inside Merkle proofs and Merkle updates, after some branches of the tree of cells representing a value of an abstract data type are pruned.
3.1.4. Standard cell representation
When a cell needs to be transferred by a network protocol or stored in a disk file, it must be serialized. The standard representation \text{CellRepr}(c)=\text{CellRepr}_\infty(c) of a cell c as an octet (byte) sequence is constructed as follows:
-
Two descriptor bytes
d_1andd_2are serialized first. Byted_1equalsr+8s+32l, where0\leq r\leq 4is the quantity of cell references contained in the cell,0\leq l\leq 3is the level of the cell, and0\leq s\leq 1is1for exotic cells and0for ordinary cells. Byted_2equals\lfloor b/8\rfloor+\lceil b/8\rceil, where0\leq b\leq 1023is the quantity of data bits inc. -
Then the data bits are serialized as
\lceil b/8\rceil8-bit octets (bytes). Ifbis not a multiple of eight, a binary\texttt{1}and up to six binary\texttt{0}s are appended to the data bits. After that, the data is split into\lceil b/8\rceileight-bit groups, and each group is interpreted as an unsigned big-endian integer0\ldots 255and stored into an octet. -
Finally, each of the
rcell references is represented by 32 bytes containing the 256-bit representation hash\text{Hash}(c_i)of the cellc_ireferred to.
In this way, 2+\lceil b/8\rceil+32r bytes of \text{CellRepr}(c) are obtained.
3.1.5. The representation hash of a cell
The 256-bit representation hash or simply hash \text{Hash}(c) of a cell c is recursively defined as the SHA-256 of the standard representation of the cell c:
\text{Hash}(c):=\text{Sha256}\bigl(\text{CellRepr}(c)\bigr) \tag{2}Notice that cyclic cell references are not allowed and cannot be created by means of the TVM, so this recursion always ends, and the representation hash of any cell is well-defined.
3.1.6. The higher hashes of a cell
Recall that a cell c of level l has l higher hashes \text{Hash}_i(c), 1\leq i\leq l, as well. Exotic cells have their own rules for computing their higher hashes. Higher hashes \text{Hash}_i(c) of an ordinary cell c are computed similarly to its representation hash, but using the higher hashes \text{Hash}_i(c_j) of its children c_j instead of their representation hashes \text{Hash}(c_j). By convention, we set \text{Hash}_\infty(c):=\text{Hash}(c), and \text{Hash}_i(c):=\text{Hash}_\infty(c)=\text{Hash}(c) for all i>l.12
3.1.7. Types of exotic cells
TVM currently supports the following cell types:
-
Type
-1: Ordinary cell — Contains up to 1023 bits of data and up to four cell references. -
Type 1: Pruned branch cell
c— May have any level1\leq l\leq 3. It contains exactly8+256ldata bits: first an 8-bit integer equal to 1 (representing the cell's type), then itslhigher hashes\text{Hash}_1(c),\ldots,\text{Hash}_l(c). The levellof a pruned branch cell may be called its de Brujn index, because it determines the outer Merkle proof or Merkle update during the construction of which the branch has been pruned. An attempt to load a pruned branch cell usually leads to an exception. -
Type 2: Library reference cell — Always has level 0, and contains
8+256data bits, including its 8-bit type integer 2 and the representation hash\text{Hash}(c')of the library cell being referred to. When loaded, a library reference cell may be transparently replaced by the cell it refers to, if found in the current library context. -
Type 3: Merkle proof cell
c— Has exactly one referencec_1and level0\leq l\leq 3, which must be one less than the level of its only childc_1:\text{Lvl}(c)=\max(\text{Lvl}(c_1)-1,0) \tag{3}The
8+256data bits of a Merkle proof cell contain its 8-bit type integer 3, followed by\text{Hash}_1(c_1)(assumed to be equal to\text{Hash}(c_1)if\text{Lvl}(c_1)=0). The higher hashes\text{Hash}_i(c)ofcare computed similarly to the higher hashes of an ordinary cell, but with\text{Hash}_{i+1}(c_1)used instead of\text{Hash}_i(c_1). When loaded, a Merkle proof cell is replaced byc_1. -
Type 4: Merkle update cell
c— Has two childrenc_1andc_2. Its level0\leq l\leq 3is given by\text{Lvl}(c)=\max(\text{Lvl}(c_1)-1,\text{Lvl}(c_2)-1,0) \tag{4}A Merkle update behaves like a Merkle proof for both
c_1andc_2, and contains8+256+256data bits with\text{Hash}_1(c_1)and\text{Hash}_1(c_2). However, an extra requirement is that all pruned branch cellsc'that are descendants ofc_2and are bound bycmust also be descendants ofc_1.13 When a Merkle update cell is loaded, it is replaced byc_2.
3.1.8. All values of algebraic data types are trees of cells
Arbitrary values of arbitrary algebraic data types (e.g., all types used in functional programming languages) can be serialized into trees of cells (of level 0), and such representations are used for representing such values within TVM. The copy-on-write mechanism allows TVM to identify cells containing the same data and references, and to keep only one copy of such cells. This actually transforms a tree of cells into a directed acyclic graph (with the additional property that all its vertices be accessible from a marked vertex called the "root"). However, this is a storage optimization rather than an essential property of TVM. From the perspective of a TVM code programmer, one should think of TVM data structures as trees of cells.
3.1.9. TVM code is a tree of cells
The TVM code itself is also represented by a tree of cells. Indeed, TVM code is simply a value of some complex algebraic data type, and as such, it can be serialized into a tree of cells.
The exact way in which the TVM code (e.g., TVM assembly code) is transformed into a tree of cells is explained later (4.1.4 and 5.2), in sections discussing control flow instructions, continuations, and TVM instruction encoding.
3.1.10. "Everything is a bag of cells" paradigm
All the data used by the TON Blockchain, including the blocks themselves and the blockchain state, can be represented—and are represented—as collections, or bags of cells. We see that TVM's structure of data and code nicely fits into this "everything is a bag of cells" paradigm. In this way, TVM can naturally be used to execute smart contracts in the TON Blockchain, and the TON Blockchain can be used to store the code and persistent data of these smart contracts between invocations of TVM. (Of course, both TVM and the TON Blockchain have been designed so that this would become possible.)
3.2 Data manipulation instructions and cells
The next large group of TVM instructions consists of data manipulation instructions, also known as cell manipulation instructions or simply cell instructions. They correspond to memory access instructions of other architectures.
3.2.1. Classes of cell manipulation instructions
The TVM cell instructions are naturally subdivided into two principal classes:
- Cell creation instructions or serialization instructions, used to construct new cells from values previously kept in the stack and previously constructed cells.
- Cell parsing instructions or deserialization instructions, used to extract data previously stored into cells by cell creation instructions.
Additionally, there are exotic cell instructions used to create and inspect exotic cells, which in particular are used to represent pruned branches of Merkle proofs and Merkle proofs themselves.
3.2.2. Builder and Slice values
Cell creation instructions usually work with Builder values, which can be kept only in the stack. Such values represent partially constructed cells, for which fast operations for appending bitstrings, integers, other cells, and references to other cells can be defined. Similarly, cell parsing instructions make heavy use of Slice values, which represent either the remainder of a partially parsed cell, or a value (subcell) residing inside such a cell and extracted from it by a parsing instruction.
3.2.3. Builder and Slice values exist only as stack values
Notice that Builder and Slice objects appear only as values in a TVM stack. They cannot be stored in "memory" (i.e., trees of cells) or "persistent storage" (which is also a bag of cells). In this sense, there are far more Cell objects than Builder or Slice objects in a TVM environment, but, somewhat paradoxically, a TVM program sees Builder and Slice objects in its stack more often than Cells. In fact, a TVM program does not have much use for Cell values, because they are immutable and opaque; all cell manipulation primitives require that a Cell value be transformed into either a Builder or a Slice first, before it can be modified or inspected.
3.2.4. TVM has no separate Bitstring value type
Notice that TVM offers no separate bitstring value type. Instead, bitstrings are represented by Slices that happen to have no references at all, but can still contain up to 1023 data bits.
3.2.5. Cells and cell primitives are bit-oriented, not byte-oriented
An important point is that TVM regards data kept in cells as sequences (strings, streams) of (up to 1023) bits, not of bytes. In other words, TVM is a bit-oriented machine, not a byte-oriented machine. If necessary, an application is free to use, say, 21-bit integer fields inside records serialized into TVM cells, thus using fewer persistent storage bytes to represent the same data.
3.2.6. Taxonomy of cell creation (serialization) primitives
Cell creation primitives usually accept a Builder argument and an argument representing the value to be serialized. Additional arguments controlling some aspects of the serialization process (e.g., how many bits should be used for serialization) can be also provided, either in the stack or as an immediate value inside the instruction. The result of a cell creation primitive is usually another Builder, representing the concatenation of the original builder and the serialization of the value provided.
Therefore, one can suggest a classification of cell serialization primitives according to the answers to the following questions:
- Which is the type of values being serialized?
- How many bits are used for serialization? If this is a variable number, does it come from the stack, or from the instruction itself?
- What happens if the value does not fit into the prescribed number of bits? Is an exception generated, or is a success flag equal to zero silently returned in the top of stack?
- What happens if there is insufficient space left in the Builder? Is an exception generated, or is a zero success flag returned along with the unmodified original Builder?
The mnemonics of cell serialization primitives usually begin with \texttt{ST}. Subsequent letters describe the following attributes:
- The type of values being serialized and the serialization format (e.g.,
\texttt{I}for signed integers,\texttt{U}for unsigned integers). - The source of the field width in bits to be used (e.g.,
\texttt{X}for integer serialization instructions means that the bit widthnis supplied in the stack; otherwise it has to be embedded into the instruction as an immediate value). - The action to be performed if the operation cannot be completed (by default, an exception is generated; "quiet" versions of serialization instructions are marked by a
\texttt{Q}letter in their mnemonics).
This classification scheme is used to create a more complete taxonomy of cell serialization primitives.
3.2.7. Integer serialization primitives
Integer serialization primitives can be classified according to the above taxonomy as well. For example:
- There are signed and unsigned (big-endian) integer serialization primitives.
- The size
nof the bit field to be used (1\leq n\leq 257for signed integers,0\leq n\leq 256for unsigned integers) can either come from the top of stack or be embedded into the instruction itself. - If the integer
xto be serialized is not in the range-2^{n-1}\leq x<2^{n-1}(for signed integer serialization) or0\leq x<2^n(for unsigned integer serialization), a range check exception is usually generated, and ifnbits cannot be stored into the provided Builder, a cell overflow exception is usually generated. - Quiet versions of serialization instructions do not throw exceptions; instead, they push
-1on top of the resulting Builder upon success, or return the original Builder with0on top of it to indicate failure.
Integer serialization instructions have mnemonics like \texttt{STU 20} ("store an unsigned 20-bit integer value") or \texttt{STIXQ} ("quietly store an integer value of variable length provided in the stack"). The full list of these instructions—includes their mnemonics, descriptions, and opcodes.
3.2.8. Integers in cells are big-endian by default
Notice that the default order of bits in Integers serialized into Cells is big-endian, not little-endian.14 In this respect TVM is a big-endian machine. However, this affects only the serialization of integers inside cells. The internal representation of the Integer value type is implementation-dependent and irrelevant for the operation of TVM. Besides, there are some special primitives such as \texttt{STULE} for (de)serializing little-endian integers, which must be stored into an integral number of bytes (otherwise "little-endianness" does not make sense, unless one is also willing to revert the order of bits inside octets). Such primitives are useful for interfacing with the little-endian world—for instance, for parsing custom-format messages arriving to a TON Blockchain smart contract from the outside world.
3.2.9. Other serialization primitives
Other cell creation primitives serialize bitstrings (i.e., cell slices without references), either taken from the stack or supplied as literal arguments; cell slices (which are concatenated to the cell builder in an obvious way); other Builders (which are also concatenated); and cell references (\texttt{STREF}).
3.2.10. Other cell creation primitives
In addition to the cell serialization primitives for certain built-in value types described above, there are simple primitives that create a new empty Builder and push it into the stack (\texttt{NEWC}), or transform a Builder into a Cell (\texttt{ENDC}), thus finishing the cell creation process. An \texttt{ENDC} can be combined with a \texttt{STREF} into a single instruction \texttt{ENDCST}, which finishes the creation of a cell and immediately stores a reference to it in an "outer" Builder. There are also primitives that obtain the quantity of data bits or references already stored in a Builder, and check how many data bits or references can be stored.
3.2.11. Taxonomy of cell deserialisation primitives
Cell parsing, or deserialization, primitives can be classified with the following modifications:
- They work with Slices (representing the remainder of the cell being parsed) instead of Builders.
- They return deserialized values instead of accepting them as arguments.
- They may come in two flavors, depending on whether they remove the deserialized portion from the Slice supplied ("fetch operations") or leave it unmodified ("prefetch operations").
- Their mnemonics usually begin with
\texttt{LD}(or\texttt{PLD}for prefetch operations) instead of\texttt{ST}.
For example, an unsigned big-endian 20-bit integer previously serialized into a cell by a \texttt{STU 20} instruction is likely to be deserialized later by a matching \texttt{LDU 20} instruction.
Again, more detailed information about these instructions is provided in A.7.2.
3.2.12. Other cell slice primitives
In addition to the cell deserialisation primitives outlined above, TVM provides some obvious primitives for initializing and completing the cell deserialization process. For instance, one can convert a Cell into a Slice (\texttt{CTOS}), so that its deserialisation might begin; or check whether a Slice is empty, and generate an exception if it is not (\texttt{ENDS}); or deserialize a cell reference and immediately convert it into a Slice (\texttt{LDREFTOS}, equivalent to two instructions \texttt{LDREF} and \texttt{CTOS}).
3.2.13. Modifying a serialized value in a cell
The reader might wonder how the values serialized inside a cell may be modified. Suppose a cell contains three serialized 29-bit integers, (x,y,z), representing the coordinates of a point in space, and we want to replace y with y'=y+1, leaving the other coordinates intact. How would we achieve this?
TVM does not offer any ways to modify existing values (2.3.4 and 2.3.5), so our example can only be accomplished with a series of operations as follows:
- Deserialize the original cell into three Integers
x,y,zin the stack (e.g., by\texttt{CTOS};\texttt{LDI 29};\texttt{LDI 29};\texttt{LDI 29};\texttt{ENDS}). - Increase
yby one (e.g., by\texttt{SWAP};\texttt{INC};\texttt{SWAP}). - Finally, serialize the resulting Integers into a new cell (e.g., by
\texttt{XCHG s2};\texttt{NEWC};\texttt{STI 29};\texttt{STI 29};\texttt{STI 29};\texttt{ENDC}).
3.2.14. Modifying the persistent storage of a smart contract
If the TVM code wants to modify its persistent storage, represented by the tree of cells rooted at \texttt{c4}, it simply needs to rewrite control register \texttt{c4} by the root of the tree of cells containing the new value of its persistent storage. (If only part of the persistent storage needs to be modified.)
3.3 Hashmaps, or dictionaries
Hashmaps, or dictionaries, are a specific data structure represented by a tree of cells. Essentially, a hashmap represents a map from keys, which are bitstrings of either fixed or variable length, into values of an arbitrary type X, in such a way that fast lookups and modifications be possible. While any such structure might be inspected or modified with the aid of generic cell serialization and deserialization primitives, TVM introduces special primitives to facilitate working with these hashmaps.
3.3.1. Basic hashmap types
The two most basic hashmap types predefined in TVM are HashmapE n X or HashmapE(n,X), which represents a partially defined map from n-bit strings (called keys) for some fixed 0\leq n\leq 1023 into values of some type X, and Hashmap(n,X), which is similar to HashmapE(n,X) but is not allowed to be empty (i.e., it must contain at least one key-value pair).
Other hashmap types are also available—for example, one with keys of arbitrary length up to some predefined bound (up to 1023 bits).
3.3.2. Hashmaps as Patricia trees
The abstract representation of a hashmap in TVM is a Patricia tree, or a compact binary trie. It is a binary tree with edges labelled by bitstrings, such that the concatenation of all edge labels on a path from the root to a leaf equals a key of the hashmap. The corresponding value is kept in this leaf (for hashmaps with keys of fixed length), or optionally in the intermediate vertices as well (for hashmaps with keys of variable length). Furthermore, any intermediate vertex must have two children, and the label of the left child must begin with a binary zero, while the label of the right child must begin with a binary one. This enables us not to store the first bit of the edge labels explicitly.
It is easy to see that any collection of key-value pairs (with distinct keys) is represented by a unique Patricia tree.
3.3.3. Serialization of hashmaps
The serialization of a hashmap into a tree of cells (or, more generally, into a Slice) is defined by the following TL-B scheme:15
bit#_ _:(## 1) = Bit;
hm_edge#_ {n:#} {X:Type} {l:#} {m:#} label:(HmLabel ~l n)
{n = (~m) + l} node:(HashmapNode m X) = Hashmap n X;
hmn_leaf#_ {X:Type} value:X = HashmapNode 0 X;
hmn_fork#_ {n:#} {X:Type} left:^(Hashmap n X)
right:^(Hashmap n X) = HashmapNode (n + 1) X;
hml_short$0 {m:#} {n:#} len:(Unary ~n)
s:(n * Bit) = HmLabel ~n m;
hml_long$10 {m:#} n:(#<= m) s:(n * Bit) = HmLabel ~n m;
hml_same$11 {m:#} v:Bit n:(#<= m) = HmLabel ~n m;
unary_zero$0 = Unary ~0;
unary_succ$1 {n:#} x:(Unary ~n) = Unary ~(n + 1);
hme_empty$0 {n:#} {X:Type} = HashmapE n X;
hme_root$1 {n:#} {X:Type} root:^(Hashmap n X) = HashmapE n X;
true#_ = True;
_ {n:#} _:(Hashmap n True) = BitstringSet n;3.3.4. Brief explanation of TL-B schemes
A TL-B scheme, like the one above, includes the following components.
The right-hand side of each "equation" is a type, either simple (such as \texttt{Bit} or \texttt{True}) or parametrized (such as \texttt{Hashmap} n X). The parameters of a type must be either natural numbers (i.e., non-negative integers, which are required to fit into 32 bits in practice), such as n in \texttt{Hashmap} n X, or other types, such as X in \texttt{Hashmap} n X.
The left-hand side of each equation describes a way to define, or even to serialize, a value of the type indicated in the right-hand side. Such a description begins with the name of a constructor, such as \texttt{hm\_edge} or \texttt{hml\_long}, immediately followed by an optional constructor tag, such as #_ or $10, which describes the bitstring used to encode (serialize) the constructor in question. Such tags may be given in either binary (after a dollar sign) or hexadecimal notation (after a hash sign), using the conventions. If a tag is not explicitly provided, TL-B computes a default 32-bit constructor tag by hashing the text of the "equation" defining this constructor in a certain fashion. Therefore, empty tags must be explicitly provided by #_ or $_. All constructor names must be distinct, and constructor tags for the same type must constitute a prefix code (otherwise the deserialization would not be unique).
The constructor and its optional tag are followed by field definitions. Each field definition is of the form \textit{ident}:\textit{type-expr}, where \textit{ident} is an identifier with the name of the field16 (replaced by an underscore for anonymous fields), and \textit{type-expr} is the field's type. The type provided here is a type expression, which may include simple types or parametrized types with suitable parameters. Variables—i.e., the (identifiers of the) previously defined fields of types \# (natural numbers) or \textit{Type} (type of types)—may be used as parameters for the parametrized types. The serialization process recursively serializes each field according to its type, and the serialization of a value ultimately consists of the concatenation of bitstrings representing the constructor (i.e., the constructor tag) and the field values.
Some fields may be implicit. Their definitions are surrounded by curly braces, which indicate that the field is not actually present in the serialization, but that its value must be deduced from other data (usually the parameters of the type being serialized).
Some occurrences of "variables" (i.e., already-defined fields) are prefixed by a tilde. This indicates that the variable's occurrence is used in the opposite way of the default behavior: in the left-hand side of the equation, it means that the variable will be deduced (computed) based on this occurrence, instead of substituting its previously computed value; in the right-hand side, conversely, it means that the variable will not be deduced from the type being serialized, but rather that it will be computed during the deserialization process. In other words, a tilde transforms an "input argument" into an "output argument", and vice versa.17
Finally, some equalities may be included in curly brackets as well. These are certain "equations", which must be satisfied by the "variables" included in them. If one of the variables is prefixed by a tilde, its value will be uniquely determined by the values of all other variables participating in the equation (which must be known at this point) when the definition is processed from the left to the right.
A caret (^) preceding a type X means that instead of serializing a value of type X as a bitstring inside the current cell, we place this value into a separate cell, and add a reference to it into the current cell. Therefore ^X means "the type of references to cells containing values of type X".
Parametrized type \texttt{\#<= }p with p:\texttt{\#} (this notation means "p of type \texttt{\#}", i.e., a natural number) denotes the subtype of the natural numbers type \texttt{\#}, consisting of integers 0\ldots p; it is serialized into \lceil\log_2(p+1)\rceil bits as an unsigned big-endian integer. Type \texttt{\#} by itself is serialized as an unsigned 32-bit integer. Parametrized type \texttt{\#\# }b with b:\texttt{\#<=}31 is equivalent to \texttt{\#<= }2^b-1 (i.e., it is an unsigned b-bit integer).
3.3.5. Application to the serialization of hashmaps
Let us explain the net result of applying the general rules to the TL-B scheme.
Suppose we wish to serialize a value of type HashmapE n X for some integer 0\leq n\leq 1023 and some type X (i.e., a dictionary with n-bit keys and values of type X, admitting an abstract representation as a Patricia tree).
First of all, if our dictionary is empty, it is serialized into a single binary \texttt{0}, which is the tag of nullary constructor \texttt{hme\_empty}. Otherwise, its serialization consists of a binary \texttt{1} (the tag of \texttt{hme\_root}), along with a reference to a cell containing the serialization of a value of type Hashmap n X (i.e., a necessarily non-empty dictionary).
The only way to serialize a value of type Hashmap n X is given by the \texttt{hm\_edge} constructor, which instructs us to serialize first the label \texttt{label} of the edge leading to the root of the subtree under consideration (i.e., the common prefix of all keys in our (sub)dictionary). This label is of type \texttt{HmLabel} l^\perp n, which means that it is a bitstring of length at most n, serialized in such a way that the true length l of the label, 0\leq l\leq n, becomes known from the serialization of the label.
The label must be followed by the serialization of a \texttt{node} of type \texttt{HashmapNode} m X, where m=n-l. It corresponds to a vertex of the Patricia tree, representing a non-empty subdictionary of the original dictionary with m-bit keys, obtained by removing from all the keys of the original subdictionary their common prefix of length l.
If m=0, a value of type \texttt{HashmapNode} 0 X is given by the \texttt{hmn\_leaf} constructor, which describes a leaf of the Patricia tree—or, equivalently, a subdictionary with 0-bit keys. A leaf simply consists of the corresponding \texttt{value} of type X and is serialized accordingly.
On the other hand, if m>0, a value of type \texttt{HashmapNode} m X corresponds to a fork (i.e., an intermediate node) in the Patricia tree, and is given by the \texttt{hmn\_fork} constructor. Its serialization consists of \texttt{left} and \texttt{right}, two references to cells containing values of type \texttt{Hashmap} m-1 X, which correspond to the left and the right child of the intermediate node in question—or, equivalently, to the two subdictionaries of the original dictionary consisting of key-value pairs with keys beginning with a binary \texttt{0} or a binary \texttt{1}, respectively. Because the first bit of all keys in each of these subdictionaries is known and fixed, it is removed, and the resulting (necessarily non-empty) subdictionaries are recursively serialized as values of type \texttt{Hashmap} m-1 X.
3.3.6. Serialization of labels
There are several ways to serialize a label of length at most n, if its exact length is l\leq n (recall that the exact length must be deducible from the serialization of the label itself, while the upper bound n is known before the label is serialized or deserialized). These ways are described by the three constructors \texttt{hml\_short}, \texttt{hml\_long}, and \texttt{hml\_same} of type \texttt{HmLabel} l^\perp n:
-
\texttt{hml\_short}— Describes a way to serialize "short" labels, of small lengthl\leq n. Such a serialization consists of a binary\texttt{0}(the constructor tag of\texttt{hml\_short}), followed bylbinary\texttt{1}s and one binary\texttt{0}(the unary representation of the lengthl), followed bylbits comprising the label itself. -
\texttt{hml\_long}— Describes a way to serialize "long" labels, of arbitrary lengthl\leq n. Such a serialization consists of a binary\texttt{10}(the constructor tag of\texttt{hml\_long}), followed by the big-endian binary representation of the length0\leq l\leq nin\lceil\log_2(n+1)\rceilbits, followed bylbits comprising the label itself. -
\texttt{hml\_same}— Describes a way to serialize "long" labels, consisting oflrepetitions of the same bitv. Such a serialization consists of\texttt{11}(the constructor tag of\texttt{hml\_same}), followed by the bitv, followed by the lengthlstored in\lceil\log_2(n+1)\rceilbits as before.
Each label can always be serialized in at least two different fashions, using \texttt{hml\_short} or \texttt{hml\_long} constructors. Usually the shortest serialization (and in the case of a tie—the lexicographically smallest among the shortest) is preferred and is generated by TVM hashmap primitives, while the other variants are still considered valid.
This label encoding scheme has been designed to be efficient for dictionaries with "random" keys (e.g., hashes of some data), as well as for dictionaries with "regular" keys (e.g., big-endian representations of integers in some range).
3.3.7. An example of dictionary serialization
Consider a dictionary with three 16-bit keys 13, 17, and 239 (considered as big-endian integers) and corresponding 16-bit values 169, 289, and 57121.
In binary form:
0000000000001101 => 0000000010101001
0000000000010001 => 0000000100100001
0000000011101111 => 1101111100100001The corresponding Patricia tree consists of a root A, two intermediate nodes B and C, and three leaf nodes D, E, and F, corresponding to 13, 17, and 239, respectively. The root A has only one child, B; the label on the edge AB is 00000000=0^8. The node B has two children: its left child is an intermediate node C with the edge BC labelled by (0)00, while its right child is the leaf F with BF labelled by (1)1101111. Finally, C has two leaf children D and E, with CD labelled by (0)1101 and CE—by (1)0001.
The corresponding value of type \texttt{HashmapE} 16 (\texttt{\#\#} 16) may be written in human-readable form as:
(hme_root$1
root:^(hm_edge label:(hml_same$11 v:0 n:8) node:(hm_fork
left:^(hm_edge label:(hml_short$0 len:$110 s:$00)
node:(hm_fork
left:^(hm_edge label:(hml_long$10 n:4 s:$1101)
node:(hm_leaf value:169))
right:^(hm_edge label:(hml_long$10 n:4 s:$0001)
node:(hm_leaf value:289))))
right:^(hm_edge label:(hml_long$10 n:7 s:$1101111)
node:(hm_leaf value:57121)))))The serialization of this data structure into a tree of cells consists of six cells with the following binary data contained in them:
A := 1
A.0 := 11 0 01000
A.0.0 := 0 110 00
A.0.0.0 := 10 100 1101 0000000010101001
A.0.0.1 := 10 100 0001 0000000100100001
A.0.1 := 10 111 1101111 1101111100100001Here A is the root cell, A.0 is the cell at the first reference of A, A.1 is the cell at the second reference of A, and so on. This tree of cells can be represented more compactly using the hexadecimal notation, using indentation to reflect the tree-of-cells structure:
C_
C8
62_
A68054C_
A08090C_
BEFDF21A total of 93 data bits and 5 references in 6 cells have been used to serialize this dictionary. Notice that a straightforward representation of three 16-bit keys and their corresponding 16-bit values would already require 96 bits (albeit without any references), so this particular serialization turns out to be quite efficient.
3.3.8. Ways to describe the serialization of type X
Notice that the built-in TVM primitives for dictionary manipulation need to know something about the serialization of type X; otherwise, they would not be able to work correctly with \mathit{Hashmap} n X, because values of type X are immediately contained in the Patricia tree leaf cells. There are several options available to describe the serialization of type X:
- The simplest case is when
X = \text{\textasciicircum}Yfor some other typeY. In this case the serialization ofXitself always consists of one reference to a cell, which in fact must contain a value of typeY, something that is not relevant for dictionary manipulation primitives. - Another simple case is when the serialization of any value of type
Xalways consists of0\leq b\leq 1023data bits and0\leq r\leq 4references. Integersbandrcan then be passed to a dictionary manipulation primitive as a simple description ofX. (Notice that the previous case corresponds tob=0,r=1.) - A more sophisticated case can be described by four integers
1\leq b_0,b_1\leq 1023,0\leq r_0,r_1\leq 4, withb_iandr_iused when the first bit of the serialization equalsi. Whenb_0=b_1andr_0=r_1, this case reduces to the previous one. - Finally, the most general description of the serialization of a type
Xis given by a splitting function\mathit{split}_XforX, which accepts one Slice parameters, and returns two Slices,s'ands'', wheres'is the only prefix ofsthat is the serialization of a value of typeX, ands''is the remainder ofs. If no such prefix exists, the splitting function is expected to throw an exception. Notice that a compiler for a high-level language, which supports some or all algebraic TL-B types, is likely to automatically generate splitting functions for all types defined in the program.
3.3.9. A simplifying assumption on the serialization of X
One may notice that values of type X always occupy the remaining part of an \texttt{hm\_edge}/\texttt{hme\_leaf} cell inside the serialization of a HashmapE n X. Therefore, if we do not insist on strict validation of all dictionaries accessed, we may assume that everything left unparsed in an \texttt{hm\_edge}/\texttt{hme\_leaf} cell after deserializing its \texttt{label} is a value of type X. This greatly simplifies the creation of dictionary manipulation primitives, because in most cases they turn out not to need any information about X at all.
3.3.10. Basic dictionary operations
Let us present a classification of basic operations with dictionaries (i.e., values D of type HashmapE n X):
-
\text{Get}(D,k)— GivenD:HashmapE(n,X)and a keyk:n\cdot\texttt{bit}, returns the corresponding valueD[k]:X^?kept inD. -
\text{Set}(D,k,x)— GivenD:HashmapE(n,X), a keyk:n\cdot\texttt{bit}, and a valuex:X, setsD'[k]toxin a copyD'ofD, and returns the resulting dictionaryD'. -
\text{Add}(D,k,x)— Similar to\text{Set}, but adds the key-value pair(k,x)toDonly if keykis absent inD. -
\text{Replace}(D,k,x)— Similar to\text{Set}, but changesD'[k]toxonly if keykis already present inD. -
\text{GetSet},\text{GetAdd},\text{GetReplace}— Similar to\text{Set},\text{Add}, and\text{Replace}, respectively, but returns the old value ofD[k]as well. -
\text{Delete}(D,k)— Deletes keykfrom dictionaryD, and returns the resulting dictionaryD'. -
\text{GetMin}(D),\text{GetMax}(D)— Gets the minimal or maximal keykfrom dictionaryD, along with the associated valuex:X. -
\text{RemoveMin}(D),\text{RemoveMax}(D)— Similar to\text{GetMin}and\text{GetMax}, but also removes the key in question from dictionaryD, and returns the modified dictionaryD'. May be used to iterate over all elements ofD, effectively using (a copy of)Ditself as an iterator. -
\text{GetNext}(D,k)— Computes the minimal keyk'>k(ork'\geq kin a variant) and returns it along with the corresponding valuex':X. May be used to iterate over all elements ofD. -
\text{GetPrev}(D,k)— Computes the maximal keyk'<k(ork'\leq kin a variant) and returns it along with the corresponding valuex':X. -
\text{Empty}(n)— Creates an empty dictionaryD:HashmapE(n,X). -
\text{IsEmpty}(D)— Checks whether a dictionary is empty. -
\text{Create}(n,\{(k_i,x_i)\})— Givenn, creates a dictionary from a list(k_i,x_i)of key-value pairs passed in stack. -
\text{GetSubdict}(D,l,k_0)— GivenD:HashmapE(n,X)and somel-bit stringk_0:l\cdot\texttt{bit}for0\leq l\leq n, returns subdictionaryD'=D/k_0ofD, consisting of keys beginning withk_0. The resultD'may be of either type HashmapE(n,X)or type HashmapE(n-l,X). -
\text{ReplaceSubdict}(D,l,k_0,D')— GivenD:HashmapE(n,X),0\leq l\leq n,k_0:l\cdot\texttt{bit}, andD':HashmapE(n-l,X), replaces withD'the subdictionaryD/k_0ofDconsisting of keys beginning withk_0, and returns the resulting dictionaryD'':HashmapE(n,X). Some variants of\text{ReplaceSubdict}may also return the old value of the subdictionaryD/k_0in question. -
\text{DeleteSubdict}(D,l,k_0)— Equivalent to\text{ReplaceSubdict}withD'being an empty dictionary. -
\text{Split}(D)— GivenD:HashmapE(n,X), returnsD_0:=D/0andD_1:=D/1:HashmapE(n-1,X), the two subdictionaries ofDconsisting of all keys beginning with0and1, respectively. -
\text{Merge}(D_0,D_1)— GivenD_0andD_1:HashmapE(n-1,X), computesD:HashmapE(n,X), such thatD/0=D_0andD/1=D_1. -
\text{ForEach}(D,f)— Executes a functionfwith two argumentskandx, with(k,x)running over all key-value pairs of a dictionaryDin lexicographical order.18 -
\text{ForEachRev}(D,f)— Similar to\text{ForEach}, but processes all key-value pairs in reverse order. -
\text{TreeReduce}(D,o,f,g)— GivenD:HashmapE(n,X), a valueo:X, and two functionsf:X\to Yandg:Y\times Y\to Y, performs a "tree reduction" ofDby first applyingfto all the leaves, and then usinggto compute the value corresponding to a fork starting from the values assigned to its children.19
3.3.11. Taxonomy of dictionary primitives
The dictionary primitives, can be classified according to the following categories:
- Which dictionary operation do they perform?
- Are they specialized for the case
X = \text{\textasciicircum}YIf so, do they represent values of typeYby Cells or by Slices? (Generic versions always represent values of typeXas Slices.) - Are the dictionaries themselves passed and returned as Cells or as Slices? (Most primitives represent dictionaries as Slices.)
- Is the key length
nfixed inside the primitive, or is it passed in the stack? - Are the keys represented by Slices, or by signed or unsigned Integers?
In addition, TVM includes special serialization/deserialization primitives, such as \texttt{STDICT}, \texttt{LDDICT}, and \texttt{PLDDICT}. They can be used to extract a dictionary from a serialization of an encompassing object, or to insert a dictionary into such a serialization.
3.4 Hashmaps with variable-length keys
TVM provides some support for dictionaries, or hashmaps, with variable-length keys, in addition to its support for dictionaries with fixed-length keys.
3.4.1. Serialization of dictionaries with variable-length keys
The serialization of a VarHashmap into a tree of cells (or, more generally, into a Slice) is defined by a TL-B scheme:
vhm_edge#_ {n:#} {X:Type} {l:#} {m:#} label:(HmLabel ~l n)
{n = (~m) + l} node:(VarHashmapNode m X)
= VarHashmap n X;
vhmn_leaf$00 {n:#} {X:Type} value:X = VarHashmapNode n X;
vhmn_fork$01 {n:#} {X:Type} left:^(VarHashmap n X)
right:^(VarHashmap n X) value:(Maybe X)
= VarHashmapNode (n + 1) X;
vhmn_cont$1 {n:#} {X:Type} branch:bit child:^(VarHashmap n X)
value:X = VarHashmapNode (n + 1) X;
nothing$0 {X:Type} = Maybe X;
just$1 {X:Type} value:X = Maybe X;
vhme_empty$0 {n:#} {X:Type} = VarHashmapE n X;
vhme_root$1 {n:#} {X:Type} root:^(VarHashmap n X)
= VarHashmapE n X;3.4.2. Serialization of prefix codes
One special case of a dictionary with variable-length keys is that of a prefix code, where the keys cannot be prefixes of each other. Values in such dictionaries may occur only in the leaves of a Patricia tree.
The serialization of a prefix code is defined by the following TL-B scheme:
phm_edge#_ {n:#} {X:Type} {l:#} {m:#} label:(HmLabel ~l n)
{n = (~m) + l} node:(PfxHashmapNode m X)
= PfxHashmap n X;
phmn_leaf$0 {n:#} {X:Type} value:X = PfxHashmapNode n X;
phmn_fork$1 {n:#} {X:Type} left:^(PfxHashmap n X)
right:^(PfxHashmap n X) = PfxHashmapNode (n + 1) X;
phme_empty$0 {n:#} {X:Type} = PfxHashmapE n X;
phme_root$1 {n:#} {X:Type} root:^(PfxHashmap n X)
= PfxHashmapE n X;4 Control flow, continuations, and exceptions
This chapter describes continuations, which may represent execution tokens and exception handlers in TVM. Continuations are deeply involved with the control flow of a TVM program; in particular, subroutine calls and conditional and iterated execution are implemented in TVM using special primitives that accept one or more continuations as their arguments.
We conclude this chapter with a discussion of the problem of recursion and of families of mutually recursive functions, exacerbated by the fact that cyclic references are not allowed in TVM data structures (including TVM code).
4.1 Continuations and subroutines
Recall that Continuation values represent "execution tokens" that can be executed later—for example, by \texttt{EXECUTE}=\texttt{CALLX} ("execute" or "call indirect") or \texttt{JMPX} ("jump indirect") primitives. As such, the continuations are heavily used by control flow primitives, enabling subroutine calls, conditional expressions, loops, and so on.
4.1.1. Ordinary continuations
The most common kind of continuations are the ordinary continuations, containing the following data:
- A Slice
\texttt{code}(1.1.3 and 3.2.2), containing (the remainder of) the TVM code to be executed. - A (possibly empty) Stack
\texttt{stack}, containing the original contents of the stack for the code to be executed. - A (possibly empty) list
\texttt{save}of pairs(\texttt{c}(i),v_i)(also called "savelist"), containing the values of control registers to be restored before the execution of the code. - A 16-bit integer value
\texttt{cp}, selecting the TVM codepage used to interpret the TVM code from\texttt{code}. - An optional non-negative integer
\texttt{nargs}, indicating the number of arguments expected by the continuation.
4.1.2. Simple ordinary continuations
In most cases, the ordinary continuations are the simplest ones, having empty \texttt{stack} and \texttt{save}. They consist essentially of a reference \texttt{code} to (the remainder of) the code to be executed, and of the codepage \texttt{cp} to be used while decoding the instructions from this code.
4.1.3. Current continuation cc
The "current continuation" \texttt{cc} is an important part of the total state of TVM, representing the code being executed right now. In particular, what we call the current stack (or simply "the stack") when discussing all other primitives is in fact the stack of the current continuation. All other components of the total state of TVM may be also thought of as parts of the current continuation \texttt{cc}; however, they may be extracted from the current continuation and kept separately as part of the total state for performance reasons. This is why we describe the stack, the control registers, and the codepage as separate parts of the TVM state.
4.1.4. Normal work of TVM, or the main loop
TVM usually performs the following operations:
If the current continuation \texttt{cc} is an ordinary one, it decodes the first instruction from the Slice \texttt{code}, similarly to the way other cells are deserialized by TVM \texttt{LD*} primitives (3.2 and 3.2.11): it decodes the opcode first, and then the parameters of the instruction (e.g., 4-bit fields indicating "stack registers" involved for stack manipulation primitives, or constant values for "push constant" or "literal" primitives). The remainder of the Slice is then put into the \texttt{code} of the new \texttt{cc}, and the decoded operation is executed on the current stack. This entire process is repeated until there are no operations left in \texttt{cc.code}.
If the \texttt{code} is empty (i.e., contains no bits of data and no references), or if a (rarely needed) explicit subroutine return (\texttt{RET}) instruction is encountered, the current continuation is discarded, and the "return continuation" from control register \texttt{c0} is loaded into \texttt{cc} instead.20 Then the execution continues by parsing operations from the new current continuation.
4.1.5. Extraordinary continuations
In addition to the ordinary continuations, TVM includes some extraordinary continuations, representing certain less common states. Examples of extraordinary continuations include:
- The continuation
\texttt{ec\_quit}with its parameter set to zero, which represents the end of the work of TVM. This continuation is the original value of\texttt{c0}when TVM begins executing the code of a smart contract. - The continuation
\texttt{ec\_until}, which contains references to two other continuations (ordinary or not) representing the body of the loop being executed and the code to be executed after the loop.
Execution of an extraordinary continuation by TVM depends on its specific class, and differs from the operations for ordinary continuations.21
4.1.6. Switching to another continuation: \texttt{JMP} and \texttt{RET}
The process of switching to another continuation c may be performed by such instructions as \texttt{JMPX} (which takes c from the stack) or \texttt{RET} (which uses \texttt{c0} as c). This process is slightly more complex than simply setting the value of \texttt{cc} to c: before doing this, either all values or the top n values in the current stack are moved to the stack of the continuation c, and only then is the remainder of the current stack discarded.
If all values need to be moved (the most common case), and if the continuation c has an empty stack (also the most common case; notice that extraordinary continuations are assumed to have an empty stack), then the new stack of c equals the stack of the current continuation, so we can simply transfer the current stack in its entirety to c. (If we keep the current stack as a separate part of the total state of TVM, we have to do nothing at all.)
4.1.7. Determining the number n of arguments passed to the next continuation c
By default, n equals the depth of the current stack. However, if c has an explicit value of \texttt{nargs} (number of arguments to be provided), then n is computed as n', equal to c.\texttt{nargs} minus the current depth of c's stack.
Furthermore, there are special forms of \texttt{JMPX} and \texttt{RET} that provide an explicit value n'', the number of parameters from the current stack to be passed to continuation c. If n'' is provided, it must be less than or equal to the depth of the current stack, or else a stack underflow exception occurs. If both n' and n'' are provided, we must have n'\leq n'', in which case n=n' is used. If n'' is provided and n' is not, then n=n'' is used.
One could also imagine that the default value of n'' equals the depth of the original stack, and that n'' values are always removed from the top of the original stack even if only n' of them are actually moved to the stack of the next continuation c. Even though the remainder of the current stack is discarded afterwards, this description will become useful later.
4.1.8. Restoring control registers from the new continuation c
After the new stack is computed, the values of control registers present in c.\texttt{save} are restored accordingly, and the current codepage \texttt{cp} is also set to c.\texttt{cp}. Only then does TVM set \texttt{cc} equal to the new c and begin its execution.22
4.1.9. Subroutine calls: \texttt{CALLX} or \texttt{EXECUTE} primitives
The execution of continuations as subroutines is slightly more complicated than switching to continuations.
Consider the \texttt{CALLX} or \texttt{EXECUTE} primitive, which takes a continuation c from the (current) stack and executes it as a subroutine.
Apart from doing the stack manipulations described in 4.1.6 and 4.1.7 and setting the new control registers, these primitives perform several additional steps:
-
After the top
n''values are removed from the current stack, the (usually empty) remainder is not discarded, but instead is stored in the (old) current continuation\texttt{cc}. -
The old value of the special register
\texttt{c0}is saved into the (previously empty) savelist\texttt{cc.save}. -
The continuation
\texttt{cc}thus modified is not discarded, but instead is set as the new\texttt{c0}, which performs the role of "next continuation" or "return continuation" for the subroutine being called. -
After that, the switching to
ccontinues as before. In particular, some control registers are restored fromc.\texttt{save}, potentially overwriting the value of\texttt{c0}set in the previous step. (Therefore, a good optimization would be to check that\texttt{c0}is present inc.\texttt{save}from the very beginning, and skip the three previous steps as useless in this case.)
In this way, the called subroutine can return control to the caller by switching the current continuation to the return continuation saved in \texttt{c0}. Nested subroutine calls work correctly because the previous value of \texttt{c0} ends up saved into the new \texttt{c0}'s control register savelist \texttt{c0.save}, from which it is restored later.
4.1.10. Determining the number of arguments passed to and/or return values accepted from a subroutine
Similarly to \texttt{JMPX} and \texttt{RET}, \texttt{CALLX} also has special (rarely used) forms, which allow us to explicitly specify the number n'' of arguments passed from the current stack to the called subroutine (by default, n'' equals the depth of the current stack, i.e., it is passed in its entirety). Furthermore, a second number n''' can be specified, used to set \texttt{nargs} of the modified \texttt{cc} continuation before storing it into the new \texttt{c0}; the new \texttt{nargs} equals the depth of the old stack minus n'' plus n'''. This means that the caller is willing to pass exactly n'' arguments to the called subroutine, and is willing to accept exactly n''' results in their stead.
Such forms of \texttt{CALLX} and \texttt{RET} are mostly intended for library functions that accept functional arguments and want to invoke them safely. Another application is related to the "virtualization support" of TVM, which enables TVM code to run other TVM code inside a "virtual TVM machine". Such virtualization techniques might be useful for implementing sophisticated payment channels in the TON Blockchain.
4.1.11. \texttt{CALLCC}: call with current continuation
Notice that TVM supports a form of the "call with current continuation" primitive. Namely, primitive \texttt{CALLCC} is similar to \texttt{CALLX} or \texttt{JMPX} in that it takes a continuation c from the stack and switches to it; however, \texttt{CALLCC} does not discard the previous current continuation c' (as \texttt{JMPX} does) and does not write c' to \texttt{c0} (as \texttt{CALLX} does), but rather pushes c' into the (new) stack as an extra argument to c. The primitive \texttt{JMPXDATA} does a similar thing, but pushes only the (remainder of the) code of the previous current continuation as a Slice.
4.2 Control flow primitives: conditional and iterated execution
4.2.1. Conditional execution: \texttt{IF}, \texttt{IFNOT}, \texttt{IFELSE}
An important modification of \texttt{EXECUTE} (or \texttt{CALLX}) consists in its conditional forms. For example, \texttt{IF} accepts an integer x and a continuation c, and executes c (in the same way as \texttt{EXECUTE} would do it) only if x is non-zero; otherwise both values are simply discarded from the stack. Similarly, \texttt{IFNOT} accepts x and c, but executes c only if x=0. Finally, \texttt{IFELSE} accepts x, c, and c', removes these values from the stack, and executes c if x\neq0 or c' if x=0.
4.2.2. Iterated execution and loops
More sophisticated modifications of \texttt{EXECUTE} include:
\texttt{REPEAT}— Takes an integernand a continuationc, and executescntimes.23\texttt{WHILE}— Takesc'andc'', executesc', and then takes the top valuexfrom the stack. Ifxis non-zero, it executesc''and then begins a new loop by executingc'again; ifxis zero, it stops.\texttt{UNTIL}— Takesc, executes it, and then takes the top integerxfrom the stack. Ifxis zero, a new iteration begins; ifxis non-zero, the previously executed code is resumed.
4.2.3. Constant, or literal, continuations
We see that we can create arbitrarily complex conditional expressions and loops in the TVM code, provided we have a means to push constant continuations into the stack. In fact, TVM includes special versions of "literal" or "constant" primitives that cut the next n bytes or bits from the remainder of the current code \texttt{cc.code} into a cell slice, and then push it into the stack not as a Slice (as a \texttt{PUSHSLICE} does) but as a simple ordinary Continuation (which has only \texttt{code} and \texttt{cp}).
The simplest of these primitives is \texttt{PUSHCONT}, which has an immediate argument n describing the number of subsequent bytes (in a byte-oriented version of TVM) or bits to be converted into a simple continuation. Another primitive is \texttt{PUSHREFCONT}, which removes the first cell reference from the current continuation \texttt{cc.code}, converts the cell referred to into a cell slice, and finally converts the cell slice into a simple continuation.
4.2.4. Constant continuations combined with conditional or iterated execution primitives
Because constant continuations are very often used as arguments to conditional or iterated execution primitives, combined versions of these primitives (e.g., \texttt{IFCONT} or \texttt{UNTILREFCONT}) may be defined in a future revision of TVM, which combine a \texttt{PUSHCONT} or \texttt{PUSHREFCONT} with another primitive. If one inspects the resulting code, \texttt{IFCONT} looks very much like the more customary "conditional-branch-forward" instruction.
4.3 Operations with continuations
4.3.1 Continuations are opaque
Notice that all continuations are opaque, at least in the current version of TVM, meaning that there is no way to modify a continuation or inspect its internal data. Almost the only use of a continuation is to supply it to a control flow primitive.
While there are some arguments in favor of including support for non-opaque continuations in TVM (along with opaque continuations, which are required for virtualization), the current revision offers no such support.
4.3.2. Allowed operations with continuations
However, some operations with opaque continuations are still possible, mostly because they are equivalent to operations of the kind "create a new continuation, which will do something special, and then invoke the original continuation". Allowed operations with continuations include:
- Push one or several values into the stack of a continuation
c(thus creating a partial application of a function, or a closure). - Set the saved value of a control register
\texttt{c}(i)inside the savelistc.\texttt{save}of a continuationc. If there is already a value for the control register in question, this operation silently does nothing.
4.3.3. Example: operations with control registers
TVM has some primitives to set and inspect the values of control registers. The most important of them are \texttt{PUSH c}(i) (pushes the current value of \texttt{c}(i) into the stack) and \texttt{POP c}(i) (sets the value of \texttt{c}(i) from the stack, if the supplied value is of the correct type). However, there is also a modified version of the latter instruction, called \texttt{POPSAVE c}(i), which saves the old value of \texttt{c}(i) (for i>0) into the continuation at \texttt{c0} before setting the new value.
4.3.4. Example: setting the number of arguments to a function in its code
The primitive \texttt{LEAVEARGS} n demonstrates another application of continuations in an operation: it leaves only the top n values of the current stack, and moves the remainder to the stack of the continuation in \texttt{c0}. This primitive enables a called function to "return" unneeded arguments to its caller's stack, which is useful in some situations (e.g., those related to exception handling).
4.3.5. Boolean circuits
A continuation c may be thought of as a piece of code with two optional exit points kept in the savelist of c: the principal exit point given by c.\texttt{c0}:=c.\texttt{save}(\texttt{c0}), and the auxiliary exit point given by c.\texttt{c1}:=c.\texttt{save}(\texttt{c1}). If executed, a continuation performs whatever action it was created for, and then (usually) transfers control to the principal exit point, or, on some occasions, to the auxiliary exit point. We sometimes say that a continuation c with both exit points c.\texttt{c0} and c.\texttt{c1} defined is a two-exit continuation, or a boolean circuit, especially if the choice of the exit point depends on some internally-checked condition.
4.3.6. Composition of continuations
One can compose two continuations c and c' simply by setting c.\texttt{c0} or c.\texttt{c1} to c'. This creates a new continuation denoted by c\circ_0c' or c\circ_1c', which differs from c in its savelist. (Recall that if the savelist of c already has an entry corresponding to the control register in question, such an operation silently does nothing.
By composing continuations, one can build chains or other graphs, possibly with loops, representing the control flow. In fact, the resulting graph resembles a flow chart, with the boolean circuits corresponding to the "condition nodes" (containing code that will transfer control either to \texttt{c0} or to \texttt{c1} depending on some condition), and the one-exit continuations corresponding to the "action nodes".
4.3.7. Basic continuation composition primitives
Two basic primitives for composing continuations are \texttt{COMPOS} (also known as \texttt{SETCONT c0} and \texttt{BOOLAND}) and \texttt{COMPOSALT} (also known as \texttt{SETCONT c1} and \texttt{BOOLOR}), which take c and c' from the stack, set c.\texttt{c0} or c.\texttt{c1} to c', and return the resulting continuation c''=c\circ_0c' or c\circ_1c'. All other continuation composition operations can be expressed in terms of these two primitives.
4.3.8. Advanced continuation composition primitives
However, TVM can compose continuations not only taken from stack, but also taken from \texttt{c0} or \texttt{c1}, or from the current continuation \texttt{cc}; likewise, the result may be pushed into the stack, stored into either \texttt{c0} or \texttt{c1}, or used as the new current continuation (i.e., the control may be transferred to it). Furthermore, TVM can define conditional composition primitives, performing some of the above actions only if an integer value taken from the stack is non-zero.
For instance, \texttt{EXECUTE} can be described as \texttt{cc}\leftarrow c\circ_0\texttt{cc}, with continuation c taken from the original stack. Similarly, \texttt{JMPX} is \texttt{cc}\leftarrow c, and \texttt{RET} (also known as \texttt{RETTRUE} in a boolean circuit context) is \texttt{cc}\leftarrow\texttt{c0}. Other interesting primitives include \texttt{THENRET} (c'\leftarrow c\circ_0\texttt{c0}) and \texttt{ATEXIT} (\texttt{c0}\leftarrow c\circ_0\texttt{c0}).
Finally, some "experimental" primitives also involve \texttt{c1} and \circ_1. For example:
\texttt{RETALT}or\texttt{RETFALSE}does\texttt{cc}\leftarrow\texttt{c1}.- Conditional versions of
\texttt{RET}and\texttt{RETALT}may also be useful:\texttt{RETBOOL}takes an integerxfrom the stack, and performs\texttt{RETTRUE}ifx\neq0,\texttt{RETFALSE}otherwise. \texttt{INVERT}does\texttt{c0}\leftrightarrow\texttt{c1}; if the two continuations in\texttt{c0}and\texttt{c1}represent the two branches we should select depending on some boolean expression,\texttt{INVERT}negates this expression on the outer level.\texttt{INVERTCONT}doesc.\texttt{c0}\leftrightarrow c.\texttt{c1}to a continuationctaken from the stack.- Variants of
\texttt{ATEXIT}include\texttt{ATEXITALT}(\texttt{c1}\leftarrow c\circ_1\texttt{c1}) and\texttt{SETEXITALT}(\texttt{c1}\leftarrow (c\circ_0\texttt{c0})\circ_1\texttt{c1}). \texttt{BOOLEVAL}takes a continuationcfrom the stack and does\texttt{cc}\leftarrow \bigl((c\circ_0(\texttt{PUSH}\,-1))\circ_1(\texttt{PUSH}\,0)\bigr)\circ_0\texttt{cc}. Ifcrepresents a boolean circuit, the net effect is to evaluate it and push either-1or0into the stack before continuing.
4.4 Continuations as objects
4.4.1. Representing objects using continuations
Object-oriented programming in Smalltalk (or Objective C) style may be implemented with the aid of continuations. For this, an object is represented by a special continuation o. If it has any data fields, they can be kept in the stack of o, making o a partial application (i.e., a continuation with a non-empty stack).
When somebody wants to invoke a method m of o with arguments x_1, x_2, \ldots, x_n, she pushes the arguments into the stack, then pushes a magic number corresponding to the method m, and then executes o passing n+1 arguments. Then o uses the top-of-stack integer m to select the branch with the required method, and executes it. If o needs to modify its state, it simply computes a new continuation o' of the same sort (perhaps with the same code as o, but with a different initial stack). The new continuation o' is returned to the caller along with whatever other return values need to be returned.
4.4.2. Serializable objects
Another way of representing Smalltalk-style objects as continuations, or even as trees of cells, consists in using the \texttt{JMPREFDATA} primitive (a variant of \texttt{JMPXDATA}), which takes the first cell reference from the code of the current continuation, transforms the cell referred to into a simple ordinary continuation, and transfers control to it, first pushing the remainder of the current continuation as a Slice into the stack. In this way, an object might be represented by a cell \tilde o that contains \texttt{JMPREFDATA} at the beginning of its data, and the actual code of the object in the first reference (one might say that the first reference of cell \tilde o is the class of object \tilde o). Remaining data and references of this cell will be used for storing the fields of the object.
Such objects have the advantage of being trees of cells, and not just continuations, meaning that they can be stored into the persistent storage of a TON smart contract.
4.4.3. Unique continuations and capabilities
It might make sense (in a future revision of TVM) to mark some continuations as unique, meaning that they cannot be copied, even in a delayed manner, by increasing their reference counter to a value greater than one. If an opaque continuation is unique, it essentially becomes a capability, which can either be used by its owner exactly once or be transferred to somebody else.
For example, imagine a continuation that represents the output stream to a printer (this is an example of a continuation used as an object). When invoked with one integer argument n, this continuation outputs the character with code n to the printer, and returns a new continuation of the same kind reflecting the new state of the stream. Obviously, copying such a continuation and using the two copies in parallel would lead to some unintended side effects; marking it as unique would prohibit such adverse usage.
4.5 Exception handling
TVM's exception handling is quite simple and consists in a transfer of control to the continuation kept in control register \texttt{c2}.
4.5.1. Two arguments of the exception handler: exception parameter and exception number
Every exception is characterized by two arguments: the exception number (an Integer) and the exception parameter (any value, most often a zero Integer). Exception numbers 0–31 are reserved for TVM, while all other exception numbers are available for user-defined exceptions.
4.5.2. Primitives for throwing an exception
There are several special primitives used for throwing an exception. The most general of them, \texttt{THROWANY}, takes two arguments, v and 0\leq n<2^{16}, from the stack, and throws the exception with number n and value v. There are variants of this primitive that assume v to be a zero integer, store n as a literal value, and/or are conditional on an integer value taken from the stack. User-defined exceptions may use arbitrary values as v (e.g., trees of cells) if needed.
4.5.3. Exceptions generated by TVM
Of course, some exceptions are generated by normal primitives. For example, an arithmetic overflow exception is generated whenever the result of an arithmetic operation does not fit into a signed 257-bit integer. In such cases, the arguments of the exception, v and n, are determined by TVM itself.
4.5.4. Exception handling
The exception handling itself consists in a control transfer to the exception handler—i.e., the continuation specified in control register \texttt{c2}, with v and n supplied as the two arguments to this continuation, as if a \texttt{JMP} to \texttt{c2} had been requested with n''=2 arguments (4.1.7 and 4.1.6). As a consequence, v and n end up in the top of the stack of the exception handler. The remainder of the old stack is discarded.
Notice that if the continuation in \texttt{c2} has a value for \texttt{c2} in its savelist, it will be used to set up the new value of \texttt{c2} before executing the exception handler. In particular, if the exception handler invokes \texttt{THROWANY}, it will re-throw the original exception with the restored value of \texttt{c2}. This trick enables the exception handler to handle only some exceptions, and pass the rest to an outer exception handler.
4.5.5. Default exception handler
When an instance of TVM is created, \texttt{c2} contains a reference to the "default exception handler continuation", which is an \texttt{ec\_fatal} extraordinary continuation. Its execution leads to the termination of the execution of TVM, with the arguments v and n of the exception returned to the outside caller. In the context of the TON Blockchain, n will be stored as a part of the transaction's result.
4.5.6. \texttt{TRY} primitive
A \texttt{TRY} primitive can be used to implement C++- like exception handling. This primitive accepts two continuations, c and c'. It stores the old value of \texttt{c2} into the savelist of c', sets \texttt{c2} to c', and executes c just as \texttt{EXECUTE} would, but additionally saving the old value of \texttt{c2} into the savelist of the new \texttt{c0} as well. Usually a version of the \texttt{TRY} primitive with an explicit number of arguments n'' passed to the continuation c is used.
The net result is roughly equivalent to C++'s \texttt{try} { c } \texttt{catch(...)} { c' } operator.
4.5.7. List of predefined exceptions
Predefined exceptions of TVM correspond to exception numbers n in the range 0–31. They include:
- Normal termination (
n=0) — Should never be generated, but it is useful for some tricks. - Alternative termination (
n=1) — Again, should never be generated. - Stack underflow (
n=2) — Not enough arguments in the stack for a primitive. - Stack overflow (
n=3) — More values have been stored on a stack than allowed by this version of TVM. - Integer overflow (
n=4) — Integer does not fit into-2^{256}\leq x<2^{256}, or a division by zero has occurred. - Range check error (
n=5) — Integer out of expected range. - Invalid opcode (
n=6) — Instruction or its immediate arguments cannot be decoded. - Type check error (
n=7) — An argument to a primitive is of incorrect value type. - Cell overflow (
n=8) — Error in one of the serialization primitives. - Cell underflow (
n=9) — Deserialization error. - Dictionary error (
n=10) — Error while deserializing a dictionary object. - Unknown error (
n=11) — Unknown error, may be thrown by user programs. - Fatal error (
n=12) — Thrown by TVM in situations deemed impossible. - Out of gas (
n=13) — Thrown by TVM when the remaining gas (g_r) becomes negative. This exception usually cannot be caught and leads to an immediate termination of TVM.
Most of these exceptions have no parameter (i.e., use a zero integer instead). The order in which these exceptions are checked is outlined below.
4.5.8. Order of stack underflow, type check, and range check exceptions
All TVM primitives first check whether the stack contains the required number of arguments, generating a stack underflow exception if this is not the case. Only then are the type tags of the arguments and their ranges (e.g., if a primitive expects an argument not only to be an Integer, but also to be in the range from 0 to 256) checked, starting from the value in the top of the stack (the last argument) and proceeding deeper into the stack. If an argument's type is incorrect, a type-checking exception is generated; if the type is correct, but the value does not fall into the expected range, a range check exception is generated.
Some primitives accept a variable number of arguments, depending on the values of some small fixed subset of arguments located near the top of the stack. In this case, the above procedure is first run for all arguments from this small subset. Then it is repeated for the remaining arguments, once their number and types have been determined from the arguments already processed.
4.6 Functions, recursion, and dictionaries
4.6.1. The problem of recursion
The conditional and iterated execution primitives—along with the unconditional branch, call, and return primitives—enable one to implement more or less arbitrary code with nested loops and conditional expressions, with one notable exception: one can only create new constant continuations from parts of the current continuation. (In particular, one cannot invoke a subroutine from itself in this way.) Therefore, the code being executed—i.e., the current continuation—gradually becomes smaller and smaller.24
4.6.2. Y-combinator solution: pass a continuation as an argument to itself
One way of dealing with the problem of recursion is by passing a copy of the continuation representing the body of a recursive function as an extra argument to itself. Consider, for example, the following code for a factorial function:
71 PUSHINT 1
9C PUSHCONT {
22 PUSH s2
72 PUSHINT 2
B9 LESS
DC IFRET
59 ROTREV
21 PUSH s1
A8 MUL
01 SWAP
A5 DEC
02 XCHG s2
20 DUP
D9 JMPX
}
20 DUP
D8 EXECUTE
30 DROP
31 NIPThis roughly corresponds to defining an auxiliary function \mathit{body} with three arguments n, x, and f, such that \mathit{body}(n,x,f) equals x if n<2 and f(n-1,nx,f) otherwise, then invoking \mathit{body}(n,1,\mathit{body}) to compute the factorial of n.
The recursion is then implemented with the aid of the \texttt{DUP}; \texttt{EXECUTE} construction, or \texttt{DUP}; \texttt{JMPX} in the case of tail recursion. This trick is equivalent to applying Y-combinator to a function \mathit{body}.
4.6.3. A variant of Y-combinator solution
Another way of recursively computing the factorial, more closely following the classical recursive definition
\mathit{fact}(n):=\begin{cases}1&\quad\text{if }n<2,\\n\cdot\mathit{fact}(n-1)&\quad\text{otherwise}\end{cases} \tag{5}is as follows:
9D PUSHCONT {
21 OVER
C102 LESSINT 2
92 PUSHCONT {
5B 2DROP
71 PUSHINT 1
}
E0 IFJMP
21 OVER
A5 DEC
01 SWAP
20 DUP
D8 EXECUTE
A8 MUL
}
20 DUP
D9 JMPXThis definition of the factorial function is two bytes shorter than the previous one, but it uses general recursion instead of tail recursion, so it cannot be easily transformed into a loop.
4.6.4. Comparison: non-recursive definition of the factorial function
Incidentally, a non-recursive definition of the factorial with the aid of a \texttt{REPEAT} loop is also possible, and it is much shorter than both recursive definitions:
71 PUSHINT 1
01 SWAP
20 DUP
94 PUSHCONT {
66 TUCK
A8 MUL
01 SWAP
A5 DEC
}
E4 REPEAT
30 DROP4.6.5. Several mutually recursive functions
If one has a collection f_1, \ldots, f_n of mutually recursive functions, one can use the same trick by passing the whole collection of continuations \{f_i\} in the stack as an extra n arguments to each of these functions. However, as n grows, this becomes more and more cumbersome, since one has to reorder these extra arguments in the stack to work with the "true" arguments, and then push their copies into the top of the stack before any recursive call.
4.6.6. Combining several functions into one tuple
One might also combine a collection of continuations representing functions f_1, \ldots, f_n into a "tuple" {\mathbf f}:=(f_1,\ldots,f_n), and pass this tuple as one stack element {\mathbf f}. For instance, when n\leq4, each function can be represented by a cell \tilde f_i (along with the tree of cells rooted in this cell), and the tuple may be represented by a cell \tilde{\mathbf f}, which has references to its component cells \tilde f_i. However, this would lead to the necessity of "unpacking" the needed component from this tuple before each recursive call.
4.6.7. Combining several functions into a selector function
Another approach is to combine several functions f_1, \ldots, f_n into one "selector function" f, which takes an extra argument i, 1\leq i\leq n, from the top of the stack, and invokes the appropriate function f_i. Stack machines such as TVM are well-suited to this approach, because they do not require the functions f_i to have the same number and types of arguments. Using this approach, one would need to pass only one extra argument, f, to each of these functions, and push into the stack an extra argument i before each recursive call to f to select the correct function to be called.
4.6.8. Using a dedicated register to keep the selector function
However, even if we use one of the two previous approaches to combine all functions into one extra argument, passing this argument to all mutually recursive functions is still quite cumbersome and requires a lot of additional stack manipulation operations. Because this argument changes very rarely, one might use a dedicated register to keep it and transparently pass it to all functions called. This is the approach used by TVM by default.
4.6.9. Special register c3 for the selector function
In fact, TVM uses a dedicated register \texttt{c3} to keep the continuation representing the current or global "selector function", which can be used to invoke any of a family of mutually recursive functions. Special primitives \texttt{CALL} nn or \texttt{CALLDICT} nn are equivalent to \texttt{PUSHINT} nn; \texttt{PUSH c3}; \texttt{EXECUTE}, and similarly \texttt{JMP} nn or \texttt{JMPDICT} nn are equivalent to \texttt{PUSHINT} nn; \texttt{PUSH c3}; \texttt{JMPX}. In this way a TVM program, which ultimately is a large collection of mutually recursive functions, may initialize \texttt{c3} with the correct selector function representing the family of all the functions in the program, and then use \texttt{CALL} nn to invoke any of these functions by its index (sometimes also called the selector of a function).
4.6.10. Initialization of c3
A TVM program might initialize \texttt{c3} by means of a \texttt{POP c3} instruction. However, because this usually is the very first action undertaken by a program (e.g., a smart contract), TVM makes some provisions for the automatic initialization of \texttt{c3}. Namely, \texttt{c3} is initialized by the code (the initial value of \texttt{cc}) of the program itself, and an extra zero (or, in some cases, some other predefined number s) is pushed into the stack before the program's execution. This is approximately equivalent to invoking \texttt{JMPDICT 0} (or \texttt{JMPDICT} s) at the very beginning of a program—i.e., the function with index zero is effectively the main() function for the program.
4.6.11. Creating selector functions and \texttt{switch} statements
TVM makes special provisions for simple and concise implementation of selector functions (which usually constitute the top level of a TVM program) or, more generally, arbitrary switch or case statements (which are also useful in TVM programs). The most important primitives included for this purpose are \texttt{IFBITJMP}, \texttt{IFNBITJMP}, \texttt{IFBITJMPREF}, and \texttt{IFNBITJMPREF}. They effectively enable one to combine subroutines, kept either in separate cells or as subslices of certain cells, into a binary decision tree with decisions made according to the indicated bits of the integer passed in the top of the stack.
Another instruction, useful for the implementation of sum-product types, is \texttt{PLDUZ}. This instruction preloads the first several bits of a Slice into an Integer, which can later be inspected by \texttt{IFBITJMP} and other similar instructions.
4.6.12. Alternative: using a hashmap to select the correct function
Yet another alternative is to use a Hashmap to hold the "collection" or "dictionary" of the code of all functions in a program, and use the hashmap lookup primitives to select the code of the required function, which can then be \texttt{BLESS}ed into a continuation and executed. Special combined "lookup, bless, and execute" primitives, such as \texttt{DICTIGETJMP} and \texttt{DICTIGETEXEC}, are also available. This approach may be more efficient for larger programs and switch statements.
5. Codepages and instruction encoding
This chapter describes the codepage mechanism, which allows TVM to be flexible and extendable while preserving backward compatibility with respect to previously generated code.
We also discuss some general considerations about instruction encodings (applicable to arbitrary machine code, not just TVM), as well as the implications of these considerations for TVM and the choices made while designing TVM's (experimental) codepage zero. The instruction encodings themselves are presented later in Appendix A.
5.1 Codepages and interoperability of different TVM versions
The codepages are an essential mechanism of backward compatibility and of future extensions to TVM. They enable transparent execution of code written for different revisions of TVM, with transparent interaction between instances of such code. The mechanism of the codepages, however, is general and powerful enough to enable some other originally unintended applications.
5.1.1. Codepages in continuations
Every ordinary continuation contains a 16-bit codepage field \texttt{cp}, which determines the codepage that will be used to execute its code. If a continuation is created by a \texttt{PUSHCONT} or similar primitive, it usually inherits the current codepage (i.e., the codepage of \texttt{cc}).25
5.1.2. Current codepage
The current codepage \texttt{cp} is the codepage of the current continuation \texttt{cc}. It determines the way the next instruction will be decoded from \texttt{cc.code}, the remainder of the current continuation's code. Once the instruction has been decoded and executed, it determines the next value of the current codepage. In most cases, the current codepage is left unchanged.
On the other hand, all primitives that switch the current continuation load the new value of \texttt{cp} from the new current continuation. In this way, all code in continuations is always interpreted exactly as it was intended to be.
5.1.3. Different versions of TVM may use different codepages
Different versions of TVM may use different codepages for their code. For example, the original version of TVM might use codepage zero. A newer version might use codepage one, which contains all the previously defined opcodes, along with some newly defined ones, using some of the previously unused opcode space. A subsequent version might use yet another codepage, and so on.
However, a newer version of TVM will execute old code for codepage zero exactly as before. If the old code contained an opcode used for some new operations that were undefined in the original version of TVM, it will still generate an invalid opcode exception, because the new operations are absent in codepage zero.
5.1.4. Changing the behavior of old operations
New codepages can also change the effects of some operations present in the old codepages while preserving their opcodes and mnemonics.
For example, imagine a future 513-bit upgrade of TVM (replacing the current 257-bit design). It might use a 513-bit Integer type within the same arithmetic primitives as before. However, while the opcodes and instructions in the new codepage would look exactly like the old ones, they would work differently, accepting 513-bit integer arguments and results. On the other hand, during the execution of the same code in codepage zero, the new machine would generate exceptions whenever the integers used in arithmetic and other primitives do not fit into 257 bits.26 In this way, the upgrade would not change the behavior of the old code.
5.1.5. Improving instruction encoding
Another application for codepages is to change instruction encodings, reflecting improved knowledge of the actual frequencies of such instructions in the code base. In this case, the new codepage will have exactly the same instructions as the old one, but with different encodings, potentially of differing lengths. For example, one might create an experimental version of the first version of TVM, using a (prefix) bitcode instead of the original bytecode, aiming to achieve higher code density.
5.1.6. Making instruction encoding context-dependent
Another way of using codepages to improve code density is to use several codepages with different subsets of the whole instruction set defined in each of them, or with the whole instruction set defined, but with different length encodings for the same instructions in different codepages.
Imagine, for instance, a "stack manipulation" codepage, where stack manipulation primitives have short encodings at the expense of all other operations, and a "data processing" codepage, where all other operations are shorter at the expense of stack manipulation operations. If stack manipulation operations tend to come one after another, we can automatically switch to "stack manipulation" codepage after executing any such instruction. When a data processing instruction occurs, we switch back to "data processing" codepage. If conditional probabilities of the class of the next instruction depending on the class of the previous instruction are considerably different from corresponding unconditional probabilities, this technique—automatically switching into stack manipulation mode to rearrange the stack with shorter instructions, then switching back—might considerably improve the code density.
5.1.7. Using codepages for status and control flags
Another potential application of multiple codepages inside the same revision of TVM consists in switching between several codepages depending on the result of the execution of some instructions.
For example, imagine a version of TVM that uses two new codepages, 2 and 3. Most operations do not change the current codepage. However, the integer comparison operations will switch to codepage 2 if the condition is false, and to codepage 3 if it is true. Furthermore, a new operation \texttt{?EXECUTE}, similar to \texttt{EXECUTE}, will indeed be equivalent to \texttt{EXECUTE} in codepage 3, but will instead be a \texttt{DROP} in codepage 2. Such a trick effectively uses bit 0 of the current codepage as a status flag.
Alternatively, one might create a couple of codepages—say, 4 and 5—which differ only in their cell deserialisation primitives. For instance, in codepage 4 they might work as before, while in codepage 5 they might deserialize data not from the beginning of a Slice, but from its end. Two new instructions—say, \texttt{CLD} and \texttt{STD}—might be used for switching to codepage 4 or codepage 5. Clearly, we have now described a status flag, affecting the execution of some instructions in a certain new manner.
5.1.8. Setting the codepage in the code itself
For convenience, we reserve some opcode in all codepages—say, \texttt{FF} n—for the instruction \texttt{SETCP} n, with n from 0 to 255. Then by inserting such an instruction into the very beginning of (the main function of) a program (e.g., a TON Blockchain smart contract) or a library function, we can ensure that the code will always be executed in the intended codepage.
5.2 Instruction encoding
This section discusses the general principles of instruction encoding valid for all codepages and all versions of TVM. Later, 5.3 discusses the choices made for the experimental "codepage zero".
5.2.1. Instructions are encoded by a binary prefix code
All complete instructions (i.e., instructions along with all their parameters, such as the names of stack registers s(i) or other embedded constants) of a TVM codepage are encoded by a binary prefix code. This means that a (finite) binary string (i.e., a bitstring) corresponds to each complete instruction, in such a way that binary strings corresponding to different complete instructions do not coincide, and no binary string among the chosen subset is a prefix of another binary string from this subset.
5.2.2. Determining the first instruction from a code stream
As a consequence of this encoding method, any binary string admits at most one prefix, which is an encoding of some complete instruction. In particular, the code \texttt{cc.code} of the current continuation (which is a Slice, and thus a bitstring along with some cell references) admits at most one such prefix, which corresponds to the (uniquely determined) instruction that TVM will execute first. After execution, this prefix is removed from the code of the current continuation, and the next instruction can be decoded.
5.2.3. Invalid opcode
If no prefix of \texttt{cc.code} encodes a valid instruction in the current codepage, an invalid opcode exception is generated. However, the case of an empty \texttt{cc.code} is treated separately (the exact behavior may depend on the current codepage).
5.2.4. Special case: end-of-code padding
As an exception to the above rule, some codepages may accept some values of \texttt{cc.code} that are too short to be valid instruction encodings as additional variants of \texttt{NOP}, thus effectively using the same procedure for them as for an empty \texttt{cc.code}. Such bitstrings may be used for padding the code near its end.
For example, if binary string \texttt{00000000} (i.e., \texttt{x00}, is used in a codepage to encode \texttt{NOP}, its proper prefixes cannot encode any instructions. So this codepage may accept \texttt{0}, \texttt{00}, \texttt{000}, \ldots, \texttt{0000000} as variants of \texttt{NOP} if this is all that is left in \texttt{cc.code}, instead of generating an invalid opcode exception.
Such a padding may be useful, for example, if the \texttt{PUSHCONT} primitive creates only continuations with code consisting of an integral number of bytes, but not all instructions are encoded by an integral number of bytes.
5.2.5. TVM code is a bitcode, not a bytecode
Recall that TVM is a bit-oriented machine in the sense that its Cells (and Slices) are naturally considered as sequences of bits, not just of octets (bytes). Because the TVM code is also kept in cells (3.1.9 and 4.1.4), there is no reason to use only bitstrings of length divisible by eight as encodings of complete instructions. In other words, generally speaking, the TVM code is a bitcode, not a bytecode.
That said, some codepages (such as our experimental codepage zero) may opt to use a bytecode (i.e., to use only encodings consisting of an integral number of bytes)—either for simplicity, or for the ease of debugging and of studying memory (i.e., cell) dumps.27
5.2.6. Opcode space used by a complete instruction
Recall from coding theory that the lengths of bitstrings l_i used in a binary prefix code satisfy Kraft–McMillan inequality \sum_i2^{-l_i}\leq1. This is applicable in particular to the (complete) instruction encoding used by a TVM codepage. We say that a particular complete instruction (or, more precisely, the encoding of a complete instruction) utilizes the portion 2^{-l} of the opcode space, if it is encoded by an l-bit string. One can see that all complete instructions together utilize at most 1 (i.e., "at most the whole opcode space").
5.2.7. Opcode space used by an instruction, or a class of instructions
The above terminology is extended to instructions (considered with all admissible values of their parameters), or even classes of instructions (e.g., all arithmetic instructions). We say that an (incomplete) instruction, or a class of instructions, occupies portion \alpha of the opcode space, if \alpha is the sum of the portions of the opcode space occupied by all complete instructions belonging to that class.
5.2.8. Opcode space for bytecodes
A useful approximation of the above definitions is as follows: Consider all 256 possible values for the first byte of an instruction encoding. Suppose that k of these values correspond to the specific instruction or class of instructions we are considering. Then this instruction or class of instructions occupies approximately the portion k/256 of the opcode space.
This approximation shows why all instructions cannot occupy together more than the portion 256/256=1 of the opcode space, at least without compromising the uniqueness of instruction decoding.
5.2.9. Almost optimal encodings
Coding theory tells us that in an optimally dense encoding, the portion of the opcode space used by a complete instruction (2^{-l}, if the complete instruction is encoded in l bits) should be approximately equal to the probability or frequency of its occurrence in real programs.28 The same should hold for (incomplete) instructions, or primitives (i.e., generic instructions without specified values of parameters), and for classes of instructions.
5.2.10. Example: stack manipulation primitives
For instance, if stack manipulation instructions constitute approximately half of all instructions in a typical TVM program, one should allocate approximately half of the opcode space for encoding stack manipulation instructions. One might reserve the first bytes ("opcodes") \texttt{0x00}-\texttt{0x7f} for such instructions. If a quarter of these instructions are \texttt{XCHG}, it would make sense to reserve \texttt{0x00}-\texttt{0x1f} for \texttt{XCHG}s. Similarly, if half of all \texttt{XCHG}s involve the top of stack \texttt{s0}, it would make sense to use \texttt{0x00}-\texttt{0x0f} to encode \texttt{XCHG s0,s}(i).
5.2.11. Simple encodings of instructions
In most cases, simple encodings of complete instructions are used. Simple encodings begin with a fixed bitstring called the opcode of the instruction, followed by, say, 4-bit fields containing the indices i of stack registers \texttt{s}(i) specified in the instruction, followed by all other constant (literal, immediate) parameters included in the complete instruction. While simple encodings may not be exactly optimal, they admit short descriptions, and their decoding and encoding can be easily implemented.
If a (generic) instruction uses a simple encoding with an l-bit opcode, then the instruction will utilize 2^{-l} portion of the opcode space. This observation might be useful for considerations described in 5.2.9 and 5.2.10.
5.2.12. Optimizing code density further: Huffman codes
One might construct optimally dense binary code for the set of all complete instructions, provided their probabilities or frequences in real code are known. This is the well-known Huffman code (for the given probability distribution). However, such code would be highly unsystematic and hard to decode.
5.2.13. Practical instruction encodings
In practice, instruction encodings used in TVM and other virtual machines offer a compromise between code density and ease of encoding and decoding. Such a compromise may be achieved by selecting simple encodings for all instructions (maybe with separate simple encodings for some often used variants, such as \texttt{XCHG s0,s}(i) among all \texttt{XCHG s}(i)\texttt{,s}(j)), and allocating opcode space for such simple encodings using the heuristics outlined in 5.2.9 and 5.2.10; this is the approach currently used in TVM.
5.3 Instruction encoding in codepage zero
This section provides details about the experimental instruction encoding for codepage zero, as described elsewhere in this document and used in the preliminary test version of TVM.
5.3.1. Upgradability
First of all, even if this preliminary version somehow gets into the production version of the TON Blockchain, the codepage mechanism enables us to introduce better versions later without compromising backward compatibility.29 So in the meantime, we are free to experiment.
5.3.2. Choice of instructions
We opted to include many "experimental" and not strictly necessary instructions in codepage zero just to see how they might be used in real code. For example, we have both the basic and the compound stack manipulation primitives, as well as some "unsystematic" ones such as \texttt{ROT} (mostly borrowed from Forth). If such primitives are rarely used, their inclusion just wastes some part of the opcode space and makes the encodings of other instructions slightly less effective, something we can afford at this stage of TVM's development.
5.3.3. Using experimental instructions
Some of these experimental instructions have been assigned quite long opcodes, just to fit more of them into the opcode space. One should not be afraid to use them just because they are long; if these instructions turn out to be useful, they will receive shorter opcodes in future revisions. Codepage zero is not meant to be fine-tuned in this respect.
5.3.4. Choice of bytecode
We opted to use a bytecode (i.e., to use encodings of complete instructions of lengths divisible by eight). While this may not produce optimal code density, because such a length restriction makes it more difficult to match portions of opcode space used for the encoding of instructions with estimated frequencies of these instructions in TVM code 5.2.11 and 5.2.9), such an approach has its advantages: it admits a simpler instruction decoder and simplifies debugging.
After all, we do not have enough data on the relative frequencies of different instructions right now, so our code density optimizations are likely to be very approximate at this stage. The ease of debugging and experimenting and the simplicity of implementation are more important at this point.
5.3.5. Simple encodings for all instructions
For similar reasons, we opted to use simple encodings for all instructions (5.2.11 and 5.2.13), with separate simple encodings for some very frequently used subcases. That said, we tried to distribute opcode space using the heuristics described in 5.2.9 and 5.2.10.
5.3.6. Lack of context-dependent encodings
This version of TVM also does not use context-dependent encodings. They may be added at a later stage, if deemed useful.
5.3.7. The list of all instructions
The list of all instructions available in codepage zero, along with their encodings and (in some cases) short descriptions, may be found in Appendix A.
A Instructions and opcodes
This appendix lists all instructions available in the (experimental) codepage zero of TVM.
We list the instructions in lexicographical opcode order. However, the opcode space is distributed in such way as to make all instructions in each category (e.g., arithmetic primitives) have neighboring opcodes. So we first list a number of stack manipulation primitives, then constant primitives, arithmetic primitives, comparison primitives, cell primitives, continuation primitives, dictionary primitives, and finally application-specific primitives.
We use hexadecimal notation for bitstrings. Stack registers \texttt{s}(i) usually have 0\leq i\leq 15, and i is encoded in a 4-bit field (or, on a few rare occasions, in an 8-bit field). Other immediate parameters are usually 4-bit, 8-bit, or variable length.
The stack notation is extensively used throughout this appendix.
A.1 Gas prices
The gas price for most primitives equals the basic gas price, computed as P_b:=10+b+5r, where b is the instruction length in bits and r is the number of cell references included in the instruction. When the gas price of an instruction differs from this basic price, it is indicated in parentheses after its mnemonics, either as (x), meaning that the total gas price equals x, or as (+x), meaning P_b+x. Apart from integer constants, the following expressions may appear:
C_r— The total price of "reading" cells (i.e., transforming cell references into cell slices). Currently equal to 100 or 25 gas units per cell depending on whether it is the first time a cell with this hash is being "read" during the current run of the VM or not.L— The total price of loading cells. Depends on the loading action required.B_w— The total price of creating new Builders. Currently equal to 0 gas units per builder.C_w— The total price of creating new Cells from Builders. Currently equal to 500 gas units per cell.
By default, the gas price of an instruction equals P:=P_b+C_r+L+B_w+C_w.
A.2 Stack manipulation primitives
This section includes both the basic and the compound stack manipulation primitives, as well as some "unsystematic" ones. Some compound stack manipulation primitives, such as \texttt{XCPU} or \texttt{XCHG2}, turn out to have the same length as an equivalent sequence of simpler operations. We have included these primitives regardless, so that they can easily be allocated shorter opcodes in a future revision of TVM—or removed for good.
Some stack manipulation instructions have two mnemonics: one Forth-style (e.g., \texttt{-ROT}), the other conforming to the usual rules for identifiers (e.g., \texttt{ROTREV}). Whenever a stack manipulation primitive (e.g., \texttt{PICK}) accepts an integer parameter n from the stack, it must be within the range 0\ldots255; otherwise a range check exception happens before any further checks.
A.2.1. Basic stack manipulation primitives
\texttt{00}—\texttt{NOP}, does nothing.\texttt{01}—\texttt{XCHG s1}, also known as\texttt{SWAP}.\texttt{0}i—\texttt{XCHG s}(i)or\texttt{XCHG s0,s}(i), interchanges the top of the stack with\texttt{s}(i),1\leq i\leq 15.\texttt{10}ij—\texttt{XCHG s}(i)\texttt{,s}(j),1\leq i<j\leq15, interchanges\texttt{s}(i)with\texttt{s}(j).\texttt{11}ii—\texttt{XCHG s0,s}(ii), with0\leq ii\leq255.\texttt{1}i—\texttt{XCHG s1,s}(i),2\leq i\leq 15.\texttt{2}i—\texttt{PUSH s}(i),0\leq i\leq 15, pushes a copy of the old\texttt{s}(i)into the stack.\texttt{20}—\texttt{PUSH s0}, also known as\texttt{DUP}.\texttt{21}—\texttt{PUSH s1}, also known as\texttt{OVER}.\texttt{3}i—\texttt{POP s}(i),0\leq i\leq 15, pops the old top-of-stack value into the old\texttt{s}(i).\texttt{30}—\texttt{POP s0}, also known as\texttt{DROP}, discards the top-of-stack value.\texttt{31}—\texttt{POP s1}, also known as\texttt{NIP}.
A.2.2. Compound stack manipulation primitives
Parameters i, j, and k of the following primitives all are 4-bit integers in the range 0\ldots15.
\texttt{4}ijk—\texttt{XCHG3 s}(i)\texttt{,s}(j)\texttt{,s}(k), equivalent to\texttt{XCHG s2,s}(i);\texttt{XCHG s1, s}(j);\texttt{XCHG s0,s}(k), with0\leq i,j,k\leq 15.\texttt{50}ij—\texttt{XCHG2 s}(i)\texttt{,s}(j), equivalent to\texttt{XCHG s1,s}(i);\texttt{XCHG s}(j).\texttt{51}ij—\texttt{XCPU s}(i)\texttt{,s}(j), equivalent to\texttt{XCHG s}(i);\texttt{PUSH s}(j).\texttt{52}ij—\texttt{PUXC s}(i)\texttt{,s}(j-1), equivalent to\texttt{PUSH s}(i);\texttt{SWAP};\texttt{XCHG s}(j).\texttt{53}ij—\texttt{PUSH2 s}(i)\texttt{,s}(j), equivalent to\texttt{PUSH s}(i);\texttt{PUSH s}(j+1).\texttt{540}ijk—\texttt{XCHG3 s}(i)\texttt{,s}(j)\texttt{,s}(k)(long form).\texttt{541}ijk—\texttt{XC2PU s}(i)\texttt{,s}(j)\texttt{,s}(k), equivalent to\texttt{XCHG2 s}(i)\texttt{,s}(j);\texttt{PUSH s}(k).\texttt{542}ijk—\texttt{XCPUXC s}(i)\texttt{,s}(j)\texttt{,s}(k-1), equivalent to\texttt{XCHG s1,s}(i);\texttt{PUXC s}(j)\texttt{,s}(k-1).\texttt{543}ijk—\texttt{XCPU2 s}(i)\texttt{,s}(j)\texttt{,s}(k), equivalent to\texttt{XCHG s}(i);\texttt{PUSH2 s}(j)\texttt{, s}(k).\texttt{544}ijk—\texttt{PUXC2 s}(i)\texttt{,s}(j-1)\texttt{,s}(k-1), equivalent to\texttt{PUSH s}(i);\texttt{XCHG s2};\texttt{XCHG2 s}(j)\texttt{,s}(k).\texttt{545}ijk—\texttt{PUXCPU s}(i)\texttt{,s}(j-1)\texttt{,s}(k-1), equivalent to\texttt{PUXC s}(i)\texttt{,s}(j-1);\texttt{PUSH s}(k).\texttt{546}ijk—\texttt{PU2XC s}(i)\texttt{,s}(j-1)\texttt{,s}(k-2), equivalent to\texttt{PUSH s}(i);\texttt{SWAP};\texttt{PUXC s}(j)\texttt{,s}(k-1).\texttt{547}ijk—\texttt{PUSH3 s}(i)\texttt{,s}(j)\texttt{,s}(k), equivalent to\texttt{PUSH s}(i);\texttt{PUSH2 s}(j+1)\texttt{,s}(k+1).\texttt{54C\_}— unused.
A.2.3. Exotic stack manipulation primitives
\texttt{55}ij—\texttt{BLKSWAP}i+1,j+1, permutes two blocks\texttt{s}(j+i+1)\ldots\texttt{s}(j+1)and\texttt{s}(j)\ldots\texttt{s0}, for0\leq i,j\leq15. Equivalent to\texttt{REVERSE}i+1,j+1;\texttt{REVERSE}j+1,0;\texttt{REVERSE}i+j+2,0.\texttt{5513}—\texttt{ROT2}or\texttt{2ROT}(abcdef--cdefab), rotates the three topmost pairs of stack entries.\texttt{550}i—\texttt{ROLL}i+1, rotates the topi+1stack entries. Equivalent to\texttt{BLKSWAP 1},i+1.\texttt{55}i\texttt{0}—\texttt{ROLLREV}i+1or\texttt{-ROLL}i+1, rotates the topi+1stack entries in the other direction. Equivalent to\texttt{BLKSWAP}i+1,1.\texttt{56}ii—\texttt{PUSH s}(ii)for0\leq ii\leq 255.\texttt{57}ii—\texttt{POP s}(ii)for0\leq ii\leq 255.\texttt{58}—\texttt{ROT}(abc--bca), equivalent to\texttt{BLKSWAP 1,2}or to\texttt{XCHG2 s2,s1}.\texttt{59}—\texttt{ROTREV}or\texttt{-ROT}(abc--cab), equivalent to\texttt{BLKSWAP 2,1}or to\texttt{XCHG2 s2,s2}.\texttt{5A}—\texttt{SWAP2}or\texttt{2SWAP}(abcd--cdab), equivalent to\texttt{BLKSWAP 2,2}or to\texttt{XCHG2 s3,s2}.\texttt{5B}—\texttt{DROP2}or\texttt{2DROP}(ab-- ), equivalent to\texttt{DROP};\texttt{DROP}.\texttt{5C}—\texttt{DUP2}or\texttt{2DUP}(ab--abab), equivalent to\texttt{PUSH2 s1,s0}.\texttt{5D}—\texttt{OVER2}or\texttt{2OVER}(abcd--abcdab), equivalent to\texttt{PUSH2 s3,s2}.\texttt{5E}ij—\texttt{REVERSE}i+2,j, reverses the order of\texttt{s}(j+i+1)\ldots\texttt{s}(j)for0\leq i,j\leq 15; equivalent to a sequence of\lfloor i/2\rfloor+1\texttt{XCHG}s.\texttt{5F0}i—\texttt{BLKDROP}i, equivalent to\texttt{DROP}performeditimes.\texttt{5F}ij—\texttt{BLKPUSH}i,j, equivalent to\texttt{PUSH s}(j)performeditimes,1\leq i\leq 15,0\leq j\leq 15.\texttt{60}—\texttt{PICK}or\texttt{PUSHX}, pops integerifrom the stack, then performs\texttt{PUSH s}(i).\texttt{61}—\texttt{ROLLX}, pops integerifrom the stack, then performs\texttt{BLKSWAP 1},i.\texttt{62}—\texttt{-ROLLX}or\texttt{ROLLREVX}, pops integerifrom the stack, then performs\texttt{BLKSWAP}i,1.\texttt{63}—\texttt{BLKSWX}, pops integersi,jfrom the stack, then performs\texttt{BLKSWAP}i,j.\texttt{64}—\texttt{REVX}, pops integersi,jfrom the stack, then performs\texttt{REVERSE}i,j.\texttt{65}—\texttt{DROPX}, pops integerifrom the stack, then performs\texttt{BLKDROP}i.\texttt{66}—\texttt{TUCK}(ab-bab), equivalent to\texttt{SWAP};\texttt{OVER}or to\texttt{XCPU s1,s1}.\texttt{67}—\texttt{XCHGX}, pops integerifrom the stack, then performs\texttt{XCHG s}(i).\texttt{68}—\texttt{DEPTH}, pushes the current depth of the stack.\texttt{69}—\texttt{CHKDEPTH}, pops integerifrom the stack, then checks whether there are at leastielements, generating a stack underflow exception otherwise.\texttt{6A}—\texttt{ONLYTOPX}, pops integerifrom the stack, then removes all but the topielements.\texttt{6B}—\texttt{ONLYX}, pops integerifrom the stack, then leaves only the bottomielements. Approximately equivalent to\texttt{DEPTH};\texttt{SWAP};\texttt{SUB};\texttt{DROPX}.\texttt{6C00}--\texttt{6C0F}— reserved for stack operations.\texttt{6C}ij—\texttt{BLKDROP2}i,j, dropsistack elements under the topjelements, where1\leq i\leq15and0\leq j\leq 15. Equivalent to\texttt{REVERSE}i+j,0;\texttt{BLKDROP}i;\texttt{REVERSE}j,0.
A.3 Tuple, List, and Null primitives
Tuples are ordered collections consisting of at most 255 TVM stack values of arbitrary types (not necessarily the same). Tuple primitives create, modify, and unpack Tuples; they manipulate values of arbitrary types in the process, similarly to the stack primitives. We do not recommend using Tuples of more than 15 elements.
When a Tuple t contains elements x_1, ..., x_n (in that order), we write t=(x_1,\ldots,x_n); number n\geq0 is the length of Tuple t. It is also denoted by |t|. Tuples of length two are called pairs, and Tuples of length three are triples.
Lisp-style lists are represented with the aid of pairs, i.e., tuples consisting of exactly two elements. An empty list is represented by a Null value, and a non-empty list is represented by pair (h,t), where h is the first element of the list, and t is its tail.
A.3.1. Null primitives
The following primitives work with (the only) value \bot of type Null, useful for representing empty lists, empty branches of binary trees, and absence of values in Maybe X types. An empty Tuple created by \texttt{NIL} could have been used for the same purpose; however, Null is more efficient and costs less gas.
\texttt{6D}—\texttt{NULL}or\texttt{PUSHNULL}( --\bot), pushes the only value of type Null.\texttt{6E}—\texttt{ISNULL}(x--?), checks whetherxis a Null, and returns-1or0accordingly.
A.3.2. Tuple primitives
\texttt{6F0}n—\texttt{TUPLE}n(x_1\ldotsx_n--t), creates a new Tuplet=(x_1,\ldots,x_n)containingnvaluesx_1,\ldots,x_n, where0\leq n\leq 15.\texttt{6F00}—\texttt{NIL}( --t), pushes the only Tuplet=()of length zero.\texttt{6F01}—\texttt{SINGLE}(x--t), creates a singletont:=(x), i.e., a Tuple of length one.\texttt{6F02}—\texttt{PAIR}or\texttt{CONS}(xy--t), creates pairt:=(x,y).\texttt{6F03}—\texttt{TRIPLE}(xyz--t), creates triplet:=(x,y,z).\texttt{6F1}k—\texttt{INDEX}k(t--x), returns thek-th element of a Tuplet, where0\leq k\leq 15. In other words, returnsx_{k+1}ift=(x_1,\ldots,x_n). Ifk\geq n, throws a range check exception.\texttt{6F10}—\texttt{FIRST}or\texttt{CAR}(t--x), returns the first element of a Tuple.\texttt{6F11}—\texttt{SECOND}or\texttt{CDR}(t--y), returns the second element of a Tuple.\texttt{6F12}—\texttt{THIRD}(t--z), returns the third element of a Tuple.\texttt{6F2}n—\texttt{UNTUPLE}n(t--x_1\ldotsx_n), unpacks a Tuplet=(x_1,\ldots,x_n)of length equal to0\leq n\leq 15. Iftis not a Tuple, of if|t|\neq n, a type check exception is thrown.\texttt{6F21}—\texttt{UNSINGLE}(t--x), unpacks a singletont=(x).\texttt{6F22}—\texttt{UNPAIR}or\texttt{UNCONS}(t--xy), unpacks a pairt=(x,y).\texttt{6F23}—\texttt{UNTRIPLE}(t--xyz), unpacks a triplet=(x,y,z).\texttt{6F3}k—\texttt{UNPACKFIRST}k(t--x_1\ldotsx_k), unpacks first0\leq k\leq 15elements of a Tuplet. If|t|<k, throws a type check exception.\texttt{6F30}—\texttt{CHKTUPLE}(t-- ), checks whethertis a Tuple.\texttt{6F4}n—\texttt{EXPLODE}n(t--x_1\ldotsx_mm), unpacks a Tuplet=(x_1,\ldots,x_m)and returns its lengthm, but only ifm\leq n\leq 15. Otherwise throws a type check exception.\texttt{6F5}k—\texttt{SETINDEX}k(tx--t'), computes Tuplet'that differs fromtonly at positiont'_{k+1}, which is set tox. In other words,|t'|=|t|,t'_i=t_ifori\neq k+1, andt'_{k+1}=x, for given0\leq k\leq15. Ifk\geq|t|, throws a range check exception.\texttt{6F50}—\texttt{SETFIRST}(tx--t'), sets the first component of Tuplettoxand returns the resulting Tuplet'.\texttt{6F51}—\texttt{SETSECOND}(tx--t'), sets the second component of Tuplettoxand returns the resulting Tuplet'.\texttt{6F52}—\texttt{SETTHIRD}(tx--t'), sets the third component of Tuplettoxand returns the resulting Tuplet'.\texttt{6F6}k—\texttt{INDEXQ}k(t--x), returns thek-th element of a Tuplet, where0\leq n\leq 15. In other words, returnsx_{k+1}ift=(x_1,\ldots,x_n). Ifk\geq n, or iftis Null, returns a Null instead ofx.\texttt{6F7}k—\texttt{SETINDEXQ}k(tx--t'), sets thek-th component of Tuplettox, where0\leq k<16, and returns the resulting Tuplet'. If|t|\leq k, first extends the original Tuple to lengthk+1by setting all new components to Null. If the original value oftis Null, treats it as an empty Tuple. Iftis not Null or Tuple, throws an exception. Ifxis Null and either|t|\leq kortis Null, then always returnst'=t(and does not consume tuple creation gas).\texttt{6F80}—\texttt{TUPLEVAR}(x_1\ldotsx_nn--t), creates a new Tupletof lengthnsimilarly to\texttt{TUPLE}, but with0\leq n\leq 255taken from the stack.\texttt{6F81}—\texttt{INDEXVAR}(tk--x), similar to\texttt{INDEX}k, but with0\leq k\leq 254taken from the stack.\texttt{6F82}—\texttt{UNTUPLEVAR}(tn--x_1\ldotsx_n), similar to\texttt{UNTUPLE}n, but with0\leq n\leq 255taken from the stack.\texttt{6F83}—\texttt{UNPACKFIRSTVAR}(tn--x_1\ldotsx_n), similar to\texttt{UNPACKFIRST}n, but with0\leq n\leq 255taken from the stack.\texttt{6F84}—\texttt{EXPLODEVAR}(tn--x_1\ldotsx_mm), similar to\texttt{EXPLODE}n, but with0\leq n\leq 255taken from the stack.\texttt{6F85}—\texttt{SETINDEXVAR}(txk--t'), similar to\texttt{SETINDEX}k, but with0\leq k\leq 254taken from the stack.\texttt{6F86}—\texttt{INDEXVARQ}(tk--x), similar to\texttt{INDEXQ}n, but with0\leq k\leq 254taken from the stack.\texttt{6F87}—\texttt{SETINDEXVARQ}(txk--t'), similar to\texttt{SETINDEXQ}k, but with0\leq k\leq 254taken from the stack.\texttt{6F88}—\texttt{TLEN}(t--n), returns the length of a Tuple.\texttt{6F89}—\texttt{QTLEN}(t--nor-1), similar to\texttt{TLEN}, but returns-1iftis not a Tuple.\texttt{6F8A}—\texttt{ISTUPLE}(t--?), returns-1or0depending on whethertis a Tuple.\texttt{6F8B}—\texttt{LAST}(t--x), returns the last elementt_{|t|}of a non-empty Tuplet.\texttt{6F8C}—\texttt{TPUSH}or\texttt{COMMA}(tx--t'), appends a valuexto a Tuplet=(x_1,\ldots,x_n), but only if the resulting Tuplet'=(x_1,\ldots,x_n,x)is of length at most 255. Otherwise throws a type check exception.\texttt{6F8D}—\texttt{TPOP}(t--t'x), detaches the last elementx=x_nfrom a non-empty Tuplet=(x_1,\ldots,x_n), and returns both the resulting Tuplet'=(x_1,\ldots,x_{n-1})and the original last elementx.\texttt{6FA0}—\texttt{NULLSWAPIF}(x--xor\botx), pushes a Null under the topmost Integerx, but only ifx\neq0.\texttt{6FA1}—\texttt{NULLSWAPIFNOT}(x--xor\botx), pushes a Null under the topmost Integerx, but only ifx=0. May be used for stack alignment after quiet primitives such as\texttt{PLDUXQ}.\texttt{6FA2}—\texttt{NULLROTRIF}(xy--xyor\botxy), pushes a Null under the second stack entry from the top, but only if the topmost Integeryis non-zero.\texttt{6FA3}—\texttt{NULLROTRIFNOT}(xy--xyor\botxy), pushes a Null under the second stack entry from the top, but only if the topmost Integeryis zero. May be used for stack alignment after quiet primitives such as\texttt{LDUXQ}.\texttt{6FA4}—\texttt{NULLSWAPIF2}(x--xor\bot\botx), pushes two Nulls under the topmost Integerx, but only ifx\neq0. Equivalent to\texttt{NULLSWAPIF};\texttt{NULLSWAPIF}.\texttt{6FA5}—\texttt{NULLSWAPIFNOT2}(x--xor\bot\botx), pushes two Nulls under the topmost Integerx, but only ifx=0. Equivalent to\texttt{NULLSWAPIFNOT};\texttt{NULLSWAPIFNOT}.\texttt{6FA6}—\texttt{NULLROTRIF2}(xy--xyor\bot\botxy), pushes two Nulls under the second stack entry from the top, but only if the topmost Integeryis non-zero. Equivalent to\texttt{NULLROTRIF};\texttt{NULLROTRIF}.\texttt{6FA7}—\texttt{NULLROTRIFNOT2}(xy--xyor\bot\botxy), pushes two Nulls under the second stack entry from the top, but only if the topmost Integeryis zero. Equivalent to\texttt{NULLROTRIFNOT};\texttt{NULLROTRIFNOT}.\texttt{6FB}ij—\texttt{INDEX2}i,j(t--x), recoversx=(t_{i+1})_{j+1}for0\leq i,j\leq 3. Equivalent to\texttt{INDEX}i;\texttt{INDEX}j.\texttt{6FB4}—\texttt{CADR}(t--x), recoversx=(t_2)_1.\texttt{6FB5}—\texttt{CDDR}(t--x), recoversx=(t_2)_2.\texttt{6FE\_}ijk—\texttt{INDEX3}i,j,k(t--x), recoversx=\bigl((t_{i+1})_{j+1}\bigr)_{k+1}for0\leq i,j,k\leq3. Equivalent to\texttt{INDEX2}i,j;\texttt{INDEX}k.\texttt{6FD4}—\texttt{CADDR}(t--x), recoversx=\bigl((t_2)_2\bigr)_1.\texttt{6FD5}—\texttt{CDDDR}(t--x), recoversx=\bigl((t_2)_2\bigr)_2.
A.4 Constant, or literal primitives
The following primitives push into the stack one literal (or unnamed constant) of some type and range, stored as a part (an immediate argument) of the instruction. Therefore, if the immediate argument is absent or too short, an "invalid or too short opcode" exception (code 6) is thrown.
A.4.1. Integer and boolean constants
\texttt{7}i—\texttt{PUSHINT}xwith-5\leq x\leq 10, pushes integerxinto the stack; hereiequals four lower-order bits ofx(i.e.,i=x\bmod 16).\texttt{70}—\texttt{ZERO},\texttt{FALSE}, or\texttt{PUSHINT 0}, pushes a zero.\texttt{71}—\texttt{ONE}or\texttt{PUSHINT 1}.\texttt{72}—\texttt{TWO}or\texttt{PUSHINT 2}.\texttt{7A}—\texttt{TEN}or\texttt{PUSHINT}10.\texttt{7F}—\texttt{TRUE}or\texttt{PUSHINT -1}.\texttt{80}xx—\texttt{PUSHINT}xxwith-128\leq xx\leq127.\texttt{81}xxxx—\texttt{PUSHINT}xxxxwith-2^{15}\leq xxxx<2^{15}a signed 16-bit big-endian integer.\texttt{81FC18}—\texttt{PUSHINT}-1000.\texttt{82}lxxx—\texttt{PUSHINT}xxx, where 5-bit0\leq l\leq30determines the lengthn=8l+19of signed big-endian integerxxx. The total length of this instruction isl+4bytes orn+13=8l+32bits.\texttt{821005F5E100}—\texttt{PUSHINT}10^8.\texttt{83}xx—\texttt{PUSHPOW2}xx+1, (quietly) pushes2^{xx+1}for0\leq xx\leq255.\texttt{83FF}—\texttt{PUSHNAN}, pushes a\texttt{NaN}.\texttt{84}xx—\texttt{PUSHPOW2DEC}xx+1, pushes2^{xx+1}-1for0\leq xx\leq 255.\texttt{85}xx—\texttt{PUSHNEGPOW2}xx+1, pushes-2^{xx+1}for0\leq xx\leq 255.\texttt{86},\texttt{87}— reserved for integer constants.
A.4.2. Constant slices, continuations, cells, and references
Most of the instructions listed below push literal slices, continuations, cells, and cell references, stored as immediate arguments to the instruction. Therefore, if the immediate argument is absent or too short, an "invalid or too short opcode" exception (code 6) is thrown.
\texttt{88}—\texttt{PUSHREF}, pushes the first reference of\texttt{cc.code}into the stack as a Cell (and removes this reference from the current continuation).\texttt{89}—\texttt{PUSHREFSLICE}, similar to\texttt{PUSHREF}, but converts the cell into a Slice.\texttt{8A}—\texttt{PUSHREFCONT}, similar to\texttt{PUSHREFSLICE}, but makes a simple ordinary Continuation out of the cell.\texttt{8B}xsss—\texttt{PUSHSLICE sss}, pushes the (prefix) subslice of\texttt{cc.code}consisting of its first8x+4bits and no references (i.e., essentially a bitstring), where0\leq x\leq15. A completion tag is assumed, meaning that all trailing zeroes and the last binary one (if present) are removed from this bitstring. If the original bitstring consists only of zeroes, an empty slice will be pushed.\texttt{8B08}—\texttt{PUSHSLICE x8\_}, pushes an empty slice (bitstring ' ').\texttt{8B04}—\texttt{PUSHSLICE x4\_}, pushes bitstring\texttt{'0'}.\texttt{8B0C}—\texttt{PUSHSLICE xC\_}, pushes bitstring\texttt{'1'}.\texttt{8C}rxxssss—\texttt{PUSHSLICE ssss}, pushes the (prefix) subslice of\texttt{cc.code}consisting of its first1\leq r+1\leq 4references and up to first8xx+1bits of data, with0\leq xx\leq 31. A completion tag is also assumed.\texttt{8C01}is equivalent to\texttt{PUSHREFSLICE}.\texttt{8D}rxxsssss—\texttt{PUSHSLICE sssss}, pushes the subslice of\texttt{cc.code}consisting of0\leq r\leq 4references and up to8xx+6bits of data, with0\leq xx\leq 127. A completion tag is assumed.\texttt{8DE\_}— unused (reserved).\texttt{8F\_}rxxcccc—\texttt{PUSHCONT cccc}, whereccccis the simple ordinary continuation made from the first0\leq r\leq 3references and the first0\leq xx\leq 127bytes of\texttt{cc.code}.\texttt{9}xccc—\texttt{PUSHCONT ccc}, pushes anx-byte continuation for0\leq x\leq 15.
A.5 Arithmetic primitives
A.5.1. Addition, subtraction, multiplication
\texttt{A0}—\texttt{ADD}(xy--x+y), adds together two integers.\texttt{A1}—\texttt{SUB}(xy--x-y).\texttt{A2}—\texttt{SUBR}(xy--y-x), equivalent to\texttt{SWAP};\texttt{SUB}.\texttt{A3}—\texttt{NEGATE}(x---x), equivalent to\texttt{MULCONST}-1or to\texttt{ZERO};\texttt{SUBR}. Notice that it triggers an integer overflow exception ifx=-2^{256}.\texttt{A4}—\texttt{INC}(x--x+1), equivalent to\texttt{ADDCONST 1}.\texttt{A5}—\texttt{DEC}(x--x-1), equivalent to\texttt{ADDCONST}-1.\texttt{A6}cc—\texttt{ADDCONST}cc(x--x+cc),-128\leq cc\leq127.\texttt{A7}cc—\texttt{MULCONST}cc(x--x\cdot cc),-128\leq cc\leq127.\texttt{A8}—\texttt{MUL}(xy--xy).
A.5.2. Division
The general encoding of a \texttt{DIV}, \texttt{DIVMOD}, or \texttt{MOD} operation is \texttt{A9}mscdf, with an optional pre-multiplication and an optional replacement of the division or multiplication by a shift. Variable one- or two-bit fields m, s, c, d, and f are as follows:
0\leq m\leq1— Indicates whether there is pre-multiplication (\texttt{MULDIV}operation and its variants), possibly replaced by a left shift.0\leq s\leq2— Indicates whether either the multiplication or the division have been replaced by shifts:s=0—no replacement,s=1—division replaced by a right shift,s=2—multiplication replaced by a left shift (possible only form=1).0\leq c\leq1— Indicates whether there is a constant one-byte argumentttfor the shift operator (ifs\neq0). Fors=0,c=0. Ifc=1, then0\leq tt\leq 255, and the shift is performed bytt+1bits. Ifs\neq0andc=0, then the shift amount is provided to the instruction as a top-of-stack Integer in range0\ldots256.1\leq d\leq3— Indicates which results of division are required:1—only the quotient,2—only the remainder,3—both.0\leq f\leq2— Rounding mode:0—floor,1—nearest integer,2—ceiling (1.5.6).
Examples:
\texttt{A904}—\texttt{DIV}(xy--q:=\lfloor x/y\rfloor).\texttt{A905}—\texttt{DIVR}(xy--q':=\lfloor x/y+1/2\rfloor).\texttt{A906}—\texttt{DIVC}(xy--q'':=\lceil x/y\rceil).\texttt{A908}—\texttt{MOD}(xy--r), whereq:=\lfloor x/y\rfloor,r:=x\bmod y:=x-yq.\texttt{A90C}—\texttt{DIVMOD}(xy--qr), whereq:=\lfloor x/y\rfloor,r:=x-yq.\texttt{A90D}—\texttt{DIVMODR}(xy--q'r'), whereq':=\lfloor x/y+1/2\rfloor,r':=x-yq'.\texttt{A90E}—\texttt{DIVMODC}(xy--q''r''), whereq'':=\lceil x/y\rceil,r'':=x-yq''.\texttt{A924}— same as\texttt{RSHIFT}: (xy--\lfloor x\cdot 2^{-y}\rfloor) for0\leq y\leq 256.\texttt{A934}tt— same as\texttt{RSHIFT}tt+1: (x--\lfloor x\cdot 2^{-tt-1}\rfloor).\texttt{A938}tt—\texttt{MODPOW2}tt+1: (x--x\bmod 2^{tt+1}).\texttt{A985}—\texttt{MULDIVR}(xyz--q'), whereq'=\lfloor xy/z+1/2\rfloor.\texttt{A988}—\texttt{MULMOD}(xyz--r), wherer=xy\bmod z=xy-qz,q=\lfloor xy/z\rfloor. This operation always succeeds forz\neq0and returns the correct value ofr, even if the intermediate resultxyor the quotientqdo not fit into 257 bits.\texttt{A98C}—\texttt{MULDIVMOD}(xyz--qr), whereq:=\lfloor x\cdot y/z\rfloor,r:=x\cdot y\bmod z(same as\texttt{*/MOD}in Forth).\texttt{A9A4}—\texttt{MULRSHIFT}(xyz--\lfloor xy\cdot2^{-z}\rfloor) for0\leq z\leq 256.\texttt{A9A5}—\texttt{MULRSHIFTR}(xyz--\lfloor xy\cdot2^{-z}+1/2\rfloor) for0\leq z\leq 256.\texttt{A9B4}tt—\texttt{MULRSHIFT}tt+1(xy--\lfloor xy\cdot 2^{-tt-1}\rfloor).\texttt{A9B5}tt—\texttt{MULRSHIFTR}tt+1(xy--\lfloor xy\cdot 2^{-tt-1}+1/2\rfloor).\texttt{A9C4}—\texttt{LSHIFTDIV}(xyz--\lfloor 2^zx/y\rfloor) for0\leq z\leq 256.\texttt{A9C5}—\texttt{LSHIFTDIVR}(xyz--\lfloor 2^zx/y+1/2\rfloor) for0\leq z\leq 256.\texttt{A9D4}tt—\texttt{LSHIFTDIV}tt+1(xy--\lfloor 2^{tt+1}x/y\rfloor).\texttt{A9D5}tt—\texttt{LSHIFTDIVR}tt+1(xy--\lfloor 2^{tt+1}x/y+1/2\rfloor).
The most useful of these operations are \texttt{DIV}, \texttt{DIVMOD}, \texttt{MOD}, \texttt{DIVR}, \texttt{DIVC}, \texttt{MODPOW2} t, and \texttt{RSHIFTR} t (for integer arithmetic); and \texttt{MULDIVMOD}, \texttt{MULDIV}, \texttt{MULDIVR}, \texttt{LSHIFTDIVR} t, and \texttt{MULRSHIFTR} t (for fixed-point arithmetic).
A.5.3. Shifts, logical operations
\texttt{AA}cc—\texttt{LSHIFT}cc+1(x--x\cdot2^{cc+1}),0\leq cc\leq255.\texttt{AA00}—\texttt{LSHIFT 1}, equivalent to\texttt{MULCONST 2}or to Forth's\texttt{2*}.\texttt{AB}cc—\texttt{RSHIFT}cc+1(x--\lfloor x\cdot2^{-cc-1}\rfloor),0\leq cc\leq255.\texttt{AC}—\texttt{LSHIFT}(xy--x\cdot 2^y),0\leq y\leq 1023.\texttt{AD}—\texttt{RSHIFT}(xy--\lfloor x\cdot 2^{-y}\rfloor),0\leq y\leq 1023.\texttt{AE}—\texttt{POW2}(y--2^y),0\leq y\leq1023, equivalent to\texttt{ONE};\texttt{SWAP};\texttt{LSHIFT}.\texttt{AF}— reserved.\texttt{B0}—\texttt{AND}(xy--x\&y), bitwise "and" of two signed integersxandy, sign-extended to infinity.\texttt{B1}—\texttt{OR}(xy--x\vee y), bitwise "or" of two integers.\texttt{B2}—\texttt{XOR}(xy--x\oplus y), bitwise "xor" of two integers.\texttt{B3}—\texttt{NOT}(x--x\oplus-1=-1-x), bitwise "not" of an integer.\texttt{B4}cc—\texttt{FITS}cc+1(x--x), checks whetherxis acc+1-bit signed integer for0\leq cc\leq 255(i.e., whether-2^{cc}\leq x<2^{cc}). If not, either triggers an integer overflow exception, or replacesxwith a\texttt{NaN}(quiet version).\texttt{B400}—\texttt{FITS 1}or\texttt{CHKBOOL}(x--x), checks whetherxis a "boolean value" (i.e., either 0 or -1).\texttt{B5}cc—\texttt{UFITS}cc+1(x--x), checks whetherxis acc+1-bit unsigned integer for0\leq cc\leq 255(i.e., whether0\leq x<2^{cc+1}).\texttt{B500}—\texttt{UFITS 1}or\texttt{CHKBIT}, checks whetherxis a binary digit (i.e., zero or one).\texttt{B600}—\texttt{FITSX}(xc--x), checks whetherxis ac-bit signed integer for0\leq c\leq 1023.\texttt{B601}—\texttt{UFITSX}(xc--x), checks whetherxis ac-bit unsigned integer for0\leq c\leq 1023.\texttt{B602}—\texttt{BITSIZE}(x--c), computes smallestc\geq0such thatxfits into ac-bit signed integer (-2^{c-1}\leq c<2^{c-1}).\texttt{B603}—\texttt{UBITSIZE}(x--c), computes smallestc\geq0such thatxfits into ac-bit unsigned integer (0\leq x<2^c), or throws a range check exception.\texttt{B608}—\texttt{MIN}(xy--xory), computes the minimum of two integersxandy.\texttt{B609}—\texttt{MAX}(xy--xory), computes the maximum of two integersxandy.\texttt{B60A}—\texttt{MINMAX}or\texttt{INTSORT2}(xy--xyoryx), sorts two integers. Quiet version of this operation returns two\texttt{NaN}s if any of the arguments are\texttt{NaN}s.\texttt{B60B}—\texttt{ABS}(x--|x|), computes the absolute value of an integerx.
A.5.4. Quiet arithmetic primitives
We opted to make all arithmetic operations "non-quiet" (signaling) by default, and create their quiet counterparts by means of a prefix. Such an encoding is definitely sub-optimal. It is not yet clear whether it should be done in this way, or in the opposite way by making all arithmetic operations quiet by default, or whether quiet and non-quiet operations should be given opcodes of equal length; this can only be settled by practice.
\texttt{B7}xx—\texttt{QUIET}prefix, transforming any arithmetic operation into its "quiet" variant, indicated by prefixing aQto its mnemonic. Such operations return\texttt{NaN}s instead of throwing integer overflow exceptions if the results do not fit in Integers, or if one of their arguments is a\texttt{NaN}. Notice that this does not extend to shift amounts and other parameters that must be within a small range (e.g., 0--1023). Also notice that this does not disable type-checking exceptions if a value of a type other than Integer is supplied.\texttt{B7A0}—\texttt{QADD}(xy--x+y), always works ifxandyare Integers, but returns a\texttt{NaN}if the addition cannot be performed.\texttt{B7A8}—\texttt{QMUL}(xy--xy), returns the product ofxandyif-2^{256}\leq xy<2^{256}. Otherwise returns a\texttt{NaN}, even ifx=0andyis a\texttt{NaN}.\texttt{B7A904}—\texttt{QDIV}(xy--\lfloor x/y\rfloor), returns a\texttt{NaN}ify=0, or ify=-1andx=-2^{256}, or if either ofxoryis a\texttt{NaN}.\texttt{B7A98C}—\texttt{QMULDIVMOD}(xyz--qr), whereq:=\lfloor x\cdot y/z\rfloor,r:=x\cdot y\bmod z. Ifz=0, or if at least one ofx,y, orzis a\texttt{NaN}, bothqandrare set to\texttt{NaN}. Otherwise the correct value ofris always returned, butqis replaced with\texttt{NaN}ifq<-2^{256}orq\geq2^{256}.\texttt{B7B0}—\texttt{QAND}(xy--x\&y), bitwise "and" (similar to\texttt{AND}), but returns a\texttt{NaN}if eitherxoryis a\texttt{NaN}instead of throwing an integer overflow exception. However, if one of the arguments is zero, and the other is a\texttt{NaN}, the result is zero.\texttt{B7B1}—\texttt{QOR}(xy--x\vee y), bitwise "or". Ifx=-1ory=-1, the result is always-1, even if the other argument is a\texttt{NaN}.\texttt{B7B507}—\texttt{QUFITS 8}(x--x'), checks whetherxis an unsigned byte (i.e., whether0\leq x<2^8), and replacesxwith a\texttt{NaN}if this is not the case; leavesxintact otherwise (i.e., ifxis an unsigned byte or a\texttt{NaN}).
A.6 Comparison primitives
A.6.1. Integer comparison
All integer comparison primitives return integer -1 ("true") or 0 ("false") to indicate the result of the comparison. We do not define their "boolean circuit" counterparts, which would transfer control to \texttt{c0} or \texttt{c1} depending on the result of the comparison. If needed, such instructions can be simulated with the aid of \texttt{RETBOOL}.
Quiet versions of integer comparison primitives are also available, encoded with the aid of the \texttt{QUIET} prefix (\texttt{B7}). If any of the integers being compared are \texttt{NaN}s, the result of a quiet comparison will also be a \texttt{NaN} ("undefined"), instead of a -1 ("yes") or 0 ("no"), thus effectively supporting ternary logic.
\texttt{B8}—\texttt{SGN}(x--\text{sgn}(x)), computes the sign of an integerx:-1ifx<0,0ifx=0,1ifx>0.\texttt{B9}—\texttt{LESS}(xy--x<y), returns-1ifx<y,0otherwise.\texttt{BA}—\texttt{EQUAL}(xy--x=y), returns-1ifx=y,0otherwise.\texttt{BB}—\texttt{LEQ}(xy--x\leq y).\texttt{BC}—\texttt{GREATER}(xy--x>y).\texttt{BD}—\texttt{NEQ}(xy--x\neq y), equivalent to\texttt{EQUAL};\texttt{NOT}.\texttt{BE}—\texttt{GEQ}(xy--x\geq y), equivalent to\texttt{LESS};\texttt{NOT}.\texttt{BF}—\texttt{CMP}(xy--\text{sgn}(x-y)), computes the sign ofx-y:-1ifx<y,0ifx=y,1ifx>y. No integer overflow can occur here unlessxoryis a\texttt{NaN}.\texttt{C0}yy—\texttt{EQINT}yy(x--x=yy) for-2^7\leq yy<2^7.\texttt{C000}—\texttt{ISZERO}, checks whether an integer is zero. Corresponds to Forth's\texttt{0=}.\texttt{C1}yy—\texttt{LESSINT}yy(x--x<yy) for-2^7\leq yy<2^7.\texttt{C100}—\texttt{ISNEG}, checks whether an integer is negative. Corresponds to Forth's\texttt{0<}.\texttt{C101}—\texttt{ISNPOS}, checks whether an integer is non-positive.\texttt{C2}yy—\texttt{GTINT}yy(x--x>yy) for-2^7\leq yy<2^7.\texttt{C200}—\texttt{ISPOS}, checks whether an integer is positive. Corresponds to Forth's\texttt{0>}.\texttt{C2FF}—\texttt{ISNNEG}, checks whether an integer is non-negative.\texttt{C3}yy—\texttt{NEQINT}yy(x--x\neq yy) for-2^7\leq yy<2^7.\texttt{C4}—\texttt{ISNAN}(x--x=\texttt{NaN}), checks whetherxis a\texttt{NaN}.\texttt{C5}—\texttt{CHKNAN}(x--x), throws an arithmetic overflow exception ifxis a\texttt{NaN}.\texttt{C6}— reserved for integer comparison.
A.6.2. Other comparison
Most of these "other comparison" primitives actually compare the data portions of Slices as bitstrings.
\texttt{C700}—\texttt{SEMPTY}(s--s=\emptyset), checks whether a Slicesis empty (i.e., contains no bits of data and no cell references).\texttt{C701}—\texttt{SDEMPTY}(s--s\approx\emptyset), checks whether Sliceshas no bits of data.\texttt{C702}—\texttt{SREMPTY}(s--r(s)=0), checks whether Sliceshas no references.\texttt{C703}—\texttt{SDFIRST}(s--s_0=1), checks whether the first bit of Slicesis a one.\texttt{C704}—\texttt{SDLEXCMP}(ss'--c), compares the data ofslexicographically with the data ofs', returning-1, 0, or 1 depending on the result.\texttt{C705}—\texttt{SDEQ}(ss'--s\approx s'), checks whether the data parts ofsands'coincide, equivalent to\texttt{SDLEXCMP};\texttt{ISZERO}.\texttt{C708}—\texttt{SDPFX}(ss'--?), checks whethersis a prefix ofs'.\texttt{C709}—\texttt{SDPFXREV}(ss'--?), checks whethers'is a prefix ofs, equivalent to\texttt{SWAP};\texttt{SDPFX}.\texttt{C70A}—\texttt{SDPPFX}(ss'--?), checks whethersis a proper prefix ofs'(i.e., a prefix distinct froms').\texttt{C70B}—\texttt{SDPPFXREV}(ss'--?), checks whethers'is a proper prefix ofs.\texttt{C70C}—\texttt{SDSFX}(ss'--?), checks whethersis a suffix ofs'.\texttt{C70D}—\texttt{SDSFXREV}(ss'--?), checks whethers'is a suffix ofs.\texttt{C70E}—\texttt{SDPSFX}(ss'--?), checks whethersis a proper suffix ofs'.\texttt{C70F}—\texttt{SDPSFXREV}(ss'--?), checks whethers'is a proper suffix ofs.\texttt{C710}—\texttt{SDCNTLEAD0}(s--n), returns the number of leading zeroes ins.\texttt{C711}—\texttt{SDCNTLEAD1}(s--n), returns the number of leading ones ins.\texttt{C712}—\texttt{SDCNTTRAIL0}(s--n), returns the number of trailing zeroes ins.\texttt{C713}—\texttt{SDCNTTRAIL1}(s--n), returns the number of trailing ones ins.
A.7 Cell primitives
The cell primitives are mostly either cell serialization primitives, which work with Builders, or cell deserialization primitives, which work with Slices.
A.7.1. Cell serialization primitives
All these primitives first check whether there is enough space in the Builder, and only then check the range of the value being serialized.
\texttt{C8}—\texttt{NEWC}( --b), creates a new empty Builder.\texttt{C9}—\texttt{ENDC}(b--c), converts a Builder into an ordinary Cell.\texttt{CA}cc—\texttt{STI}cc+1(xb--b'), stores a signedcc+1-bit integerxinto Builderbfor0\leq cc\leq 255, throws a range check exception ifxdoes not fit intocc+1bits.\texttt{CB}cc—\texttt{STU}cc+1(xb--b'), stores an unsignedcc+1-bit integerxinto Builderb. In all other respects it is similar to\texttt{STI}.\texttt{CC}—\texttt{STREF}(cb--b'), stores a reference to Cellcinto Builderb.\texttt{CD}—\texttt{STBREFR}or\texttt{ENDCST}(bb''--b), equivalent to\texttt{ENDC};\texttt{SWAP};\texttt{STREF}.\texttt{CE}—\texttt{STSLICE}(sb--b'), stores Slicesinto Builderb.\texttt{CF00}—\texttt{STIX}(xbl--b'), stores a signedl-bit integerxintobfor0\leq l\leq 257.\texttt{CF01}—\texttt{STUX}(xbl--b'), stores an unsignedl-bit integerxintobfor0\leq l\leq 256.\texttt{CF02}—\texttt{STIXR}(bxl--b'), similar to\texttt{STIX}, but with arguments in a different order.\texttt{CF03}—\texttt{STUXR}(bxl--b'), similar to\texttt{STUX}, but with arguments in a different order.\texttt{CF04}—\texttt{STIXQ}(xbl--xbforb'0), a quiet version of\texttt{STIX}. If there is no space inb, setsb'=bandf=-1. Ifxdoes not fit intolbits, setsb'=bandf=1. If the operation succeeds,b'is the new Builder andf=0. However,0\leq l\leq 257, with a range check exception if this is not so.\texttt{CF05}—\texttt{STUXQ}(xbl--b'f).\texttt{CF06}—\texttt{STIXRQ}(bxl--bxforb'0).\texttt{CF07}—\texttt{STUXRQ}(bxl--bxforb'0).\texttt{CF08}cc— a longer version of\texttt{STI}cc+1.\texttt{CF09}cc— a longer version of\texttt{STU}cc+1.\texttt{CF0A}cc—\texttt{STIR}cc+1(bx--b'), equivalent to\texttt{SWAP};\texttt{STI}cc+1.\texttt{CF0B}cc—\texttt{STUR}cc+1(bx--b'), equivalent to\texttt{SWAP};\texttt{STU}cc+1.\texttt{CF0C}cc—\texttt{STIQ}cc+1(xb--xbforb'0).\texttt{CF0D}cc—\texttt{STUQ}cc+1(xb--xbforb'0).\texttt{CF0E}cc—\texttt{STIRQ}cc+1(bx--bxforb'0).\texttt{CF0F}cc—\texttt{STURQ}cc+1(bx--bxforb'0).\texttt{CF10}— a longer version of\texttt{STREF}(cb--b').\texttt{CF11}—\texttt{STBREF}(b'b--b''), equivalent to\texttt{SWAP};\texttt{STBREFREV}.\texttt{CF12}— a longer version of\texttt{STSLICE}(sb--b').\texttt{CF13}—\texttt{STB}(b'b--b''), appends all data from Builderb'to Builderb.\texttt{CF14}—\texttt{STREFR}(bc--b').\texttt{CF15}—\texttt{STBREFR}(bb'--b''), a longer encoding of\texttt{STBREFR}.\texttt{CF16}—\texttt{STSLICER}(bs--b').\texttt{CF17}—\texttt{STBR}(bb'--b''), concatenates two Builders, equivalent to\texttt{SWAP};\texttt{STB}.\texttt{CF18}—\texttt{STREFQ}(cb--cb-1orb'0).\texttt{CF19}—\texttt{STBREFQ}(b'b--b'b-1orb''0).\texttt{CF1A}—\texttt{STSLICEQ}(sb--sb-1orb'0).\texttt{CF1B}—\texttt{STBQ}(b'b--b'b-1orb''0).\texttt{CF1C}—\texttt{STREFRQ}(bc--bc-1orb'0).\texttt{CF1D}—\texttt{STBREFRQ}(bb'--bb'-1orb''0).\texttt{CF1E}—\texttt{STSLICERQ}(bs--bs-1orb''0).\texttt{CF1F}—\texttt{STBRQ}(bb'--bb'-1orb''0).\texttt{CF20}—\texttt{STREFCONST}, equivalent to\texttt{PUSHREF};\texttt{STREFR}.\texttt{CF21}—\texttt{STREF2CONST}, equivalent to\texttt{STREFCONST};\texttt{STREFCONST}.\texttt{CF23}—\texttt{ENDXC}(bx--c), ifx\neq0, creates a special or exotic cell from Builderb. The type of the exotic cell must be stored in the first 8 bits ofb. Ifx=0, it is equivalent to\texttt{ENDC}. Otherwise some validity checks on the data and references ofbare performed before creating the exotic cell.\texttt{CF28}—\texttt{STILE4}(xb--b'), stores a little-endian signed 32-bit integer.\texttt{CF29}—\texttt{STULE4}(xb--b'), stores a little-endian unsigned 32-bit integer.\texttt{CF2A}—\texttt{STILE8}(xb--b'), stores a little-endian signed 64-bit integer.\texttt{CF2B}—\texttt{STULE8}(xb--b'), stores a little-endian unsigned 64-bit integer.\texttt{CF30}—\texttt{BDEPTH}(b--x), returns the depth of Builderb. If no cell references are stored inb, thenx=0; otherwisexis one plus the maximum of depths of cells referred to fromb.\texttt{CF31}—\texttt{BBITS}(b--x), returns the number of data bits already stored in Builderb.\texttt{CF32}—\texttt{BREFS}(b--y), returns the number of cell references already stored inb.\texttt{CF33}—\texttt{BBITREFS}(b--xy), returns the numbers of both data bits and cell references inb.\texttt{CF35}—\texttt{BREMBITS}(b--x'), returns the number of data bits that can still be stored inb.\texttt{CF36}—\texttt{BREMREFS}(b--y').\texttt{CF37}—\texttt{BREMBITREFS}(b--x'y').\texttt{CF38}cc—\texttt{BCHKBITS}cc+1(b--), checks whethercc+1bits can be stored intob, where0\leq cc\leq 255.\texttt{CF39}—\texttt{BCHKBITS}(bx-- ), checks whetherxbits can be stored intob,0\leq x\leq 1023. If there is no space forxmore bits inb, or ifxis not within the range0\ldots1023, throws an exception.\texttt{CF3A}—\texttt{BCHKREFS}(by-- ), checks whetheryreferences can be stored intob,0\leq y\leq 7.\texttt{CF3B}—\texttt{BCHKBITREFS}(bxy-- ), checks whetherxbits andyreferences can be stored intob,0\leq x\leq 1023,0\leq y\leq 7.\texttt{CF3C}cc—\texttt{BCHKBITSQ}cc+1(b--?), checks whethercc+1bits can be stored intob, where0\leq cc\leq 255.\texttt{CF3D}—\texttt{BCHKBITSQ}(bx--?), checks whetherxbits can be stored intob,0\leq x\leq 1023.\texttt{CF3E}—\texttt{BCHKREFSQ}(by--?), checks whetheryreferences can be stored intob,0\leq y\leq 7.\texttt{CF3F}—\texttt{BCHKBITREFSQ}(bxy--?), checks whetherxbits andyreferences can be stored intob,0\leq x\leq 1023,0\leq y\leq 7.\texttt{CF40}—\texttt{STZEROES}(bn--b'), storesnbinary zeroes into Builderb.\texttt{CF41}—\texttt{STONES}(bn--b'), storesnbinary ones into Builderb.\texttt{CF42}—\texttt{STSAME}(bnx--b'), storesnbinaryxes (0\leq x\leq1) into Builderb.\texttt{CFC0\_xysss}—\texttt{STSLICECONST}sss(b--b'), stores a constant subslicesssconsisting of0\leq x\leq 3references and up to8y+1data bits, with0\leq y\leq 7. Completion bit is assumed.\texttt{CF81}—\texttt{STSLICECONST '0'}or\texttt{STZERO}(b--b'), stores one binary zero.\texttt{CF83}—\texttt{STSLICECONST '1'}or\texttt{STONE}(b--b'), stores one binary one.\texttt{CFA2}— equivalent to\texttt{STREFCONST}.\texttt{CFA3}— almost equivalent to\texttt{STSLICECONST '1'};\texttt{STREFCONST}.\texttt{CFC2}— equivalent to\texttt{STREF2CONST}.\texttt{CFE2}—\texttt{STREF3CONST}.
A.7.2. Cell deserialization primitives
\texttt{D0}—\texttt{CTOS}(c--s), converts a Cell into a Slice. Notice thatcmust be either an ordinary cell, or an exotic cell which is automatically loaded to yield an ordinary cellc', converted into a Slice afterwards.\texttt{D1}—\texttt{ENDS}(s-- ), removes a Slicesfrom the stack, and throws an exception if it is not empty.\texttt{D2}cc—\texttt{LDI}cc+1(s--xs'), loads (i.e., parses) a signedcc+1-bit integerxfrom Slices, and returns the remainder ofsass'.\texttt{D3}cc—\texttt{LDU}cc+1(s--xs'), loads an unsignedcc+1-bit integerxfrom Slices.\texttt{D4}—\texttt{LDREF}(s--cs'), loads a cell referencecfroms.\texttt{D5}—\texttt{LDREFRTOS}(s--s's''), equivalent to\texttt{LDREF};\texttt{SWAP};\texttt{CTOS}.\texttt{D6}cc—\texttt{LDSLICE}cc+1(s--s''s'), cuts the nextcc+1bits ofsinto a separate Slices''.\texttt{D700}—\texttt{LDIX}(sl--xs'), loads a signedl-bit (0\leq l\leq 257) integerxfrom Slices, and returns the remainder ofsass'.\texttt{D701}—\texttt{LDUX}(sl--xs'), loads an unsignedl-bit integerxfrom (the firstlbits of)s, with0\leq l\leq 256.\texttt{D702}—\texttt{PLDIX}(sl--x), preloads a signedl-bit integer from Slices, for0\leq l\leq 257.\texttt{D703}—\texttt{PLDUX}(sl--x), preloads an unsignedl-bit integer froms, for0\leq l\leq 256.\texttt{D704}—\texttt{LDIXQ}(sl--xs'-1ors0), quiet version of\texttt{LDIX}: loads a signedl-bit integer fromssimilarly to\texttt{LDIX}, but returns a success flag, equal to-1on success or to0on failure (ifsdoes not havelbits), instead of throwing a cell underflow exception.\texttt{D705}—\texttt{LDUXQ}(sl--xs'-1ors0), quiet version of\texttt{LDUX}.\texttt{D706}—\texttt{PLDIXQ}(sl--x-1or0), quiet version of\texttt{PLDIX}.\texttt{D707}—\texttt{PLDUXQ}(sl--x-1or0), quiet version of\texttt{PLDUX}.\texttt{D708}cc—\texttt{LDI}cc+1(s--xs'), a longer encoding for\texttt{LDI}.\texttt{D709}cc—\texttt{LDU}cc+1(s--xs'), a longer encoding for\texttt{LDU}.\texttt{D70A}cc—\texttt{PLDI}cc+1(s--x), preloads a signedcc+1-bit integer from Slices.\texttt{D70B}cc—\texttt{PLDU}cc+1(s--x), preloads an unsignedcc+1-bit integer froms.\texttt{D70C}cc—\texttt{LDIQ}cc+1(s--xs'-1ors0), a quiet version of\texttt{LDI}.\texttt{D70D}cc—\texttt{LDUQ}cc+1(s--xs'-1ors0), a quiet version of\texttt{LDU}.\texttt{D70E}cc—\texttt{PLDIQ}cc+1(s--x-1or0), a quiet version of\texttt{PLDI}.\texttt{D70F}cc—\texttt{PLDUQ}cc+1(s--x-1or0), a quiet version of\texttt{PLDU}.\texttt{D714\_}c—\texttt{PLDUZ}32(c+1)(s--sx), preloads the first32(c+1)bits of Slicesinto an unsigned integerx, for0\leq c\leq 7. Ifsis shorter than necessary, missing bits are assumed to be zero. This operation is intended to be used along with\texttt{IFBITJMP}and similar instructions.\texttt{D718}—\texttt{LDSLICEX}(sl--s''s'), loads the first0\leq l\leq 1023bits from Slicesinto a separate Slices'', returning the remainder ofsass'.\texttt{D719}—\texttt{PLDSLICEX}(sl--s''), returns the first0\leq l\leq 1023bits ofsass''.\texttt{D71A}—\texttt{LDSLICEXQ}(sl--s''s'-1ors0), a quiet version of\texttt{LDSLICEX}.\texttt{D71B}—\texttt{PLDSLICEXQ}(sl--s'-1or0), a quiet version of\texttt{LDSLICEXQ}.\texttt{D71C}cc—\texttt{LDSLICE}cc+1(s--s''s'), a longer encoding for\texttt{LDSLICE}.\texttt{D71D}cc—\texttt{PLDSLICE}cc+1(s--s''), returns the first0<cc+1\leq 256bits ofsass''.\texttt{D71E}cc—\texttt{LDSLICEQ}cc+1(s--s''s'-1ors0), a quiet version of\texttt{LDSLICE}.\texttt{D71F}cc—\texttt{PLDSLICEQ}cc+1(s--s''-1or0), a quiet version of\texttt{PLDSLICE}.\texttt{D720}—\texttt{SDCUTFIRST}(sl--s'), returns the first0\leq l\leq 1023bits ofs. It is equivalent to\texttt{PLDSLICEX}.\texttt{D721}—\texttt{SDSKIPFIRST}(sl--s'), returns all but the first0\leq l\leq 1023bits ofs. It is equivalent to\texttt{LDSLICEX};\texttt{NIP}.\texttt{D722}—\texttt{SDCUTLAST}(sl--s'), returns the last0\leq l\leq 1023bits ofs.\texttt{D723}—\texttt{SDSKIPLAST}(sl--s'), returns all but the last0\leq l\leq 1023bits ofs.\texttt{D724}—\texttt{SDSUBSTR}(sll'--s'), returns0\leq l'\leq 1023bits ofsstarting from offset0\leq l\leq 1023, thus extracting a bit substring out of the data ofs.\texttt{D726}—\texttt{SDBEGINSX}(ss'--s''), checks whethersbegins with (the data bits of)s', and removess'fromson success. On failure throws a cell deserialization exception. Primitive\texttt{SDPFXREV}can be considered a quiet version of\texttt{SDBEGINSX}.\texttt{D727}—\texttt{SDBEGINSXQ}(ss'--s''-1ors0), a quiet version of\texttt{SDBEGINSX}.\texttt{D72A\_}xsss—\texttt{SDBEGINS}(s--s''), checks whethersbegins with constant bitstringsssof length8x+3(with continuation bit assumed), where0\leq x\leq 127, and removessssfromson success.\texttt{D72802}—\texttt{SDBEGINS '0'}(s--s''), checks whethersbegins with a binary zero.\texttt{D72806}—\texttt{SDBEGINS '1'}(s--s''), checks whethersbegins with a binary one.\texttt{D72E\_}xsss—\texttt{SDBEGINSQ}(s--s''-1ors0), a quiet version of\texttt{SDBEGINS}.\texttt{D730}—\texttt{SCUTFIRST}(slr--s'), returns the first0\leq l\leq 1023bits and first0\leq r\leq 4references ofs.\texttt{D731}—\texttt{SSKIPFIRST}(slr--s').\texttt{D732}—\texttt{SCUTLAST}(slr--s'), returns the last0\leq l\leq 1023data bits and last0\leq r\leq 4references ofs.\texttt{D733}—\texttt{SSKIPLAST}(slr--s').\texttt{D734}—\texttt{SUBSLICE}(slrl'r'--s'), returns0\leq l'\leq 1023bits and0\leq r'\leq 4references from Slices, after skipping the first0\leq l\leq 1023bits and first0\leq r\leq 4references.\texttt{D736}—\texttt{SPLIT}(slr--s's''), splits the first0\leq l\leq 1023data bits and first0\leq r\leq 4references fromsintos', returning the remainder ofsass''.\texttt{D737}—\texttt{SPLITQ}(slr--s's''-1ors0), a quiet version of\texttt{SPLIT}.\texttt{D739}—\texttt{XCTOS}(c--s?), transforms an ordinary or exotic cell into a Slice, as if it were an ordinary cell. A flag is returned indicating whethercis exotic. If that be the case, its type can later be deserialized from the first eight bits ofs.\texttt{D73A}—\texttt{XLOAD}(c--c'), loads an exotic cellcand returns an ordinary cellc'. Ifcis already ordinary, does nothing. Ifccannot be loaded, throws an exception.\texttt{D73B}—\texttt{XLOADQ}(c--c'-1orc0), loads an exotic cellcas\texttt{XLOAD}, but returns 0 on failure.\texttt{D741}—\texttt{SCHKBITS}(sl-- ), checks whether there are at leastldata bits in Slices. If this is not the case, throws a cell deserialisation (i.e., cell underflow) exception.\texttt{D742}—\texttt{SCHKREFS}(sr-- ), checks whether there are at leastrreferences in Slices.\texttt{D743}—\texttt{SCHKBITREFS}(slr-- ), checks whether there are at leastldata bits andrreferences in Slices.\texttt{D745}—\texttt{SCHKBITSQ}(sl--?), checks whether there are at leastldata bits in Slices.\texttt{D746}—\texttt{SCHKREFSQ}(sr--?), checks whether there are at leastrreferences in Slices.\texttt{D747}—\texttt{SCHKBITREFSQ}(slr--?), checks whether there are at leastldata bits andrreferences in Slices.\texttt{D748}—\texttt{PLDREFVAR}(sn--c), returns then-th cell reference of Slicesfor0\leq n\leq 3.\texttt{D749}—\texttt{SBITS}(s--l), returns the number of data bits in Slices.\texttt{D74A}—\texttt{SREFS}(s--r), returns the number of references in Slices.\texttt{D74B}—\texttt{SBITREFS}(s--lr), returns both the number of data bits and the number of references ins.\texttt{D74E\_}n—\texttt{PLDREFIDX}n(s--c), returns then-th cell reference of Slices, where0\leq n\leq 3.\texttt{D74C}—\texttt{PLDREF}(s--c), preloads the first cell reference of a Slice.\texttt{D750}—\texttt{LDILE4}(s--xs'), loads a little-endian signed 32-bit integer.\texttt{D751}—\texttt{LDULE4}(s--xs'), loads a little-endian unsigned 32-bit integer.\texttt{D752}—\texttt{LDILE8}(s--xs'), loads a little-endian signed 64-bit integer.\texttt{D753}—\texttt{LDULE8}(s--xs'), loads a little-endian unsigned 64-bit integer.\texttt{D754}—\texttt{PLDILE4}(s--x), preloads a little-endian signed 32-bit integer.\texttt{D755}—\texttt{PLDULE4}(s--x), preloads a little-endian unsigned 32-bit integer.\texttt{D756}—\texttt{PLDILE8}(s--x), preloads a little-endian signed 64-bit integer.\texttt{D757}—\texttt{PLDULE8}(s--x), preloads a little-endian unsigned 64-bit integer.\texttt{D758}—\texttt{LDILE4Q}(s--xs'-1ors0), quietly loads a little-endian signed 32-bit integer.\texttt{D759}—\texttt{LDULE4Q}(s--xs'-1ors0), quietly loads a little-endian unsigned 32-bit integer.\texttt{D75A}—\texttt{LDILE8Q}(s--xs'-1ors0), quietly loads a little-endian signed 64-bit integer.\texttt{D75B}—\texttt{LDULE8Q}(s--xs'-1ors0), quietly loads a little-endian unsigned 64-bit integer.\texttt{D75C}—\texttt{PLDILE4Q}(s--x-1or0), quietly preloads a little-endian signed 32-bit integer.\texttt{D75D}—\texttt{PLDULE4Q}(s--x-1or0), quietly preloads a little-endian unsigned 32-bit integer.\texttt{D75E}—\texttt{PLDILE8Q}(s--x-1or0), quietly preloads a little-endian signed 64-bit integer.\texttt{D75F}—\texttt{PLDULE8Q}(s--x-1or0), quietly preloads a little-endian unsigned 64-bit integer.\texttt{D760}—\texttt{LDZEROES}(s--ns'), returns the countnof leading zero bits ins, and removes these bits froms.\texttt{D761}—\texttt{LDONES}(s--ns'), returns the countnof leading one bits ins, and removes these bits froms.\texttt{D762}—\texttt{LDSAME}(sx--ns'), returns the countnof leading bits equal to0\leq x\leq 1ins, and removes these bits froms.\texttt{D764}—\texttt{SDEPTH}(s--x), returns the depth of Slices. Ifshas no references, thenx=0; otherwisexis one plus the maximum of depths of cells referred to froms.\texttt{D765}—\texttt{CDEPTH}(c--x), returns the depth of Cellc. Ifchas no references, thenx=0; otherwisexis one plus the maximum of depths of cells referred to fromc. Ifcis a Null instead of a Cell, returns zero.
A.8 Continuation and control flow primitives
A.8.1. Unconditional control flow primitives
\texttt{D8}—\texttt{EXECUTE}or\texttt{CALLX}(c- ), calls or executes continuationc(i.e.,\texttt{cc}\leftarrow c\circ_0\texttt{cc}).\texttt{D9}—\texttt{JMPX}(c- ), jumps, or transfers control, to continuationc(i.e.,\texttt{cc}\leftarrow c\circ_0\texttt{c0}, or rather\texttt{cc}\leftarrow(c\circ_0\texttt{c0})\circ_1\texttt{c1}). The remainder of the previous current continuation\texttt{cc}is discarded.\texttt{DA}pr—\texttt{CALLXARGS}p,r(c- ), calls continuationcwithpparameters and expectingrreturn values,0\leq p\leq15,0\leq r\leq 15.\texttt{DB0}p—\texttt{CALLXARGS}p,-1(c- ), calls continuationcwith0\leq p\leq15parameters, expecting an arbitrary number of return values.\texttt{DB1}p—\texttt{JMPXARGS}p(c- ), jumps to continuationc, passing only the top0\leq p\leq 15values from the current stack to it (the remainder of the current stack is discarded).\texttt{DB2}r—\texttt{RETARGS}r, returns to\texttt{c0}, with0\leq r\leq 15return values taken from the current stack.\texttt{DB30}—\texttt{RET}or\texttt{RETTRUE}, returns to the continuation at\texttt{c0}(i.e., performs\texttt{cc}\leftarrow\texttt{c0}). The remainder of the current continuation\texttt{cc}is discarded. Approximately equivalent to\texttt{PUSH c0};\texttt{JMPX}.\texttt{DB31}—\texttt{RETALT}or\texttt{RETFALSE}, returns to the continuation at\texttt{c1}(i.e.,\texttt{cc}\leftarrow\texttt{c1}). Approximately equivalent to\texttt{PUSH c1};\texttt{JMPX}.\texttt{DB32}—\texttt{BRANCH}or\texttt{RETBOOL}(f- ), performs\texttt{RETTRUE}if integerf\neq 0, or\texttt{RETFALSE}iff=0.\texttt{DB34}—\texttt{CALLCC}(c- ), call with current continuation, transfers control toc, pushing the old value of\texttt{cc}intoc's stack (instead of discarding it or writing it into new\texttt{c0}).\texttt{DB35}—\texttt{JMPXDATA}(c- ), similar to\texttt{CALLCC}, but the remainder of the current continuation (the old value of\texttt{cc}) is converted into a Slice before pushing it into the stack ofc.\texttt{DB36}pr—\texttt{CALLCCARGS}p,r(c- ), similar to\texttt{CALLXARGS}, but pushes the old value of\texttt{cc}(along with the top0\leq p\leq 15values from the original stack) into the stack of newly-invoked continuationc, setting\texttt{cc.nargs}to-1\leq r\leq 14.\texttt{DB38}—\texttt{CALLXVARARGS}(cpr- ), similar to\texttt{CALLXARGS}, but takes-1\leq p,r\leq254from the stack. The next three operations also takepandrfrom the stack, both in the range-1\ldots254.\texttt{DB39}—\texttt{RETVARARGS}(pr- ), similar to\texttt{RETARGS}.\texttt{DB3A}—\texttt{JMPXVARARGS}(cpr- ), similar to\texttt{JMPXARGS}.\texttt{DB3B}—\texttt{CALLCCVARARGS}(cpr- ), similar to\texttt{CALLCCARGS}.\texttt{DB3C}—\texttt{CALLREF}, equivalent to\texttt{PUSHREFCONT};\texttt{CALLX}.\texttt{DB3D}—\texttt{JMPREF}, equivalent to\texttt{PUSHREFCONT};\texttt{JMPX}.\texttt{DB3E}—\texttt{JMPREFDATA}, equivalent to\texttt{PUSHREFCONT};\texttt{JMPXDATA}.\texttt{DB3F}—\texttt{RETDATA}, equivalent to\texttt{PUSH c0};\texttt{JMPXDATA}. In this way, the remainder of the current continuation is converted into a Slice and returned to the caller.
A.8.2. Conditional control flow primitives
\texttt{DC}—\texttt{IFRET}(f- ), performs a\texttt{RET}, but only if integerfis non-zero. Iffis a\texttt{NaN}, throws an integer overflow exception.\texttt{DD}—\texttt{IFNOTRET}(f- ), performs a\texttt{RET}, but only if integerfis zero.\texttt{DE}—\texttt{IF}(fc- ), performs\texttt{EXECUTE}forc(i.e., executesc), but only if integerfis non-zero. Otherwise simply discards both values.\texttt{DF}—\texttt{IFNOT}(fc- ), executes continuationc, but only if integerfis zero. Otherwise simply discards both values.\texttt{E0}—\texttt{IFJMP}(fc- ), jumps toc(similarly to\texttt{JMPX}), but only iffis non-zero.\texttt{E1}—\texttt{IFNOTJMP}(fc- ), jumps toc(similarly to\texttt{JMPX}), but only iffis zero.\texttt{E2}—\texttt{IFELSE}(fcc'- ), if integerfis non-zero, executesc, otherwise executesc'. Equivalent to\texttt{CONDSELCHK};\texttt{EXECUTE}.\texttt{E300}—\texttt{IFREF}(f- ), equivalent to\texttt{PUSHREFCONT};\texttt{IF}, with the optimization that the cell reference is not actually loaded into a Slice and then converted into an ordinary Continuation iff=0. Similar remarks apply to the next three primitives.\texttt{E301}—\texttt{IFNOTREF}(f- ), equivalent to\texttt{PUSHREFCONT};\texttt{IFNOT}.\texttt{E302}—\texttt{IFJMPREF}(f- ), equivalent to\texttt{PUSHREFCONT};\texttt{IFJMP}.\texttt{E303}—\texttt{IFNOTJMPREF}(f- ), equivalent to\texttt{PUSHREFCONT};\texttt{IFNOTJMP}.\texttt{E304}—\texttt{CONDSEL}(fxy-xory), if integerfis non-zero, returnsx, otherwise returnsy. Notice that no type checks are performed onxandy; as such, it is more like a conditional stack operation. Roughly equivalent to\texttt{ROT};\texttt{ISZERO};\texttt{INC};\texttt{ROLLX};\texttt{NIP}.\texttt{E305}—\texttt{CONDSELCHK}(fxy-xory), same as\texttt{CONDSEL}, but first checks whetherxandyhave the same type.\texttt{E308}—\texttt{IFRETALT}(f-), performs\texttt{RETALT}if integerf\neq0.\texttt{E309}—\texttt{IFNOTRETALT}(f-), performs\texttt{RETALT}if integerf=0.\texttt{E30D}—\texttt{IFREFELSE}(fc-), equivalent to\texttt{PUSHREFCONT};\texttt{SWAP};\texttt{IFELSE}, with the optimization that the cell reference is not actually loaded into a Slice and then converted into an ordinary Continuation iff=0. Similar remarks apply to the next two primitives: Cells are converted into Continuations only when necessary.\texttt{E30E}—\texttt{IFELSEREF}(fc-), equivalent to\texttt{PUSHREFCONT};\texttt{IFELSE}.\texttt{E30F}—\texttt{IFREFELSEREF}(f-), equivalent to\texttt{PUSHREFCONT};\texttt{PUSHREFCONT};\texttt{IFELSE}.\texttt{E310}--\texttt{E31F}— reserved for loops with break operators, A.8.4 below.\texttt{E39\_}n—\texttt{IFBITJMP}n(xc-x), checks whether bit0\leq n\leq 31is set in integerx, and if so, performs\texttt{JMPX}to continuationc. Valuexis left in the stack.\texttt{E3B\_}n—\texttt{IFNBITJMP}n(xc-x), jumps tocif bit0\leq n\leq 31is not set in integerx.\texttt{E3D\_}n—\texttt{IFBITJMPREF}n(x-x), performs a\texttt{JMPREF}if bit0\leq n\leq 31is set in integerx.\texttt{E3F\_}n—\texttt{IFNBITJMPREF}n(x-x), performs a\texttt{JMPREF}if bit0\leq n\leq 31is not set in integerx.
A.8.3. Control flow primitives: loops
Most of the loop primitives listed below are implemented with the aid of extraordinary continuations, such as \texttt{ec\_until}, with the loop body and the original current continuation \texttt{cc} stored as the arguments to this extraordinary continuation. Typically a suitable extraordinary continuation is constructed, and then saved into the loop body continuation savelist as \texttt{c0}; after that, the modified loop body continuation is loaded into \texttt{cc} and executed in the usual fashion. All of these loop primitives have \texttt{*BRK} versions, adapted for breaking out of a loop; they additionally set \texttt{c1} to the original current continuation (or original \texttt{c0} for \texttt{*ENDBRK} versions), and save the old \texttt{c1} into the savelist of the original current continuation (or of the original \texttt{c0} for \texttt{*ENDBRK} versions).
\texttt{E4}—\texttt{REPEAT}(nc- ), executes continuationcntimes, if integernis non-negative. Ifn\geq2^{31}orn<-2^{31}, generates a range check exception. Notice that a\texttt{RET}inside the code ofcworks as a\texttt{continue}, not as a\texttt{break}. One should use either alternative (experimental) loops or alternative\texttt{RETALT}(along with a\texttt{SETEXITALT}before the loop) to\texttt{break}out of a loop.\texttt{E5}—\texttt{REPEATEND}(n- ), similar to\texttt{REPEAT}, but it is applied to the current continuation\texttt{cc}.\texttt{E6}—\texttt{UNTIL}(c- ), executes continuationc, then pops an integerxfrom the resulting stack. Ifxis zero, performs another iteration of this loop. The actual implementation of this primitive involves an extraordinary continuation\texttt{ec\_until}with its arguments set to the body of the loop (continuationc) and the original current continuation\texttt{cc}. This extraordinary continuation is then saved into the savelist ofcasc.\texttt{c0}and the modifiedcis then executed. The other loop primitives are implemented similarly with the aid of suitable extraordinary continuations.\texttt{E7}—\texttt{UNTILEND}( - ), similar to\texttt{UNTIL}, but executes the current continuation\texttt{cc}in a loop. When the loop exit condition is satisfied, performs a\texttt{RET}.\texttt{E8}—\texttt{WHILE}(c'c- ), executesc'and pops an integerxfrom the resulting stack. Ifxis zero, exists the loop and transfers control to the original\texttt{cc}. Ifxis non-zero, executesc, and then begins a new iteration.\texttt{E9}—\texttt{WHILEEND}(c'- ), similar to\texttt{WHILE}, but uses the current continuation\texttt{cc}as the loop body.\texttt{EA}—\texttt{AGAIN}(c- ), similar to\texttt{REPEAT}, but executescinfinitely many times. A\texttt{RET}only begins a new iteration of the infinite loop, which can be exited only by an exception, or a\texttt{RETALT}(or an explicit\texttt{JMPX}).\texttt{EB}—\texttt{AGAINEND}( - ), similar to\texttt{AGAIN}, but performed with respect to the current continuation\texttt{cc}.\texttt{E314}—\texttt{REPEATBRK}(nc- ), similar to\texttt{REPEAT}, but also sets\texttt{c1}to the original\texttt{cc}after saving the old value of\texttt{c1}into the savelist of the original\texttt{cc}. In this way\texttt{RETALT}could be used to break out of the loop body.\texttt{E315}—\texttt{REPEATENDBRK}(n- ), similar to\texttt{REPEATEND}, but also sets\texttt{c1}to the original\texttt{c0}after saving the old value of\texttt{c1}into the savelist of the original\texttt{c0}. Equivalent to\texttt{SAMEALTSAVE};\texttt{REPEATEND}.\texttt{E316}—\texttt{UNTILBRK}(c- ), similar to\texttt{UNTIL}, but also modifies\texttt{c1}in the same way as\texttt{REPEATBRK}.\texttt{E317}—\texttt{UNTILENDBRK}( - ), equivalent to\texttt{SAMEALTSAVE};\texttt{UNTILEND}.\texttt{E318}—\texttt{WHILEBRK}(c'c- ), similar to\texttt{WHILE}, but also modifies\texttt{c1}in the same way as\texttt{REPEATBRK}.\texttt{E319}—\texttt{WHILEENDBRK}(c- ), equivalent to\texttt{SAMEALTSAVE};\texttt{WHILEEND}.\texttt{E31A}—\texttt{AGAINBRK}(c- ), similar to\texttt{AGAIN}, but also modifies\texttt{c1}in the same way as\texttt{REPEATBRK}.\texttt{E31B}—\texttt{AGAINENDBRK}( - ), equivalent to\texttt{SAMEALTSAVE};\texttt{AGAINEND}.
A.8.4. Manipulating the stack of continuations
\texttt{EC}rn—\texttt{SETCONTARGS}r,n(x_1x_2\ldots x_rc-c'), similar to\texttt{SETCONTARGS}r, but setsc.\texttt{nargs}to the final size of the stack ofc'plusn. In other words, transformscinto a closure or a partially applied function, with0\leq n\leq 14arguments missing.\texttt{EC0}n—\texttt{SETNUMARGS}nor\texttt{SETCONTARGS}0,n(c-c'), setsc.\texttt{nargs}tonplus the current depth ofc's stack, where0\leq n\leq 14. Ifc.\texttt{nargs}is already set to a non-negative value, does nothing.\texttt{EC}r\texttt{F}—\texttt{SETCONTARGS}ror\texttt{SETCONTARGS}r,-1(x_1x_2\ldots x_rc-c'), pushes0\leq r\leq 15valuesx_1\ldots x_rinto the stack of (a copy of) the continuationc, starting withx_1. If the final depth ofc's stack turns out to be greater thanc.\texttt{nargs}, a stack overflow exception is generated.\texttt{ED0}p—\texttt{RETURNARGS}p( - ), leaves only the top0\leq p\leq 15values in the current stack (somewhat similarly to\texttt{ONLYTOPX}), with all the unused bottom values not discarded, but saved into continuation\texttt{c0}in the same way as\texttt{SETCONTARGS}does.\texttt{ED10}—\texttt{RETURNVARARGS}(p- ), similar to\texttt{RETURNARGS}, but with Integer0\leq p\leq 255taken from the stack.\texttt{ED11}—\texttt{SETCONTVARARGS}(x_1x_2\ldots x_rcrn-c'), similar to\texttt{SETCONTARGS}, but with0\leq r\leq 255and-1\leq n\leq 255taken from the stack.\texttt{ED12}—\texttt{SETNUMVARARGS}(cn-c'), where-1\leq n\leq 255. Ifn=-1, this operation does nothing (c'=c). Otherwise its action is similar to\texttt{SETNUMARGS}n, but withntaken from the stack.
A.8.5. Creating simple continuations and closures
\texttt{ED1E}—\texttt{BLESS}(s-c), transforms a Slicesinto a simple ordinary continuationc, withc.\texttt{code}=sand an empty stack and savelist.\texttt{ED1F}—\texttt{BLESSVARARGS}(x_1\ldots x_rsrn-c), equivalent to\texttt{ROT};\texttt{BLESS};\texttt{ROTREV};\texttt{SETCONTVARARGS}.\texttt{EE}rn—\texttt{BLESSARGS}r,n(x_1\ldots x_rs-c), where0\leq r\leq 15,-1\leq n\leq 14, equivalent to\texttt{BLESS};\texttt{SETCONTARGS}r,n. The value ofnis represented inside the instruction by the 4-bit integern\bmod16.\texttt{EE0}n—\texttt{BLESSNUMARGS}nor\texttt{BLESSARGS 0},n(s-c), also transforms a Slicesinto a Continuationc, but setsc.\texttt{nargs}to0\leq n\leq 14.
A.8.6. Operations with continuation savelists and control registers
\texttt{ED4}i—\texttt{PUSH c}(i)or\texttt{PUSHCTR c}(i)( -x), pushes the current value of control register\texttt{c}(i). If the control register is not supported in the current codepage, or if it does not have a value, an exception is triggered.\texttt{ED44}—\texttt{PUSH c4}or\texttt{PUSHROOT}, pushes the "global data root" cell reference, thus enabling access to persistent smart-contract data.\texttt{ED5}i—\texttt{POP c}(i)or\texttt{POPCTR c}(i)(x- ), pops a valuexfrom the stack and stores it into control register\texttt{c}(i), if supported in the current codepage. Notice that if a control register accepts only values of a specific type, a type-checking exception may occur.\texttt{ED54}—\texttt{POP c4}or\texttt{POPROOT}, sets the "global data root" cell reference, thus allowing modification of persistent smart-contract data.\texttt{ED6}i—\texttt{SETCONT c}(i)or\texttt{SETCONTCTR c}(i)(xc-c'), storesxinto the savelist of continuationcas\texttt{c}(i), and returns the resulting continuationc'. Almost all operations with continuations may be expressed in terms of\texttt{SETCONTCTR},\texttt{POPCTR}, and\texttt{PUSHCTR}.\texttt{ED7}i—\texttt{SETRETCTR c}(i)(x- ), equivalent to\texttt{PUSH c0};\texttt{SETCONTCTR c}(i);\texttt{POP c0}.\texttt{ED8}i—\texttt{SETALTCTR c}(i)(x- ), equivalent to\texttt{PUSH c1};\texttt{SETCONTCTR c}(i);\texttt{POP c0}.\texttt{ED9}i—\texttt{POPSAVE c}(i)or\texttt{POPCTRSAVE c}(i)(x-), similar to\texttt{POP c}(i), but also saves the old value of\texttt{c}(i)into continuation\texttt{c0}. Equivalent (up to exceptions) to\texttt{SAVECTR c}(i);\texttt{POP c}(i).\texttt{EDA}i—\texttt{SAVE c}(i)or\texttt{SAVECTR c}(i)( - ), saves the current value of\texttt{c}(i)into the savelist of continuation\texttt{c0}. If an entry for\texttt{c}(i)is already present in the savelist of\texttt{c0}, nothing is done. Equivalent to\texttt{PUSH c}(i);\texttt{SETRETCTR c}(i).\texttt{EDB}i—\texttt{SAVEALT c}(i)or\texttt{SAVEALTCTR c}(i)( - ), similar to\texttt{SAVE c}(i), but saves the current value of\texttt{c}(i)into the savelist of\texttt{c1}, not\texttt{c0}.\texttt{EDC}i—\texttt{SAVEBOTH c}(i)or\texttt{SAVEBOTHCTR c}(i)( - ), equivalent to\texttt{DUP};\texttt{SAVE c}(i);\texttt{SAVEALT c}(i).\texttt{EDE0}—\texttt{PUSHCTRX}(i-x), similar to\texttt{PUSHCTR c}(i), but withi,0\leq i\leq255, taken from the stack. Notice that this primitive is one of the few "exotic" primitives, which are not polymorphic like stack manipulation primitives, and at the same time do not have well-defined types of parameters and return values, because the type ofxdepends oni.\texttt{EDE1}—\texttt{POPCTRX}(xi- ), similar to\texttt{POPCTR c}(i), but with0\leq i\leq255from the stack.\texttt{EDE2}—\texttt{SETCONTCTRX}(xci-c'), similar to\texttt{SETCONTCTR c}(i), but with0\leq i\leq 255from the stack.\texttt{EDF0}—\texttt{COMPOS}or\texttt{BOOLAND}(cc'-c''), computes the compositionc\circ_0c', which has the meaning of "performc, and, if successful, performc'" (ifcis a boolean circuit) or simply "performc, thenc'". Equivalent to\texttt{SWAP};\texttt{SETCONT c0}.\texttt{EDF1}—\texttt{COMPOSALT}or\texttt{BOOLOR}(cc'-c''), computes the alternative compositionc\circ_1 c', which has the meaning of "performc, and, if not successful, performc'" (ifcis a boolean circuit). Equivalent to\texttt{SWAP};\texttt{SETCONT c1}.\texttt{EDF2}—\texttt{COMPOSBOTH}(cc'-c''), computes(c\circ_0c')\circ_1c', which has the meaning of "compute boolean circuitc, then computec', regardless of the result ofc".\texttt{EDF3}—\texttt{ATEXIT}(c- ), sets\texttt{c0}\leftarrow c\circ_0\texttt{c0}. In other words,cwill be executed before exiting current subroutine.\texttt{EDF4}—\texttt{ATEXITALT}(c- ), sets\texttt{c1}\leftarrow c\circ_1\texttt{c1}. In other words,cwill be executed before exiting current subroutine by its alternative return path.\texttt{EDF5}—\texttt{SETEXITALT}(c- ), sets\texttt{c1}\leftarrow (c\circ_0\texttt{c0})\circ_1\texttt{c1}. In this way, a subsequent\texttt{RETALT}will first executec, then transfer control to the original\texttt{c0}. This can be used, for instance, to exit from nested loops.\texttt{EDF6}—\texttt{THENRET}(c-c'), computesc':=c\circ_0\texttt{c0}\texttt{EDF7}—\texttt{THENRETALT}(c-c'), computesc':=c\circ_0\texttt{c1}\texttt{EDF8}—\texttt{INVERT}( - ), interchanges\texttt{c0}and\texttt{c1}.\texttt{EDF9}—\texttt{BOOLEVAL}(c-?), performs\texttt{cc}\leftarrow \bigl(c\circ_0((\texttt{PUSH}\,-1)\circ_0\texttt{cc})\bigr)\circ_1((\texttt{PUSH}\,0)\circ_0\texttt{cc}). Ifcrepresents a boolean circuit, the net effect is to evaluate it and push either-1or0into the stack before continuing.\texttt{EDFA}—\texttt{SAMEALT}( - ), setsc_1:=c_0. Equivalent to\texttt{PUSH c0};\texttt{POP c1}.\texttt{EDFB}—\texttt{SAMEALTSAVE}( - ), setsc_1:=c_0, but first saves the old value ofc_1into the savelist ofc_0. Equivalent to\texttt{SAVE c1};\texttt{SAMEALT}.\texttt{EE}rn—\texttt{BLESSARGS}r,n(x_1\ldots x_rs--c), described in A.8.4.
A.8.7. Dictionary subroutine calls and jumps
\texttt{F0}n—\texttt{CALL}nor\texttt{CALLDICT}n( -n), calls the continuation in\texttt{c3}, pushing integer0\leq n\leq 255into its stack as an argument. Approximately equivalent to\texttt{PUSHINT}n;\texttt{PUSH c3};\texttt{EXECUTE}.\texttt{F12\_}n—\texttt{CALL}nfor0\leq n<2^{14}( -n), an encoding of\texttt{CALL}nfor larger values ofn.\texttt{F16\_}n—\texttt{JMP}nor\texttt{JMPDICT}n( -n), jumps to the continuation in\texttt{c3}, pushing integer0\leq n<2^{14}as its argument. Approximately equivalent to\texttt{PUSHINT}n;\texttt{PUSH c3};\texttt{JMPX}.\texttt{F1A\_}n—\texttt{PREPARE}nor\texttt{PREPAREDICT}n( -nc), equivalent to\texttt{PUSHINT}n;\texttt{PUSH c3}, for0\leq n<2^{14}. In this way,\texttt{CALL}nis approximately equivalent to\texttt{PREPARE}n;\texttt{EXECUTE}, and\texttt{JMP}nis approximately equivalent to\texttt{PREPARE}n;\texttt{JMPX}. One might use, for instance,\texttt{CALLARGS}or\texttt{CALLCC}instead of\texttt{EXECUTE}here.
A.9 Exception generating and handling primitives
A.9.1. Throwing exceptions
\texttt{F22\_}nn—\texttt{THROW}nn( -0nn), throws exception0\leq nn\leq 63with parameter zero. In other words, it transfers control to the continuation in\texttt{c2}, pushing0andnninto its stack, and discarding the old stack altogether.\texttt{F26\_}nn—\texttt{THROWIF}nn(f- ), throws exception0\leq nn\leq 63with parameter zero only if integerf\neq0.\texttt{F2A\_}nn—\texttt{THROWIFNOT}nn(f- ), throws exception0\leq nn\leq 63with parameter zero only if integerf=0.\texttt{F2C4\_}nn—\texttt{THROW}nnfor0\leq nn<2^{11}, an encoding of\texttt{THROW}nnfor larger values ofnn.\texttt{F2CC\_}nn—\texttt{THROWARG}nn(x-xnn), throws exception0\leq nn<2^{11}with parameterx, by copyingxandnninto the stack of\texttt{c2}and transferring control to\texttt{c2}.\texttt{F2D4\_}nn—\texttt{THROWIF}nn(f- ) for0\leq nn<2^{11}.\texttt{F2DC\_}nn—\texttt{THROWARGIF}nn(xf- ), throws exception0\leq nn<2^{11}with parameterxonly if integerf\neq0.\texttt{F2E4\_}nn—\texttt{THROWIFNOT}nn(f- ) for0\leq nn<2^{11}.\texttt{F2EC\_}nn—\texttt{THROWARGIFNOT}nn(xf- ), throws exception0\leq nn<2^{11}with parameterxonly if integerf=0.\texttt{F2F0}—\texttt{THROWANY}(n-0n), throws exception0\leq n<2^{16}with parameter zero. Approximately equivalent to\texttt{PUSHINT 0};\texttt{SWAP};\texttt{THROWARGANY}.\texttt{F2F1}—\texttt{THROWARGANY}(xn-xn), throws exception0\leq n<2^{16}with parameterx, transferring control to the continuation in\texttt{c2}. Approximately equivalent to\texttt{PUSH c2};\texttt{JMPXARGS 2}.\texttt{F2F2}—\texttt{THROWANYIF}(nf- ), throws exception0\leq n<2^{16}with parameter zero only iff\neq0.\texttt{F2F3}—\texttt{THROWARGANYIF}(xnf- ), throws exception0\leq n<2^{16}with parameterxonly iff\neq0.\texttt{F2F4}—\texttt{THROWANYIFNOT}(nf- ), throws exception0\leq n<2^{16}with parameter zero only iff=0.\texttt{F2F5}—\texttt{THROWARGANYIFNOT}(xnf- ), throws exception0\leq n<2^{16}with parameterxonly iff=0.
A.9.2. Catching and handling exceptions
\texttt{F2FF}—\texttt{TRY}(cc'- ), sets\texttt{c2}toc', first saving the old value of\texttt{c2}both into the savelist ofc'and into the savelist of the current continuation, which is stored intoc.\texttt{c0}andc'.\texttt{c0}. Then runscsimilarly to\texttt{EXECUTE}. Ifcdoes not throw any exceptions, the original value of\texttt{c2}is automatically restored on return fromc. If an exception occurs, the execution is transferred toc', but the original value of\texttt{c2}is restored in the process, so thatc'can re-throw the exception by\texttt{THROWANY}if it cannot handle it by itself.\texttt{F3}pr—\texttt{TRYARGS}p,r(cc'- ), similar to\texttt{TRY}, but with\texttt{CALLARGS}p,rinternally used instead of\texttt{EXECUTE}. In this way, all but the top0\leq p\leq 15stack elements will be saved into current continuation's stack, and then restored upon return from eithercorc', with the top0\leq r\leq 15values of the resulting stack ofcorc'copied as return values.
A.10 Dictionary manipulation primitives
TVM's dictionary support is discussed at length in 3.3. The basic operations with dictionaries, while the taxonomy of dictionary manipulation primitives. Here we use the concepts and notation introduced in those sections.
Dictionaries admit two different representations as TVM stack values:
-
A Slice
swith a serialization of a TL-B value of type\mathit{HashmapE}(n,X). In other words,sconsists either of one bit equal to zero (if the dictionary is empty), or of one bit equal to one and a reference to a Cell containing the root of the binary tree, i.e., a serialized value of type\mathit{Hashmap}(n,X). -
A "maybe Cell"
c^?, i.e., a value that is either a Cell (containing a serialized value of type\mathit{Hashmap}(n,X)as before) or a Null (corresponding to an empty dictionary). When a "maybe Cell"c^?is used to represent a dictionary, we usually denote it byDin the stack notation.
Most of the dictionary primitives listed below accept and return dictionaries in the second form, which is more convenient for stack manipulation. However, serialized dictionaries inside larger TL-B objects use the first representation.
Opcodes starting with \texttt{F4} and \texttt{F5} are reserved for dictionary operations.
A.10.1. Dictionary creation
\texttt{6D}—\texttt{NEWDICT}( -D), returns a new empty dictionary. It is an alternative mnemonics for\texttt{PUSHNULL}, A.3.1.\texttt{6E}—\texttt{DICTEMPTY}(D-?), checks whether dictionaryDis empty, and returns-1or0accordingly. It is an alternative mnemonics for\texttt{ISNULL}, A.3.1.
A.10.2. Dictionary serialization and deserialization
\texttt{CE}—\texttt{STDICTS}(sb-b'), stores a Slice-represented dictionarysinto Builderb. It is actually a synonym for\texttt{STSLICE}.\texttt{F400}—\texttt{STDICT}or\texttt{STOPTREF}(Db-b'), stores dictionaryDinto Builderb, returning the resulting Builderb'. In other words, ifDis a cell, performs\texttt{STONE}and\texttt{STREF}; ifDis Null, performs\texttt{NIP}and\texttt{STZERO}; otherwise throws a type checking exception.\texttt{F401}—\texttt{SKIPDICT}or\texttt{SKIPOPTREF}(s-s'), equivalent to\texttt{LDDICT};\texttt{NIP}.\texttt{F402}—\texttt{LDDICTS}(s-s's''), loads (parses) a (Slice-represented) dictionarys'from Slices, and returns the remainder ofsass''. This is a "split function" for all\mathit{HashmapE}(n,X)dictionary types.\texttt{F403}—\texttt{PLDDICTS}(s-s'), preloads a (Slice-represented) dictionarys'from Slices. Approximately equivalent to\texttt{LDDICTS};\texttt{DROP}.\texttt{F404}—\texttt{LDDICT}or\texttt{LDOPTREF}(s-Ds'), loads (parses) a dictionaryDfrom Slices, and returns the remainder ofsass'. May be applied to dictionaries or to values of arbitrary(\text{\textasciicircum}Y) types.\texttt{F405}—\texttt{PLDDICT}or\texttt{PLDOPTREF}(s-D), preloads a dictionaryDfrom Slices. Approximately equivalent to\texttt{LDDICT};\texttt{DROP}.\texttt{F406}—\texttt{LDDICTQ}(s-Ds'-1ors0), a quiet version of\texttt{LDDICT}.\texttt{F407}—\texttt{PLDDICTQ}(s-D-1or0), a quiet version of\texttt{PLDDICT}.
A.10.3. \texttt{Get} dictionary operations
\texttt{F40A}—\texttt{DICTGET}(kDn-x-1or0), looks up keyk(represented by a Slice, the first0\leq n\leq 1023data bits of which are used as a key) in dictionaryDof type\mathit{HashmapE}(n,X)withn-bit keys. On success, returns the value found as a Slicex.\texttt{F40B}—\texttt{DICTGETREF}(kDn-c-1or0), similar to\texttt{DICTGET}, but with a\texttt{LDREF};\texttt{ENDS}applied toxon success. This operation is useful for dictionaries of type\mathit{HashmapE}(n, \text{\textasciicircum}Y).\texttt{F40C}—\texttt{DICTIGET}(iDn-x-1or0), similar to\texttt{DICTGET}, but with a signed (big-endian)n-bit Integerias a key. Ifidoes not fit intonbits, returns0. Ifiis a\texttt{NaN}, throws an integer overflow exception.\texttt{F40D}—\texttt{DICTIGETREF}(iDn-c-1or0), combines\texttt{DICTIGET}with\texttt{DICTGETREF}: it uses signedn-bit Integerias a key and returns a Cell instead of a Slice on success.\texttt{F40E}—\texttt{DICTUGET}(iDn-x-1or0), similar to\texttt{DICTIGET}, but with unsigned (big-endian)n-bit Integeriused as a key.\texttt{F40F}—\texttt{DICTUGETREF}(iDn-c-1or0), similar to\texttt{DICTIGETREF}, but with an unsignedn-bit Integer keyi.
A.10.4. \texttt{Set}/\texttt{Replace}/\texttt{Add} dictionary operations
The mnemonics of the following dictionary primitives are constructed in a systematic fashion according to the regular expression \texttt{DICT}[,\texttt{I},\texttt{U}](\texttt{SET},\texttt{REPLACE},\texttt{ADD})[\texttt{GET}][\texttt{REF}] depending on the type of the key used (a Slice or a signed or unsigned Integer), the dictionary operation to be performed, and the way the values are accepted and returned (as Cells or as Slices). Therefore, we provide a detailed description only for some primitives, assuming that this information is sufficient for the reader to understand the precise action of the remaining primitives.
\texttt{F412}—\texttt{DICTSET}(xkDn-D'), sets the value associated withn-bit keyk(represented by a Slice as in\texttt{DICTGET}) in dictionaryD(also represented by a Slice) to valuex(again a Slice), and returns the resulting dictionary asD'.\texttt{F413}—\texttt{DICTSETREF}(ckDn-D'), similar to\texttt{DICTSET}, but with the value set to a reference to Cellc.\texttt{F414}—\texttt{DICTISET}(xiDn-D'), similar to\texttt{DICTSET}, but with the key represented by a (big-endian) signedn-bit integeri. Ifidoes not fit intonbits, a range check exception is generated.\texttt{F415}—\texttt{DICTISETREF}(ciDn-D'), similar to\texttt{DICTSETREF}, but with the key a signedn-bit integer as in\texttt{DICTISET}.\texttt{F416}—\texttt{DICTUSET}(xiDn-D'), similar to\texttt{DICTISET}, but withian unsignedn-bit integer.\texttt{F417}—\texttt{DICTUSETREF}(ciDn-D'), similar to\texttt{DICTISETREF}, but withiunsigned.\texttt{F41A}—\texttt{DICTSETGET}(xkDn-D'y-1orD'0), combines\texttt{DICTSET}with\texttt{DICTGET}: it sets the value corresponding to keyktox, but also returns the old valueyassociated with the key in question, if present.\texttt{F41B}—\texttt{DICTSETGETREF}(ckDn-D'c'-1orD'0), combines\texttt{DICTSETREF}with\texttt{DICTGETREF}similarly to\texttt{DICTSETGET}.\texttt{F41C}—\texttt{DICTISETGET}(xiDn-D'y-1orD'0), similar to\texttt{DICTSETGET}, but with the key represented by a big-endian signedn-bit Integeri.\texttt{F41D}—\texttt{DICTISETGETREF}(ciDn-D'c'-1orD'0), a version of\texttt{DICTSETGETREF}with signed Integerias a key.\texttt{F41E}—\texttt{DICTUSETGET}(xiDn-D'y-1orD'0), similar to\texttt{DICTISETGET}, but withian unsignedn-bit integer.\texttt{F41F}—\texttt{DICTUSETGETREF}(ciDn-D'c'-1orD'0).\texttt{F422}—\texttt{DICTREPLACE}(xkDn-D'-1orD0), a\texttt{Replace}operation, which is similar to\texttt{DICTSET}, but sets the value of keykin dictionaryDtoxonly if the keykwas already present inD.\texttt{F423}—\texttt{DICTREPLACEREF}(ckDn-D'-1orD0), a\texttt{Replace}counterpart of\texttt{DICTSETREF}.\texttt{F424}—\texttt{DICTIREPLACE}(xiDn-D'-1orD0), a version of\texttt{DICTREPLACE}with signedn-bit Integeriused as a key.\texttt{F425}—\texttt{DICTIREPLACEREF}(ciDn-D'-1orD0).\texttt{F426}—\texttt{DICTUREPLACE}(xiDn-D'-1orD0).\texttt{F427}—\texttt{DICTUREPLACEREF}(ciDn-D'-1orD0).\texttt{F42A}—\texttt{DICTREPLACEGET}(xkDn-D'y-1orD0), a\texttt{Replace}counterpart of\texttt{DICTSETGET}: on success, also returns the old value associated with the key in question.\texttt{F42B}—\texttt{DICTREPLACEGETREF}(ckDn-D'c'-1orD0).\texttt{F42C}—\texttt{DICTIREPLACEGET}(xiDn-D'y-1orD0).\texttt{F42D}—\texttt{DICTIREPLACEGETREF}(ciDn-D'c'-1orD0).\texttt{F42E}—\texttt{DICTUREPLACEGET}(xiDn-D'y-1orD0).\texttt{F42F}—\texttt{DICTUREPLACEGETREF}(ciDn-D'c'-1orD0).\texttt{F432}—\texttt{DICTADD}(xkDn-D'-1orD0), an Add counterpart of\texttt{DICTSET}: sets the value associated with keykin dictionaryDtox, but only if it is not already present inD.\texttt{F433}—\texttt{DICTADDREF}(ckDn-D'-1orD0).\texttt{F434}—\texttt{DICTIADD}(xiDn-D'-1orD0).\texttt{F435}—\texttt{DICTIADDREF}(ciDn-D'-1orD0).\texttt{F436}—\texttt{DICTUADD}(xiDn-D'-1orD0).\texttt{F437}—\texttt{DICTUADDREF}(ciDn-D'-1orD0).\texttt{F43A}—\texttt{DICTADDGET}(xkDn-D'-1orDy0), an Add counterpart of\texttt{DICTSETGET}: sets the value associated with keykin dictionaryDtox, but only if keykis not already present inD. Otherwise, just returns the old valueywithout changing the dictionary.\texttt{F43B}—\texttt{DICTADDGETREF}(ckDn-D'-1orDc'0), an Add counterpart of\texttt{DICTSETGETREF}.\texttt{F43C}—\texttt{DICTIADDGET}(xiDn-D'-1orDy0).\texttt{F43D}—\texttt{DICTIADDGETREF}(ciDn-D'-1orDc'0).\texttt{F43E}—\texttt{DICTUADDGET}(xiDn-D'-1orDy0).\texttt{F43F}—\texttt{DICTUADDGETREF}(ciDn-D'-1orDc'0).
A.10.5. Builder-accepting variants of \texttt{Set} dictionary operations
The following primitives accept the new value as a Builder b instead of a Slice x, which often is more convenient if the value needs to be serialized from several components computed in the stack. (This is reflected by appending a \texttt{B} to the mnemonics of the corresponding \texttt{Set} primitives that work with Slices.) The net effect is roughly equivalent to converting b into a Slice by \texttt{ENDC}; \texttt{CTOS} and executing the corresponding primitive.
\texttt{F441}—\texttt{DICTSETB}(bkDn-D').\texttt{F442}—\texttt{DICTISETB}(biDn-D').\texttt{F443}—\texttt{DICTUSETB}(biDn-D').\texttt{F445}—\texttt{DICTSETGETB}(bkDn-D'y-1orD'0).\texttt{F446}—\texttt{DICTISETGETB}(biDn-D'y-1orD'0).\texttt{F447}—\texttt{DICTUSETGETB}(biDn-D'y-1orD'0).\texttt{F449}—\texttt{DICTREPLACEB}(bkDn-D'-1orD0).\texttt{F44A}—\texttt{DICTIREPLACEB}(biDn-D'-1orD0).\texttt{F44B}—\texttt{DICTUREPLACEB}(biDn-D'-1orD0).\texttt{F44D}—\texttt{DICTREPLACEGETB}(bkDn-D'y-1orD0).\texttt{F44E}—\texttt{DICTIREPLACEGETB}(biDn-D'y-1orD0).\texttt{F44F}—\texttt{DICTUREPLACEGETB}(biDn-D'y-1orD0).\texttt{F451}—\texttt{DICTADDB}(bkDn-D'-1orD0).\texttt{F452}—\texttt{DICTIADDB}(biDn-D'-1orD0).\texttt{F453}—\texttt{DICTUADDB}(biDn-D'-1orD0).\texttt{F455}—\texttt{DICTADDGETB}(bkDn-D'-1orDy0).\texttt{F456}—\texttt{DICTIADDGETB}(biDn-D'-1orDy0).\texttt{F457}—\texttt{DICTUADDGETB}(biDn-D'-1orDy0).
A.10.6. \texttt{Delete} dictionary operations
\texttt{F459}—\texttt{DICTDEL}(kDn-D'-1orD0), deletesn-bit key, represented by a Slicek, from dictionaryD. If the key is present, returns the modified dictionaryD'and the success flag-1. Otherwise, returns the original dictionaryDand0.\texttt{F45A}—\texttt{DICTIDEL}(iDn-D'?), a version of\texttt{DICTDEL}with the key represented by a signedn-bit Integeri. Ifidoes not fit intonbits, simply returnsD0("key not found, dictionary unmodified").\texttt{F45B}—\texttt{DICTUDEL}(iDn-D'?), similar to\texttt{DICTIDEL}, but withian unsignedn-bit integer.\texttt{F462}—\texttt{DICTDELGET}(kDn-D'x-1orD0), deletesn-bit key, represented by a Slicek, from dictionaryD. If the key is present, returns the modified dictionaryD', the original valuexassociated with the keyk(represented by a Slice), and the success flag-1. Otherwise, returns the original dictionaryDand0.\texttt{F463}—\texttt{DICTDELGETREF}(kDn-D'c-1orD0), similar to\texttt{DICTDELGET}, but with\texttt{LDREF};\texttt{ENDS}applied toxon success, so that the value returnedcis a Cell.\texttt{F464}—\texttt{DICTIDELGET}(iDn-D'x-1orD0), a variant of primitive\texttt{DICTDELGET}with signedn-bit integerias a key.\texttt{F465}—\texttt{DICTIDELGETREF}(iDn-D'c-1orD0), a variant of primitive\texttt{DICTIDELGET}returning a Cell instead of a Slice.\texttt{F466}—\texttt{DICTUDELGET}(iDn-D'x-1orD0), a variant of primitive\texttt{DICTDELGET}with unsignedn-bit integerias a key.\texttt{F467}—\texttt{DICTUDELGETREF}(iDn-D'c-1orD0), a variant of primitive\texttt{DICTUDELGET}returning a Cell instead of a Slice.
A.10.7. "Maybe reference" dictionary operations
The following operations assume that a dictionary is used to store values c^? of type \mathit{Cell}^? ("Maybe Cell"), which can be used in particular to store dictionaries as values in other dictionaries. The representation is as follows: if c^? is a Cell, it is stored as a value with no data bits and exactly one reference to this Cell. If c^? is Null, then the corresponding key must be absent from the dictionary altogether.
\texttt{F469}—\texttt{DICTGETOPTREF}(kDn-c^?), a variant of\texttt{DICTGETREF}that returns Null instead of the valuec^?if the keykis absent from dictionaryD.\texttt{F46A}—\texttt{DICTIGETOPTREF}(iDn-c^?), similar to\texttt{DICTGETOPTREF}, but with the key given by signedn-bit Integeri. If the keyiis out of range, also returns Null.\texttt{F46B}—\texttt{DICTUGETOPTREF}(iDn-c^?), similar to\texttt{DICTGETOPTREF}, but with the key given by unsignedn-bit Integeri.\texttt{F46D}—\texttt{DICTSETGETOPTREF}(c^?kDn-D'\tilde{c}^?), a variant of both\texttt{DICTGETOPTREF}and\texttt{DICTSETGETREF}that sets the value corresponding to keykin dictionaryDtoc^?(ifc^?is Null, then the key is deleted instead), and returns the old value\tilde{c}^?(if the keykwas absent before, returns Null instead).\texttt{F46E}—\texttt{DICTISETGETOPTREF}(c^?iDn-D'\tilde{c}^?), similar to primitive\texttt{DICTSETGETOPTREF}, but using signedn-bit Integerias a key. Ifidoes not fit intonbits, throws a range checking exception.\texttt{F46F}—\texttt{DICTUSETGETOPTREF}(c^?iDn-D'\tilde{c}^?), similar to primitive\texttt{DICTSETGETOPTREF}, but using unsignedn-bit Integerias a key.
A.10.8. Prefix code dictionary operations
These are some basic operations for constructing prefix code dictionaries. The primary application for prefix code dictionaries is deserializing TL-B serialized data structures, or, more generally, parsing prefix codes. Therefore, most prefix code dictionaries will be constant and created at compile time, not by the following primitives.
Some \texttt{Get} operations for prefix code dictionaries may be found in A.10.11. Other prefix code dictionary operations include:
\texttt{F470}—\texttt{PFXDICTSET}(xkDn-D'-1orD0).\texttt{F471}—\texttt{PFXDICTREPLACE}(xkDn-D'-1orD0).\texttt{F472}—\texttt{PFXDICTADD}(xkDn-D'-1orD0).\texttt{F473}—\texttt{PFXDICTDEL}(kDn-D'-1orD0).
These primitives are completely similar to their non-prefix code counterparts \texttt{DICTSET} etc, with the obvious difference that even a \texttt{Set} may fail in a prefix code dictionary, so a success flag must be returned by \texttt{PFXDICTSET} as well.
A.10.9. Variants of \texttt{GetNext} and \texttt{GetPrev} operations
\texttt{F474}—\texttt{DICTGETNEXT}(kDn-x'k'-1or0), computes the minimal keyk'in dictionaryDthat is lexicographically greater thank, and returnsk'(represented by a Slice) along with associated valuex'(also represented by a Slice).\texttt{F475}—\texttt{DICTGETNEXTEQ}(kDn-x'k'-1or0), similar to\texttt{DICTGETNEXT}, but computes the minimal keyk'that is lexicographically greater than or equal tok.\texttt{F476}—\texttt{DICTGETPREV}(kDn-x'k'-1or0), similar to\texttt{DICTGETNEXT}, but computes the maximal keyk'lexicographically smaller thank.\texttt{F477}—\texttt{DICTGETPREVEQ}(kDn-x'k'-1or0), similar to\texttt{DICTGETPREV}, but computes the maximal keyk'lexicographically smaller than or equal tok.\texttt{F478}—\texttt{DICTIGETNEXT}(iDn-x'i'-1or0), similar to\texttt{DICTGETNEXT}, but interprets all keys in dictionaryDas big-endian signedn-bit integers, and computes the minimal keyi'that is larger than Integeri(which does not necessarily fit intonbits).\texttt{F479}—\texttt{DICTIGETNEXTEQ}(iDn-x'i'-1or0).\texttt{F47A}—\texttt{DICTIGETPREV}(iDn-x'i'-1or0).\texttt{F47B}—\texttt{DICTIGETPREVEQ}(iDn-x'i'-1or0).\texttt{F47C}—\texttt{DICTUGETNEXT}(iDn-x'i'-1or0), similar to\texttt{DICTGETNEXT}, but interprets all keys in dictionaryDas big-endian unsignedn-bit integers, and computes the minimal keyi'that is larger than Integeri(which does not necessarily fit intonbits, and is not necessarily non-negative).\texttt{F47D}—\texttt{DICTUGETNEXTEQ}(iDn-x'i'-1or0).\texttt{F47E}—\texttt{DICTUGETPREV}(iDn-x'i'-1or0).\texttt{F47F}—\texttt{DICTUGETPREVEQ}(iDn-x'i'-1or0).
A.10.10. \texttt{GetMin}, \texttt{GetMax}, \texttt{RemoveMin}, \texttt{RemoveMax} operations
\texttt{F482}—\texttt{DICTMIN}(Dn-xk-1or0), computes the minimal keyk(represented by a Slice withndata bits) in dictionaryD, and returnskalong with the associated valuex.\texttt{F483}—\texttt{DICTMINREF}(Dn-ck-1or0), similar to\texttt{DICTMIN}, but returns the only reference in the value as a Cellc.\texttt{F484}—\texttt{DICTIMIN}(Dn-xi-1or0), somewhat similar to\texttt{DICTMIN}, but computes the minimal keyiunder the assumption that all keys are big-endian signedn-bit integers. Notice that the key and value returned may differ from those computed by\texttt{DICTMIN}and\texttt{DICTUMIN}.\texttt{F485}—\texttt{DICTIMINREF}(Dn-ci-1or0).\texttt{F486}—\texttt{DICTUMIN}(Dn-xi-1or0), similar to\texttt{DICTMIN}, but returns the key as an unsignedn-bit Integeri.\texttt{F487}—\texttt{DICTUMINREF}(Dn-ci-1or0).\texttt{F48A}—\texttt{DICTMAX}(Dn-xk-1or0), computes the maximal keyk(represented by a Slice withndata bits) in dictionaryD, and returnskalong with the associated valuex.\texttt{F48B}—\texttt{DICTMAXREF}(Dn-ck-1or0).\texttt{F48C}—\texttt{DICTIMAX}(Dn-xi-1or0).\texttt{F48D}—\texttt{DICTIMAXREF}(Dn-ci-1or0).\texttt{F48E}—\texttt{DICTUMAX}(Dn-xi-1or0).\texttt{F48F}—\texttt{DICTUMAXREF}(Dn-ci-1or0).\texttt{F492}—\texttt{DICTREMMIN}(Dn-D'xk-1orD0), computes the minimal keyk(represented by a Slice withndata bits) in dictionaryD, removeskfrom the dictionary, and returnskalong with the associated valuexand the modified dictionaryD'.\texttt{F493}—\texttt{DICTREMMINREF}(Dn-D'ck-1orD0), similar to\texttt{DICTREMMIN}, but returns the only reference in the value as a Cellc.\texttt{F494}—\texttt{DICTIREMMIN}(Dn-D'xi-1orD0), somewhat similar to\texttt{DICTREMMIN}, but computes the minimal keyiunder the assumption that all keys are big-endian signedn-bit integers. Notice that the key and value returned may differ from those computed by\texttt{DICTREMMIN}and\texttt{DICTUREMMIN}.\texttt{F495}—\texttt{DICTIREMMINREF}(Dn-D'ci-1orD0).\texttt{F496}—\texttt{DICTUREMMIN}(Dn-D'xi-1orD0), similar to\texttt{DICTREMMIN}, but returns the key as an unsignedn-bit Integeri.\texttt{F497}—\texttt{DICTUREMMINREF}(Dn-D'ci-1orD0).\texttt{F49A}—\texttt{DICTREMMAX}(Dn-D'xk-1orD0), computes the maximal keyk(represented by a Slice withndata bits) in dictionaryD, removeskfrom the dictionary, and returnskalong with the associated valuexand the modified dictionaryD'.\texttt{F49B}—\texttt{DICTREMMAXREF}(Dn-D'ck-1orD0).\texttt{F49C}—\texttt{DICTIREMMAX}(Dn-D'xi-1orD0).\texttt{F49D}—\texttt{DICTIREMMAXREF}(Dn-D'ci-1orD0).\texttt{F49E}—\texttt{DICTUREMMAX}(Dn-D'xi-1orD0).\texttt{F49F}—\texttt{DICTUREMMAXREF}(Dn-D'ci-1orD0).
A.10.11. Special \texttt{Get} dictionary and prefix code dictionary operations, and constant dictionaries
\texttt{F4A0}—\texttt{DICTIGETJMP}(iDn- ), similar to\texttt{DICTIGET}(A.10.12), but withx\texttt{BLESS}ed into a continuation with a subsequent\texttt{JMPX}to it on success. On failure, does nothing. This is useful for implementing\texttt{switch}/\texttt{case}constructions.\texttt{F4A1}—\texttt{DICTUGETJMP}(iDn- ), similar to\texttt{DICTIGETJMP}, but performs\texttt{DICTUGET}instead of\texttt{DICTIGET}.\texttt{F4A2}—\texttt{DICTIGETEXEC}(iDn- ), similar to\texttt{DICTIGETJMP}, but with\texttt{EXECUTE}instead of\texttt{JMPX}.\texttt{F4A3}—\texttt{DICTUGETEXEC}(iDn- ), similar to\texttt{DICTUGETJMP}, but with\texttt{EXECUTE}instead of\texttt{JMPX}.\texttt{F4A6\_}n—\texttt{DICTPUSHCONST}n( -Dn), pushes a non-empty constant dictionaryD(as a\mathit{Cell}^?) along with its key length0\leq n\leq 1023, stored as a part of the instruction. The dictionary itself is created from the first of remaining references of the current continuation. In this way, the complete\texttt{DICTPUSHCONST}instruction can be obtained by first serializing\texttt{xF4A8\_}, then the non-empty dictionary itself (one\texttt{1}bit and a cell reference), and then the unsigned 10-bit integern(as if by a\texttt{STU 10}instruction). An empty dictionary can be pushed by a\texttt{NEWDICT}primitive (A.10.1) instead.\texttt{F4A8}—\texttt{PFXDICTGETQ}(sDn-s'xs''-1ors0), looks up the unique prefix of Slicespresent in the prefix code dictionary represented by\mathit{Cell}^?Dand0\leq n\leq 1023. If found, the prefix ofsis returned ass', and the corresponding value (also a Slice) asx. The remainder ofsis returned as a Slices''. If no prefix ofsis a key in prefix code dictionaryD, returns the unchangedsand a zero flag to indicate failure.\texttt{F4A9}—\texttt{PFXDICTGET}(sDn-s'xs''), similar to\texttt{PFXDICTGETQ}, but throws a cell deserialization failure exception on failure.\texttt{F4AA}—\texttt{PFXDICTGETJMP}(sDn-s's''ors), similar to\texttt{PFXDICTGETQ}, but on success\texttt{BLESS}es the valuexinto a Continuation and transfers control to it as if by a\texttt{JMPX}. On failure, returnssunchanged and continues execution.\texttt{F4AB}—\texttt{PFXDICTGETEXEC}(sDn-s's''), similar to\texttt{PFXDICTGETJMP}, but\texttt{EXEC}utes the continuation found instead of jumping to it. On failure, throws a cell deserialization exception.\texttt{F4AE\_}n—\texttt{PFXDICTCONSTGETJMP}nor\texttt{PFXDICTSWITCH}n(s-s's''ors), combines\texttt{DICTPUSHCONST}nfor0\leq n\leq 1023with\texttt{PFXDICTGETJMP}.\texttt{F4BC}—\texttt{DICTIGETJMPZ}(iDn-ior nothing), a variant of\texttt{DICTIGETJMP}that returns indexion failure.\texttt{F4BD}—\texttt{DICTUGETJMPZ}(iDn-ior nothing), a variant of\texttt{DICTUGETJMP}that returns indexion failure.\texttt{F4BE}—\texttt{DICTIGETEXECZ}(iDn-ior nothing), a variant of\texttt{DICTIGETEXEC}that returns indexion failure.\texttt{F4BF}—\texttt{DICTUGETEXECZ}(iDn-ior nothing), a variant of\texttt{DICTUGETEXEC}that returns indexion failure.
A.10.12. \texttt{SubDict} dictionary operations
\texttt{F4B1}—\texttt{SUBDICTGET}(klDn-D'), constructs a subdictionary consisting of all keys beginning with prefixk(represented by a Slice, the first0\leq l\leq n\leq 1023data bits of which are used as a key) of lengthlin dictionaryDof type\mathit{HashmapE}(n,X)withn-bit keys. On success, returns the new subdictionary of the same type\mathit{HashmapE}(n,X)as a SliceD'.\texttt{F4B2}—\texttt{SUBDICTIGET}(xlDn-D'), variant of\texttt{SUBDICTGET}with the prefix represented by a signed big-endianl-bit Integerx, where necessarilyl\leq257.\texttt{F4B3}—\texttt{SUBDICTUGET}(xlDn-D'), variant of\texttt{SUBDICTGET}with the prefix represented by an unsigned big-endianl-bit Integerx, where necessarilyl\leq256.\texttt{F4B5}—\texttt{SUBDICTRPGET}(klDn-D'), similar to\texttt{SUBDICTGET}, but removes the common prefixkfrom all keys of the new dictionaryD', which becomes of type\mathit{HashmapE}(n-l,X).\texttt{F4B6}—\texttt{SUBDICTIRPGET}(xlDn-D'), variant of\texttt{SUBDICTRPGET}with the prefix represented by a signed big-endianl-bit Integerx, where necessarilyl\leq257.\texttt{F4B7}—\texttt{SUBDICTURPGET}(xlDn-D'), variant of\texttt{SUBDICTRPGET}with the prefix represented by an unsigned big-endianl-bit Integerx, where necessarilyl\leq256.\texttt{F4BC}–\texttt{F4BF}— used by\texttt{DICT\ldots Z}primitives in A.10.11.
A.11 Application-specific primitives
Opcode range \texttt{F8}...\texttt{FB} is reserved for the application-specific primitives. When TVM is used to execute TON Blockchain smart contracts, these application-specific primitives are in fact TON Blockchain-specific.
A.11.1. External actions and access to blockchain configuration data
Some of the primitives listed below pretend to produce some externally visible actions, such as sending a message to another smart contract. In fact, the execution of a smart contract in TVM never has any effect apart from a modification of the TVM state. All external actions are collected into a linked list stored in special register \texttt{c5} ("output actions"). Additionally, some primitives use the data kept in the first component of the Tuple stored in \texttt{c7}. Smart contracts are free to modify any other data kept in the cell \texttt{c7}, provided the first reference remains intact (otherwise some application-specific primitives would be likely to throw exceptions when invoked).
Most of the primitives listed below use 16-bit opcodes.
A.11.2. Gas-related primitives
Of the following primitives, only the first two are "pure" in the sense that they do not use \texttt{c5} or \texttt{c7}.
\texttt{F800}—\texttt{ACCEPT}, sets current gas limitg_lto its maximal allowed valueg_m, and resets the gas creditg_cto zero (1.4), decreasing the value ofg_rbyg_cin the process. In other words, the current smart contract agrees to buy some gas to finish the current transaction. This action is required to process external messages, which bring no value (hence no gas) with themselves.\texttt{F801}—\texttt{SETGASLIMIT}(g- ), sets current gas limitg_lto the minimum ofgandg_m, and resets the gas creditg_cto zero. If the gas consumed so far (including the present instruction) exceeds the resulting value ofg_l, an (unhandled) out of gas exception is thrown before setting new gas limits. Notice that\texttt{SETGASLIMIT}with an argumentg\geq 2^{63}-1is equivalent to\texttt{ACCEPT}.\texttt{F802}—\texttt{BUYGAS}(x- ), computes the amount of gas that can be bought forxnanograms, and setsg_laccordingly in the same way as\texttt{SETGASLIMIT}.\texttt{F804}—\texttt{GRAMTOGAS}(x-g), computes the amount of gas that can be bought forxnanograms. Ifxis negative, returns 0. Ifgexceeds2^{63}-1, it is replaced with this value.\texttt{F805}—\texttt{GASTOGRAM}(g-x), computes the price ofggas in nanograms.\texttt{F806}–\texttt{F80E}— Reserved for gas-related primitives.\texttt{F80F}—\texttt{COMMIT}( - ), commits the current state of registers\texttt{c4}("persistent data") and\texttt{c5}("actions") so that the current execution is considered "successful" with the saved values even if an exception is thrown later.
A.11.3. Pseudo-random number generator primitives
The pseudo-random number generator uses the random seed (parameter #6), an unsigned 256-bit Integer, and (sometimes) other data kept in \texttt{c7}. The initial value of the random seed before a smart contract is executed in TON Blockchain is a hash of the smart contract address and the global block random seed. If there are several runs of the same smart contract inside a block, then all of these runs will have the same random seed. This can be fixed, for example, by running \texttt{LTIME}; \texttt{ADDRAND} before using the pseudo-random number generator for the first time.
\texttt{F810}—\texttt{RANDU256}( -x), generates a new pseudo-random unsigned 256-bit Integerx. The algorithm is as follows: ifris the old value of the random seed, considered as a 32-byte array (by constructing the big-endian representation of an unsigned 256-bit integer), then its\text{Sha512}(r)is computed; the first 32 bytes of this hash are stored as the new valuer'of the random seed, and the remaining 32 bytes are returned as the next random valuex.\texttt{F811}—\texttt{RAND}(y-z), generates a new pseudo-random integerzin the range0\ldots y-1(ory\ldots-1, ify<0). More precisely, an unsigned random valuexis generated as in\texttt{RAND256U}; thenz:=\lfloor xy/2^{256}\rflooris computed. Equivalent to\texttt{RANDU256};\texttt{MULRSHIFT 256}.\texttt{F814}—\texttt{SETRAND}(x- ), sets the random seed to unsigned 256-bit Integerx.\texttt{F815}—\texttt{ADDRAND}(x- ), mixes unsigned 256-bit Integerxinto the random seedrby setting the random seed to SHA-256 of the concatenation of two 32-byte strings: the first with the big-endian representation of the old seedr, and the second with the big-endian representation ofx.\texttt{F810}–\texttt{F81F}— Reserved for pseudo-random number generator primitives.
A.11.4. Configuration primitives
The following primitives read configuration data provided in the Tuple stored in the first component of the Tuple at \texttt{c7}. Whenever TVM is invoked for executing TON Blockchain smart contracts, this Tuple is initialized by a SmartContractInfo structure; configuration primitives assume that it has remained intact.
\texttt{F82}i—\texttt{GETPARAM}i( -x), returns thei-th parameter from the Tuple provided at\texttt{c7}for0\leq i<16. Equivalent to\texttt{PUSH c7};\texttt{FIRST};\texttt{INDEX}i. If one of these internal operations fails, throws an appropriate type checking or range checking exception.\texttt{F823}—\texttt{NOW}( -x), returns the current Unix time as an Integer. If it is impossible to recover the requested value starting from\texttt{c7}, throws a type checking or range checking exception as appropriate. Equivalent to\texttt{GETPARAM 3}.\texttt{F824}—\texttt{BLOCKLT}( -x), returns the starting logical time of the current block. Equivalent to\texttt{GETPARAM 4}.\texttt{F825}—\texttt{LTIME}( -x), returns the logical time of the current transaction. Equivalent to\texttt{GETPARAM 5}.\texttt{F826}—\texttt{RANDSEED}( -x), returns the current random seed as an unsigned 256-bit Integer. Equivalent to\texttt{GETPARAM 6}.\texttt{F827}—\texttt{BALANCE}( -t), returns the remaining balance of the smart contract as a Tuple consisting of an Integer (the remaining Gram balance in nanograms) and a Maybe Cell (a dictionary with 32-bit keys representing the balance of "extra currencies"). Equivalent to\texttt{GETPARAM 7}. Note that\texttt{RAW}primitives such as\texttt{SENDRAWMSG}do not update this field.\texttt{F828}—\texttt{MYADDR}( -s), returns the internal address of the current smart contract as a Slice with a\texttt{MsgAddressInt}. If necessary, it can be parsed further using primitives such as\texttt{PARSESTDADDR}or\texttt{REWRITESTDADDR}. Equivalent to\texttt{GETPARAM 8}.\texttt{F829}—\texttt{CONFIGROOT}( -D), returns the Maybe CellDwith the current global configuration dictionary. Equivalent to\texttt{GETPARAM 9}.\texttt{F830}—\texttt{CONFIGDICT}( -D32), returns the global configuration dictionary along with its key length (32). Equivalent to\texttt{CONFIGROOT};\texttt{PUSHINT 32}.\texttt{F832}—\texttt{CONFIGPARAM}(i-c-1or0), returns the value of the global configuration parameter with integer indexias a Cellc, and a flag to indicate success. Equivalent to\texttt{CONFIGDICT};\texttt{DICTIGETREF}.\texttt{F833}—\texttt{CONFIGOPTPARAM}(i-c^?), returns the value of the global configuration parameter with integer indexias a Maybe Cellc^?. Equivalent to\texttt{CONFIGDICT};\texttt{DICTIGETOPTREF}.\texttt{F820}-\texttt{F83F}— Reserved for configuration primitives.
A.11.5. Global variable primitives
The "global variables" may be helpful in implementing some high-level smart-contract languages. They are in fact stored as components of the Tuple at \texttt{c7}: the k-th global variable simply is the k-th component of this Tuple, for 1\leq k\leq 254. By convention, the 0-th component is used for the configuration parameters, so it is not available as a global variable.
\texttt{F840}—\texttt{GETGLOBVAR}(k-x), returns thek-th global variable for0\leq k<255. Equivalent to\texttt{PUSH c7};\texttt{SWAP};\texttt{INDEXVARQ}(A.3.2).\texttt{F85\_}k—\texttt{GETGLOB}k( -x), returns thek-th global variable for1\leq k\leq 31. Equivalent to\texttt{PUSH c7};\texttt{INDEXQ}k.\texttt{F860}—\texttt{SETGLOBVAR}(xk- ), assignsxto thek-th global variable for0\leq k<255. Equivalent to\texttt{PUSH c7};\texttt{ROTREV};\texttt{SETINDEXVARQ};\texttt{POP c7}.\texttt{F87\_}k—\texttt{SETGLOB}k(x- ), assignsxto thek-th global variable for1\leq k\leq 31. Equivalent to\texttt{PUSH c7};\texttt{SWAP};\texttt{SETINDEXQ}k;\texttt{POP c7}.
A.11.6. Hashing and cryptography primitives
\texttt{F900}—\texttt{HASHCU}(c-x), computes the representation hash of a Cellcand returns it as a 256-bit unsigned integerx. Useful for signing and checking signatures of arbitrary entities represented by a tree of cells.\texttt{F901}—\texttt{HASHSU}(s-x), computes the hash of a Slicesand returns it as a 256-bit unsigned integerx. The result is the same as if an ordinary cell containing only data and references fromshad been created and its hash computed by\texttt{HASHCU}.\texttt{F902}—\texttt{SHA256U}(s-x), computes SHA-256 of the data bits of Slices. If the bit length ofsis not divisible by eight, throws a cell underflow exception. The hash value is returned as a 256-bit unsigned integerx.\texttt{F910}—\texttt{CHKSIGNU}(hsk-?), checks the Ed25519-signaturesof a hashh(a 256-bit unsigned integer, usually computed as the hash of some data) using public keyk(also represented by a 256-bit unsigned integer). The signaturesmust be a Slice containing at least 512 data bits; only the first 512 bits are used. The result is-1if the signature is valid,0otherwise. Notice that\texttt{CHKSIGNU}is equivalent to\texttt{ROT};\texttt{NEWB};\texttt{STU 256};\texttt{ENDB};\texttt{NEWC};\texttt{ROTREV};\texttt{CHKSIGNS}, i.e., to\texttt{CHKSIGNS}with the first argumentdset to 256-bit Slice containingh. Therefore, ifhis computed as the hash of some data, these data are hashed twice, the second hashing occurring inside\texttt{CHKSIGNS}.\texttt{F911}—\texttt{CHKSIGNS}(dsk-?), checks whethersis a valid Ed25519-signature of the data portion of Slicedusing public keyk, similarly to\texttt{CHKSIGNU}. If the bit length of Slicedis not divisible by eight, throws a cell underflow exception. The verification of Ed25519 signatures is the standard one, with SHA-256 used to reducedto the 256-bit number that is actually signed.\texttt{F912}-\texttt{F93F}— Reserved for hashing and cryptography primitives.
A.11.7. Miscellaneous primitives
\texttt{F940}—\texttt{CDATASIZEQ}(cn-xyz-1or0), recursively computes the count of distinct cellsx, data bitsy, and cell referenceszin the dag rooted at Cellc, effectively returning the total storage used by this dag taking into account the identification of equal cells. The values ofx,y, andzare computed by a depth-first traversal of this dag, with a hash table of visited cell hashes used to prevent visits of already-visited cells. The total count of visited cellsxcannot exceed non-negative Integern; otherwise the computation is aborted before visiting the(n+1)-st cell and a zero is returned to indicate failure. Ifcis Null, returnsx=y=z=0.\texttt{F941}—\texttt{CDATASIZE}(cn-xyz), a non-quiet version of\texttt{CDATASIZEQ}that throws a cell overflow exception (8) on failure.\texttt{F942}—\texttt{SDATASIZEQ}(sn-xyz-1or0), similar to\texttt{CDATASIZEQ}, but accepting a Slicesinstead of a Cell. The returned value ofxdoes not take into account the cell that contains the slicesitself; however, the data bits and the cell references ofsare accounted for inyandz.\texttt{F943}—\texttt{SDATASIZE}(sn-xyz), a non-quiet version of\texttt{SDATASIZEQ}that throws a cell overflow exception (8) on failure.\texttt{F944}-\texttt{F97F}— Reserved for miscellaneous TON-specific primitives that do not fall into any other specific category.
A.11.8. Currency manipulation primitives
\texttt{FA00}—\texttt{LDGRAMS}or\texttt{LDVARUINT16}(s-xs'), loads (deserializes) a\texttt{Gram}or\texttt{VarUInteger 16}amount from CellSlices, and returns the amount as Integerxalong with the remainders'ofs. The expected serialization ofxconsists of a 4-bit unsigned big-endian integerl, followed by an8l-bit unsigned big-endian representation ofx. The net effect is approximately equivalent to\texttt{LDU 4};\texttt{SWAP};\texttt{LSHIFT 3};\texttt{LDUX}.\texttt{FA01}—\texttt{LDVARINT16}(s-xs'), similar to\texttt{LDVARUINT16}, but loads a signed Integerx. Approximately equivalent to\texttt{LDU 4};\texttt{SWAP};\texttt{LSHIFT 3};\texttt{LDIX}.\texttt{FA02}—\texttt{STGRAMS}or\texttt{STVARUINT16}(bx-b'), stores (serializes) an Integerxin the range0\ldots2^{120}-1into Builderb, and returns the resulting Builderb'. The serialization ofxconsists of a 4-bit unsigned big-endian integerl, which is the smallest integerl\geq0, such thatx<2^{8l}, followed by an8l-bit unsigned big-endian representation ofx. Ifxdoes not belong to the supported range, a range check exception is thrown.\texttt{FA03}—\texttt{STVARINT16}(bx-b'), similar to\texttt{STVARUINT16}, but serializes a signed Integerxin the range-2^{119}\ldots2^{119}-1.\texttt{FA04}—\texttt{LDVARUINT32}(s-xs'), loads (deserializes) a\texttt{VarUInteger 32}from CellSlices, and returns the deserialized value as an Integer0\leq x<2^{248}. The expected serialization ofxconsists of a 5-bit unsigned big-endian integerl, followed by an8l-bit unsigned big-endian representation ofx. The net effect is approximately equivalent to\texttt{LDU 5};\texttt{SWAP};\texttt{SHIFT 3};\texttt{LDUX}.\texttt{FA05}—\texttt{LDVARINT32}(s-xs'), deserializes a\texttt{VarInteger 32}from CellSlices, and returns the deserialized value as an Integer-2^{247}\leq x<2^{247}.\texttt{FA06}—\texttt{STVARUINT32}(bx-b'), serializes an Integer0\leq x<2^{248}as a\texttt{VarUInteger 32}.\texttt{FA07}—\texttt{STVARINT32}(bx-b'), serializes an Integer-2^{247}\leq x<2^{247}as a\texttt{VarInteger 32}.\texttt{FA08}-\texttt{FA1F}— Reserved for currency manipulation primitives.
A.11.9. Message and address manipulation primitives
The message and address manipulation primitives listed below serialize and deserialize values according to the following TL-B scheme:
addr_none$00 = MsgAddressExt;
addr_extern$01 len:(## 9) external_address:(bits len)
= MsgAddressExt;
anycast_info$_ depth:(#<= 30) { depth >= 1 }
rewrite_pfx:(bits depth) = Anycast;
addr_std$10 anycast:(Maybe Anycast)
workchain_id:int8 address:bits256 = MsgAddressInt;
addr_var$11 anycast:(Maybe Anycast) addr_len:(## 9)
workchain_id:int32 address:(bits addr_len) = MsgAddressInt;
_ _:MsgAddressInt = MsgAddress;
_ _:MsgAddressExt = MsgAddress;
int_msg_info$0 ihr_disabled:Bool bounce:Bool bounced:Bool
src:MsgAddress dest:MsgAddressInt
value:CurrencyCollection ihr_fee:Grams fwd_fee:Grams
created_lt:uint64 created_at:uint32 = CommonMsgInfoRelaxed;
ext_out_msg_info$11 src:MsgAddress dest:MsgAddressExt
created_lt:uint64 created_at:uint32 = CommonMsgInfoRelaxed;A deserialized \texttt{MsgAddress} is represented by a Tuple t as follows:
\texttt{addr\_none}is represented byt=(0), i.e., a Tuple containing exactly one Integer equal to zero.\texttt{addr\_extern}is represented byt=(1,s), where Slicescontains the field\texttt{external\_address}. In other words,tis a pair (a Tuple consisting of two entries), containing an Integer equal to one and Slices.\texttt{addr\_std}is represented byt=(2,u,x,s), whereuis either a Null (if\texttt{anycast}is absent) or a Slices'containing\texttt{rewrite\_pfx}(if\texttt{anycast}is present). Next, Integerxis the\texttt{workchain\_id}, and Slicescontains the\texttt{address}.\texttt{addr\_var}is represented byt=(3,u,x,s), whereu,x, andshave the same meaning as for\texttt{addr\_std}.
The following primitives, which use the above conventions, are defined:
\texttt{FA40}—\texttt{LDMSGADDR}(s-s's''), loads from CellSlicesthe only prefix that is a valid\texttt{MsgAddress}, and returns both this prefixs'and the remainders''ofsas CellSlices.\texttt{FA41}—\texttt{LDMSGADDRQ}(s-s's''-1ors0), a quiet version of\texttt{LDMSGADDR}: on success, pushes an extra-1; on failure, pushes the originalsand a zero.\texttt{FA42}—\texttt{PARSEMSGADDR}(s-t), decomposes CellSlicescontaining a valid\texttt{MsgAddress}into a Tupletwith separate fields of this\texttt{MsgAddress}. Ifsis not a valid\texttt{MsgAddress}, a cell deserialization exception is thrown.\texttt{FA43}—\texttt{PARSEMSGADDRQ}(s-t-1or0), a quiet version of\texttt{PARSEMSGADDR}: returns a zero on error instead of throwing an exception.\texttt{FA44}—\texttt{REWRITESTDADDR}(s-xy), parses CellSlicescontaining a valid\texttt{MsgAddressInt}(usually a\texttt{msg\_addr\_std}), applies rewriting from the\texttt{anycast}(if present) to the same-length prefix of the address, and returns both the workchainxand the 256-bit addressyas Integers. If the address is not 256-bit, or ifsis not a valid serialization of\texttt{MsgAddressInt}, throws a cell deserialization exception.\texttt{FA45}—\texttt{REWRITESTDADDRQ}(s-xy-1or0), a quiet version of primitive\texttt{REWRITESTDADDR}.\texttt{FA46}—\texttt{REWRITEVARADDR}(s-xs'), a variant of\texttt{REWRITESTDADDR}that returns the (rewritten) address as a Slice s, even if it is not exactly 256 bit long (represented by a\texttt{msg\_addr\_var}).\texttt{FA47}—\texttt{REWRITEVARADDRQ}(s-xs'-1or0), a quiet version of primitive\texttt{REWRITEVARADDR}.\texttt{FA48}-\texttt{FA5F}— Reserved for message and address manipulation primitives.
A.11.10. Outbound message and output action primitives
\texttt{FB00}—\texttt{SENDRAWMSG}(cx- ), sends a raw message contained in Cellc, which should contain a correctly serialized object\texttt{Message}X, with the only exception that the source address is allowed to have dummy value\texttt{addr\_none}(to be automatically replaced with the current smart-contract address), and\texttt{ihr\_fee},\texttt{fwd\_fee},\texttt{created\_lt}and\texttt{created\_at}fields can have arbitrary values (to be rewritten with correct values during the action phase of the current transaction). Integer parameterxcontains the flags. Currentlyx=0is used for ordinary messages;x=128is used for messages that are to carry all the remaining balance of the current smart contract (instead of the value originally indicated in the message);x=64is used for messages that carry all the remaining value of the inbound message in addition to the value initially indicated in the new message (if bit 0 is not set, the gas fees are deducted from this amount);x'=x+1means that the sender wants to pay transfer fees separately;x'=x+2means that any errors arising while processing this message during the action phase should be ignored. Finally,x'=x+32means that the current account must be destroyed if its resulting balance is zero. This flag is usually employed together with+128.\texttt{FB02}—\texttt{RAWRESERVE}(xy- ), creates an output action which would reserve exactlyxnanograms (ify=0), at mostxnanograms (ify=2), or all butxnanograms (ify=1ory=3), from the remaining balance of the account. It is roughly equivalent to creating an outbound message carryingxnanograms (orb-xnanograms, wherebis the remaining balance) to oneself, so that the subsequent output actions would not be able to spend more money than the remainder. Bit+2inymeans that the external action does not fail if the specified amount cannot be reserved; instead, all remaining balance is reserved. Bit+8inymeansx\leftarrow -xbefore performing any further actions. Bit+4inymeans thatxis increased by the original balance of the current account (before the compute phase), including all extra currencies, before performing any other checks and actions. Currentlyxmust be a non-negative integer, andymust be in the range0\ldots 15.\texttt{FB03}—\texttt{RAWRESERVEX}(xDy- ), similar to\texttt{RAWRESERVE}, but also accepts a dictionaryD(represented by a Cell or Null) with extra currencies. In this way currencies other than Grams can be reserved.\texttt{FB04}—\texttt{SETCODE}(c- ), creates an output action that would change this smart contract code to that given by Cellc. Notice that this change will take effect only after the successful termination of the current run of the smart contract.\texttt{FB06}—\texttt{SETLIBCODE}(cx- ), creates an output action that would modify the collection of this smart contract libraries by adding or removing library with code given in Cellc. Ifx=0, the library is actually removed if it was previously present in the collection (if not, this action does nothing). Ifx=1, the library is added as a private library, and ifx=2, the library is added as a public library (and becomes available to all smart contracts if the current smart contract resides in the masterchain); if the library was present in the collection before, its public/private status is changed according tox. Values ofxother than0\ldots 2are invalid.\texttt{FB07}—\texttt{CHANGELIB}(hx- ), creates an output action similarly to\texttt{SETLIBCODE}, but instead of the library code accepts its hash as an unsigned 256-bit integerh. Ifx\neq0and the library with hashhis absent from the library collection of this smart contract, this output action will fail.\texttt{FB08}-\texttt{FB3F}— Reserved for output action primitives.
A.12 Debug primitives
Opcodes beginning with \texttt{FE} are reserved for the debug primitives. These primitives have known fixed operation length, and behave as (multibyte) NOP operations. In particular, they never change the stack contents, and never throw exceptions, unless there are not enough bits to completely decode the opcode. However, when invoked in a TVM instance with debug mode enabled, these primitives can produce specific output into the text debug log of the TVM instance, never affecting the TVM state (so that from the perspective of TVM the behavior of debug primitives in debug mode is exactly the same). For instance, a debug primitive might dump all or some of the values near the top of the stack, display the current state of TVM and so on.
A.12.1. Debug primitives as multibyte NOPs
\texttt{FE}nn—\texttt{DEBUG}nn, for0\leq nn<240, is a two-byte NOP.\texttt{FEF}nssss—\texttt{DEBUGSTR}ssss, for0\leq n<16, is an(n+3)-byte NOP, with the(n+1)-byte "contents string"ssssskipped as well.
A.12.2. Debug primitives as operations without side-effect
Next we describe the debug primitives that might (and actually are) implemented in a version of TVM. Notice that another TVM implementation is free to use these codes for other debug purposes, or treat them as multibyte NOPs. Whenever these primitives need some arguments from the stack, they inspect these arguments, but leave them intact in the stack. If there are insufficient values in the stack, or they have incorrect types, debug primitives may output error messages into the debug log, or behave as NOPs, but they cannot throw exceptions.
\texttt{FE00}—\texttt{DUMPSTK}, dumps the stack (at most the top 255 values) and shows the total stack depth.\texttt{FE0}n—\texttt{DUMPSTKTOP}n,1\leq n<15, dumps the topnvalues from the stack, starting from the deepest of them. If there ared<nvalues available, dumps onlydvalues.\texttt{FE10}—\texttt{HEXDUMP}, dumps\texttt{s0}in hexadecimal form, be it a Slice or an Integer.\texttt{FE11}—\texttt{HEXPRINT}, similar to\texttt{HEXDUMP}, except the hexadecimal representation of\texttt{s0}is not immediately output, but rather concatenated to an output text buffer.\texttt{FE12}—\texttt{BINDUMP}, dumps\texttt{s0}in binary form, similarly to\texttt{HEXDUMP}.\texttt{FE13}—\texttt{BINPRINT}, outputs the binary representation of\texttt{s0}to a text buffer.\texttt{FE14}—\texttt{STRDUMP}, dumps the Slice at\texttt{s0}as an UTF-8 string.\texttt{FE15}—\texttt{STRPRINT}, similar to\texttt{STRDUMP}, but outputs the string into a text buffer (without carriage return).\texttt{FE1E}—\texttt{DEBUGOFF}, disables all debug output until it is re-enabled by a\texttt{DEBUGON}. More precisely, this primitive increases an internal counter, which disables all debug operations (except\texttt{DEBUGOFF}and\texttt{DEBUGON}) when strictly positive.\texttt{FE1F}—\texttt{DEBUGON}, enables debug output (in a debug version of TVM).\texttt{FE2}n—\texttt{DUMP s}(n),0\leq n<15, dumps\texttt{s}(n).\texttt{FE3}n—\texttt{PRINT s}(n),0\leq n<15, concatenates the text representation of\texttt{s}(n)(without any leading or trailing spaces or carriage returns) to a text buffer which will be output before the output of any other debug operation.\texttt{FEC0}-\texttt{FEEF}— Use these opcodes for custom/experimental debug operations.\texttt{FEF}nssss—\texttt{DUMPTOSFMT}ssss, dumps\texttt{s0}formatted according to the(n+1)-byte stringssss. This string might contain (a prefix of) the name of a TL-B type supported by the debugger. If the string begins with a zero byte, simply outputs it (without the first byte) into the debug log. If the string begins with a byte equal to one, concatenates it to a buffer, which will be output before the output of any other debug operation (effectively outputs a string without a carriage return).\texttt{FEF}n\texttt{00}ssss—\texttt{LOGSTR}ssss, stringssssisnbytes long.\texttt{FEF000}—\texttt{LOGFLUSH}, flushes all pending debug output from the buffer into the debug log.\texttt{FEF}n\texttt{01}ssss—\texttt{PRINTSTR}ssss, stringssssisnbytes long.
A.13. Codepage primitives
The following primitives, which begin with byte \texttt{FF}, typically are used at the very beginning of a smart contract's code or a library subroutine to select another TVM codepage. Notice that we expect all codepages to contain these primitives with the same codes, otherwise switching back to another codepage might be impossible.
\texttt{FF}nn—\texttt{SETCP}nn, selects TVM codepage0\leq nn<240. If the codepage is not supported, throws an invalid opcode exception.\texttt{FF00}—\texttt{SETCP0}, selects TVM (test) codepage zero as described in this document.\texttt{FFF}z—\texttt{SETCP}z-16, selects TVM codepagez-16for1\leq z\leq 15. Negative codepages-13\ldots-1are reserved for restricted versions of TVM needed to validate runs of TVM in other codepages. Negative codepage-14is reserved for experimental codepages, not necessarily compatible between different TVM implementations, and should be disabled in the production versions of TVM.\texttt{FFF0}—\texttt{SETCPX}(c- ), selects codepagecwith-2^{15}\leq c<2^{15}passed in the top of the stack.
B Formal properties and specifications of TVM
This appendix discusses certain formal properties of TVM that are necessary for executing smart contracts in the TON Blockchain and validating such executions afterwards.
B.1 Serialization of the TVM state
Recall that a virtual machine used for executing smart contracts in a blockchain must be deterministic, otherwise the validation of each execution would require the inclusion of all intermediate steps of the execution into a block, or at least of the choices made when indeterministic operations have been performed.
Furthermore, the state of such a virtual machine must be (uniquely) serializable, so that even if the state itself is not usually included in a block, its hash is still well-defined and can be included into a block for verification purposes.
B.1.1. TVM stack values
TVM stack values can be serialized as follows:
vm_stk_tinyint#01 value:int64 = VmStackValue;
vm_stk_int#0201_ value:int257 = VmStackValue;
vm_stk_nan#02FF = VmStackValue;
vm_stk_cell#03 cell:^Cell = VmStackValue;
_ cell:^Cell st_bits:(## 10) end_bits:(## 10)
{ st_bits <= end_bits }
st_ref:(#<= 4) end_ref:(#<= 4)
{ st_ref <= end_ref } = VmCellSlice;
vm_stk_slice#04 _:VmCellSlice = VmStackValue;
vm_stk_builder#05 cell:^Cell = VmStackValue;
vm_stk_cont#06 cont:VmCont = VmStackValue;Of these, \texttt{vm\_stk\_tinyint} is never used by TVM in codepage zero; it is used only in restricted modes.
B.1.2. TVM stack
The TVM stack can be serialized as follows:
vm_stack#_ depth:(## 24) stack:(VmStackList depth) = VmStack;
vm_stk_cons#_ {n:#} head:VmStackValue tail:^(VmStackList n)
= VmStackList (n + 1);
vm_stk_nil#_ = VmStackList 0;B.1.3. TVM control registers
Control registers in TVM can be serialized as follows:
_ cregs:(HashmapE 4 VmStackValue) = VmSaveList;B.1.4. TVM gas limits
Gas limits in TVM can be serialized as follows:
gas_limits#_ remaining:int64 _:^[
max_limit:int64 cur_limit:int64 credit:int64 ]
= VmGasLimits;B.1.5. TVM library environment
The TVM library environment can be serialized as follows:
_ libraries:(HashmapE 256 ^Cell) = VmLibraries;B.1.6. TVM continuations
Continuations in TVM can be serialized as follows:
vmc_std$00 nargs:(## 22) stack:(Maybe VmStack) save:VmSaveList
cp:int16 code:VmCellSlice = VmCont;
vmc_envelope$01 nargs:(## 22) stack:(Maybe VmStack)
save:VmSaveList next:^VmCont = VmCont;
vmc_quit$1000 exit_code:int32 = VmCont;
vmc_quit_exc$1001 = VmCont;
vmc_until$1010 body:^VmCont after:^VmCont = VmCont;
vmc_again$1011 body:^VmCont = VmCont;
vmc_while_cond$1100 cond:^VmCont body:^VmCont
after:^VmCont = VmCont;
vmc_while_body$1101 cond:^VmCont body:^VmCont
after:^VmCont = VmCont;
vmc_pushint$1111 value:int32 next:^VmCont = VmCont;B.1.7. TVM state
The total state of TVM can be serialized as follows:
vms_init$00 cp:int16 step:int32 gas:GasLimits
stack:(Maybe VmStack) save:VmSaveList code:VmCellSlice
lib:VmLibraries = VmState;
vms_exception$01 cp:int16 step:int32 gas:GasLimits
exc_no:int32 exc_arg:VmStackValue
save:VmSaveList lib:VmLibraries = VmState;
vms_running$10 cp:int16 step:int32 gas:GasLimits stack:VmStack
save:VmSaveList code:VmCellSlice lib:VmLibraries
= VmState;
vms_finished$11 cp:int16 step:int32 gas:GasLimits
exit_code:int32 no_gas:Boolean stack:VmStack
save:VmSaveList lib:VmLibraries = VmState;When TVM is initialized, its state is described by a \texttt{vms\_init}, usually with \texttt{step} set to zero. The step function of TVM does nothing to a \texttt{vms\_finished} state, and transforms all other states into \texttt{vms\_running}, \texttt{vms\_exception}, or \texttt{vms\_finished}, with \texttt{step} increased by one.
B.2 Step function of TVM
A formal specification of TVM would be completed by the definition of a step function f:\textit{VmState}\to\textit{VmState}. This function deterministically transforms a valid VM state into a valid subsequent VM state, and is allowed to throw exceptions or return an invalid subsequent state if the original state was invalid.
B.2.1. A high-level definition of the step function
We might present a very long formal definition of the TVM step function in a high-level functional programming language. Such a specification, however, would mostly be useful as a reference for the (human) developers. We have chosen another approach, better adapted to automated formal verification by computers.
B.2.2. An operational definition of the step function
Notice that the step function f is a well-defined computable function from trees of cells into trees of cells. As such, it can be computed by a universal Turing machine. Then a program P computing f on such a machine would provide a machine-checkable specification of the step function f. This program P effectively is an emulator of TVM on this Turing machine.
B.2.3. A reference implementation of the TVM emulator inside TVM
We see that the step function of TVM may be defined by a reference implementation of a TVM emulator on another machine. An obvious idea is to use TVM itself, since it is well-adapted to working with trees of cells. However, an emulator of TVM inside itself is not very useful if we have doubts about a particular implementation of TVM and want to check it. For instance, if such an emulator interpreted a \texttt{DICTISET} instruction simply by invoking this instruction itself, then a bug in the underlying implementation of TVM would remain unnoticed.
B.2.4. Reference implementation inside a minimal version of TVM
We see that using TVM itself as a host machine for a reference implementation of TVM emulator would yield little insight. A better idea is to define a stripped-down version of TVM, which supports only the bare minimum of primitives and 64-bit integer arithmetic, and provide a reference implementation P of the TVM step function f for this stripped-down version of TVM.
In that case, one must carefully implement and check only a handful of primitives to obtain a stripped-down version of TVM, and compare the reference implementation P running on this stripped-down version to the full custom TVM implementation being verified. In particular, if there are any doubts about the validity of a specific run of a custom TVM implementation, they can now be easily resolved with the aid of the reference implementation.
B.2.5. Relevance for the TON Blockchain
The TON Blockchain adopts this approach to validate the runs of TVM (e.g., those used for processing inbound messages by smart contracts) when the validators' results do not match one another. In this case, a reference implementation of TVM, stored inside the masterchain as a configurable parameter (thus defining the current revision of TVM), is used to obtain the correct result.
B.2.6. Codepage -1
Codepage -1 of TVM is reserved for the stripped-down version of TVM. Its main purpose is to execute the reference implementation of the step function of the full TVM. This codepage contains only special versions of arithmetic primitives working with "tiny integers" (64-bit signed integers); therefore, TVM's 257-bit Integer arithmetic must be defined in terms of 64-bit arithmetic. Elliptic curve cryptography primitives are also implemented directly in codepage -1, without using any third-party libraries. Finally, a reference implementation of the \texttt{Sha} hash function is also provided in codepage -1.
B.2.7. Codepage -2
This bootstrapping process could be iterated even further, by providing an emulator of the stripped-down version of TVM written for an even simpler version of TVM that supports only boolean values (or integers 0 and 1)—a "codepage -2". All 64-bit arithmetic used in codepage -1 would then need to be defined by means of boolean operations, thus providing a reference implementation for the stripped-down version of TVM used in codepage -1. In this way, if some of the TON Blockchain validators did not agree on the results of their 64-bit arithmetic, they could regress to this reference implementation to find the correct answer.30
C Code density of stack and register machines
This appendix extends the general consideration of stack manipulation primitives, explaining the choice of such primitives for TVM, with a comparison of stack machines and register machines in terms of the quantity of primitives used and the code density. We do this by comparing the machine code that might be generated by an optimizing compiler for the same source files, for different (abstract) stack and register machines.
It turns out that the stack machines (at least those equipped with the basic stack manipulation primitives have far superior code density. Furthermore, the stack machines have excellent extendability with respect to additional arithmetic and arbitrary data processing operations, especially if one considers machine code automatically generated by optimizing compilers.
C.1 Sample leaf function
We start with a comparison of machine code generated by an (imaginary) optimizing compiler for several abstract register and stack machines, corresponding to the same high-level language source code that contains the definition of a leaf function (i.e., a function that does not call any other functions). For both the register machines and stack machines, we observe the notation and conventions.
C.1.1. Sample source file for a leaf function
The source file we consider contains one function f that takes six integer arguments a, b, c, d, e, and f and returns two integer values, x and y, which are the solutions of the system of two linear equations:
\begin{cases}
ax + by = e \\
cx + dy = f
\end{cases}
\tag{6}The source code of the function, in a programming language similar to C, might look as follows:
(int, int) f(int a, int b, int c, int d, int e, int f) {
int D = a*d - b*c;
int Dx = e*d - b*f;
int Dy = a*f - e*c;
return (Dx / D, Dy / D);
}We assume that the register machines we consider accept the six parameters a \ldots f in registers \texttt{r0} \ldots \texttt{r5}, and return the two values x and y in \texttt{r0} and \texttt{r1}. We also assume that the register machines have 16 registers, and that the stack machine can directly access \texttt{s0} to \texttt{s15} by its stack manipulation primitives; the stack machine will accept the parameters in \texttt{s5} to \texttt{s0}, and return the two values in \texttt{s0} and \texttt{s1}, somewhat similarly to the register machine. Finally, we assume at first that the register machine is allowed to destroy values in all registers (which is slightly unfair towards the stack machine); this assumption will be revisited later.
C.1.2. Three-address register machine
The machine code (or rather the corresponding assembly code) for a three-address register machine) might look as follows:
IMUL r6,r0,r3 // r6 := r0 * r3 = ad
IMUL r7,r1,r2 // r7 := bc
SUB r6,r6,r7 // r6 := ad-bc = D
IMUL r3,r4,r3 // r3 := ed
IMUL r1,r1,r5 // r1 := bf
SUB r3,r3,r1 // r3 := ed-bf = Dx
IMUL r1,r0,r5 // r1 := af
IMUL r7,r4,r2 // r7 := ec
SUB r1,r1,r7 // r1 := af-ec = Dy
IDIV r0,r3,r6 // x := Dx/D
IDIV r1,r1,r6 // y := Dy/D
RETWe have used 12 operations and at least 23 bytes (each operation uses 3\times 4=12 bits to indicate the three registers involved, and at least 4 bits to indicate the operation performed; thus we need two or three bytes to encode each operation). A more realistic estimate would be 34 (three bytes for each arithmetic operation) or 31 bytes (two bytes for addition and subtraction, three bytes for multiplication and division).
C.1.3. Two-address register machine
The machine code for a two-address register machine might look as follows:
MOV r6,r0 // r6 := r0 = a
MOV r7,r1 // r7 := b
IMUL r6,r3 // r6 := r6*r3 = ad
IMUL r7,r2 // r7 := bc
IMUL r3,r4 // r3 := de
IMUL r1,r5 // r1 := bf
SUB r6,r7 // r6 := ad-bc = D
IMUL r5,r0 // r5 := af
SUB r3,r1 // r3 := de-bf = Dx
IMUL r2,r4 // r2 := ce
MOV r0,r3 // r0 := Dx
SUB r5,r2 // r5 := af-ce = Dy
IDIV r0,r6 // r0 := x = Dx/D
MOV r1,r5 // r1 := Dy
IDIV r1,r6 // r1 := Dy/D
RETWe have used 16 operations; optimistically assuming each of them (with the exception of \texttt{RET}) can be encoded by two bytes, this code would require 31 bytes.31
C.1.4. One-address register machine
The machine code for a one-address register machine might look as follows:
MOV r8,r0 // r8 := r0 = a
XCHG r1 // r0 <-> r1; r0 := b, r1 := a
MOV r6,r0 // r6 := b
IMUL r2 // r0 := r0*r2; r0 := bc
MOV r7,r0 // r7 := bc
MOV r0,r8 // r0 := a
IMUL r3 // r0 := ad
SUB r7 // r0 := ad-bc = D
XCHG r1 // r1 := D, r0 := b
IMUL r5 // r0 := bf
XCHG r3 // r0 := d, r3 := bf
IMUL r4 // r0 := de
SUB r3 // r0 := de-bf = Dx
IDIV r1 // r0 := Dx/D = x
XCHG r2 // r0 := c, r2 := x
IMUL r4 // r0 := ce
XCHG r5 // r0 := f, r5 := ce
IMUL r8 // r0 := af
SUB r5 // r0 := af-ce = Dy
IDIV r1 // r0 := Dy/D = y
MOV r1,r0 // r1 := y
MOV r0,r2 // r0 := x
RETWe have used 23 operations; if we assume one-byte encoding for all arithmetic operations and \texttt{XCHG}, and two-byte encodings for \texttt{MOV}, the total size of the code will be 29 bytes. Notice, however, that to obtain the compact code shown above we had to choose a specific order of computation, and made heavy use of the commutativity of multiplication. (For example, we compute bc before af, and af-bc immediately after af.) It is not clear whether a compiler would be able to make all such optimizations by itself.
C.1.5. Stack machine with basic stack primitives
The machine code for a stack machine equipped with basic stack manipulation primitives might look as follows:
PUSH s5 // a b c d e f a
PUSH s3 // a b c d e f a d
IMUL // a b c d e f ad
PUSH s5 // a b c d e f ad b
PUSH s5 // a b c d e f ad b c
IMUL // a b c d e f ad bc
SUB // a b c d e f ad-bc
XCHG s3 // a b c ad-bc e f d
PUSH s2 // a b c ad-bc e f d e
IMUL // a b c ad-bc e f de
XCHG s5 // a de c ad-bc e f b
PUSH s1 // a de c ad-bc e f b f
IMUL // a de c ad-bc e f bf
XCHG s1,s5 // a f c ad-bc e de bf
SUB // a f c ad-bc e de-bf
XCHG s3 // a f de-bf ad-bc e c
IMUL // a f de-bf ad-bc ec
XCHG s3 // a ec de-bf ad-bc f
XCHG s1,s4 // ad-bc ec de-bf a f
IMUL // D ec Dx af
XCHG s1 // D ec af Dx
XCHG s2 // D Dx af ec
SUB // D Dx Dy
XCHG s1 // D Dy Dx
PUSH s2 // D Dy Dx D
IDIV // D Dy x
XCHG s2 // x Dy D
IDIV // x y
RETWe have used 29 operations; assuming one-byte encodings for all stack operations involved (including \texttt{XCHG s1,s}(i)), we have used 29 code bytes as well. Notice that with one-byte encoding, the "unsystematic" operation \texttt{ROT} (equivalent to \texttt{XCHG s1}; \texttt{XCHG s2}) would reduce the operation and byte count to 28. This shows that such "unsystematic" operations, borrowed from Forth, may indeed reduce the code size on some occasions.
Notice as well that we have implicitly used the commutativity of multiplication in this code, computing de-bf instead of ed-bf as specified in high-level language source code. If we were not allowed to do so, an extra \texttt{XCHG s1} would need to be inserted before the third \texttt{IMUL}, increasing the total size of the code by one operation and one byte.
The code presented above might have been produced by a rather unsophisticated compiler that simply computed all expressions and subexpressions in the order they appear, then rearranged the arguments near the top of the stack before each operation. The only "manual" optimization done here involves computing ec before af; one can check that the other order would lead to slightly shorter code of 28 operations and bytes (or 29, if we are not allowed to use the commutativity of multiplication), but the \texttt{ROT} optimization would not be applicable.
C.1.6. Stack machine with compound stack primitives
A stack machine with compound stack primitives would not significantly improve code density of the code presented above, at least in terms of bytes used. The only difference is that, if we were not allowed to use commutativity of multiplication, the extra \texttt{XCHG s1} inserted before the third \texttt{IMUL} might be combined with two previous operations \texttt{XCHG s3}, \texttt{PUSH s2} into one compound operation \texttt{PUXC s2,s3}; we provide the resulting code below. To make this less redundant, we show a version of the code that computes subexpression af before ec as specified in the original source file. We see that this replaces six operations (starting from line 15) with five other operations, and disables the \texttt{ROT} optimization:
PUSH s5 // a b c d e f a
PUSH s3 // a b c d e f a d
IMUL // a b c d e f ad
PUSH s5 // a b c d e f ad b
PUSH s5 // a b c d e f ad b c
IMUL // a b c d e f ad bc
SUB // a b c d e f ad-bc
PUXC s2,s3 // a b c ad-bc e f e d
IMUL // a b c ad-bc e f ed
XCHG s5 // a ed c ad-bc e f b
PUSH s1 // a ed c ad-bc e f b f
IMUL // a ed c ad-bc e f bf
XCHG s1,s5 // a f c ad-bc e ed bf
SUB // a f c ad-bc e ed-bf
XCHG s4 // a ed-bf c ad-bc e f
XCHG s1,s5 // e Dx c D a f
IMUL // e Dx c D af
XCHG s2 // e Dx af D c
XCHG s1,s4 // D Dx af e c
IMUL // D Dx af ec
SUB // D Dx Dy
XCHG s1 // D Dy Dx
PUSH s2 // D Dy Dx D
IDIV // D Dy x
XCHG s2 // x Dy D
IDIV // x y
RETWe have used a total of 27 operations and 28 bytes, the same as the previous version (with the \texttt{ROT} optimization). However, we did not use the commutativity of multiplication here, so we can say that compound stack manipulation primitives enable us to reduce the code size from 29 to 28 bytes.
Yet again, notice that the above code might have been generated by an unsophisticated compiler. Manual optimizations might lead to more compact code; for instance, we could use compound operations such as \texttt{XCHG3} to prepare in advance not only the correct values of \texttt{s0} and \texttt{s1} for the next arithmetic operation, but also the value of \texttt{s2} for the arithmetic operation after that. The next section provides an example of such an optimization.
C.1.7. Stack machine with compound stack primitives and manually optimized code
The previous version of code for a stack machine with compound stack primitives can be manually optimized as follows.
By interchanging \texttt{XCHG} operations with preceding \texttt{XCHG}, \texttt{PUSH}, and arithmetic operations whenever possible, we obtain code fragment \texttt{XCHG s2,s6}; \texttt{XCHG s1,s0}; \texttt{XCHG s0,s5}, which can then be replaced by compound operation \texttt{XCHG3 s6,s0,s5}. This compound operation would admit a two-byte encoding, thus leading to 27-byte code using only 21 operations:
PUSH2 s5,s2 // a b c d e f a d
IMUL // a b c d e f ad
PUSH2 s5,s4 // a b c d e f ad b c
IMUL // a b c d e f ad bc
SUB // a b c d e f ad-bc
PUXC s2,s3 // a b c ad-bc e f e d
IMUL // a b c D e f ed
XCHG3 s6,s0,s5 // (same as XCHG s2,s6; XCHG s1,s0; XCHG s0,s5)
// e f c D a ed b
PUSH s5 // e f c D a ed b f
IMUL // e f c D a ed bf
SUB // e f c D a ed-bf
XCHG s4 // e Dx c D a f
IMUL // e Dx c D af
XCHG2 s4,s2 // D Dx af e c
IMUL // D Dx af ec
SUB // D Dx Dy
XCPU s1,s2 // D Dy Dx D
IDIV // D Dy x
XCHG s2 // x Dy D
IDIV // x y
RETIt is interesting to note that this version of stack machine code contains only 9 stack manipulation primitives for 11 arithmetic operations. It is not clear, however, whether an optimizing compiler would be able to reorganize the code in such a manner by itself.
C.2 Comparison of machine code for sample leaf function
Table 1 summarizes the properties of machine code corresponding to the same source file, generated for a hypothetical three-address register machine, with both "optimistic" and "realistic" instruction encodings; a two-address machine; a one-address machine; and a stack machine, similar to TVM, using either only the basic stack manipulation primitives or both the basic and the composite stack primitives.
The meaning of the columns in Table 1 is as follows:
- "Operations" — The quantity of instructions used, split into "data" (i.e., register move and exchange instructions for register machines, and stack manipulation instructions for stack machines) and "arithmetic" (instructions for adding, subtracting, multiplying and dividing integer numbers). The "total" is one more than the sum of these two, because there is also a one-byte
\texttt{RET}instruction at the end of machine code. - "Code bytes" — The total amount of code bytes used.
- "Opcode space" — The portion of "opcode space" (i.e., of possible choices for the first byte of the encoding of an instruction) used by data and arithmetic instructions in the assumed instruction encoding. For example, the "optimistic" encoding for the three-address machine assumes two-byte encodings for all arithmetic instructions op
\texttt{r}(i)\texttt{, r}(j)\texttt{, r}(k). Each arithmetic instruction would then consume portion16/256=1/16of the opcode space. Notice that for the stack machine we have assumed one-byte encodings for\texttt{XCHG s}(i),\texttt{PUSH s}(i)and\texttt{POP s}(i)in all cases, augmented by\texttt{XCHG s1,s}(i)for the basic stack instructions case only. As for the compound stack operations, we have assumed two-byte encodings for\texttt{PUSH3},\texttt{XCHG3},\texttt{XCHG2},\texttt{XCPU},\texttt{PUXC},\texttt{PUSH2}, but not for\texttt{XCHG s1,s}(i).
| Machine | Operations | Code bytes | Opcode space | ||||||
|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | |
| 3-addr. (opt.) | 0 | 11 | 12 | 0 | 22 | 23 | 0/256 | 64/256 | 65/256 |
| 3-addr. (real.) | 0 | 11 | 12 | 0 | 30 | 31 | 0/256 | 34/256 | 35/256 |
| 2-addr. | 4 | 11 | 16 | 8 | 22 | 31 | 1/256 | 4/256 | 6/256 |
| 1-addr. | 11 | 11 | 23 | 17 | 11 | 29 | 17/256 | 64/256 | 82/256 |
| stack (basic) | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
Table 1. A summary of machine code properties for hypothetical 3-address, 2-address, 1-address, and stack machines, generated for a sample leaf function. The two most important columns, reflecting code density and extendability to other operations, are marked by bold font. Smaller values are better in both of these columns.
The "code bytes" column reflects the density of the code for the specific sample source. However, "opcode space" is also important, because it reflects the extendability of the achieved density to other classes of operations (e.g., if one were to complement arithmetic operations with string manipulation operations and so on). Here the "arithmetic" subcolumn is more important than the "data" subcolumn, because no further data manipulation operations would be required for such extensions.
We see that the three-address register machine with the "optimistic" encoding, assuming two-byte encodings for all three-register arithmetic operations, achieves the best code density, requiring only 23 bytes. However, this comes at a price: each arithmetic operation consumes 1/16 of the opcode space, so the four operations already use a quarter of the opcode space. At most 11 other operations, arithmetic or not, might be added to this architecture while preserving such high code density. On the other hand, when we consider the "realistic" encoding for the three-address machine, using two-byte encodings only for the most frequently used addition/subtraction operations (and longer encodings for less frequently used multiplication/division operations, reflecting the fact that the possible extension operations would likely fall in this class), then the three-address machine ceases to offer such attractive code density.
In fact, the two-address machine becomes equally attractive at this point: it is capable of achieving the same code size of 31 bytes as the three-address machine with the "realistic" encoding, using only 6/256 of the opcode space for this! However, 31 bytes is the worst result in this table.
The one-address machine uses 29 bytes, slightly less than the two-address machine. However, it utilizes a quarter of the opcode space for its arithmetic operations, hampering its extendability. In this respect it is similar to the three-address machine with the "optimistic" encoding, but requires 29 bytes instead of 23! So there is no reason to use the one-address machine at all, in terms of extendability (reflected by opcode space used for arithmetic operations) compared to code density.
Finally, the stack machine wins the competition in terms of code density (27 or 28 bytes), losing only to the three-address machine with the "optimistic" encoding (which, however, is terrible in terms of extendability).
To summarize: the two-address machine and stack machine achieve the best extendability with respect to additional arithmetic or data processing instructions (using only 1/256 of code space for each such instruction), while the stack machine additionally achieves the best code density by a small margin. The stack machine utilizes a significant part of its code space (more than a quarter) for data (i.e., stack) manipulation instructions; however, this does not seriously hamper extendability, because the stack manipulation instructions occupy a constant part of the opcode space, regardless of all other instructions and extensions.
While one might still be tempted to use a two-address register machine, we will explain shortly why the two-address register machine offers worse code density and extendability in practice than it appears based on this table.
As for the choice between a stack machine with only basic stack manipulation primitives or one supporting compound stack primitives as well, the case for the more sophisticated stack machine appears to be weaker: it offers only one or two fewer bytes of code at the expense of using considerably more opcode space for stack manipulation, and the optimized code using these additional instructions is hard for programmers to write and for compilers to automatically generate.
C.2.1. Register calling conventions: some registers must be preserved by functions
Up to this point, we have considered the machine code of only one function, without taking into account the interplay between this function and other functions in the same program.
Usually a program consists of more than one function, and when a function is not a "simple" or "leaf" function, it must call other functions. Therefore, it becomes important whether a called function preserves all or at least some registers. If it preserves all registers except those used to return results, the caller can safely keep its local and temporary variables in certain registers; however, the callee needs to save all the registers it will use for its temporary values somewhere (usually into the stack, which also exists on register machines), and then restore the original values. On the other hand, if the called function is allowed to destroy all registers, it can be written in the manner described in C.1.2, C.1.3, and C.1.4, but the caller will now be responsible for saving all its temporary values into the stack before the call, and restoring these values afterwards.
In most cases, calling conventions for register machines require preservation of some but not all registers. We will assume that m\leq n registers will be preserved by functions (unless they are used for return values), and that these registers are \texttt{r} (n-m) \ldots \texttt{r} (n-1). Case m=0 corresponds to the case "the callee is free to destroy all registers" considered so far; it is quite painful for the caller. Case m=n corresponds to the case "the callee must preserve all registers"; it is quite painful for the callee, as we will see in a moment. Usually a value of m around n/2 is used in practice.
The following sections consider cases m=0, m=8, and m=16 for our register machines with n=16 registers.
C.2.2. Case m=0: no registers to preserve
This case has been considered and summarized in C.2 and Table 1 above.
C.2.3. Case m=n=16: all registers must be preserved
This case is the most painful one for the called function. It is especially difficult for leaf functions like the one we have been considering, which do not benefit at all from the fact that other functions preserve some registers when called—they do not call any functions, but instead must preserve all registers themselves.
In order to estimate the consequences of assuming m=n=16, we will assume that all our register machines are equipped with a stack, and with one-byte instructions \texttt{PUSH r}(i) and \texttt{POP r}(i), which push or pop a register into/from the stack. For example, the three-address machine code destroys the values in registers \texttt{r2}, \texttt{r3}, \texttt{r6}, and \texttt{r7}; this means that the code of this function must be augmented by four instructions \texttt{PUSH r2}; \texttt{PUSH r3}; \texttt{PUSH r6}; \texttt{PUSH r7} at the beginning, and by four instructions \texttt{POP r7}; \texttt{POP r6}; \texttt{POP r3}; \texttt{POP r2} right before the \texttt{RET} instruction, in order to restore the original values of these registers from the stack. These four additional \texttt{PUSH}/\texttt{POP} pairs would increase the operation count and code size in bytes by 4\times 2=8. A similar analysis can be done for other register machines as well, leading to Table 2.
| Machine | \mathit{r} | Operations | Code bytes | Opcode space | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | ||
| 3-addr. (opt.) | 4 | 8 | 11 | 20 | 8 | 22 | 31 | 32/256 | 64/256 | 97/256 |
| 3-addr. (real.) | 4 | 8 | 11 | 20 | 8 | 30 | 39 | 32/256 | 34/256 | 67/256 |
| 2-addr. | 5 | 14 | 11 | 26 | 18 | 22 | 41 | 33/256 | 4/256 | 38/256 |
| 1-addr. | 6 | 23 | 11 | 35 | 29 | 11 | 41 | 49/256 | 64/256 | 114/256 |
| stack (basic) | 0 | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 0 | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
Table 2. A summary of machine code properties for hypothetical 3-address, 2-address, 1-address, and stack machines, generated for a sample leaf function, assuming all of the 16 registers must be preserved by called functions (m=n=16). The new column labeled r denotes the number of registers to be saved and restored, leading to 2r more operations and code bytes compared to Table 1. Newly-added \texttt{PUSH} and \texttt{POP} instructions for register machines also utilize 32/256 of the opcode space. The two rows corresponding to stack machines remain unchanged.
We see that under these assumptions the stack machines are the obvious winners in terms of code density, and are in the winning group with respect to extendability.
C.2.4. Case m = 8, n = 16: registers \texttt{r8} \ldots \texttt{r15} must be preserved
The analysis of this case is similar to the previous one. The results are summarized in Table 3.
| Machine | \mathit{r} | Operations | Code bytes | Opcode space | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | ||
| 3-addr. (opt.) | 0 | 0 | 11 | 12 | 0 | 22 | 23 | 32/256 | 64/256 | 97/256 |
| 3-addr. (real.) | 0 | 0 | 11 | 12 | 0 | 30 | 31 | 32/256 | 34/256 | 67/256 |
| 2-addr. | 0 | 4 | 11 | 16 | 8 | 22 | 31 | 33/256 | 4/256 | 38/256 |
| 1-addr. | 1 | 13 | 11 | 25 | 19 | 11 | 31 | 49/256 | 64/256 | 114/256 |
| stack (basic) | 0 | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 0 | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
Table 3. A summary of machine code properties for hypothetical 3-address, 2-address, 1-address and stack machines, generated for a sample leaf function, assuming that only the last 8 of the 16 registers must be preserved by called functions (m=8, n=16). This table is similar to Table 2, but has smaller values of r.
Notice that the resulting table is very similar to Table 1, apart from the "Opcode space" columns and the row for the one-address machine. Therefore, the conclusions of C.2 still apply in this case, with some minor modifications. We must emphasize, however, that these conclusions are valid only for leaf functions, i.e., functions that do not call other functions. Any program aside from the very simplest will have many non-leaf functions, especially if we are minimizing resulting machine code size (which prevents inlining of functions in most cases).
C.2.5. A fairer comparison using a binary code instead of a byte code
The reader may have noticed that our preceding discussion of k-address register machines and stack machines depended very much on our insistence that complete instructions be encoded by an integer number of bytes. If we had been allowed to use a "bit" or "binary code" instead of a byte code for encoding instructions, we could more evenly balance the opcode space used by different machines. For instance, the opcode of SUB for a three-address machine had to be either 4-bit (good for code density, bad for opcode space) or 12-bit (very bad for code density), because the complete instruction has to occupy a multiple of eight bits (e.g., 16 or 24 bits), and 3\cdot 4=12 of those bits have to be used for the three register names.
Therefore, let us get rid of this restriction.
Now that we can use any number of bits to encode an instruction, we can choose all opcodes of the same length for all the machines considered. For instance, all arithmetic instructions can have 8-bit opcodes, as the stack machine does, using 1/256 of the opcode space each; then the three-address register machine will use 20 bits to encode each complete arithmetic instruction. All \texttt{MOV}s, \texttt{XCHG}s, \texttt{PUSH}es, and \texttt{POP}s on register machines can be assumed to have 4-bit opcodes, because this is what we do for the most common stack manipulation primitives on a stack machine. The results of these changes are shown in Table 4.
We can see that the performance of the various machines is much more balanced, with the stack machine still the winner in terms of the code density, but with the three-address machine enjoying the second place it really merits. If we were to consider the decoding speed and the possibility of parallel execution of instructions, we would have to choose the three-address machine, because it uses only 12 instructions instead of 21.
| Machine | \mathit{r} | Operations | Code bytes | Opcode space | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | arith | total | data | arith | total | data | arith | total | ||
| 3-addr. | 0 | 0 | 11 | 12 | 0 | 27.5 | 28.5 | 64/256 | 4/256 | 69/256 |
| 2-addr. | 0 | 4 | 11 | 16 | 6 | 22 | 29 | 64/256 | 4/256 | 69/256 |
| 1-addr. | 1 | 13 | 11 | 25 | 16 | 16.5 | 32.5 | 64/256 | 4/256 | 69/256 |
| stack (basic) | 0 | 16 | 11 | 28 | 16 | 11 | 28 | 64/256 | 4/256 | 69/256 |
| stack (comp.) | 0 | 9 | 11 | 21 | 15 | 11 | 27 | 84/256 | 4/256 | 89/256 |
Table 4. A summary of machine code properties for hypothetical 3-address, 2-address, 1-address and stack machines, generated for a sample leaf function, assuming that only 8 of the 16 registers must be preserved by functions (m=8, n=16). This time we can use fractions of bytes to encode instructions, so as to match opcode space used by different machines. All arithmetic instructions have 8-bit opcodes, all data/stack manipulation instructions have 4-bit opcodes. In other respects this table is similar to Table 3.
C.3 Sample non-leaf function
This section compares the machine code for different register machines for a sample non-leaf function. Again, we assume that either m=0, m=8, or m=16 registers are preserved by called functions, with m=8 representing the compromise made by most modern compilers and operating systems.
C.3.1. Sample source code for a non-leaf function
A sample source file may be obtained by replacing the built-in integer type with a custom Rational type, represented by a pointer to an object in memory, in our function for solving systems of two linear equations:
struct Rational;
typedef struct Rational *num;
extern num r_add(num, num);
extern num r_sub(num, num);
extern num r_mul(num, num);
extern num r_div(num, num);
(num, num) r_f(num a, num b, num c, num d, num e, num f) {
num D = r_sub(r_mul(a, d), r_mul(b, c)); // a*d-b*c
num Dx = r_sub(r_mul(e, d), r_mul(b, f)); // e*d-b*f
num Dy = r_sub(r_mul(a, f), r_mul(e, c)); // a*f-e*c
return (r_div(Dx, D), r_div(Dy, D)); // Dx/D, Dy/D
}We will ignore all questions related to allocating new objects of type Rational in memory (e.g., in heap), and to preventing memory leaks. We may assume that the called subroutines \texttt{r\_sub}, \texttt{r\_mul}, and so on allocate new objects simply by advancing some pointer in a pre-allocated buffer, and that unused objects are later freed by a garbage collector, external to the code being analysed.
Rational numbers will now be represented by pointers, addresses, or references, which will fit into registers of our hypothetical register machines or into the stack of our stack machines. If we want to use TVM as an instance of these stack machines, we should use values of type Cell to represent such references to objects of type Rational in memory.
We assume that subroutines (or functions) are called by a special \texttt{CALL} instruction, which is encoded by three bytes, including the specification of the function to be called (e.g., the index in a "global function table").
C.3.2. Three-address and two-address register machines, m=0 preserved registers
Because our sample function does not use built-in arithmetic instructions at all, compilers for our hypothetical three-address and two-address register machines will produce the same machine code. Apart from the previously introduced \texttt{PUSH r}(i) and \texttt{POP r}(i) one-byte instructions, we assume that our two- and three-address machines support the following two-byte instructions: \texttt{MOV r}(i)\texttt{,s}(j), \texttt{MOV s}(j)\texttt{,r}(i), and \texttt{XCHG r}(i)\texttt{,s}(j), for 0\leq i,j\leq 15. Such instructions occupy only 3/256 of the opcode space, so their addition seems quite natural.
We first assume that m=0 (i.e., that all subroutines are free to destroy the values of all registers). In this case, our machine code for \texttt{r\_f} does not have to preserve any registers, but has to save all registers containing useful values into the stack before calling any subroutines. A size-optimizing compiler might produce the following code:
PUSH r4 // STACK: e
PUSH r1 // STACK: e b
PUSH r0 // .. e b a
PUSH r6 // .. e b a f
PUSH r2 // .. e b a f c
PUSH r3 // .. e b a f c d
MOV r0,r1 // b
MOV r1,r2 // c
CALL r_mul // bc
PUSH r0 // .. e b a f c d bc
MOV r0,s4 // a
MOV r1,s1 // d
CALL r_mul // ad
POP r1 // bc; .. e b a f c d
CALL r_sub // D:=ad-bc
XCHG r0,s4 // b ; .. e D a f c d
MOV r1,s2 // f
CALL r_mul // bf
POP r1 // d ; .. e D a f c
PUSH r0 // .. e D a f c bf
MOV r0,s5 // e
CALL r_mul // ed
POP r1 // bf; .. e D a f c
CALL r_sub // Dx:=ed-bf
XCHG r0,s4 // e ; .. Dx D a f c
POP r1 // c ; .. Dx D a f
CALL r_mul // ec
XCHG r0,s1 // a ; .. Dx D ec f
POP r1 // f ; .. Dx D ec
CALL r_mul // af
POP r1 // ec; .. Dx D
CALL r_sub // Dy:=af-ec
XCHG r0,s1 // Dx; .. Dy D
MOV r1,s0 // D
CALL r_div // x:=Dx/D
XCHG r0,s1 // Dy; .. x D
POP r1 // D ; .. x
CALL r_div // y:=Dy/D
MOV r1,r0 // y
POP r0 // x ; ..
RETWe have used 41 instructions: 17 one-byte (eight \texttt{PUSH}/\texttt{POP} pairs and one \texttt{RET}), 13 two-byte (\texttt{MOV} and \texttt{XCHG}; out of them 11 "new" ones, involving the stack), and 11 three-byte (\texttt{CALL}), for a total of 17\cdot1+13\cdot2+11\cdot3=76 bytes.32
C.3.3. Three-address and two-address register machines, m=8 preserved registers
Now we have eight registers, \texttt{r8} to \texttt{r15}, that are preserved by subroutine calls. We might keep some intermediate values there instead of pushing them into the stack. However, the penalty for doing so consists in a \texttt{PUSH}/\texttt{POP} pair for every such register that we choose to use, because our function is also required to preserve its original value. It seems that using these registers under such a penalty does not improve the density of the code, so the optimal code for three- and two-address machines for m=8 preserved registers is the same, with a total of 42 instructions and 74 code bytes.
C.3.4. Three-address and two-address register machines, m=16 preserved registers
This time all registers must be preserved by the subroutines, excluding those used for returning the results. This means that our code must preserve the original values of \texttt{r2} to \texttt{r5}, as well as any other registers it uses for temporary values. A straightforward way of writing the code of our subroutine would be to push registers \texttt{r2} up to, say, \texttt{r8} into the stack, then perform all the operations required, using \texttt{r6}—\texttt{r8} for intermediate values, and finally restore registers from the stack. However, this would not optimize code size. We choose another approach:
PUSH r0 // STACK: a
PUSH r1 // STACK: a b
MOV r0,r1 // b
MOV r1,r2 // c
CALL r_mul // bc
PUSH r0 // .. a b bc
MOV r0,s2 // a
MOV r1,r3 // d
CALL r_mul // ad
POP r1 // bc; .. a b
CALL r_sub // D:=ad-bc
XCHG r0,s0 // b; .. a D
MOV r1,r5 // f
CALL r_mul // bf
PUSH r0 // .. a D bf
MOV r0,r4 // e
MOV r1,r3 // d
CALL r_mul // ed
POP r1 // bf; .. a D
CALL r_sub // Dx:=ed-bf
XCHG r0,s1 // a ; .. Dx D
MOV r1,r5 // f
CALL r_mul // af
PUSH r0 // .. Dx D af
MOV r0,r4 // e
MOV r1,r2 // c
CALL r_mul // ec
MOV r1,r0 // ec
POP r0 // af; .. Dx D
CALL r_sub // Dy:=af-ec
XCHG r0,s1 // Dx; .. Dy D
MOV r1,s0 // D
CALL r_div // x:=Dx/D
XCHG r0,s1 // Dy; .. x D
POP r1 // D ; .. x
CALL r_div // y:=Dy/D
MOV r1,r0 // y
POP r0 // x
RETWe have used 39 instructions: 11 one-byte, 17 two-byte (among them 5 "new" instructions), and 11 three-byte, for a total of 11\cdot1+17\cdot2+11\cdot3=78 bytes. Somewhat paradoxically, the code size in bytes is slightly longer than in the previous case, contrary to what one might have expected. This is partially due to the fact that we have assumed two-byte encodings for "new" \texttt{MOV} and \texttt{XCHG} instructions involving the stack, similarly to the "old" instructions. Most existing architectures (such as x86-64) use longer encodings (maybe even twice as long) for their counterparts of our "new" move and exchange instructions compared to the "usual" register-register ones. Taking this into account, we see that we would have obtained here 83 bytes (versus 87 for the code) assuming three-byte encodings of new operations, and 88 bytes (versus 98) assuming four-byte encodings. This shows that, for two-address architectures without optimized encodings for register-stack move and exchange operations, m=16 preserved registers might result in slightly shorter code for some non-leaf functions, at the expense of leaf functions C.2.3 and C.2.4, which would become considerably longer.
C.3.5. One-address register machine, m=0 preserved registers
For our one-address register machine, we assume that new register-stack instructions work through the accumulator only. Therefore, we have three new instructions, \texttt{LD s}(j) (equivalent to \texttt{MOV r0,s}(j) of two-address machines), \texttt{ST s}(j) (equivalent to \texttt{MOV s}(j)\texttt{,r0}), and \texttt{XCHG s}(j) (equivalent to \texttt{XCHG r0,s}(j)). To make the comparison with two-address machines more interesting, we assume one-byte encodings for these new instructions, even though this would consume 48/256=3/16 of the opcode space.
By adapting the code to the one-address machine, we obtain the following:
PUSH r4 // STACK: e
PUSH r1 // STACK: e b
PUSH r0 // .. e b a
PUSH r6 // .. e b a f
PUSH r2 // .. e b a f c
PUSH r3 // .. e b a f c d
LD s1 // r0:=c
XCHG r1 // r0:=b, r1:=c
CALL r_mul // bc
PUSH r0 // .. e b a f c d bc
LD s1 // d
XCHG r1 // r1:=d
LD s4 // a
CALL r_mul // ad
POP r1 // bc; .. e b a f c d
CALL r_sub // D:=ad-bc
XCHG s4 // b ; .. e D a f c d
XCHG r1
LD s2 // f
XCHG r1 // r0:=b, r1:=f
CALL r_mul // bf
POP r1 // d ; .. e D a f c
PUSH r0 // .. e D a f c bf
LD s5 // e
CALL r_mul // ed
POP r1 // bf; .. e D a f c
CALL r_sub // Dx:=ed-bf
XCHG s4 // e ; .. Dx D a f c
POP r1 // c ; .. Dx D a f
CALL r_mul // ec
XCHG s1 // a ; .. Dx D ec f
POP r1 // f ; .. Dx D ec
CALL r_mul // af
POP r1 // ec; .. Dx D
CALL r_sub // Dy:=af-ec
XCHG s1 // Dx; .. Dy D
POP r1 // D ; .. Dy
PUSH r1 // .. Dy D
CALL r_div // x:=Dx/D
XCHG s1 // Dy; .. x D
POP r1 // D ; .. x
CALL r_div // y:=Dy/D
XCHG r1 // r1:=y
POP r0 // r0:=x ; ..
RETWe have used 45 instructions: 34 one-byte and 11 three-byte, for a total of 67 bytes. Compared to the 76 bytes used by two- and three-address machines, we see that, again, the one-address register machine code may be denser than that of two-register machines, at the expense of utilizing more opcode space. However, this time the extra 3/16 of the opcode space was used for data manipulation instructions, which do not depend on specific arithmetic operations or user functions invoked.
C.3.6. One-address register machine, m=8 preserved registers
As explained above, the preservation of \texttt{r8}—\texttt{r15} between subroutine calls does not improve the size of our previously written code, so the one-address machine will use for m=8 the same code.
C.3.7. One-address register machine, m=16 preserved registers
We simply adapt the code to the one-address register machine:
PUSH r0 // STACK: a
PUSH r1 // STACK: a b
MOV r0,r1 // b
MOV r1,r2 // c
CALL r_mul // bc
PUSH r0 // .. a b bc
LD s2 // a
MOV r1,r3 // d
CALL r_mul // ad
POP r1 // bc; .. a b
CALL r_sub // D:=ad-bc
XCHG s0 // b; .. a D
MOV r1,r5 // f
CALL r_mul // bf
PUSH r0 // .. a D bf
MOV r0,r4 // e
MOV r1,r3 // d
CALL r_mul // ed
POP r1 // bf; .. a D
CALL r_sub // Dx:=ed-bf
XCHG s1 // a ; .. Dx D
MOV r1,r5 // f
CALL r_mul // af
PUSH r0 // .. Dx D af
MOV r0,r4 // e
MOV r1,r2 // c
CALL r_mul // ec
MOV r1,r0 // ec
POP r0 // af; .. Dx D
CALL r_sub // Dy:=af-ec
XCHG s1 // Dx; .. Dy D
POP r1 // D ; .. Dy
PUSH r1 // .. Dy D
CALL r_div // x:=Dx/D
XCHG s1 // Dy; .. x D
POP r1 // D ; .. x
CALL r_div // y:=Dy/D
MOV r1,r0 // y
POP r0 // x
RETWe have used 40 instructions: 18 one-byte, 11 two-byte, and 11 three-byte, for a total of 18\cdot1+11\cdot2+11\cdot3=73 bytes.
C.3.8. Stack machine with basic stack primitives
We reuse the code, simply replacing arithmetic primitives (VM instructions) with subroutine calls. The only substantive modification is the insertion of the previously optional \texttt{XCHG s1} before the third multiplication, because even an optimizing compiler cannot now know whether \texttt{CALL r\_mul} is a commutative operation. We have also used the "tail recursion optimization" by replacing the final \texttt{CALL r\_div} followed by \texttt{RET} with \texttt{JMP r\_div}.
PUSH s5 // a b c d e f a
PUSH s3 // a b c d e f a d
CALL r_mul // a b c d e f ad
PUSH s5 // a b c d e f ad b
PUSH s5 // a b c d e f ad b c
CALL r_mul // a b c d e f ad bc
CALL r_sub // a b c d e f ad-bc
XCHG s3 // a b c ad-bc e f d
PUSH s2 // a b c ad-bc e f d e
XCHG s1 // a b c ad-bc e f e d
CALL r_mul // a b c ad-bc e f ed
XCHG s5 // a ed c ad-bc e f b
PUSH s1 // a ed c ad-bc e f b f
CALL r_mul // a ed c ad-bc e f bf
XCHG s1,s5 // a f c ad-bc e ed bf
CALL r_sub // a f c ad-bc e ed-bf
XCHG s3 // a f ed-bf ad-bc e c
CALL r_mul // a f ed-bf ad-bc ec
XCHG s3 // a ec ed-bf ad-bc f
XCHG s1,s4 // ad-bc ec ed-bf a f
CALL r_mul // D ec Dx af
XCHG s1 // D ec af Dx
XCHG s2 // D Dx af ec
CALL r_sub // D Dx Dy
XCHG s1 // D Dy Dx
PUSH s2 // D Dy Dx D
CALL r_div // D Dy x
XCHG s2 // x Dy D
JMP r_div // x yWe have used 29 instructions; assuming one-byte encodings for all stack operations, and three-byte encodings for \texttt{CALL} and \texttt{JMP} instructions, we end up with 51 bytes.
C.3.9. Stack machine with compound stack primitives
We again reuse the code, replacing arithmetic primitives with subroutine calls and making the tail recursion optimization:
PUSH2 s5,s2 // a b c d e f a d
CALL r_mul // a b c d e f ad
PUSH2 s5,s4 // a b c d e f ad b c
CALL r_mul // a b c d e f ad bc
CALL r_sub // a b c d e f ad-bc
PUXC s2,s3 // a b c ad-bc e f e d
CALL r_mul // a b c D e f ed
XCHG3 s6,s0,s5 // (same as XCHG s2,s6; XCHG s1,s0; XCHG s0,s5)
// e f c D a ed b
PUSH s5 // e f c D a ed b f
CALL r_mul // e f c D a ed bf
CALL r_sub // e f c D a ed-bf
XCHG s4 // e Dx c D a f
CALL r_mul // e Dx c D af
XCHG2 s4,s2 // D Dx af e c
CALL r_mul // D Dx af ec
CALL r_sub // D Dx Dy
XCPU s1,s2 // D Dy Dx D
CALL r_div // D Dy x
XCHG s2 // x Dy D
JMP r_div // x yThis code uses only 20 instructions, 9 stack-related and 11 control flow-related (\texttt{CALL} and \texttt{JMP}), for a total of 48 bytes.
C.4 Comparison of machine code for sample non-leaf function
Table 5 summarizes the properties of machine code corresponding to the same source file. We consider only the "realistically" encoded three-address machines. Three-address and two-address machines have the same code density properties, but differ in the utilization of opcode space. The one-address machine, somewhat surprisingly, managed to produced shorter code than the two-address and three-address machines, at the expense of using up more than half of all opcode space. The stack machine is the obvious winner in this code density contest, without compromizing its excellent extendability (measured in opcode space used for arithmetic and other data transformation instructions).
| Machine | m | Operations | Code bytes | Opcode space | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | cont. | total | data | cont. | total | data | arith | total | ||
| 3-addr. | 0,8 | 29 | 12 | 41 | 42 | 34 | 76 | 35/256 | 34/256 | 72/256 |
| 16 | 27 | 12 | 39 | 44 | 34 | 78 | ||||
| 2-addr. | 0,8 | 29 | 12 | 41 | 42 | 34 | 76 | 37/256 | 4/256 | 44/256 |
| 16 | 27 | 12 | 39 | 44 | 34 | 78 | ||||
| 1-addr. | 0,8 | 33 | 12 | 45 | 33 | 34 | 67 | 97/256 | 64/256 | 164/256 |
| 16 | 28 | 12 | 40 | 39 | 34 | 73 | ||||
| stack (basic) | - | 18 | 11 | 29 | 18 | 33 | 51 | 64/256 | 4/256 | 71/256 |
| stack (comp.) | - | 9 | 11 | 20 | 15 | 33 | 48 | 84/256 | 4/256 | 91/256 |
Table 5: A summary of machine code properties for hypothetical 3-address, 2-address, 1-address, and stack machines, generated for a sample non-leaf function, assuming m of the 16 registers must be preserved by called subroutines.
C.4.1. Combining with results for leaf functions
It is instructive to compare this table with the results in C.2 for a sample leaf function, summarized in Table 1 (for m=0 preserved registers) and the very similar Table 3 (for m=8 preserved registers), and, if one is still interested in case m=16 (which turned out to be worse than m=8 in almost all situations), also to Table 2.
We see that the stack machine beats all register machines on non-leaf functions. As for the leaf functions, only the three-address machine with the "optimistic" encoding of arithmetic instructions was able to beat the stack machine, winning by 15%, by compromising its extendability. However, the same three-address machine produces 25% longer code for non-leaf functions. If a typical program consists of a mixture of leaf and non-leaf functions in approximately equal proportion, then the stack machine will still win.
C.4.2. A fairer comparison using a binary code instead of a byte code
Similarly to C.2.5, we may offer a fairer comparison of different register machines and the stack machine by using arbitrary binary codes instead of byte codes to encode instructions, and matching the opcode space used for data manipulation and arithmetic instructions by different machines. The results of this modified comparison are summarized in Table 6. We see that the stack machines still win by a large margin, while using less opcode space for stack/data manipulation.
| Machine | m | Operations | Code bytes | Opcode space | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| data | cont. | total | data | cont. | total | data | arith | total | ||
| 3-addr. | 0,8 | 29 | 12 | 41 | 35.5 | 34 | 69.5 | 110/256 | 4/256 | 117/256 |
| 16 | 27 | 12 | 39 | 35.5 | 34 | 69.5 | ||||
| 2-addr. | 0,8 | 29 | 12 | 41 | 35.5 | 34 | 69.5 | 110/256 | 4/256 | 117/256 |
| 16 | 27 | 12 | 39 | 35.5 | 34 | 69.5 | ||||
| 1-addr. | 0,8 | 33 | 12 | 45 | 33 | 34 | 67 | 112/256 | 4/256 | 119/256 |
| 16 | 28 | 12 | 40 | 33.5 | 34 | 67.5 | ||||
| stack (basic) | - | 18 | 11 | 29 | 18 | 33 | 51 | 64/256 | 4/256 | 71/256 |
| stack (comp.) | - | 9 | 11 | 20 | 15 | 33 | 48 | 84/256 | 4/256 | 91/256 |
Table 6: A summary of machine code properties for hypothetical 3-address, 2-address, 1-address, and stack machines, generated for a sample non-leaf function, assuming m of the 16 registers must be preserved by called subroutines. This time we use fractions of bytes to encode instructions, enabling a fairer comparison. Otherwise, this table is similar to Table 5.
C.4.3. Comparison with real machines
Note that our hypothetical register machines have been considerably optimized to produce shorter code than actually existing register machines; the latter are subject to other design considerations apart from code density and extendability, such as backward compatibility, faster instruction decoding, parallel execution of neighboring instructions, ease of automatically producing optimized code by compilers, and so on.
For example, the very popular two-address register architecture x86-64 produces code that is approximately twice as long as our "ideal" results for the two-address machines. On the other hand, our results for the stack machines are directly applicable to TVM, which has been explicitly designed with the considerations presented in this appendix in mind. Furthermore, the actual TVM code is even shorter (in bytes) than shown in Table 5 because of the presence of the two-byte \texttt{CALL} instruction, allowing TVM to call up to 256 user-defined functions from the dictionary at \texttt{c3}. This means that one should subtract 10 bytes from the results for stack machines in Table 5 if one wants to specifically consider TVM, rather than an abstract stack machine; this produces a code size of approximately 40 bytes (or shorter), almost half that of an abstract two-address or three-address machine.
C.4.4. Automatic generation of optimized code
An interesting point is that the stack machine code in our samples might have been generated automatically by a very simple optimizing compiler, which rearranges values near the top of the stack appropriately before invoking each primitive or calling a function as explained in 2.2.2 and 2.2.5. The only exception is the unimportant manual \texttt{XCHG3} optimization, which enabled us to shorten the code by one more byte.
By contrast, the heavily optimized (with respect to size) code for register machines shown in C.3.2 and C.3.3 is unlikely to be produced automatically by an optimizing compiler. Therefore, if we had compared compiler-generated code instead of manually-generated code, the advantages of stack machines with respect to code density would have been even more striking.
References
[1] N. Durov, Telegram Open Network, 2017.
Footnotes
1 For example, there are no floating-point arithmetic operations (which could be efficiently implemented using hardware-supported double type on most modern CPUs) present in TVM, because the result of performing such operations is dependent on the specific underlying hardware implementation and rounding mode settings. Instead, TVM supports special integer arithmetic operations, which can be used to simulate fixed-point arithmetic if needed. Back ↑
2 The production version will likely require some tweaks and modifications prior to launch, which will become apparent only after using the experimental version in the test environment for some time. Back ↑
3 A high-level smart-contract language might create a visibility of variables for the ease of programming; however, the high-level source code working with variables will be translated into TVM machine code keeping all the values of these variables in the TVM stack. Back ↑
4 In the TON Blockchain context, c7 is initialized with a singleton Tuple, the only component of which is a Tuple containing blockchain-specific data. The smart contract is free to modify c7 to store its temporary data provided the first component of this Tuple remains intact. Back ↑
5 Strictly speaking, there is also the current library context, which consists of a dictionary with 256-bit keys and cell values, used to load library reference cells of Types of exotic cells. Back ↑
6 Our inclusion of r0 here creates a minor conflict with our assumption that the accumulator register, if present, is also r0; for simplicity, we will resolve this problem by assuming that the first argument to a function is passed in the accumulator. Back ↑
7 For instance, if one writes a function for extracting square roots, this function will always accept its argument and return its result in the same registers, in contrast with a hypothetical built-in square root instruction, which could allow the programmer to arbitrarily choose the source and destination registers. Therefore, a user-defined function is tremendously less flexible than a built-in instruction on a register machine. Back ↑
8 Of course, if the second option is used, this will destroy the original arrangement of x in the top of the stack. In this case, one should either issue a SWAP before XCHG s(j'), or replace the previous operation XCHG s(i) with XCHG s1, s(i), so that x is exchanged with s1 from the beginning. Back ↑
9 Notice that the most common XCHG s(i) operation is not really required here if we do not insist on keeping the same temporary value or variable always in the same stack location, but rather keep track of its subsequent locations. We will move it to some other location while preparing the arguments to the next primitive or function call. Back ↑
10 An alternative, arguably better, translation of PUO' s(i_1),...,s(i_γ) consists of the translation of O' s(i_2),...,s(i_γ), followed by PUSH s(i_1+α-1); XCHG s(γ-1). Back ↑
11 From the perspective of low-level cell operations, these data bits and cell references are not intermixed. In other words, an (ordinary) cell essentially is a couple consisting of a list of up to 1023 bits and of a list of up to four cell references, without prescribing an order in which the references and the data bits should be deserialized, even though TL-B schemes appear to suggest such an order. Back ↑
12 From a theoretical perspective, we might say that a cell c has an infinite sequence of hashes (\text{Hash}_i(c))_{i\geq1}, which eventually stabilizes: \text{Hash}_i(c)\to\text{Hash}_\infty(c). Then the level l is simply the largest index i, such that \text{Hash}_i(c)\neq\text{Hash}_\infty(c). Back ↑
13 A pruned branch cell c' of level l is bound by a Merkle (proof or update) cell c if there are exactly l Merkle cells on the path from c to its descendant c', including c. Back ↑
14 Negative numbers are represented using two's complement. For instance, integer -17 is serialized by instruction STI 8 into bitstring xEF. Back ↑
15 A description of an older version of TL may be found at https://core.telegram.org/mtproto/TL. Back ↑
16 The field's name is useful for representing values of the type being defined in human-readable form, but it does not affect the binary serialization. Back ↑
17 This is the "linear negation" operation (-)^\perp of linear logic, hence our notation ~. Back ↑
18 In fact, f may receive m extra arguments and return m modified values, which are passed to the next invocation of f. This may be used to implement "map" and "reduce" operations with dictionaries. Back ↑
19 Versions of this operation may be introduced where f and g receive an additional bitstring argument, equal to the key (for leaves) or to the common prefix of all keys (for forks) in the corresponding subtree. Back ↑
20 If there are no bits of data left in code, but there is still exactly one reference, an implicit JMP to the cell at that reference is performed instead of an implicit RET. Back ↑
21 Technically, TVM may simply invoke a virtual method run() of the continuation currently in cc. Back ↑
22 The already used savelist cc.save of the new cc is emptied before the execution starts. Back ↑
23 The implementation of REPEAT involves an extraordinary continuation that remembers the remaining number of iterations, the body of the loop c, and the return continuation c'. (The latter term represents the remainder of the body of the function that invoked REPEAT, which would be normally stored in c0 of the new cc.) Back ↑
24 An important point here is that the tree of cells representing a TVM program cannot have cyclic references, so using CALLREF along with a reference to a cell higher up the tree would not work. Back ↑
25 This is not exactly true. A more precise statement is that usually the codepage of the newly-created continuation is a known function of the current codepage. Back ↑
26 This is another important mechanism of backward compatibility. All values of newly-added types, as well as values belonging to extended original types that do not belong to the original types (e.g., 513-bit integers that do not fit into 257 bits in the example above), are treated by all instructions (except stack manipulation instructions, which are naturally polymorphic, Polymorphism of stack manipulation primitives) in the old codepages as "values of incorrect type", and generate type-checking exceptions accordingly. Back ↑
27 If the cell dumps are hexadecimal, encodings consisting of an integral number of hexadecimal digits (i.e., having length divisible by four bits) might be equally convenient. Back ↑
28 Notice that it is the probability of occurrence in the code that counts, not the probability of being executed. An instruction occurring in the body of a loop executed a million times is still counted only once. Back ↑
29 Notice that any modifications after launch cannot be done unilaterally; rather they would require the support of at least two-thirds of validators. Back ↑
30 The preliminary version of TVM does not use codepage -2 for this purpose. This may change in the future. Back ↑
31 It is interesting to compare this code with that generated by optimizing C compilers for the x86-64 architecture. First of all, the integer division operation for x86-64 uses the one-address form, with the (double-length) dividend to be supplied in accumulator pair r2:r0. The quotient is also returned in r0. As a consequence, two single-to-double extension operations (CDQ or CQO) and at least one move operation need to be added. Secondly, the encoding used for arithmetic and move operations is less optimistic than in our example above, requiring about three bytes per operation on average. As a result, we obtain a total of 43 bytes for 32-bit integers, and 68 bytes for 64-bit integers. Back ↑
32 Code produced for this function by an optimizing compiler for x86-64 architecture with size-optimization enabled actually occupied 150 bytes, due mostly to the fact that actual instruction encodings are about twice as long as we had optimistically assumed. Back ↑
Last updated on