Neural Networks as Products of Boolean Functions

Setup

• A (deployed) neural network computes, at inference time, a total function $$F:{\{0,1\}}^n\to {\{0,1\}}^m$$ once you fix a finite bit-encoding of its inputs, parameters, and numerical operations (fixed/float with fixed width, rounding mode, etc.). (If inputs are reals, you use a fixed finite encoding of them; if you allow unbounded precision, the claim is false as stated because the domain is no longer finite.)
• A product of multivariable boolean functions means simply a tuple of boolean functions, i.e., a map into a finite product type: $$⟨f_1,\dots ,f_m⟩:{\{0,1\}}^n\to {\{0,1\}}^m,f_j:{\{0,1\}}^n\to \{0,1\}.$$ "Product" here is categorical/product-type, not arithmetic product.
• Goal: Show that every such network $F$ is extensionally equal to such a product $⟨f_1,\dots ,f_m⟩$, where each $f_j$ is a multivariable boolean function (i.e. a boolean expression over the input bits).

A second, reversible variant replaces non-reversible primitives by a reversible embedding using ancillary bits.

Reduction of a Neural Net to a Straight-Line Program

Model. Fix:

• finite directed acyclic computation graph (feedforward net; for RNNs, fix a finite unrolling),
• finite set of primitive ops: add, multiply, compare, copy/move, constant load, and activations (ReLU, step, sigmoid, etc.); for non-polynomial activations we fix a finite-precision implementation,
• finite bit-width arithmetic with deterministic rounding.

Observation. Under these choices, a forward pass is a straight-line program (no data-dependent loops; conditionals arise only from comparisons inside activations like ReLU or explicit branches). Thus the net computes some function $F$ by composing a finite number of primitives, each of which itself is a finite function on bit-vectors.

Each Primitive is a Product of Boolean Functions

Any k-bit adder, multiplier, comparator, mux, table lookup, and each fixed-width activation implementation has a known combinational boolean circuit. Therefore:

• For each primitive $P:{\{0,1\}}^a\to {\{0,1\}}^b$ there exist boolean formulas $$p_1(\mathbf{x}),\dots ,p_b(\mathbf{x})$$ such that $P(\mathbf{x})=⟨p_1(\mathbf{x}),\dots ,p_b(\mathbf{x})⟩$.
• Equivalently, $P$ is a product of multivariable boolean functions over its input bits.

Induction on the Abstract Syntax Tree (AST)

Define a minimal typed functional core for forward passes:

• Atoms: input variables $x_1,\dots ,x_n$ (bits), constants.
• Constructors:
  1. Primitive application: $t:=P(t_1,\dots ,t_r)$.
  2. Pairing/product: $⟨t_1,\dots ,t_k⟩$.
  3. Projection: ${\pi }_j(t)$ to select components.
  4. Conditional: $\mathrm{if}b\mathrm{then}t\mathrm{else}u$, where $b$ is a boolean term (arises from comparators/branches inside activations). (No general recursion is needed for a finite forward pass.)

Claim (by structural induction).
For every term $t$ denoting a b-bit value, there exists a tuple of boolean formulas $$\left[t\right]=⟨f_1^{(t)}(\mathbf{x}),\dots ,f_b^{(t)}(\mathbf{x})⟩$$ over the input bits $\mathbf{x}$ such that evaluation of $t$ equals this tuple.

Proof sketch.

• Base cases.
  • Variable bit $x_i$: already a boolean function (projection).
  • Constant bit $c\in \{0,1\}$: constant boolean function.
• Inductive steps.
  1. Primitive application. Each primitive $P$ is itself $⟨p_1,\dots ,p_b⟩$. By IH, each argument $t_j$ yields a tuple of formulas for its bits. Substituting those formulas for the input wires of $P$ gives boolean formulas for the output bits of $t:=P(t_1,\dots ,t_r)$.
  2. Pairing/product. If IH gives formulas for $t_1,\dots ,t_k$, then concatenating the tuples yields formulas for $⟨t_1,\dots ,t_k⟩$.
  3. Projection. Simply select the appropriate subset of formulas from the IH for $t$.
  4. Conditional. Let $b(\mathbf{x})$ be the IH-formula for the guard bit and $\left[t\right]=⟨f_i⟩$, $\left[u\right]=⟨g_i⟩$. Each output bit is $$h_i(\mathbf{x})=(b(\mathbf{x})\wedge f_i(\mathbf{x}))\vee (\neg b(\mathbf{x})\wedge g_i(\mathbf{x})),$$ a boolean formula.

Thus, by induction on the AST, the whole network’s output $\left[\text{net}\right]$ is a product of boolean functions of the input bits.
This also immediately yields a boolean circuit (replace boolean formulas by gates), and hence extensional equality on ${\{0,1\}}^n$.

Reversible Version (with Ancilla)

Neural primitives are generally non-bijective. To embed computation reversibly, use the standard Bennett trick:

• For any function $C:{\{0,1\}}^A\to {\{0,1\}}^B$ realized by a boolean circuit, there exists a reversible circuit $\tilde{C}$ and finite ancilla $z$ such that $$(\mathbf{x},\mathbf{0})\mapsto (\mathbf{x},C(\mathbf{x}),\text{garbage})$$ and by uncomputing (run the circuit that produced garbage in reverse after copying out $C(\mathbf{x})$) we obtain $$(\mathbf{x},\mathbf{0},\mathbf{0})\mapsto (\mathbf{x},C(\mathbf{x}),\mathbf{0}).$$
• Universality: the Toffoli (CCNOT) gate generates all classical reversible computation; with ancillas you can implement AND, OR, XOR, MUX, adders, multipliers, comparators, etc., hence any finite-width primitive and thus the whole network.

Therefore the network’s function embeds into a reversible boolean permutation on a larger space, i.e., a product of reversible boolean functions with ancillas.

Converse Direction (for Completeness)

Any boolean function $f:{\{0,1\}}^n\to \{0,1\}$ can be computed by a small neural net:

• Threshold/step activations: single-layer perceptrons implement AND/OR/NOT; a two-layer net implements XOR; composition yields any circuit.
• ReLU/sigmoid: standard gadget constructions simulate threshold gates with polynomial overhead; therefore any boolean circuit (hence any product of boolean functions) is computable by a finite ReLU/sigmoid net.

Thus we have equivalence up to the chosen bit-encoding.

Edge Cases and Objections

• Infinite precision / real RAM model. If you allow infinite precision reals as a primitive, the state space is uncountable and not boolean-finite; the claim then fails as stated. Remedy: all physical/simulated NNs use finite encodings; choosing that encoding is part of the premise.
• Randomness (dropout, stochastic layers). During inference, networks are typically run deterministically. If randomness remains, you get a stochastic map; you can still encode it as a deterministic boolean function of input bits plus random seed bits (product with extra inputs).
• RNNs / loops. For fixed time horizon $T$, the unrolled function remains finite & extensionally equivalent up to the unrolled time horizon. For unbounded $T$ you are in partial-function/Turing territory; the extensional claim then applies to each finite, terminating run given a step bound.
• Non-measurable niceties / sigmoids with irrational constants. Finite implementation uses rational approximations and fixed tables/polynomials; these are boolean-circuit realizable.

Summary (the "One-Line" Induction)

Fix a finite bit-encoding of all values. Interpret inference as a term in a first-order functional language whose leaves are input bits/constants and whose internal nodes are finite-arity primitives. Each primitive denotes a product of boolean functions; pairing, projection, composition, and conditional preserve "being a product of boolean functions." By structural induction on the AST, the whole network denotes a product of boolean functions of the input bits. Moreover, by Bennett’s method and Toffoli universality, the same computation admits a reversible embedding with ancillas.