A scheme for building generative models using tensor trees — Enabling the determination of causal structures such as phylogenetic trees

Posted on by Katsuya Akamatsu

Building on previous work [1], we proposed [2] a novel approach for building a generative model using a nonnegative tensor tree and demonstrated its ability to capture the hidden structure of various synthetic and real-world datasets. Machine learning models based on tensor networks typically use a Born machine ansatz, where the wavefunction is directly modeled and the probability of a given sample is determined by squaring the wavefunction (the Born rule). However, in this scheme, we cannot directly interpret the model since it is a type of quantum model.

In our work [2], we considered an adaptive tensor tree constrained to have nonnegative elements. We showed that there is an equivalence between our proposed model and hidden Markov models, and that this establishes a basis for the probabilistic interpretation of our proposed model, which we call a “nonnegative adaptive tensor tree” (NATT). We call the tensor tree an adaptive tensor tree because it can change its structure according to the training data. We perform local rearrangements of the network structure favoring

To demonstrate a concrete application of our method, we consider real data drawn from bioinformatics, where we reconstruct a cladogram of species in the taxonomic family Carnivora (dogs, cats, and related species) using mitochondrial DNA sequence data from a specific gene common to all the considered organisms. We found that the resulting network showed a generally correct hierarchical clustering of species even if the only information provided was unlabeled DNA sequence alignment data. The figure shows an example network structure generated using our method. Our method correctly determines the hidden structure of the data by clustering similar species based entirely on genetic information. The advantage of our method is in the observation that the network structure and weights correspond to a probabilistic model that describes the mutation of nucleotides, which means that the generated tensor tree is fully interpretable as a model of a stochastic process. We expect that the NATT would be a useful tool in exploratory data science applications and transparent AI, where developing interpretable generative models for large datasets containing hidden correlation structures is of particular interest.

[1] Kenji Harada, Tsuyoshi Okubo and Naoki Kawashima, “Tensor tree learns hidden relational structures in data to construct generative models”, Mach. Learn.: Sci. Technol. 6 (2025) 025002.
[2] Katsuya O. Akamatsu, Kenji Harada, Tsuyoshi Okubo and Naoki Kawashima, “Plastic tensor networks for interpretable generative modeling”, Mach. Learn.: Sci. Technol. 7 (2026) 015014.

(https://doi.org/10.1088/2632-2153/ae3048)

Generative model using tensor networks — Correlation structure emerges from rise and fall patterns of stock prices

Posted on by Naoki Kawashima

We proposed [1] a new method for constructing a generative model based on a tensor network (TN) and demonstrated its effectiveness. In most cases, generative models are based on neural networks, and optimization of the network structure has not been fully investigated. We considered a Born machine that uses the correspondence between wave functions expressed as TNs and probability distributions represented as their amplitude. Based on this correspondence, we proposed a generative model construction method (adaptive tensor tree, ATT) that optimizes the network structure, and demonstrated its effectiveness. The structure optimization was performed to minimize the mutual information carried by the tree branches. In other words, if the mutual information of each branch decreases when the surrounding branches are rearranged, the rearrangement is performed. As an example, when a generative model was constructed using ATT from data on stock price rise and fall patterns, it was observed that the correlation between stocks was naturally reflected in the network structure as learning progressed. The figure shows a tree structure generated by a Born machine based on an adaptive tensor tree (ATT) learning about 10 years of rise and fall patterns of the S&P 500. Each dot represents a company, and the color changes depending on the industry category (called “sector”). Although sector information was not provided as learning information for ATT, it can be seen that it has “discovered” that there is a close relationship between the sector and the branch structure. Because ATT can build a generative model for any sample, it is expected to be useful for elucidating various correlation structures that were previously difficult to capture, and may provide a new framework for building AI.

[1] Kenji Harada, Tsuyoshi Okubo and Naoki Kawashima, “Tensor tree learns hidden relational structures in data to construct generative models”, Mach. Learn.: Sci. Technol. 6 (2025) 025002.

(https://doi.org/10.1088/2632-2153/adc2c7)

Does the majority rule transformation bring us to the correct fixed point?

Posted on by Katsuya Akamatsu
Fig. 1: A block-spin transformation according to the majority rule performed on an Ising model configuration.

Block-spin transformations (Fig. 1) are a simple way to implement the real-space renormalization group (RSRG) for spin models. They work by dividing the lattice into blocks and determining representative spins for each block according to some rule or map. Two common choices of these maps are decimation, where the spin at a particular site is always chosen to represent the block, and the majority rule, where the most frequent spin in a block is chosen to represent it. By repeatedly applying each of these two maps to a large critical configuration, we can see that there is a qualitative difference in their effect (Fig. 2). Decimation is relatively well-understood, as it can be treated analytically, but despite the popularity and accessibility of the majority rule map, its behavior has not been thoroughly studied analytically or even numerically.

Fig. 2: The effect of repeatedly applying the decimation map (top) and majority rule map (bottom) to the same initial critical configuration. The renormalized configuration generated by the majority rule appears to be “cleaner” than its counterpart.

In our paper [1], we obtained numerical data to answer the question of whether the majority rule map brings us to the correct fixed point. We simulated large critical \(q=2,3,4\) Potts model configurations and repeatedly applied \(2\times2\) RSRG map iterations to the configurations to examine the spin and energy correlation functions across different choices of the RSRG map, including the decimation map and the majority rule map. We found that when the renormalized lattice size \(L_g\) is fixed, allowing the initial lattice size \(L_0\) and number of iterations \(g\) to vary, the RG flow of the aforementioned correlation functions converge to a nontrivial curve with the correct scaling behavior, under the majority rule map on critical Potts configurations (Fig. 3). We denote this property as “faithfulness”, as the RSRG map correctly preserves the critical behavior of the model under repeated iteration. We also provided an explanation for why the majority rule correctly reproduces the scaling of the magnetization in [1].

Fig. 3: Plots of the spin (left) and energy (right) correlation functions for the majority rule map on the critical \(q=2\) Potts (Ising) model, fixing a renormalized lattice size of \(L_g=256\) and allowing \(L_0\) and \(g\) to vary appropriately.

References

  1. K. O. Akamatsu and N. Kawashima, J. Stat. Phys. 191, 109 (2024).

Exact tensor-network ground state of a Bose Hubbard model and its 2+0 dimensional quantum criticality

Posted on by Naoki Kawashima

We considered a Bose-Hubbard model on a two-dimensional lattice. We showed that the model parameters (the lattice structure, the hopping constants, and the chemical potential) can be tuned in such a way that the ground state is generated by applying, to the vacuum, N creation operators of locally-defined quasi-particles, where N is proportional to the total number of lattice points. The resulting model has a free parameter β that controls the degree of quantum fluctuation; β=0 for no fluctuation and β=1/2 for the maximum fluctuation. We carried out Monte Carlo simulation of this model at zero temperature, and discovered that the model undergoes a quantum phase transition when we change this parameter. While the system is 2+1 dimensional, the new phase transition exhibits the characteristics of 2 dimensions, i.e., the Kosterlitz-Thouless phase transition analogous to the classical XY model in 2 dimensions. Accordingly, the large β phase is the quasi Bose-Einstein condensation phase.

The above-mentioned ground state is expressed exactly as a tensor network with the bond dimension 2. Tracing out this tensor network is equivalent to calculating the partition function of some classical loop gas model. This is why the quantum criticality exhibits the purely 2-dimensional characteristics. It also allows us to construct a Monte Carlo algorithm for efficient sampling from the space of classical occupation-number configurations. In other words, we can stochastically trace out the tensor network to obtain various physical quantities of interest, such as the static structure factors and the helicity modulus.

Fig. 1: The lattice considered (p=1). Within each cluster specified by x or u, a tuned hopping between the black and the white sites is defined.

The lattice is generated from the square lattice by duplicating each bond into p copies and introducing an additional site in the middle of each copy. (Fig. 1) The direct hopping is allowed only within local clusters labelled by either x or u. While bosons interact with each other, the model is constructed in such a way that the dispersion of a single particle has a flat band if influence of the other paricles could be neglected. The result of the Monte Carlo simulation are shown in Fig.2. The static structure factor diverges around β=0.2 and, at the same time, the helicity modulus jumps from 0 to 2/π, the universal value characteristic to the KT transition.

Fig. 2. The static structure factor S (top) and the helicity modulus (bottom) as functions of β for p=2. The horizontal line in the bottom panel indicates the universal jump 2/π, the thermodynamic value characteristic to the KT transition point.
[1] H. Katsura, N. Kawashima, S. Morita, A. Tanaka and H. Tasaki: Physical Review Research, 3 033190 (2021)

A classical loop-gas model represents the Kitaev spin liquid

Posted on by Naoki Kawashima

Quantum spin liquids (QSLs) represent a state of quantum matter which is not characterized by any local order parameters even at zero temperature. These novel states are expected to exhibit non-trivial entanglement structure leading to the topological order and fractionalized excitations. For example, the Haldane phase, which is known as a symmetry-protected topological phase, is a fascinating phase one can find in the S=1 quantum spin chain. The novel character that discriminates the Haldane phase from trivial gapped states was clearly revealed by the discovery of Affleck-Kennedy-Lieb-Tasaki (AKLT) model and its exact ground state known as the AKLT state. Its compact representation in the form of a matrix-product state provided a new insight into the Haldane phase. As a two-dimensional system with spin-liquid state, the Kitaev model on honeycomb lattice (KHM) is now a subject of active research, due to recent successful realizations of Kitaev materials. The model is exactly solvable and exhibits gapless and gapped Kitaev spin liquid (KSL) ground states with fractionalized excitations. One may naturally wonder if there is a simple tensor-network representation that captures the essence of the KSL, i.e., a 2D analogue of the correspondence between the AKLT state and the Haldane state.

In this work, we propose a “loop-gas” state, i.e., an exactly-solvable quantum state that has a compact and exact tensor-network representation, as well as a series of ansatze that starts from the loop-gas state and converges rapidly to the KHM ground-state. The loop gas state is defined by the loop-gas projector applied to a product state in which all spins are aligned in the (111) direction. The loop-gas projector is defined as tensor-network operator with bond-dimension 2. Specifically, it is expressed as a sum of all loop configurations, each corresponding to a product of Pauli operators on all lattice points. (Fig.1) We find that the norm of the loop gas state has exactly the same as the classical loop gas model at the critical fugacity on honeycomb lattice. The classical loop gas model appears in the low-temperature expansion of the Ising model, and therefore shares the same critical properties as the latter. These observations establishes that the correlation function of the loop-gas state is exactly the same as the correlation function of the critical Ising model. Furthermore, by defining the dimer-gas operator in an analogous way to the loop-gas operator, and applying it to the loop-gas state, we can improve the loop-gas state as an approximate to the KHM ground-state. (Fig.2) For example, the second order approximation, which is obtained by applying the dimer-gas operator twice, achieves more than 4-digits accuracy in the energy, which is better than any other previous variational wave function. Also, note that there are only two adjustable parameters in the ansatz.

From these results, we conclude that the series of ansatze starting from the loop-gas state correctly reflects the essential properties of the family of KSLs, which is known to belong to the Ising universality class. Since the present description of KSLs does not rely on the mapping to fermions, it may provide us with an alternative view point for studying KSLs.

References
[1] Hyun-Yong Lee, Ryui Kaneko, Tsuyoshi Okubo and Naoki Kawashima: Phys. Rev. Lett. 123, 087203 (2019)  (arXiv:1901.05786).

Fig. 1: Schematic representation of the “loop-gas” state. It is a sum of all loop-gas configurations. Each configuration corresponds to a product of Pauli operators. The loop-gas configuration determines which Pauli operator is assigned on a given lattice point.
Fig. 2: The series of ansatze converging to the exact KHM ground-state. The first (the zero-th order) one is the loop-gas state. The energy of the second order approximation with only 2 parameters already attains 4-digit accuracy, which has never been achieved by other numerical variational methods with many variational parameters.

Boundary CFT and tensor network approach to surface critical phenomena of the tricritical 3-state Potts model

Posted on by Shumpei Iino

It is widely known that there is no spontaneous symmetry breaking in 1d classical systems at finite temperature, which suggests that 1d classical many body problems are almost trivial and boring.
Then, let us consider 1d classical `edges’ in 2d systems, or 1d classical systems attached to a 2d bulk.
Does something interesting happen in these 1d edges?
In fact, the situation is a bit different from the simple 1d cases.

When the 2d bulk is at criticality, in particular, the strong correlation at the bulk helps the edge ordering, which results in a variety of phase transitions on the 1d edges.
For example, the 2d classical tricritical Ising model exhibits a finite-surface-temperature symmetry breaking on its 1d edge, when the bulk is fine-tuned at the tricritical point [1].
It is also known that another surface transition occurs driven by the surface external fields.

As a theoretical way of studying surface criticality, boundary conformal field theory (BCFT) is very powerful, particularly for 2d classical / 1+1d quantum systems at criticality.
It could be essential to understand surface critical behavior precisely, since the restriction of the conformal invariance may enable us not only to classify the conformal boundary conditions but also to investigate stability of boundary fixed points and exact values of scaling dimensions.

Now, let us consider an extension of the above classical tricritical Ising model in 2d, called the tricritical 3-state Potts model, whose symmetry of the spin is \(S_3\) rather than \(Z_2\).
The Monte Carlo simulation of this model suggests, similarly to the tricritical Ising, rich surface phase transitions when the bulk is at tricriticality [2].
However, the precise study from the view point of BCFT has been missing for a long time.

In order to investigate the more detailed surface phase diagram of this model, we utilize the \(ADE\) classification of the minimal-series BCFTs and perform numerical simulation with tensor network renormaization (TNR) [3,4,5].
The tricritical 3-state Potts model can be described by the \(D\)-type minimal CFT with \(c=6/7\), whose modular invariant partition function is labeled as the pair of Lie algebra \((D_4, A_6)\) in the \(ADE\) classification.
Using the complete classification of the conformal boundary states in Ref. [3], one can derive the twelve conformal boundary conditions labeled by the nodes of the Dynkin diagrams \(D_4\) and \(A_6\).
The triality of \(D_4\) implies that those can be classified into three \(Z_3\)-symmetric boundary states and nine \(Z_3\)-broken ones.

To understand the physical picture of the above twelve boundary fixed points, we simulate the 3-state dilute Potts model on lattices with the TNR technique.
The extended TNR method suited for open-boundary systems enables us to extract accurate conformal spectrum from lattice models, by which one can study the correspondence between the boundary fixed points from BCFT and the surface phases in the phase diagram of the tricritical 3-state Potts model.
Our numerical analysis reveals that the eleven boundary fixed points among the twelve can be realized on the lattice by controlling the external field and coupling strength at the boundary.
The last unfound fixed point would be out of the physically sound region in the parameter space, and we leave it an open question to consider the realization of that boundary condition on the lattice model.
(by Shumpei Iino)

Fig. The surface phase diagram of the tricritical 3-state Potts model obtained by the TNR simulation.

References:
[1] I. Affleck, J. Phys. A 33(37), 6473 (2000).
[2] Y. Deng and H. W. J. Blöte, Phys. Rev. E 70, 035107 (2004); Phys. Rev. E 71, 026109 (2005).
[3] R. E. Behrend, P. A. Pearce, V. B. Petkova, and J.-B. Zuber, Nucl. Phys. B 579(3), 707 (2000).
[4] G. Evenbly and G. Vidal, Phys. Rev. Lett. 115, 180405 (2015); S. Iino, S. Morita, and N. Kawashima, Phys. Rev. B 101, 155418 (2020).
[5] S. Iino, arXiv:2007.03182; J. Stat. Phys. 182, 56 (2021).

Textbook renormalization group prescription in tensor-network language

Posted on by Xinliang Lyu

The Kadanoff’s block-spin method gives a nice and simple physical picture for renormalization group (RG) transformations. Combined with Migdal’s bond-moving trick, it serves as a first approximation to estimate phase diagram and critical exponents. However, it is difficult to improve the estimations systematically.

Modern tensor-network-based methods open up new possibilities for RG transformations. They generalize the conventional real-space RG transformations and are more versatile and systematically improvable. They are most convenient for calculations of free energy.

In this paper [1], we fully exploit the RG interpretation of these tensor-network-based methods and show a way to carry out the textbook RG prescription in tensor-network language: identify a fixed point, linearize the RG equation around this fixed point, and diagonalize this linearized RG equation to obtain scaling dimensions, to which the critical exponents are related. The proposed method works equally well compared with the current bread-and-butter tensor method for extracting scaling dimensions in the context of the 2D classical Ising model. The advantage is its potential applications to 3D systems, where the bread-and-butter method is inapplicable.

Figure 1: Schematic diagram of the linearized tensor RG equation. In the proposed method, it can be generated using automatic differentiation once the coarse graining is implemented.

References

  1. X. Lyu, R.Q.G. Xu, and N. Kawashima, Phys. Rev. Research 3, 023048 (2021)

Fast Algorithm for Generating Random Bit Strings

Posted on by Hiroshi Watanabe

Suppose we want to generate a bit string with length \(N\) in which each bit is set with arbitrary probability \(p\). If we adopt the simple algorithm, which repeats check for each bit, we need to generate \(N\) random numbers. However, we can reduce the number of random number generations by adopting more sophisticated algorithms [1]. We proposed three algorithm, the Binomial-Shuffle, the Poisson-OR, and the finite-digit algorithms.

Fig. 1: Three algorithms to generate a random bit string.

The Binomial-Shuffle algorithm is an algorithm that first calculates how many bits are set and then shuffles them. Adopting Walker’s alias method and Floyd’s sampling method, it is possible to generate a random bit string with \(pN+1\) random number generations. The Poisson-OR algorithm generates \(k\) bit strings with one of the bits is set randomly and takes the logical OR of them. This is a special case of the algorithm proposed by Todo and Suwa [2]. The average number of the random number generations is \(-N \log(1-p)+1\). The finite digit method is effective when the probability \(p\) can be represented by a finite digit in the binary notation. When the probability p can be expressed by \(n\) digits in the binary notation, we can generate a random bit string with that probability by generating \(n\) bit strings in which each bit is set with a probability \(1/2\) and properly taking the logical operations. Therefore, the finite-digit algorithm with n digits involves the random number generates \(n\) times.

While the Binomial-Shuffle and the Poisson-OR algorithms are effective when the value of probability \(p\) is small, the required number of random numbers increases as \(p\) increases. Meanwhile, the required number for the finite-digit method does not depend on \(p\), although the method cannot be applied directly unless p can be expressed by \(n\) digits in the binary notation. Therefore, we constructed a hybrid algorithm, i.e., which first generates a bit string with probability \(\tilde{p}\) close to \(p\) with the finite digit algorithm and corrects the difference \(\tilde{p}-p\) by the Binomial-Shuffle or Poisson-OR algorithms. The expected number of random numbers generated to generate a string in which each bit is set with arbitrary probability is at most 7 for the 32-bit case and 8 for 64-bit case.

Fig. 2: Multi-spin coding of the directed-percolation. Times required for the simulations with 32768 steps and 1000 independent samples with L = 32768. The cases for the scalar implementation (Scalar), finite digit + Binomial-Shuffle (BS) algorithms, and finite digit + Poisson-OR (PO) algorithms are shown both for 32- and 64-bits.

Employing the developed algorithm, we applied the multispin coding technique to the one-dimensional bond-directed percolation and obtained a factor of 14 speed-up.

References

  1. H. Watanabe, S. Morita, S. Todo, and N. Kawashima, J. Phys. Soc. Jpn. 88, 024004 (2019)
  2. S. Todo and H. Suwa, J. Phys.: Conf. Ser. 473, 012013 (2013).

Spin Thermal Hall Conductivity of a Kagome Antiferromagnet

Posted on by Hyunyong Lee

After the discovery of the high-temperature superconductors, their parent compounds are conjectured to be in a spin liquid phase which becomes superconducting when charge carriers are doped. Spin liquid is a highly entangled quantum disordered state. Due to its strong geometrical frustration, the Kagome quantum magnets are expected to host such novel phase. Searching for an ideal Kagome quantum magnet has led to recent discovery of new materials from structural polymorphs of herbertsmithite, i.e., Ca-kapellasite which is described in Fig.1(a) and (b).

Figure 1. Crystal structure of Ca kapellasite viewed along the c axis (a) and a axis (b). (c) The illustration of thermal Hall experiment setup.

However, according to exhaustive theoretical investigations, it may host many different types of spin liquid phase including RVB, Z2 and Dirac spin liquids. Therefore, it is very important to find the elementary excitations expected to emerge in these unknown phases through experiments on ideal Kagome materials, though it has been remained as a very challenging problem. As an evidence of emergence of spin liquid phase and its excitation, the thermal Hall response, which is an analogue of Hall response in electronic system, has been suggested and, in fact, the finite thermal Hall conductivity was measured. However, its origin was not so clear since the phonon also can transport the heat.

In this study, a clear thermal Hall signal was observed in the spin-liquid phase of the S=1/2 kagome antiferromagnet Ca-kapellasite and provide a strong evidence suggesting the heat is carried by the spinon excitation on top of the spin liquid state. In order to obtain a theoretical estimation of thermal Hall conductivity, we employ the Schwinger-Boson mean field theory (SBMFT) and the U(1) spin liquid ansatz. The thermal Hall conductivity can be measured by the following formula

Where c2 is a distribution function of the Schwinger boson, and nB the Bose-Einstein distribution function, E is the Schwinger boson band, and Ω is the corresponding Berry curvature.  Remarkably, the (dimensionless) thermal Hall conductivity obtained from the SBMFT is perfectly fitted by the ones obtained from the experiment as shown in Fig. 2.

Figure 2. Dimensionless thermal Hall conductivity.

Also, we find all observed thermal Hall conductivity curves from many samples converge onto a single curve by adjusting only the exchange interaction and Dzyloshinkii-Moriya interaction, which suggests that it has a common temperature dependence on the Kagome quantum magnet. This excellent agreement, both qualitative and quantitative, demonstrates that the thermal Hall effect in these kagome antiferromagnets derives from spins in the spin-liquid phase.

Reference

Hayato Doki, Masatoshi Akazawa, Hyun-Yong Lee, Jung Hoon Han, Kaori Sugii, Masaaki Shimozawa, Naoki Kawashima, Migaku Oda, Hiroyuki Yoshida, and Minoru Yamashita, Phys. Rev. Lett. 121, 097203 (2018)

Tensor Renormalization Group with Randomized Algorithm for Singular Value Decomposition

Posted on by Satoshi Morita

Tensor network methods are becoming powerful tools in the study of many-body problems. Real-space renormalization by coarse-graining tensor networks, including tensor renormalization group (TRG) [1] and its derivatives, is an efficient numerical method for the contraction of tensor networks. Accuracy of approximation is improved by increasing the bond dimension. However the computational complexity of TRG is proportional to the sixth power of the bond dimension \(\chi\). Thus complexity reduction without loss of accuracy is desired.

Fig. 1: The chain of deformation in the TRG algorithm. Our improved algorithm skips the intermediate step of calculating the fourth-order tensor

We proposed a new algorithm of TRG based on a randomized algorithm for singular value decomposition (RSVD) [2]. Its computational complexity scales as \(O(\chi^5)\). The main operations in TRG are “decomposition” and “contraction”. Since the both have \(O(\chi^6)\) complexity, we need to reduce the complexity not only by replacing the full SVD with RSVD but also by avoiding the explicit creation of a four-leg tensor. An oversampling parameter tunes the statistical error of RSVD. We showed that the oversampling parameter larger than the bond dimension reproduced the same result as the full SVD in two-dimensional Ising model. [3]

Fig. 2: Elapsed time per TRG step as a function of bond dimension.

Since “decomposition” and “contraction” are dominant operations in most tensor network schemes, the techniques presented here would be useful in improving most of the tensor network calculations in an essential way.

References

  1. M. Levin and C. P. Nave, Phys. Rev. Lett. 99, 120601 (2007).
  2. N. Halko, P. G. Martinsson, and J. A. Tropp, SIAM Rev. 53, 217 (2011).
  3. S. Morita, R. Igarashi, H.-H. Zhao, and N. Kawashima, Phys. Rev. E 97, 033310 (2018).