fbpx
Wikipedia

Low-density parity-check code

In information theory, a low-density parity-check (LDPC) code is a linear error correcting code, a method of transmitting a message over a noisy transmission channel.[1][2] An LDPC code is constructed using a sparse Tanner graph (subclass of the bipartite graph).[3] LDPC codes are capacity-approaching codes, which means that practical constructions exist that allow the noise threshold to be set very close to the theoretical maximum (the Shannon limit) for a symmetric memoryless channel. The noise threshold defines an upper bound for the channel noise, up to which the probability of lost information can be made as small as desired. Using iterative belief propagation techniques, LDPC codes can be decoded in time linear in their block length.

LDPC codes are also known as Gallager codes, in honor of Robert G. Gallager, who developed the LDPC concept in his doctoral dissertation at the Massachusetts Institute of Technology in 1960.[4][5] However, LDPC codes require computationally expensive iterative decoding, so they went unused for decades. In 1993 the newly invented turbo codes demonstrated that codes with iterative decoding could far outperform other codes used at that time, but turbo codes were patented and required a fee for use. This raised renewed interest in LDPC codes, which were shown to have similar performance, but were much older and patent-free.[6] Now that the fundamental patent for turbo codes has expired (on August 29, 2013),[7][8] LDPC codes are still used for their technical merits.

LDPC codes have been shown to have ideal combinatorial properties. In his dissertation, Gallager showed that LDPC codes achieve the Gilbert–Varshamov bound for linear codes over binary fields with high probability. In 2020 it was shown that Gallager's LDPC codes achieve list decoding capacity and also achieve the Gilbert–Varshamov bound for linear codes over general fields.[9]

History edit

Impractical to implement when first developed by Gallager in 1963,[10] LDPC codes were forgotten until his work was rediscovered in 1996.[11] Turbo codes, another class of capacity-approaching codes discovered in 1993, became the coding scheme of choice in the late 1990s, used for applications such as the Deep Space Network and satellite communications. LDPC codes then received renewed interest as a patent-free alternative of similar performance.[6] Since then, advances in low-density parity-check codes have seen them surpass turbo codes in terms of error floor and performance in the higher code rate range, leaving turbo codes better suited for the lower code rates only.[12]

Applications edit

In 2003, an irregular repeat accumulate (IRA) style LDPC code beat six turbo codes to become the error-correcting code in the new DVB-S2 standard for digital television.[13] The DVB-S2 selection committee made decoder complexity estimates for the turbo code proposals using a much less efficient serial decoder architecture rather than a parallel decoder architecture. This forced the turbo code proposals to use frame sizes on the order of one half the frame size of the LDPC proposals.[citation needed]

In 2008, LDPC beat convolutional turbo codes as the forward error correction (FEC) system for the ITU-T G.hn standard.[14] G.hn chose LDPC codes over turbo codes because of their lower decoding complexity (especially when operating at data rates close to 1.0 Gbit/s) and because the proposed turbo codes exhibited a significant error floor at the desired range of operation.[15]

LDPC codes are also used for 10GBASE-T Ethernet, which sends data at 10 gigabits per second over twisted-pair cables. As of 2009, LDPC codes are also part of the Wi-Fi 802.11 standard as an optional part of 802.11n and 802.11ac, in the High Throughput (HT) PHY specification.[16] LDPC is a mandatory part of 802.11ax (Wi-Fi 6).[17]

Some OFDM systems add an additional outer error correction that fixes the occasional errors (the "error floor") that get past the LDPC correction inner code even at low bit error rates.

For example: The Reed-Solomon code with LDPC Coded Modulation (RS-LCM) uses a Reed-Solomon outer code.[18] The DVB-S2, the DVB-T2 and the DVB-C2 standards all use a BCH code outer code to mop up residual errors after LDPC decoding.[19]

5G NR uses polar code for the control channels and LDPC for the data channels.[20][21]

Although LDPC code has had its success in commercial hard disk drives, to fully exploit its error correction capability in SSDs demands unconventional fine-grained flash memory sensing, leading to an increased memory read latency. LDPC-in-SSD[22] is an effective approach to deploy LDPC in SSD with a very small latency increase, which turns LDPC in SSD into a reality. Since then, LDPC has been widely adopted in commercial SSDs in both customer-grades and enterprise-grades by major storage venders. Many TLC (and later) SSDs are using LDPC codes. A fast hard-decode (binary erasure) is first attempted, which can fall back into the slower but more powerful soft decoding.[23]

Operational use edit

LDPC codes functionally are defined by a sparse parity-check matrix. This sparse matrix is often randomly generated, subject to the sparsity constraints—LDPC code construction is discussed later. These codes were first designed by Robert Gallager in 1960.[5]

Below is a graph fragment of an example LDPC code using Forney's factor graph notation. In this graph, n variable nodes in the top of the graph are connected to (nk) constraint nodes in the bottom of the graph.

This is a popular way of graphically representing an (nk) LDPC code. The bits of a valid message, when placed on the T's at the top of the graph, satisfy the graphical constraints. Specifically, all lines connecting to a variable node (box with an '=' sign) have the same value, and all values connecting to a factor node (box with a '+' sign) must sum, modulo two, to zero (in other words, they must sum to an even number; or there must be an even number of odd values).

 

Ignoring any lines going out of the picture, there are eight possible six-bit strings corresponding to valid codewords: (i.e., 000000, 011001, 110010, 101011, 111100, 100101, 001110, 010111). This LDPC code fragment represents a three-bit message encoded as six bits. Redundancy is used, here, to increase the chance of recovering from channel errors. This is a (6, 3) linear code, with n = 6 and k = 3.

Again ignoring lines going out of the picture, the parity-check matrix representing this graph fragment is

 

In this matrix, each row represents one of the three parity-check constraints, while each column represents one of the six bits in the received codeword.

In this example, the eight codewords can be obtained by putting the parity-check matrix H into this form   through basic row operations in GF(2):

 

Step 1: H.

Step 2: Row 1 is added to row 3.

Step 3: Row 2 and 3 are swapped.

Step 4: Row 1 is added to row 3.

From this, the generator matrix G can be obtained as   (noting that in the special case of this being a binary code  ), or specifically:

 

Finally, by multiplying all eight possible 3-bit strings by G, all eight valid codewords are obtained. For example, the codeword for the bit-string '101' is obtained by:

 ,

where   is symbol of mod 2 multiplication.

As a check, the row space of G is orthogonal to H such that  

The bit-string '101' is found in as the first 3 bits of the codeword '101011'.

Example encoder edit

 
LDPC encoder

During the encoding of a frame, the input data bits (D) are repeated and distributed to a set of constituent encoders. The constituent encoders are typically accumulators and each accumulator is used to generate a parity symbol. A single copy of the original data (S0,K-1) is transmitted with the parity bits (P) to make up the code symbols. The S bits from each constituent encoder are discarded.

The parity bit may be used within another constituent code.

In an example using the DVB-S2 rate 2/3 code the encoded block size is 64800 symbols (N=64800) with 43200 data bits (K=43200) and 21600 parity bits (M=21600). Each constituent code (check node) encodes 16 data bits except for the first parity bit which encodes 8 data bits. The first 4680 data bits are repeated 13 times (used in 13 parity codes), while the remaining data bits are used in 3 parity codes (irregular LDPC code).

For comparison, classic turbo codes typically use two constituent codes configured in parallel, each of which encodes the entire input block (K) of data bits. These constituent encoders are recursive convolutional codes (RSC) of moderate depth (8 or 16 states) that are separated by a code interleaver which interleaves one copy of the frame.

The LDPC code, in contrast, uses many low depth constituent codes (accumulators) in parallel, each of which encode only a small portion of the input frame. The many constituent codes can be viewed as many low depth (2 state) "convolutional codes" that are connected via the repeat and distribute operations. The repeat and distribute operations perform the function of the interleaver in the turbo code.

The ability to more precisely manage the connections of the various constituent codes and the level of redundancy for each input bit give more flexibility in the design of LDPC codes, which can lead to better performance than turbo codes in some instances. Turbo codes still seem to perform better than LDPCs at low code rates, or at least the design of well performing low rate codes is easier for turbo codes.

As a practical matter, the hardware that forms the accumulators is reused during the encoding process. That is, once a first set of parity bits are generated and the parity bits stored, the same accumulator hardware is used to generate a next set of parity bits.

Decoding edit

As with other codes, the maximum likelihood decoding of an LDPC code on the binary symmetric channel is an NP-complete problem,[24] shown by reduction from 3-dimensional matching. So assuming P != NP, which is widely believed, then performing optimal decoding for an arbitrary code of any useful size is not practical.

However, sub-optimal techniques based on iterative belief propagation decoding give excellent results and can be practically implemented. The sub-optimal decoding techniques view each parity check that makes up the LDPC as an independent single parity check (SPC) code. Each SPC code is decoded separately using soft-in-soft-out (SISO) techniques such as SOVA, BCJR, MAP, and other derivates thereof. The soft decision information from each SISO decoding is cross-checked and updated with other redundant SPC decodings of the same information bit. Each SPC code is then decoded again using the updated soft decision information. This process is iterated until a valid codeword is achieved or decoding is exhausted. This type of decoding is often referred to as sum-product decoding.

The decoding of the SPC codes is often referred to as the "check node" processing, and the cross-checking of the variables is often referred to as the "variable-node" processing.

In a practical LDPC decoder implementation, sets of SPC codes are decoded in parallel to increase throughput.

In contrast, belief propagation on the binary erasure channel is particularly simple where it consists of iterative constraint satisfaction.

For example, consider that the valid codeword, 101011, from the example above, is transmitted across a binary erasure channel and received with the first and fourth bit erased to yield ?01?11. Since the transmitted message must have satisfied the code constraints, the message can be represented by writing the received message on the top of the factor graph.

In this example, the first bit cannot yet be recovered, because all of the constraints connected to it have more than one unknown bit. In order to proceed with decoding the message, constraints connecting to only one of the erased bits must be identified. In this example, only the second constraint suffices. Examining the second constraint, the fourth bit must have been zero, since only a zero in that position would satisfy the constraint.

This procedure is then iterated. The new value for the fourth bit can now be used in conjunction with the first constraint to recover the first bit as seen below. This means that the first bit must be a one to satisfy the leftmost constraint.

 

Thus, the message can be decoded iteratively. For other channel models, the messages passed between the variable nodes and check nodes are real numbers, which express probabilities and likelihoods of belief.

This result can be validated by multiplying the corrected codeword r by the parity-check matrix H:

 

Because the outcome z (the syndrome) of this operation is the three × one zero vector, the resulting codeword r is successfully validated.

After the decoding is completed, the original message bits '101' can be extracted by looking at the first 3 bits of the codeword.

While illustrative, this erasure example does not show the use of soft-decision decoding or soft-decision message passing, which is used in virtually all commercial LDPC decoders.

Updating node information edit

In recent years[when?], there has also been a great deal of work spent studying the effects of alternative schedules for variable-node and constraint-node update. The original technique that was used for decoding LDPC codes was known as flooding. This type of update required that, before updating a variable node, all constraint nodes needed to be updated and vice versa. In later work by Vila Casado et al.,[25] alternative update techniques were studied, in which variable nodes are updated with the newest available check-node information.[citation needed]

The intuition behind these algorithms is that variable nodes whose values vary the most are the ones that need to be updated first. Highly reliable nodes, whose log-likelihood ratio (LLR) magnitude is large and does not change significantly from one update to the next, do not require updates with the same frequency as other nodes, whose sign and magnitude fluctuate more widely.[citation needed] These scheduling algorithms show greater speed of convergence and lower error floors than those that use flooding. These lower error floors are achieved by the ability of the Informed Dynamic Scheduling (IDS)[25] algorithm to overcome trapping sets of near codewords.[26]

When nonflooding scheduling algorithms are used, an alternative definition of iteration is used. For an (nk) LDPC code of rate k/n, a full iteration occurs when n variable and n − k constraint nodes have been updated, no matter the order in which they were updated.[citation needed]

Code construction edit

For large block sizes, LDPC codes are commonly constructed by first studying the behaviour of decoders. As the block size tends to infinity, LDPC decoders can be shown to have a noise threshold below which decoding is reliably achieved, and above which decoding is not achieved,[27] colloquially referred to as the cliff effect. This threshold can be optimised by finding the best proportion of arcs from check nodes and arcs from variable nodes. An approximate graphical approach to visualising this threshold is an EXIT chart.[citation needed]

The construction of a specific LDPC code after this optimization falls into two main types of techniques:[citation needed]

  • Pseudorandom approaches
  • Combinatorial approaches

Construction by a pseudo-random approach builds on theoretical results that, for large block size, a random construction gives good decoding performance.[11] In general, pseudorandom codes have complex encoders, but pseudorandom codes with the best decoders can have simple encoders.[28] Various constraints are often applied to help ensure that the desired properties expected at the theoretical limit of infinite block size occur at a finite block size.[citation needed]

Combinatorial approaches can be used to optimize the properties of small block-size LDPC codes or to create codes with simple encoders.[citation needed]

Some LDPC codes are based on Reed–Solomon codes, such as the RS-LDPC code used in the 10 Gigabit Ethernet standard.[29] Compared to randomly generated LDPC codes, structured LDPC codes—such as the LDPC code used in the DVB-S2 standard—can have simpler and therefore lower-cost hardware—in particular, codes constructed such that the H matrix is a circulant matrix.[30]

Yet another way of constructing LDPC codes is to use finite geometries. This method was proposed by Y. Kou et al. in 2001.[31]

Compared to turbo codes edit

LDPC codes can be compared with other powerful coding schemes, e.g. turbo codes.[32] In one hand, BER performance of turbo codes is influenced by low codes limitations.[33] LDPC codes have no limitations of minimum distance,[34] that indirectly means that LDPC codes may be more efficient on relatively large code rates (e.g. 3/4, 5/6, 7/8) than turbo codes. However, LDPC codes are not the complete replacement: turbo codes are the best solution at the lower code rates (e.g. 1/6, 1/3, 1/2).[35][36]

See also edit

People edit

Theory edit

Applications edit

  • G.hn/G.9960 (ITU-T Standard for networking over power lines, phone lines and coaxial cable)
  • 802.3an or 10GBASE-T (10 gigabit/s Ethernet over twisted pair)
  • CMMB (China Multimedia Mobile Broadcasting)
  • DVB-S2 / DVB-T2 / DVB-C2 (digital video broadcasting, 2nd generation)
  • DMB-T/H (digital video broadcasting)[37]
  • WiMAX (IEEE 802.16e standard for microwave communications)
  • IEEE 802.11n-2009 (Wi-Fi standard)
  • DOCSIS 3.1
  • ATSC 3.0 (Next generation North America digital terrestrial broadcasting)
  • 3GPP (5G-NR data channel)

Other capacity-approaching codes edit

Capacity-achieving codes edit

So far there is only one capacity achieving code by design and proof.

References edit

  1. ^ MacKay, David J. (2003). Information theory, Inference and Learning Algorithms. Cambridge University Press. ISBN 0-521-64298-1.
  2. ^ Moon, Todd K. (2005). Error Correction Coding, Mathematical Methods and Algorithms. Wiley. ISBN 0-471-64800-0. (Includes code)
  3. ^ Amin Shokrollahi, (PDF), archived from the original (PDF) on May 17, 2017
  4. ^ Hardesty, L. (January 21, 2010). "Explained: Gallager codes". MIT News. Retrieved August 7, 2013.
  5. ^ a b Gallager, R.G. (January 1962). "Low density parity check codes". IRE Trans. Inf. Theory. 8 (1): 21–28. doi:10.1109/TIT.1962.1057683. hdl:1721.1/11804/32786367-MIT. S2CID 260490814.
  6. ^ a b Erico Guizzo (March 1, 2004). "CLOSING IN ON THE PERFECT CODE". IEEE Spectrum. "Another advantage, perhaps the biggest of all, is that the LDPC patents have expired, so companies can use them without having to pay for intellectual-property rights."
  7. ^ US 5446747 
  8. ^ Mackenzie, D. (July 9, 2005). "Communication speed nears terminal velocity". New Scientist.
  9. ^ Mosheiff, J.; Resch, N.; Ron-Zewi, N.; Silas, S.; Wootters, M. (2020). "Low-Density Parity-Check Codes Achieve List-Decoding Capacity". SIAM Journal on Computing (FOCS 2020): 38–73. doi:10.1137/20M1365934. S2CID 244549036.
  10. ^ Gallager, Robert G. (1963). Low Density Parity Check Codes (PDF). M.I.T. Press. Retrieved August 7, 2013.
  11. ^ a b David J.C. MacKay and Radford M. Neal, "Near Shannon Limit Performance of Low Density Parity Check Codes," Electronics Letters, July 1996
  12. ^ Telemetry Data Decoding, Design Handbook
  13. ^ Presentation by Hughes Systems 2006-10-08 at the Wayback Machine
  14. ^ HomePNA Blog: G.hn, a PHY For All Seasons
  15. ^ IEEE Communications Magazine paper on G.hn 2009-12-13 at the Wayback Machine
  16. ^ IEEE Standard, section 20.3.11.6 "802.11n-2009", IEEE, October 29, 2009, accessed March 21, 2011.
  17. ^ "IEEE SA - IEEE 802.11ax-2021". IEEE Standards Association. Retrieved May 22, 2022.
  18. ^ Chih-Yuan Yang, Mong-Kai Ku. http://123seminarsonly.com/Seminar-Reports/029/26540350-Ldpc-Coded-Ofdm-Modulation.pdf "LDPC coded OFDM modulation for high spectral efficiency transmission"
  19. ^ Nick Wells. "DVB-T2 in relation to the DVB-x2 Family of Standards" 2013-05-26 at the Wayback Machine
  20. ^ (PDF). Archived from the original (PDF) on December 6, 2018. Retrieved January 6, 2019.
  21. ^ Maunder, Robert (September 2016). (PDF). Archived from the original (PDF) on December 6, 2018. Retrieved January 6, 2019.
  22. ^ Kai Zhao; Wenzhe Zhao; Hongbin Sun; Tong Zhang; Xiaodong Zhang; Nanning Zheng (2013). LDPC-in-SSD: Making Advanced Error Correction Codes Work Effectively in Solid State Drives (PDF). FAST' 13. pp. 243–256.
  23. ^ "Soft-Decoding in LDPC based SSD Controllers". EE Times. 2015.
  24. ^ Robert McEliece, E. R. Berlekamp and H. Van Tilborg (1978). "On the Inherent Intractability of Certain Coding Problems". IEEE Trans. Inf. Theory. IEEE: 384–386. doi:10.1109/TIT.1978.1055873.
  25. ^ a b Casado, A.I.V.; Griot, M.; Wesel, R.D. (2007). Informed Dynamic Scheduling for Belief-Propagation Decoding of LDPC Codes. 2007 IEEE International Conference on Communications, Glasgow, UK. pp. 932–7. arXiv:cs/0702111. doi:10.1109/ICC.2007.158.
  26. ^ Richardson, T. (October 2003). "Error floors of LDPC codes" (PDF). Proceedings of the Annual Allerton Conference on Communication Control and Computing. 41 (3): 1426–35. ISSN 0732-6181.
  27. ^ Richardson, T.J.; Shokrollahi, M.A.; Urbanke, R.L. (February 2001). "Design of capacity-approaching irregular low-density parity-check codes". IEEE Transactions on Information Theory. 47 (2): 619–637. doi:10.1109/18.910578.
  28. ^ Richardson, T.J.; Urbanke, R.L. (February 2001). "Efficient encoding of low-density parity-check codes". IEEE Transactions on Information Theory. 47 (2): 638–656. doi:10.1109/18.910579.
  29. ^ Ahmad Darabiha, Anthony Chan Carusone, Frank R. Kschischang. "Power Reduction Techniques for LDPC Decoders"
  30. ^ Zhang, Z.; Anantharam, V.; Wainwright, M.J.; Nikolic, B. (April 2010). "An Efficient 10GBASE-T Ethernet LDPC Decoder Design With Low Error Floors" (PDF). IEEE Journal of Solid-State Circuits. 45 (4): 843–855. Bibcode:2010IJSSC..45..843Z. doi:10.1109/JSSC.2010.2042255. S2CID 10431486.
  31. ^ Kou, Y.; Lin, S.; Fossorier, M.P.C. (November 2001). "Low-density parity-check codes based on finite geometries: a rediscovery and new results". IEEE Transactions on Information Theory. 47 (7): 2711–36. CiteSeerX 10.1.1.100.3023. doi:10.1109/18.959255.
  32. ^ Tahir, B.; Schwarz, S.; Rupp, M. (2017). BER comparison between Convolutional, Turbo, LDPC, and Polar codes. 2017 24th International Conference on Telecommunications (ICT), Limassol, Cyprus. pp. 1–7. doi:10.1109/ICT.2017.7998249.
  33. ^ Moon Todd, K. (2005). Error correction coding: mathematical methods and algorithms. Wiley. p. 614. ISBN 0-471-64800-0.
  34. ^ Moon Todd 2005, p. 653
  35. ^ Andrews, Kenneth S., et al. "The development of turbo and LDPC codes for deep-space applications." Proceedings of the IEEE 95.11 (2007): 2142-2156.
  36. ^ Hassan, A.E.S., Dessouky, M., Abou Elazm, A. and Shokair, M., 2012. Evaluation of complexity versus performance for turbo code and LDPC under different code rates. Proc. SPACOMM, pp.93-98.
  37. ^ . spectrum.ieee.org. Archived from the original on December 12, 2009.

External links edit

  • Introducing Low-Density Parity-Check Codes (by Sarah J Johnson, 2010)
  • LDPC Codes – a brief Tutorial (by Bernhard Leiner, 2005)
  • LDPC Codes (TU Wien) February 28, 2019, at the Wayback Machine
  • MacKay, David J.C. (September 25, 2003). "47. Low-Density Parity-Check Codes". Information Theory, Inference, and Learning Algorithms. Cambridge University Press. pp. 557–573. ISBN 9780521642989.
  • Guruswami, Venkatesan (2006). "Iterative Decoding of Low-Density Parity Check Codes". arXiv:cs/0610022.
  • LDPC Codes: An Introduction (by Amin Shokrollahi, 2003)
  • Belief-Propagation Decoding of LDPC Codes (by Amir Bennatan, Princeton University)
  • Turbo and LDPC Codes: Implementation, Simulation, and Standardization (West Virginia University)
  • Information theory and coding (Marko Hennhöfer, 2011, TU Ilmenau) - discusses LDPC codes at pages 74–78.
  • LDPC codes and performance results
  • DVB-S.2 Link, Including LDPC Coding (MatLab)
  • Source code for encoding, decoding, and simulating LDPC codes is available from a variety of locations:
    • Binary LDPC codes in C
    • Binary LDPC codes for Python (core algorithm in C)
    • and in MATLAB
    • A Fast Forward Error Correction Toolbox (AFF3CT) in C++11 for fast LDPC simulations

density, parity, check, code, information, theory, density, parity, check, ldpc, code, linear, error, correcting, code, method, transmitting, message, over, noisy, transmission, channel, ldpc, code, constructed, using, sparse, tanner, graph, subclass, bipartit. In information theory a low density parity check LDPC code is a linear error correcting code a method of transmitting a message over a noisy transmission channel 1 2 An LDPC code is constructed using a sparse Tanner graph subclass of the bipartite graph 3 LDPC codes are capacity approaching codes which means that practical constructions exist that allow the noise threshold to be set very close to the theoretical maximum the Shannon limit for a symmetric memoryless channel The noise threshold defines an upper bound for the channel noise up to which the probability of lost information can be made as small as desired Using iterative belief propagation techniques LDPC codes can be decoded in time linear in their block length LDPC codes are also known as Gallager codes in honor of Robert G Gallager who developed the LDPC concept in his doctoral dissertation at the Massachusetts Institute of Technology in 1960 4 5 However LDPC codes require computationally expensive iterative decoding so they went unused for decades In 1993 the newly invented turbo codes demonstrated that codes with iterative decoding could far outperform other codes used at that time but turbo codes were patented and required a fee for use This raised renewed interest in LDPC codes which were shown to have similar performance but were much older and patent free 6 Now that the fundamental patent for turbo codes has expired on August 29 2013 7 8 LDPC codes are still used for their technical merits LDPC codes have been shown to have ideal combinatorial properties In his dissertation Gallager showed that LDPC codes achieve the Gilbert Varshamov bound for linear codes over binary fields with high probability In 2020 it was shown that Gallager s LDPC codes achieve list decoding capacity and also achieve the Gilbert Varshamov bound for linear codes over general fields 9 Contents 1 History 2 Applications 3 Operational use 4 Example encoder 5 Decoding 5 1 Updating node information 6 Code construction 7 Compared to turbo codes 8 See also 8 1 People 8 2 Theory 8 3 Applications 8 4 Other capacity approaching codes 8 5 Capacity achieving codes 9 References 10 External linksHistory editImpractical to implement when first developed by Gallager in 1963 10 LDPC codes were forgotten until his work was rediscovered in 1996 11 Turbo codes another class of capacity approaching codes discovered in 1993 became the coding scheme of choice in the late 1990s used for applications such as the Deep Space Network and satellite communications LDPC codes then received renewed interest as a patent free alternative of similar performance 6 Since then advances in low density parity check codes have seen them surpass turbo codes in terms of error floor and performance in the higher code rate range leaving turbo codes better suited for the lower code rates only 12 Applications editIn 2003 an irregular repeat accumulate IRA style LDPC code beat six turbo codes to become the error correcting code in the new DVB S2 standard for digital television 13 The DVB S2 selection committee made decoder complexity estimates for the turbo code proposals using a much less efficient serial decoder architecture rather than a parallel decoder architecture This forced the turbo code proposals to use frame sizes on the order of one half the frame size of the LDPC proposals citation needed In 2008 LDPC beat convolutional turbo codes as the forward error correction FEC system for the ITU T G hn standard 14 G hn chose LDPC codes over turbo codes because of their lower decoding complexity especially when operating at data rates close to 1 0 Gbit s and because the proposed turbo codes exhibited a significant error floor at the desired range of operation 15 LDPC codes are also used for 10GBASE T Ethernet which sends data at 10 gigabits per second over twisted pair cables As of 2009 LDPC codes are also part of the Wi Fi 802 11 standard as an optional part of 802 11n and 802 11ac in the High Throughput HT PHY specification 16 LDPC is a mandatory part of 802 11ax Wi Fi 6 17 Some OFDM systems add an additional outer error correction that fixes the occasional errors the error floor that get past the LDPC correction inner code even at low bit error rates For example The Reed Solomon code with LDPC Coded Modulation RS LCM uses a Reed Solomon outer code 18 The DVB S2 the DVB T2 and the DVB C2 standards all use a BCH code outer code to mop up residual errors after LDPC decoding 19 5G NR uses polar code for the control channels and LDPC for the data channels 20 21 Although LDPC code has had its success in commercial hard disk drives to fully exploit its error correction capability in SSDs demands unconventional fine grained flash memory sensing leading to an increased memory read latency LDPC in SSD 22 is an effective approach to deploy LDPC in SSD with a very small latency increase which turns LDPC in SSD into a reality Since then LDPC has been widely adopted in commercial SSDs in both customer grades and enterprise grades by major storage venders Many TLC and later SSDs are using LDPC codes A fast hard decode binary erasure is first attempted which can fall back into the slower but more powerful soft decoding 23 Operational use editThis section needs additional citations for verification Please help improve this article by adding citations to reliable sources in this section Unsourced material may be challenged and removed May 2023 Learn how and when to remove this message LDPC codes functionally are defined by a sparse parity check matrix This sparse matrix is often randomly generated subject to the sparsity constraints LDPC code construction is discussed later These codes were first designed by Robert Gallager in 1960 5 Below is a graph fragment of an example LDPC code using Forney s factor graph notation In this graph n variable nodes in the top of the graph are connected to n k constraint nodes in the bottom of the graph This is a popular way of graphically representing an n k LDPC code The bits of a valid message when placed on the T s at the top of the graph satisfy the graphical constraints Specifically all lines connecting to a variable node box with an sign have the same value and all values connecting to a factor node box with a sign must sum modulo two to zero in other words they must sum to an even number or there must be an even number of odd values nbsp Ignoring any lines going out of the picture there are eight possible six bit strings corresponding to valid codewords i e 000000 011001 110010 101011 111100 100101 001110 010111 This LDPC code fragment represents a three bit message encoded as six bits Redundancy is used here to increase the chance of recovering from channel errors This is a 6 3 linear code with n 6 and k 3 Again ignoring lines going out of the picture the parity check matrix representing this graph fragment is H 1 1 1 1 0 0 0 0 1 1 0 1 1 0 0 1 1 0 displaystyle mathbf H begin pmatrix 1 amp 1 amp 1 amp 1 amp 0 amp 0 0 amp 0 amp 1 amp 1 amp 0 amp 1 1 amp 0 amp 0 amp 1 amp 1 amp 0 end pmatrix nbsp In this matrix each row represents one of the three parity check constraints while each column represents one of the six bits in the received codeword In this example the eight codewords can be obtained by putting the parity check matrix H into this form P T I n k displaystyle begin bmatrix P T I n k end bmatrix nbsp through basic row operations in GF 2 H 1 1 1 1 0 0 0 0 1 1 0 1 1 0 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 0 1 0 1 1 0 1 0 2 1 1 1 1 0 0 0 1 1 0 1 0 0 0 1 1 0 1 3 1 1 1 1 0 0 0 1 1 0 1 0 1 1 0 0 0 1 4 displaystyle mathbf H begin pmatrix 1 amp 1 amp 1 amp 1 amp 0 amp 0 0 amp 0 amp 1 amp 1 amp 0 amp 1 1 amp 0 amp 0 amp 1 amp 1 amp 0 end pmatrix 1 sim begin pmatrix 1 amp 1 amp 1 amp 1 amp 0 amp 0 0 amp 0 amp 1 amp 1 amp 0 amp 1 0 amp 1 amp 1 amp 0 amp 1 amp 0 end pmatrix 2 sim begin pmatrix 1 amp 1 amp 1 amp 1 amp 0 amp 0 0 amp 1 amp 1 amp 0 amp 1 amp 0 0 amp 0 amp 1 amp 1 amp 0 amp 1 end pmatrix 3 sim begin pmatrix 1 amp 1 amp 1 amp 1 amp 0 amp 0 0 amp 1 amp 1 amp 0 amp 1 amp 0 1 amp 1 amp 0 amp 0 amp 0 amp 1 end pmatrix 4 nbsp Step 1 H Step 2 Row 1 is added to row 3 Step 3 Row 2 and 3 are swapped Step 4 Row 1 is added to row 3 From this the generator matrix G can be obtained as I k P displaystyle begin bmatrix I k P end bmatrix nbsp noting that in the special case of this being a binary code P P displaystyle P P nbsp or specifically G 1 0 0 1 0 1 0 1 0 1 1 1 0 0 1 1 1 0 displaystyle mathbf G begin pmatrix 1 amp 0 amp 0 amp 1 amp 0 amp 1 0 amp 1 amp 0 amp 1 amp 1 amp 1 0 amp 0 amp 1 amp 1 amp 1 amp 0 end pmatrix nbsp Finally by multiplying all eight possible 3 bit strings by G all eight valid codewords are obtained For example the codeword for the bit string 101 is obtained by 1 0 1 1 0 0 1 0 1 0 1 0 1 1 1 0 0 1 1 1 0 1 0 1 0 1 1 displaystyle begin pmatrix 1 amp 0 amp 1 end pmatrix odot begin pmatrix 1 amp 0 amp 0 amp 1 amp 0 amp 1 0 amp 1 amp 0 amp 1 amp 1 amp 1 0 amp 0 amp 1 amp 1 amp 1 amp 0 end pmatrix begin pmatrix 1 amp 0 amp 1 amp 0 amp 1 amp 1 end pmatrix nbsp where displaystyle odot nbsp is symbol of mod 2 multiplication As a check the row space of G is orthogonal to H such that G H T 0 displaystyle G odot H T 0 nbsp The bit string 101 is found in as the first 3 bits of the codeword 101011 Example encoder editThis section does not cite any sources Please help improve this section by adding citations to reliable sources Unsourced material may be challenged and removed May 2023 Learn how and when to remove this message nbsp LDPC encoder During the encoding of a frame the input data bits D are repeated and distributed to a set of constituent encoders The constituent encoders are typically accumulators and each accumulator is used to generate a parity symbol A single copy of the original data S0 K 1 is transmitted with the parity bits P to make up the code symbols The S bits from each constituent encoder are discarded The parity bit may be used within another constituent code In an example using the DVB S2 rate 2 3 code the encoded block size is 64800 symbols N 64800 with 43200 data bits K 43200 and 21600 parity bits M 21600 Each constituent code check node encodes 16 data bits except for the first parity bit which encodes 8 data bits The first 4680 data bits are repeated 13 times used in 13 parity codes while the remaining data bits are used in 3 parity codes irregular LDPC code For comparison classic turbo codes typically use two constituent codes configured in parallel each of which encodes the entire input block K of data bits These constituent encoders are recursive convolutional codes RSC of moderate depth 8 or 16 states that are separated by a code interleaver which interleaves one copy of the frame The LDPC code in contrast uses many low depth constituent codes accumulators in parallel each of which encode only a small portion of the input frame The many constituent codes can be viewed as many low depth 2 state convolutional codes that are connected via the repeat and distribute operations The repeat and distribute operations perform the function of the interleaver in the turbo code The ability to more precisely manage the connections of the various constituent codes and the level of redundancy for each input bit give more flexibility in the design of LDPC codes which can lead to better performance than turbo codes in some instances Turbo codes still seem to perform better than LDPCs at low code rates or at least the design of well performing low rate codes is easier for turbo codes As a practical matter the hardware that forms the accumulators is reused during the encoding process That is once a first set of parity bits are generated and the parity bits stored the same accumulator hardware is used to generate a next set of parity bits Decoding editThis section does not cite any sources Please help improve this section by adding citations to reliable sources Unsourced material may be challenged and removed May 2023 Learn how and when to remove this message As with other codes the maximum likelihood decoding of an LDPC code on the binary symmetric channel is an NP complete problem 24 shown by reduction from 3 dimensional matching So assuming P NP which is widely believed then performing optimal decoding for an arbitrary code of any useful size is not practical However sub optimal techniques based on iterative belief propagation decoding give excellent results and can be practically implemented The sub optimal decoding techniques view each parity check that makes up the LDPC as an independent single parity check SPC code Each SPC code is decoded separately using soft in soft out SISO techniques such as SOVA BCJR MAP and other derivates thereof The soft decision information from each SISO decoding is cross checked and updated with other redundant SPC decodings of the same information bit Each SPC code is then decoded again using the updated soft decision information This process is iterated until a valid codeword is achieved or decoding is exhausted This type of decoding is often referred to as sum product decoding The decoding of the SPC codes is often referred to as the check node processing and the cross checking of the variables is often referred to as the variable node processing In a practical LDPC decoder implementation sets of SPC codes are decoded in parallel to increase throughput In contrast belief propagation on the binary erasure channel is particularly simple where it consists of iterative constraint satisfaction For example consider that the valid codeword 101011 from the example above is transmitted across a binary erasure channel and received with the first and fourth bit erased to yield 01 11 Since the transmitted message must have satisfied the code constraints the message can be represented by writing the received message on the top of the factor graph In this example the first bit cannot yet be recovered because all of the constraints connected to it have more than one unknown bit In order to proceed with decoding the message constraints connecting to only one of the erased bits must be identified In this example only the second constraint suffices Examining the second constraint the fourth bit must have been zero since only a zero in that position would satisfy the constraint This procedure is then iterated The new value for the fourth bit can now be used in conjunction with the first constraint to recover the first bit as seen below This means that the first bit must be a one to satisfy the leftmost constraint nbsp Thus the message can be decoded iteratively For other channel models the messages passed between the variable nodes and check nodes are real numbers which express probabilities and likelihoods of belief This result can be validated by multiplying the corrected codeword r by the parity check matrix H z H r 1 1 1 1 0 0 0 0 1 1 0 1 1 0 0 1 1 0 1 0 1 0 1 1 0 0 0 displaystyle mathbf z mathbf H odot r begin pmatrix 1 amp 1 amp 1 amp 1 amp 0 amp 0 0 amp 0 amp 1 amp 1 amp 0 amp 1 1 amp 0 amp 0 amp 1 amp 1 amp 0 end pmatrix odot begin pmatrix 1 0 1 0 1 1 end pmatrix begin pmatrix 0 0 0 end pmatrix nbsp Because the outcome z the syndrome of this operation is the three one zero vector the resulting codeword r is successfully validated After the decoding is completed the original message bits 101 can be extracted by looking at the first 3 bits of the codeword While illustrative this erasure example does not show the use of soft decision decoding or soft decision message passing which is used in virtually all commercial LDPC decoders Updating node information edit In recent years when there has also been a great deal of work spent studying the effects of alternative schedules for variable node and constraint node update The original technique that was used for decoding LDPC codes was known as flooding This type of update required that before updating a variable node all constraint nodes needed to be updated and vice versa In later work by Vila Casado et al 25 alternative update techniques were studied in which variable nodes are updated with the newest available check node information citation needed The intuition behind these algorithms is that variable nodes whose values vary the most are the ones that need to be updated first Highly reliable nodes whose log likelihood ratio LLR magnitude is large and does not change significantly from one update to the next do not require updates with the same frequency as other nodes whose sign and magnitude fluctuate more widely citation needed These scheduling algorithms show greater speed of convergence and lower error floors than those that use flooding These lower error floors are achieved by the ability of the Informed Dynamic Scheduling IDS 25 algorithm to overcome trapping sets of near codewords 26 When nonflooding scheduling algorithms are used an alternative definition of iteration is used For an n k LDPC code of rate k n a full iteration occurs when n variable and n k constraint nodes have been updated no matter the order in which they were updated citation needed Code construction editFor large block sizes LDPC codes are commonly constructed by first studying the behaviour of decoders As the block size tends to infinity LDPC decoders can be shown to have a noise threshold below which decoding is reliably achieved and above which decoding is not achieved 27 colloquially referred to as the cliff effect This threshold can be optimised by finding the best proportion of arcs from check nodes and arcs from variable nodes An approximate graphical approach to visualising this threshold is an EXIT chart citation needed The construction of a specific LDPC code after this optimization falls into two main types of techniques citation needed Pseudorandom approaches Combinatorial approaches Construction by a pseudo random approach builds on theoretical results that for large block size a random construction gives good decoding performance 11 In general pseudorandom codes have complex encoders but pseudorandom codes with the best decoders can have simple encoders 28 Various constraints are often applied to help ensure that the desired properties expected at the theoretical limit of infinite block size occur at a finite block size citation needed Combinatorial approaches can be used to optimize the properties of small block size LDPC codes or to create codes with simple encoders citation needed Some LDPC codes are based on Reed Solomon codes such as the RS LDPC code used in the 10 Gigabit Ethernet standard 29 Compared to randomly generated LDPC codes structured LDPC codes such as the LDPC code used in the DVB S2 standard can have simpler and therefore lower cost hardware in particular codes constructed such that the H matrix is a circulant matrix 30 Yet another way of constructing LDPC codes is to use finite geometries This method was proposed by Y Kou et al in 2001 31 Compared to turbo codes editLDPC codes can be compared with other powerful coding schemes e g turbo codes 32 In one hand BER performance of turbo codes is influenced by low codes limitations 33 LDPC codes have no limitations of minimum distance 34 that indirectly means that LDPC codes may be more efficient on relatively large code rates e g 3 4 5 6 7 8 than turbo codes However LDPC codes are not the complete replacement turbo codes are the best solution at the lower code rates e g 1 6 1 3 1 2 35 36 See also editPeople edit Richard Hamming Claude Shannon David J C MacKay Irving S Reed Michael Luby Theory edit Graph theory Hamming code Sparse graph code Expander code Applications edit G hn G 9960 ITU T Standard for networking over power lines phone lines and coaxial cable 802 3an or 10GBASE T 10 gigabit s Ethernet over twisted pair CMMB China Multimedia Mobile Broadcasting DVB S2 DVB T2 DVB C2 digital video broadcasting 2nd generation DMB T H digital video broadcasting 37 WiMAX IEEE 802 16e standard for microwave communications IEEE 802 11n 2009 Wi Fi standard DOCSIS 3 1 ATSC 3 0 Next generation North America digital terrestrial broadcasting 3GPP 5G NR data channel Other capacity approaching codes edit Fountain codes LT codes Online codes Raptor codes Repeat accumulate codes a class of simple turbo codes Serial concatenated convolutional codes Tornado codes LDPC codes designed for erasure decoding Turbo codes Capacity achieving codes edit So far there is only one capacity achieving code by design and proof Polar codesReferences edit MacKay David J 2003 Information theory Inference and Learning Algorithms Cambridge University Press ISBN 0 521 64298 1 Moon Todd K 2005 Error Correction Coding Mathematical Methods and Algorithms Wiley ISBN 0 471 64800 0 Includes code Amin Shokrollahi LDPC Codes An Introduction PDF archived from the original PDF on May 17 2017 Hardesty L January 21 2010 Explained Gallager codes MIT News Retrieved August 7 2013 a b Gallager R G January 1962 Low density parity check codes IRE Trans Inf Theory 8 1 21 28 doi 10 1109 TIT 1962 1057683 hdl 1721 1 11804 32786367 MIT S2CID 260490814 a b Erico Guizzo March 1 2004 CLOSING IN ON THE PERFECT CODE IEEE Spectrum Another advantage perhaps the biggest of all is that the LDPC patents have expired so companies can use them without having to pay for intellectual property rights US 5446747 Mackenzie D July 9 2005 Communication speed nears terminal velocity New Scientist Mosheiff J Resch N Ron Zewi N Silas S Wootters M 2020 Low Density Parity Check Codes Achieve List Decoding Capacity SIAM Journal on Computing FOCS 2020 38 73 doi 10 1137 20M1365934 S2CID 244549036 Gallager Robert G 1963 Low Density Parity Check Codes PDF M I T Press Retrieved August 7 2013 a b David J C MacKay and Radford M Neal Near Shannon Limit Performance of Low Density Parity Check Codes Electronics Letters July 1996 Telemetry Data Decoding Design Handbook Presentation by Hughes Systems Archived 2006 10 08 at the Wayback Machine HomePNA Blog G hn a PHY For All Seasons IEEE Communications Magazine paper on G hn Archived 2009 12 13 at the Wayback Machine IEEE Standard section 20 3 11 6 802 11n 2009 IEEE October 29 2009 accessed March 21 2011 IEEE SA IEEE 802 11ax 2021 IEEE Standards Association Retrieved May 22 2022 Chih Yuan Yang Mong Kai Ku http 123seminarsonly com Seminar Reports 029 26540350 Ldpc Coded Ofdm Modulation pdf LDPC coded OFDM modulation for high spectral efficiency transmission Nick Wells DVB T2 in relation to the DVB x2 Family of Standards Archived 2013 05 26 at the Wayback Machine 5G Channel Coding PDF Archived from the original PDF on December 6 2018 Retrieved January 6 2019 Maunder Robert September 2016 A Vision for 5G Channel Coding PDF Archived from the original PDF on December 6 2018 Retrieved January 6 2019 Kai Zhao Wenzhe Zhao Hongbin Sun Tong Zhang Xiaodong Zhang Nanning Zheng 2013 LDPC in SSD Making Advanced Error Correction Codes Work Effectively in Solid State Drives PDF FAST 13 pp 243 256 Soft Decoding in LDPC based SSD Controllers EE Times 2015 Robert McEliece E R Berlekamp and H Van Tilborg 1978 On the Inherent Intractability of Certain Coding Problems IEEE Trans Inf Theory IEEE 384 386 doi 10 1109 TIT 1978 1055873 a b Casado A I V Griot M Wesel R D 2007 Informed Dynamic Scheduling for Belief Propagation Decoding of LDPC Codes 2007 IEEE International Conference on Communications Glasgow UK pp 932 7 arXiv cs 0702111 doi 10 1109 ICC 2007 158 Richardson T October 2003 Error floors of LDPC codes PDF Proceedings of the Annual Allerton Conference on Communication Control and Computing 41 3 1426 35 ISSN 0732 6181 Richardson T J Shokrollahi M A Urbanke R L February 2001 Design of capacity approaching irregular low density parity check codes IEEE Transactions on Information Theory 47 2 619 637 doi 10 1109 18 910578 Richardson T J Urbanke R L February 2001 Efficient encoding of low density parity check codes IEEE Transactions on Information Theory 47 2 638 656 doi 10 1109 18 910579 Ahmad Darabiha Anthony Chan Carusone Frank R Kschischang Power Reduction Techniques for LDPC Decoders Zhang Z Anantharam V Wainwright M J Nikolic B April 2010 An Efficient 10GBASE T Ethernet LDPC Decoder Design With Low Error Floors PDF IEEE Journal of Solid State Circuits 45 4 843 855 Bibcode 2010IJSSC 45 843Z doi 10 1109 JSSC 2010 2042255 S2CID 10431486 Kou Y Lin S Fossorier M P C November 2001 Low density parity check codes based on finite geometries a rediscovery and new results IEEE Transactions on Information Theory 47 7 2711 36 CiteSeerX 10 1 1 100 3023 doi 10 1109 18 959255 Tahir B Schwarz S Rupp M 2017 BER comparison between Convolutional Turbo LDPC and Polar codes 2017 24th International Conference on Telecommunications ICT Limassol Cyprus pp 1 7 doi 10 1109 ICT 2017 7998249 Moon Todd K 2005 Error correction coding mathematical methods and algorithms Wiley p 614 ISBN 0 471 64800 0 Moon Todd 2005 p 653 Andrews Kenneth S et al The development of turbo and LDPC codes for deep space applications Proceedings of the IEEE 95 11 2007 2142 2156 Hassan A E S Dessouky M Abou Elazm A and Shokair M 2012 Evaluation of complexity versus performance for turbo code and LDPC under different code rates Proc SPACOMM pp 93 98 IEEE Spectrum Does China Have the Best Digital Television Standard on the Planet spectrum ieee org Archived from the original on December 12 2009 External links editIntroducing Low Density Parity Check Codes by Sarah J Johnson 2010 LDPC Codes a brief Tutorial by Bernhard Leiner 2005 LDPC Codes TU Wien Archived February 28 2019 at the Wayback Machine MacKay David J C September 25 2003 47 Low Density Parity Check Codes Information Theory Inference and Learning Algorithms Cambridge University Press pp 557 573 ISBN 9780521642989 Guruswami Venkatesan 2006 Iterative Decoding of Low Density Parity Check Codes arXiv cs 0610022 LDPC Codes An Introduction by Amin Shokrollahi 2003 Belief Propagation Decoding of LDPC Codes by Amir Bennatan Princeton University Turbo and LDPC Codes Implementation Simulation and Standardization West Virginia University Information theory and coding Marko Hennhofer 2011 TU Ilmenau discusses LDPC codes at pages 74 78 LDPC codes and performance results DVB S 2 Link Including LDPC Coding MatLab Source code for encoding decoding and simulating LDPC codes is available from a variety of locations Binary LDPC codes in C Binary LDPC codes for Python core algorithm in C LDPC encoder and LDPC decoder in MATLAB A Fast Forward Error Correction Toolbox AFF3CT in C 11 for fast LDPC simulations Retrieved from https en wikipedia org w index php title Low density parity check code amp oldid 1222608270, wikipedia, wiki, book, books, library,

article

, read, download, free, free download, mp3, video, mp4, 3gp, jpg, jpeg, gif, png, picture, music, song, movie, book, game, games.