rsbep - Reed-Solomon FEC (forward error correction) with byte-spreading

rsbep [-d] [-v] [-q] [-B x] [-D y] [-R z]

This manual page documents briefly the rsbep tool.

This manual page was written for the Debian distribution because the original program does not have a manual page.

rsbepis a tool that protects the data-stream from stdin with Reed-Solomon FEC (forward error correction) and additional it spreads the bytes of the resulting blocks out to give some protection against burst errors (e.g from tape-recordings).

The Reed-Solomon-code is from Phil Karn (see REAMDE.rsbep.RS32), and it's a special version for i386 (and compatible) to get enough performance for realtime encoding/decoding making it suitable for use with the tool "dvbackup" - which requires 3,6MByte/sec of data to feed the MiniDV-Camcorder. It is hardwired for (255,223)-RS.

The burst error protection is done by encoding ERR_BURST_LEN blocks of 255 bytes and placing the Bytes of an RS-block with distance ERR_BURST_LEN withing a buffer of ERR_BURST_LEN*255 bytes size. (This is the minimum output size of rsbep)

During decoding the blocks are reassembled and then decoded by RS32. If any error occured in the data it is corrected if possible, otherwise written as is, the number of corrected failures and uncorrectable blocks is printed if >0 or according to commandline options (v,q).

The first line of each encoded output block contains

"rsbep 255 223 765 MAGIC_NUM"

which is the identifier for rsbep, the Reed-Solomon block-size, data-size and the ERR_BURST_LEN (that is a number >=block-size and exact multiple of block-size). MAGIC_NUM is used to resynchronize if the stream is completly broken, which makes it possible to recover later files in your (tar-)archive.

Now how much errors can it correct?

RS(255,223) can correct up to 16 incorrect bytes in unknown positions.

However, the problem is, that typical communications and storage errors are also burst errors - which is that a lot of subsequent bytes are lost. That would mean, if more than 16 subsequent bytes are damaged, the data are irreparably damaged.

The method above spreads those bytes so far out, that up to 16*255*3=12240 subsequent bytes can get garbled without loosing valuable data at all!

Test with dvbackup and the LongPlay mode of my camcorder showed that more than enough to use it - it corrected Kilo-Bytes of errors in Giga-Bytes of data.

Of course there is a probability, that the failure has just that rate, that it hit's always the bytes of the same block - i leave it to the mathematicians among you to calculate it ;-)

* It does'nt make use of "erasures" - that would double the recover rate.

* It is hardcoded for RS(255,223)*(<=255) bytes, if you change the code you may not be able to recover old data with different RS-values

* The resynchronisation takes so much CPU that it will fillup the buffer soon and anything will fall apart, (running the program 'offline' might help) - it needs a lot of work to be really of use...

* Failures in the header-line might have drastic consequences, mathematically spoken: Those bytes contain more information

* The error correction works only from stream with failures, not erasures, dvbackup has the option '-r' to take care of this to the extend MiniDV allows.

-d

Decode. The default is to encode.

-v

Verbose. Talks about the errors it encounters and gives statistics at the end.

-q

Quiet. Output only a minimum of messages.

-B x

Override the default Reed-Solomon blocksize of 255 and use 'x' instead.

-D y

Override the default Reed-Solomon datasize of 223 and use 'y' instead.

-R z

Override the default ERR_BURST_LEN (used to spread the data) of 765 and use 'z' instead.

This manual page was written by Robert Jordens jordens@debian.org for the Debian system (but may be used by others). The text is based on information from Guido Fiala. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2. On Debian systems, the full text of this license can be found in the file /usr/share/common-licenses/GPL-2.