EByte Case Study Floats Bit Fields and Palettes in WAV BMP and IPv4

EByte 0.6.0 adds three kinds of field that real formats kept asking for. There are floating-point numbers. There are bit fields, several small numbers packed into the same bytes. And there are byte fields that run up to an offset another field declares, like the colour palette in an 8-bit BMP. This case study tried each one on a different format with the released ebyte 0.6.0 Linux binary, downloaded from GitHub. Its checksum matched SHA256SUMS before anything ran. Every result was checked with a tool that knows nothing about EByte: ffmpeg, Pillow and dpkt.

The three files are small, made for the purpose: a 32-bit float WAV built with Python’s struct, a paletted BMP saved by Pillow, and the sensor packet captures from the diff case study. All three schemas are in, or built from, the schema library.

Three panels: a float WAV whose samples were halved from a peak of -6 dB to -12 dB with a NaN payload preserved, a BMP palette entry changed from red to orange, and IPv4 version, header length and flag bits matched against dpkt, with a note that file output now works on Windows.

Floats: A WAV Made of Numbers

The wav-float32 schema reads the header and then every sample as an f32, repeated until the file ends. The sample is 400 samples of a 440 Hz tone at amplitude 0.5. In the JSON, samples look like "0.16936897": the shortest decimal that turns back into exactly the same 32 bits.

A five-line Python script halved every value in the JSON, and EByte encoded the result. ffmpeg’s volume detector measured a peak of -6.0 dB on the original and -12.0 dB on the edited file, with the mean down from -9.0 to -15.1 dB, which is what halving should do.

Floats have corners that a plain decimal can’t hold. EByte writes infinities as inf and -inf, keeps -0 distinct from 0, and writes a NaN as nan: followed by all its bits. Setting sample 10 to nan:7fc0beef and encoding gave a file whose bytes at that sample are 7fc0beef, and decoding it again gave back nan:7fc0beef. A tool that turned every NaN into one standard NaN would lose that payload.

Forward Jumps: A BMP Palette

An 8-bit BMP stores up to 256 colours between its header and its pixels, and the header says where the pixels start. Pillow’s file puts them at offset 1078, after a 1,024-byte palette. The bmp-8bit schema declares the palette with one new line:

{"name": "palette", "type": "bytes", "length": {"until_offset": "pixel_offset"}}

The palette is the bytes from where the header ends up to pixel_offset. Nothing is skipped: those bytes are in the JSON, so the file still round-trips exactly. Colour 0 was red, stored as 0000ff00 in blue, green, red order. Changing it to 0080ff00 and encoding changed one byte of the 1,094. Pillow then read the top-left pixel as (255, 128, 0), orange, and every other colour was unchanged.

The offset has to tell the truth. With pixel_offset changed to 1000 in the JSON and the palette left alone, encoding stopped with constraint_mismatch at /palette, offset 54: the palette’s bytes no longer end where the header says the pixels begin. EByte doesn’t move the offset for you, and it doesn’t read backwards.

Bit Fields: An IPv4 Header

The sensor captures from 0.5.0 had the IP header’s first byte as one opaque u8 with the value 69. It’s really two numbers, the IP version and the header length in 4-byte words. Two bytes further on, three flags and a 13-bit fragment offset share 16 bits. With bit fields the schema says so:

{"name": "ip_version", "type": "bits", "bits": 4, "expect": "4"},
{"name": "ip_header_words", "type": "bits", "bits": 4, "expect": "5"},

inspect reported ip_version 4 and ip_header_words 5, both at offset 54, then ip_reserved 0, ip_dont_fragment 1, ip_more_fragments 0 and ip_fragment_offset 0, sharing bytes 60 and 61. dpkt read the same packet as version 4, header length 5, don’t-fragment set, more-fragments clear and offset 0. Bits are taken from the most significant end, which is how network protocols and most file format specs number them. Formats that count from the other end aren’t covered yet.

And Windows

Until now --output FILE refused to run on Windows, leaving standard output as the only way to get a result. In 0.6.0 it works there with the same rule as on Linux and macOS: it creates a new file and never replaces an existing one. Windows now has its own job in CI on every push. It runs the tests, including attempts to write to an alternate data stream, a device name and a directory junction, each of which must fail without touching anything. It also runs the native smoke test, which publishes a file and then checks that a second run refuses to overwrite it. The download page has the archives.