go-sciioLICENSE=BSD-3-ClauseCGO_ENABLED=0 go-sciioLICENSE=BSD-3-ClauseCGO_ENABLED=0go-sciio is the organisation the go-scikit plan names for what MATLAB's file functions and Mathematica's Import read and write: HDF5, MAT, NetCDF, FITS, npy, Matrix Market, WAV and FLAC. It is wave 2 of the plan; no library and no code exist yet.
state planned (wave 2)
library none yet
plan github.com/go-scikit/roadmap
PLAN.md, STATUS.mdEvery item starts from the roadmap's bibliography, is validated against the tool that is the reference in its domain, and lands with tests at 100% coverage and a performance table: no slower than the reference on one core.
Reader and writer for the File Format Specification 3.0, the base of MAT v7.3 and NetCDF-4. Whether it wraps an audited package or is written here is still open.
Versions 4, 5, 7 and 7.3, the last on top of the HDF5 package.
Classic (CDF-1, 2, 5) and NetCDF-4, with the CF conventions' units.
FITS 4.0: images, binary tables and compressed images.
NumPy .npy and .npz; Matrix Market coordinate and array files.
WAV, and FLAC decoding (RFC 9639).
Fuzzing and a limit on the allocation a header can ask for, in line with the fleet's decompression-bomb standard.
No cgo on any architecture the fleet's CI tests. The C libraries are oracles, never dependencies.
These are the measures the plan holds the work to. None has been run: there is no code yet.
Integers, strings and raw floats read back byte for byte equal to the reference reader’s.
Write in Go, read with h5py, astropy or SciPy, and the arrays are identical; read in Go a file those tools wrote, and the same.
go test -fuzz for panics, and a limit on the allocation a hostile header can ask for.
The names are provisional. None of these repositories exists yet.
HDF5 (File Format Specification 3.0), reader and writer. Validated against h5py, h5dump and the HDF Group's example files.
MAT-files v4, v5, v7 and v7.3, the last on top of h5. Validated against scipy.io and h5py.
NetCDF classic (CDF-1, 2, 5) and NetCDF-4 on top of h5. Validated against netcdf-c's ncdump and its test files.
FITS 4.0: images, binary tables and compressed images. Validated against astropy.io.fits and NASA's FITS samples.
NumPy .npy and .npz. Validated against NumPy.
Matrix Market, coordinate and array. Validated against scipy.io.mmread. go-sparse's wave 1 also lists Matrix Market I/O: the line between the two is yet to be drawn.
WAV, PCM and float. Validated against scipy.io.wavfile.
FLAC decoding (RFC 9639). Validated against the IETF working group's FLAC test files.
go-sciio is planned, wave 2 of the go-scikit roadmap. The plan says what it will cover and in what order, and its status says where each item stands. The documentation repeats the scope and the references.
Read the source on GitHub →