Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

One exception though. Piping unix command this way allow handling of _very_ large file (stream model), whereas R requires loading the whole content in memory.


That depends on the statistical moments you're calculating. For sums and means, yes. For percentiles, standard deviation, and other moments, you'll need to operate on the set as a whole, which either means re-reading the dataset or processing it in memory.

That said, I've got my own univariate statistics generator written in awk that can handle sets with millions of values readily on desktop/laptop hardware.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: