Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> the only big gotcha I encountered was finding, somewhat counter-intuitively, that it's often best minimize partitioning.

For parquet, I think with partitioning, it's really important to be mindful of the ordering of the data within the parquet file and also the query patterns of the main use cases. A little hard to generalize well to every pattern I guess.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: