Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Working on an analytics team myself, I strongly agree with everything in your reply. I just want to nitpick one bit:

>weakly recommend against writing production ETL pipelines in a notebook

I find prototyping and developing ETL code in notebooks to be the most efficient. Especially when dealing with new and unfamiliar data sources. The interactivity and feedback loop makes defining what an ETL process should be doing a lot easier.

That said, once it's working, everything should get moved out into a module with tests.





Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: