Working with DataFrames.jl beyond CSV files

Data scientists need to work with various data sources and sinks in their projects. During the workshop I you will learn how you can work with standard data formats using DataFrames.jl. A special focus will be put on working with data that is larger than available RAM.

Data science pipelines created in Julia typically need to be integrated into larger workflows involving various tools and technologies. Therefore an important aspect is ensuring interoperability, especially for the case of large data that does not fit in RAM of a single machine. During the workshop I plan to discuss working with the following data formats (along with their pros and cons):

• CSV;
• Apache Arrow;
• Apache Parquet;
• JSON;
• statistical packages; using examples of RData, Stata files;
• databases; using examples of SQLite and DuckDB.

All the examples will use DataFrames.jl that provides a representative implementation of Tables.jl table.