Managing large amounts of data can be a challenging task. Processing large numbers of files incur heavy overhead of IO communications. This course explores several options such as using Apptainer Overlay and SQLite to pack and reduce a large number of files to few files, and hence, improving IO performance. Python scripts are used throughout the course.
Format: Virtual
Using version control for your scripts, codes, documents, papers, and even data, allows you to track changes, keep backups, and facilitate collaboration. In this workshop, you will learn the basics of version control with the popular distributed version control software GIT. This workshop assumes that students have an understanding of basic Linux shell commands.
Format: Virtual
Using version control for your scripts, codes, documents, papers, and even data, allows you to track changes, keep backups, and facilitate collaboration. In this workshop, you will learn the basics of version control with the popular distributed version control software GIT. This workshop assumes that students have an understanding of basic Linux shell commands.
Format: In-person, but also will be broadcast and recorded.
Managing large amounts of data can be a challenging task. Processing large numbers of files incur heavy overhead of IO communications. This course explores several options such as using Apptainer Overlay and SQLite to pack and reduce a large number of files to few files, and hence, improving IO performance. Python scripts are used throughout the course.
Format: On-line (Zoom).
Using version control for your scripts, codes, documents, papers, and even data, allows you to track changes, keep backups, and facilitate collaboration. In this workshop, you will learn the basics of version control with the popular distributed version control software GIT. This workshop assumes that students have an understanding of basic Linux shell commands.
