Topic outline

  • Collapse Expand

    General

    Collapse all Expand all

    • Compute Ontario Colloquium (1 Feb 2023)

      This week's colloquium: "File Management - Packing Small Files" by Ching-Hsing Yu from SciNet. The Compute Ontario Colloquia are weekly Zoom presentations on Advanced Research Computing, High Performance Computing, Research Data Management, and Research Software topics, delivered by staff from three Compute Ontario consortia (CAC, SciNet, SHARCNET) and guest speakers. The colloquia are one hour long and include time for questions. No registration is required. 

      Enseignant: Ching-Hsing Yu
      Date: : mer., 1 févr. 2023 - 12:00 pm
      Événements:

    • Abstract

      Managing large amounts of data can be a challenging task. Processing large numbers of files incurs heavy overhead of I/O communications. In this talk, we will explore several options such as using Apptainer Overlay and SQLite to pack and reduce a large number of files to few files, and hence, improving IO performance. Python scripts will be used throughout the presentation.

    • Information on how to join the Zoom meeting can be found at 

      https://training.computeontario.ca/colloquium

      Note that this page will always contain the zoom link of the upcoming colloquium.

    • Announcements Forum