Topic outline

    • Compute Ontario Colloquium (12 Apr 2023)

      This week's colloquium: "File quotas and effective strategies to store large sets of small files" by Rakesh Raghavaraju (CAC).  The Compute Ontario Colloquia are weekly Zoom presentations on Advanced Research Computing, High Performance Computing, Research Data Management, and Research Software topics, delivered by staff from three Compute Ontario consortia (CAC, SciNet, SHARCNET) and guest speakers. The colloquia are one hour long and include time for questions. No registration is required. 

      Date: Wed., 12 Apr. 2023 - 12:00 pm
    • Abstract

      By default, every group can store up 500k files on Project space in 1TB of disk space on the General Purpose clusters. If you have a large dataset of smaller file,i.e. ~2MB/file, you can quickly run out of inode quota. This webinar presents the usage file formats such as SQL Blob and Netcdf, to effectively store relatively a large set of smaller files.