Working Filesystems¶
As soon as you have access to ZIH systems, you have to manage your data. Several filesystems are available. Each filesystem serves for special purpose according to their respective capacity, performance and permanence.
End of life of scratch and ssd
The filesystem /lustre/scratch and /lustre/ssd will be turned off on January 3 2024 for good
(no data access afterwards!).
The /beegfs filesystem will remain available to
Alpha Centauri
and
Power
users only.
All others need to migrate your data to Barnard’s new filesystem /horse. Please follow these
detailed instruction on how to migrate to Barnard.
| Filesystem Type | Usable Directory | Capacity | Availability | Remarks |
|---|---|---|---|---|
Lustre |
/data/horse |
20 PB | global | Only accessible via Workspaces. The(!) working directory to meet almost all demands |
Lustre |
/data/walrus |
20 PB | global | Only accessible via Workspaces. For moderately low bandwidth, low IOPS. Mounted read-only on compute nodes. |
WEKAio |
/data/weasel |
1 PB | global (w/o Power) | Coming 2024! For high IOPS |
BeeGFS |
/beegfs/.global0 |
280 TB | Alpha and Power9 | Only accessible via Workspaces. Fastest available filesystem, only for large parallel applications running with millions of small I/O operations |
BeeGFS |
/beegfs/.global1 |
232 TB | Alpha and Power9 | Only accessible via Workspaces. Fastest available filesystem, only for large parallel applications running with millions of small I/O operations |
ext4 |
/tmp |
95 GB | node local | Systems: tbd. Is cleaned up after the job automatically. |
Outdated filesystems /lustre/scratch and /lustre/ssd
| Filesystem | Usable directory | Capacity | Availability | Backup | Remarks |
|---|---|---|---|---|---|
Lustre |
/scratch/ |
4 PB | global | No | Only accessible via Workspaces. Not made for billions of files! |
Lustre |
/lustre/ssd |
40 TB | global | No | Only accessible via Workspaces. For small I/O operations |
Recommendations for Filesystem Usage¶
To work as efficient as possible, consider the following points
- Save source code etc. in
/homeor/projects/... - Store checkpoints and other temporary data in workspaces on
horse - Compilation in
/dev/shmor/tmp
Getting high I/O-bandwidth
- Use many clients
- Use many processes (writing in the same file at the same time is possible)
- Use large I/O transfer blocks
- Avoid reading many small files. Use data container e. g. ratarmount to bundle small files into one
Cheat Sheet for Debugging Filesystem Issues¶
Users can select from the following commands to get some idea about their data.
General¶
For the first view, you can use the command df.
marie@login$ df
Alternatively, you can use the command findmnt, which is also able to report space usage
by adding the parameter -D:
marie@login$ findmnt -D
Optionally, you can use the parameter -t to specify the filesystem type or the parameter -o to
alter the output.
Important
Do not use the du-command for this purpose. It is able to cause issues
for other users, while reading data from the filesystem.