A good start
on ccn2.

01 / The mental model

Shared storage. Separate compute.

ccn2 is a group of Linux machines, called nodes. SSH puts you on one specific node. That node has its own CPU, memory, GPU devices, processes, and local disk. The /ccn2/u/ filesystem is shared: your directory should be available at the same path from every node you use.

A mounted directory is not a backup guarantee. Ask about quotas, retention, and backup arrangements before relying on them.

02 / Your first connection

Get an account. Then SSH in.

1. Get access from the lab. Ask your lab contact or cluster administrator for your username, approved nodes, and any Stanford network/VPN or gateway requirements. This guide does not grant access.
2. Open Terminal on your Mac or Linux laptop. Replace your_username below with your cluster account, not necessarily your laptop username. Node6 is an example; use the node you were assigned.

ssh your_username@node6-ccn2cluster.stanford.edu

On first connection, verify the server’s host-key fingerprint with the administrator before accepting it. Use the approved authentication method. Never disable host-key checking to get past a warning.

hostname
whoami
pwd

ssh-keygen -t ed25519 -f ~/.ssh/ccn2_ed25519 -C "ccn2 access"

ssh -i ~/.ssh/ccn2_ed25519 your_username@node6-ccn2cluster.stanford.edu

Host ccn2-node6
    HostName node6-ccn2cluster.stanford.edu
    User your_username
    IdentityFile ~/.ssh/ccn2_ed25519
    IdentitiesOnly yes
    ServerAliveInterval 60
    ServerAliveCountMax 3

03 / Finding your way around

A node is an SSH destination.

The hostname pattern is nodeN-ccn2cluster.stanford.edu. Use a full hostname from your laptop or from another cluster node. Shared files do not imply that SSH authentication is automatically shared.

ssh your_username@node6-ccn2cluster.stanford.edu

Common node numbers: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 14, 15.

For node-to-node SSH, arrange an approved authentication method with the administrator. Do not copy your laptop’s private key onto the cluster or turn on agent forwarding by default.

ssh -i /ccn2/u/your_username/.ssh/ccn2_ed25519 \
    your_username@node7-ccn2cluster.stanford.edu

04 / One place for your work

Your home belongs under /ccn2/u/.

Each person should have their own /ccn2/u/your_username/ directory, owned by their account. Use it as the consistent home/workspace across nodes. Ask the administrator to create or confirm it and configure the account’s home path consistently on every node you use.

printf 'Shell home: %s\n' "$HOME"
getent passwd "$(id -un)" | cut -d: -f1,6
ls -ld "/ccn2/u/$(id -un)"
df -h "/ccn2/u/$(id -un)"

If a node still uses a local home, keep only the necessary startup wiring there and load a shared environment file. Do not delete, move, or replace an existing home or .ssh directory with a symlink without a migration plan. Missing shared mount? Stop and ask for help; do not create a local look-alike under /ccn2.

05 / Download once, use across nodes

Put caches beside your work.

Keep persistent configuration, environments, and large model caches under your own shared home—not duplicated in each node’s /home. The following Bash snippet assumes your shared directory already exists and is mounted. Save it as /ccn2/u/your_username/.config/ccn2/env.sh, creating the parent directory if needed.

CCN2_USER="$(id -un)"
CCN2_ROOT="/ccn2/u/$CCN2_USER"

if [ -d "$CCN2_ROOT" ] && [ -w "$CCN2_ROOT" ]; then
    export XDG_CONFIG_HOME="$CCN2_ROOT/.config"
    export XDG_CACHE_HOME="$CCN2_ROOT/.cache"
    export XDG_DATA_HOME="$CCN2_ROOT/.local/share"
    export XDG_STATE_HOME="$CCN2_ROOT/.local/state"
    export HF_HOME="$XDG_CACHE_HOME/huggingface"
    export HF_HUB_CACHE="$HF_HOME/hub"
    export TORCH_HOME="$XDG_CACHE_HOME/torch"
    export PIP_CACHE_DIR="$XDG_CACHE_HOME/pip"
    export CUDA_CACHE_PATH="$XDG_CACHE_HOME/nv/ComputeCache"
    export CONDA_ENVS_PATH="$CCN2_ROOT/envs"
    export CONDA_PKGS_DIRS="$XDG_CACHE_HOME/conda/pkgs"
else
    printf 'Shared ccn2 home unavailable; check the mount.\n' >&2
    return 1
fi

Load this file from the Bash startup file actually used on each node, and explicitly from job scripts. Merge the line into existing configuration; do not overwrite your startup files. Noninteractive SSH commands may not load the same startup files as your terminal.

source "/ccn2/u/$(id -un)/.config/ccn2/env.sh"

Activate your project’s environment after loading this file. Use your own Conda installation or the lab-approved environment setup; another person’s installation path is not a template to copy. Shared environments still need compatible CUDA libraries and node hardware.

printf '%s\n' "$XDG_CACHE_HOME" "$HF_HOME" "$HF_HUB_CACHE"
python -m pip cache dir
conda info --envs

• Pointing is not migrating. These settings affect future reads and writes; they do not move existing caches or environments. Coordinate a migration while your jobs are stopped, then verify before removing anything.
• Keep secrets private. Configuration and Hugging Face home directories can contain login tokens. Never publish them, copy another user’s credentials, or make them world-readable. Use approved login tools and owner-only permissions for credential files.
• Not everything belongs on shared storage. Leave runtime sockets and XDG_RUNTIME_DIR node-local. Use administrator-approved local scratch for heavy temporary I/O; do not blindly redirect TMPDIR. Monitor space in both locations.

References: XDG directory conventions · Hugging Face cache settings. Some tools need their own cache setting; check each project’s documentation.

06 / Be a good cluster neighbor

Check first. Use only what you need.

Follow the lab’s current reservation or scheduling process before starting work. Check the node you are permitted to use. A low utilization reading is a snapshot—not a reservation or proof that a GPU is free.

hostname
nvidia-smi
nvidia-smi --query-gpu=index,name,memory.used,utilization.gpu,temperature.gpu,power.draw --format=csv
nvidia-smi --query-compute-apps=gpu_uuid,pid,process_name,used_memory --format=csv

Look for low memory use and low utilization together; temperature and power provide additional context. A process holding memory may be someone’s active or paused experiment. Coordinate with its owner rather than assuming you can take over.

Select the GPU explicitly for your process. For example, only if GPU 2 has been cleared for you:

CUDA_VISIBLE_DEVICES=2 python train.py

train.py is a placeholder for your project. This mask limits visibility; it does not reserve a GPU. Physical GPU 2 normally appears as cuda:0 inside that process. Do not override an allocation provided by a scheduler. NVIDIA’s device visibility reference.

07 / Keep your job, not your laptop, running

Use tmux. Keep a small run record.

For an approved long-running job, start a named tmux session on the target node. It keeps your shell alive if the SSH connection drops. It does not reserve resources or survive a node reboot.

tmux new -s first-run

Inside tmux, load your shared configuration, activate your environment, and run your command. This template uses one approved GPU and a unique log filename. Replace the project and command with your own.

source "/ccn2/u/$(id -un)/.config/ccn2/env.sh" || exit 1
# Activate your project's environment here.
cd "/ccn2/u/$(id -un)/Code/your_project" || exit 1
mkdir -p "/ccn2/u/$(id -un)/runs/first-run" || exit 1
CCN2_LOG="/ccn2/u/$(id -un)/runs/first-run/$(date +%Y%m%d-%H%M%S)-$(hostname)-$$.log"
set -o pipefail
CUDA_VISIBLE_DEVICES=2 python -u train.py 2>&1 | tee "$CCN2_LOG"

Detach with Ctrl + b, then d. To return, SSH into the same node and reattach:

tmux ls
tmux attach -t first-run

Record the node, GPU IDs, tmux session, environment, command, log path, and output/checkpoint directory. Check logs and nvidia-smi while the job runs. Give concurrent jobs separate output directories.

More terminal basics: tmux’s getting-started guide.

08 / A few common snags

Pause, check, then ask.