Back Up and Migrate Conda Environments

Background

Deep learning projects rely on a web of dependencies: torch, transformers, cuda, and many more. Version mismatches often lead to obscure compatibility issues, making it hard to faithfully recreate an environment on a different server.

I already had a stable deep learning Conda environment on one server and wanted to migrate it to another. The easiest path was conda-pack, which can bundle a Conda environment into a single archive that works out of the box on another Linux host.

Export

To move the original environment off the source server, export it first.

1. List all environments

   Use conda env list to see every Conda environment on the machine. Make sure the target isn’t the default base. If you need to move base, clone it to a new name with conda clone first.

   1	conda env list

2. Activate the environment to migrate

   Suppose the environment name is torch:

   1	conda activate torch

3. Install conda-pack

   Install inside the activated environment:

   1	conda install conda-pack -c conda-forge

4. Pack the environment

   Run conda pack -n <env> to generate a tarball. Replace <env> with your environment name, e.g. torch.

   1	conda pack -n torch

   The command creates torch.tar.gz in the current directory.

Packing time scales with environment size. My torch env is large, so the job took a while—thankfully there’s a progress bar. The environment consumed ~20GB and the packed file landed just under 10GB.

Import

After packing and downloading the archive, upload it to the target server and unpack it.

Upload the file to the target

1. Download locally, then upload

   Keeps a local backup: download torch.tar.gz to your laptop, then upload to the target. Use whatever SSH/SFTP tool you like.

2. Use rsync directly

   Transfer straight between servers:

   1	rsync -avz torch.tar.gz user@target_server:/path/to/destination

Restore the environment

1. Ensure no environment with the same name exists

   1	conda env list

   If a duplicate exists, remove it first:

   1 2 3

   conda env remove -p <env_path> # or conda env remove -n <env_name>

2. Prepare the destination directory

   Check where Conda keeps environments, e.g. /mnt/nfs/yuesir/miniconda3/envs. Create a folder matching the environment name:

   1 2	mkdir /mnt/nfs/yuesir/miniconda3/envs/torch chmod 777 /mnt/nfs/yuesir/miniconda3/envs/torch

3. Extract the archive

   1	tar -xzvf torch.tar.gz -C /mnt/nfs/yuesir/miniconda3/envs/torch

4. Activate the new environment

   1	conda activate torch

Verify

After importing, run a few checks.

1. List environments

   1	conda env list

   You should see the migrated environment in the list.

2. Activate it

   1	conda activate <env_name>

3. Inspect installed packages

   1	conda list

   Confirm versions and dependencies look right.

Extra

For deep learning workloads (e.g., PyTorch) that depend on specific driver versions, verify CUDA driver compatibility between the two servers. CUDA drivers are backward compatible—if they don’t match, either upgrade the target’s driver (root required) or downgrade PyTorch (risky because of dependency complexity). In tricky cases, a clean install might be safer.

For choosing CUDA/PyTorch combos, see my earlier post:

Thanks

Using conda pack for environment migration