Computer Science Home Directory Migration

On June 30, 2021, all public-use Computer Science Linux systems will be shut down as well as the file system storing CS home directories.

Users are responsible for migrating any data they wish to keep from their personal CS home directory to either ThayerFS, DartFS or Jumbo as appropriate. The deadline for this move is April 9, 2021. Any non-user home directories (research, courses, etc.) are being handled separately - if you use any non-user account or have data in one that needs to be preserved, email computing@thayer.dartmouth.edu to discuss migration options.

When you have completed moving your CS home directory data, please fill out this Google Form to indicate you are done.

Where should I move my data?

For most people, ThayerFS (see below) will be the easiest and most appropriate location for their home directory. Thayer and CS provide several public-use linux systems for class and research use that utilize these home directories. For more information, visit our Linux Services page.

One thing to keep in mind is that with ThayerFS and DartFS home directories, data cannot be shared. Any data stored in these are only accessible to the specific user and will be removed when that user's affiliation with Dartmouth ends. Any shared data should be stored in a different location. If you have or need shared data, please email computing@thayer.dartmouth.edu to discuss your needs.

You can move your data to multiple locations while you migrate, if appropriate. And, all of these are cross-mounted and available on all Thayer, CS and IT&C Research Computing systems.

ThayerFS

If your primary use of Linux systems will be either the public-use Thayer and CS systems or other Linux systems at Thayer or the new CS building, your ThayerFS home directory is the appropriate place for your data. ThayerFS home directories can also be accessed from Mac and Windows systems.

You can find more information about ThayerFS on our File Service page.

DartFS

If your primary use of Linux systems will be the Discovery Cluster or other IT&C Research Computing systems, your DartFS home directory is the appropriate place for your data. As with ThayerFS, DartFS can also be accessed from Mac and Windows systems.

If you do not currently have a Research Computing account, you can request one by clicking the "Request an Account" button on the Research Computing page.

If you have shared data in your home directory that will primarily be used on the Discovery Cluster or other Research Computing systems, it should be moved into a shared DartFS research space. Faculty or staff can request space by clicking the "Request Storage" button on the Research Computing page.

Jumbo

If you have shared data in your home directory that will primarily be used on public-use Thayer and CS systems or other Linux systems at Thayer or the new CS building, Jumbo may be the appropriate place for this data.

Jumbo storage is primarily for use by research groups and there is a charge for this space - there is more information about this on our File Service page. If you believe this may be an appropriate place for your shared data, please email computing@thayer.dartmouth.edu to discuss your needs.

How do I move my data?

To move data to any of these destinations, you will use the galehead.cs.dartmouth.edu Linux system.

In order to access ThayerFS, DartFS or Jumbo you will need a kerberos ticket. This is a manual process on galehead, but happens automatically when logging directly into Thayer, CS, and Research Computing systems. Also, if you will need long-running processes, which may be necessary for copying large amounts of data, you will need to run a process called krenew to keep this ticket renewed.

To do this automatically, ssh into galehead and run this command:

cs-home-migration

This will add commands to your .bashrc file to run these commands and set some environment variables.

If you do not use bash as your default shell, you can still add this and then invoke a bash shell manually before copying. You can check which shell you're using by looking at the output of echo $SHELL. To invoke a bash shell manually, type bash at your existing shell prompt. If you need assistance or have questions about these commands and variables, please email computing@thayer.dartmouth.edu.

Log out and back in to galehead and you will be prompted for your Dartmouth credentials to obtain the kerberos ticket. The environment variables THAYERFS_HOME and DARTFS_HOME will be set to your home directory locations on those systems.

To copy small amounts of data, you can simply use cp:

cp ~/file ${THAYERFS_HOME}/
cp -R ~/directory ${THAYERFS_HOME}/

If you have large amounts of data to copy, we recommend using screen (so you can detach your ssh session) and rsync to copy files. When you do this, you will be re-prompted for your credentials since you are in a new shell. If you do not get re-prompted, it is likely because you are using a different shell (see above) - if this is the case, just type bash.

Note: If you are familiar with rsync, do not use the -a argument - this will result in errors because of the different authentication systems.

If you aren't sure about what rsync will copy, we suggest adding a -n argument first. This will do a "dry run" without copying anything. If things look good, use the same command without the -n argument to actually do the copy.

screen
rsync -rlptDvP ~/directory ${THAYERFS_HOME}/ <- this creates "directory" on the target and copies everything into it
rsync -rlptDvP ~/directory/ ${THAYERFS_HOME}/new_directory/ <- this copies the contents of "directory" on the source into "new_directory" on the target

To exit the screen session while rsync is running, press ctrl-a and d. To then reconnect, either in that ssh session or a new one:

screen -r

These same cp or rsync commands can be used for targets in ${DARTFS_HOME} or Jumbo shares located at /jumbo/sharename

Next Steps

Once your data is migrated, be sure to fill out and submit this Google Form so we know you're done and won't send you any follow-up reminders.

Your ThayerFS home directory is your default home directory on all Thayer and new CS systems. For more information about our public-use Linux systems and how to access them, please visit our Linux Services page.

You can also access ThayerFS and Jumbo shares from a Mac, Windows or personal Linux system. For more information about this, please visit our Thayer Shares Connecting page.

What if I need help or have questions?

Thayer Computing is here to help make this migration as easy as possible. If you have questions or need any help, please email computing@thayer.dartmouth.edu and one of our team will be in touch.