DML Analysis Part 12

Running bismark on Mox

Today I started my bismark run on Mox! Here’s how I did it:

Create a script

Since I’m running bismark with revised parameters, I’m putting these “clean” analyses in my class repository. In my class repository, I created a shell script to run bismark based on Sam’s script. Because I’m working in a script, I was able to specify variable paths for bismark, samtools, my genome folder, and my trimmed files.

Set up Mox

I navigated to the /gscratch/scrubbed/ directory and created a subdirectory called data. Within this subdirectory, I wanted to house my bisulfite genome and trimmed files. Steven and Sam said I could transfer these files to Mox in this issue. To do this, I initiated rsync transfers from Mox so I wouldn’t have to deal with two-factor authentication.

rsync --archive --progress --verbose yaamini@ . #Transfer my bisulfite genome to Mox
rsync --archive --progress --verbose yaamini@*R1*.fq.gz . #Transfer analysis files

When I used ls -F to look at my directory contents, I realized I only transferred half of the trimmed files. I used the code below to transfer the rest.

rsync --archive --progress --verbose yaamini@*_val_2.fq.gz . #Transfered rest of files
rsync --archive --progress --verbose yaamini@ . #Transfer checksums
md5sum -c checksums.md5 #Check checksums. All passed!

Based on Steven’s feedback, I transferred my shell script to my user directory.

rsync --archive --progress --verbose yaamini@ . #Transfer script to USER directory, not scrubbed directory

Start Mox run

Now that I had everything on Mox, I was ready to start running the job!

sbatch -p srlab -A srlab #Submitted batch job 381386

Then that failed…because I didn’t properly specify my SBATCH path. My output directory is within my data directory, so I specified that in my script on genefish, then moved the fixed script to Mox.

sbatch -p srlab -A srlab #Submitted batch job 381403

Now I’m running! The more I think about it, the more I don’t like analysis output in my data directory. Once my job ends, I’ll move the output directory to an analyses subdirectory.

Fingers crossed this all works well.

Written on October 17, 2018