Running bismark on Mox

Today I started my bismark run on Mox! Here’s how I did it:

Create a script

Since I’m running bismark with revised parameters, I’m putting these “clean” analyses in my class repository. In my class repository, I created a shell script to run bismark based on Sam’s script. Because I’m working in a script, I was able to specify variable paths for bismark, samtools, my genome folder, and my trimmed files.

Set up Mox

I navigated to the /gscratch/scrubbed/ directory and created a subdirectory called data. Within this subdirectory, I wanted to house my bisulfite genome and trimmed files. Steven and Sam said I could transfer these files to Mox in this issue. To do this, I initiated rsync transfers from Mox so I wouldn’t have to deal with two-factor authentication.

rsync --archive --progress --verbose yaamini@172.25.149.226:/Users/yaamini/Documents/project-virginica-oa/analyses/2018-04-27-Bismark/2018-04-27-Bismark-Inputs . #Transfer my bisulfite genome to Mox

rsync --archive --progress --verbose yaamini@172.25.149.226:/Volumes/web/Athaliana/20180411_trimgalore_10bp_Cvirginica_MBD/zr2096_*R1*.fq.gz . #Transfer analysis files

When I used ls -F to look at my directory contents, I realized I only transferred half of the trimmed files. I used the code below to transfer the rest.

rsync --archive --progress --verbose yaamini@172.25.149.226:/Volumes/web/Athaliana/20180411_trimgalore_10bp_Cvirginica_MBD/zr2096_*_val_2.fq.gz . #Transfered rest of files

rsync --archive --progress --verbose yaamini@172.25.149.226:/Volumes/web/Athaliana/20180411_trimgalore_10bp_Cvirginica_MBD/checksums.md5 . #Transfer checksums

md5sum -c checksums.md5 #Check checksums. All passed!

Based on Steven’s feedback, I transferred my shell script to my user directory.

rsync --archive --progress --verbose yaamini@172.25.149.226://Users/yaamini/Documents/yaamini-virginica/scripts/2018-10-12-Bismark-Revised-Parameters.sh . #Transfer script to USER directory, not scrubbed directory

Start Mox run

Now that I had everything on Mox, I was ready to start running the job!

sbatch -p srlab -A srlab 2018-10-12-Bismark-Revised-Parameters.sh #Submitted batch job 381386

Then that failed…because I didn’t properly specify my SBATCH path. My output directory is within my data directory, so I specified that in my script on genefish, then moved the fixed script to Mox.

sbatch -p srlab -A srlab 2018-10-12-Bismark-Revised-Parameters.sh #Submitted batch job 381403

Now I’m running! The more I think about it, the more I don’t like analysis output in my data directory. Once my job ends, I’ll move the output directory to an analyses subdirectory.

Fingers crossed this all works well.