Next Generation Sequencing - Best Practices
I found this very useful link, of what tools to use, why each step is necessary and example usage. vi-ssem.eu/BestPractice.pdf
And some Gotchas:
Academic Markdown - writing scientific articles and a whole thesis in markdown
After some very unpleasant experiences of collaboratively writing papers, proposals or my thesis either in Microsoft Word, Google Docs or Overeaf.com (previously WriteLatex), I tried to give authoring in markdown another shot. All of my documentation I already do in markdown (in Vim) on a daily basis. And whenever required I can just produce beautiful PDF documents or HTML versions of READMEs or notes. So why not use this very legible format for bigger documents?
A good primer can be found here: http://blog.kdheepak.com/writing-papers-with-markdown.html
Pandoc is pretty much required for this to work at the moment, because delivers the best ecosystem of markup extensions, filters, documentation and input/output formats. It is written in Haskell and thus a hefty installation of the Haskell compiler is required before installing Pandoc itself in most cases (my old laptop did not have enough memory to compile the package with default configurations) and Haskells nice package manager Cabal itself was kind of hard to install on older systems because of missing dependencies.
My make command for markdown files in Vim is now:
autocmd BufRead,BufNewFile *.md setlocal filetype=markdown spelllang=en_us spell makeprg=pandoc\ --filter\ pandoc-citeproc\ -o\ %<.pdf\ %
and I plan to include the following filters:
--filter pandoc-fignos \ --filter pandoc-eqnos \ --filter pandoc-tablenos \
but should really define this on a document basis in the meta-header with panflute.
Also, I will try to implement my own acronym/abbreviation filter that replaces the first occurrence of things like 'ncRNAs' with 'non-coding RNAs' if defined like ~ncRNA: non-coding RNA.
I will add to this blog post as my experience progresses.
Pandoc-Scholar is another project for setting up Pandoc for collaborative writing.
Open science platforms
I stumbled upon the following projects for creating open (or closed) science projects for collaborative work:
How to make movies with VMD, imagemagick and ffmpeg
Export frames in VMD after the molecule is setup and looking pretty. The following will produce 360 frames for a full rotation around the Y axis at fixed 720p resolution using Tachyon as external renderer. This must be installed first via sudo apt install tachyon. By default the renderer does not support JPEG output.
set i 0;
while {$i < 360} {
rotate y by 1
set d [format "%04d" $i]
render Tachyon out.dat tachyon %s -res 1280 720 -aasamples 2 -mediumshade %s -format PNG -o out_$d.png
incr i
}
Be sure to render in the resolution you want your GIF or movie to be in the end. Otherwise you loose quality or ffmpeg will not accept certain aspect ratios (you will have to pad with black pixels and you don't want that!)
Assembling the frames into a GIF with imagemagick:
convert -quality 5 -loop 0 -delay 5 out_%04d.png animation.gif
The quality option did not change my results visibly (no idea if this is working as intended). The -loop 0 parameter controls how often the animation is repeated. If your animation is large and you want to be nice to users of old PCs, limit this to a couple of times, because the CPU will have to flip through all the frames indefinitively with 0. Experiment with delay based on how many frames you have for the speed of the animation.
Creating a short movie from the same frames using ffmpeg:
ffmpeg -i out_%04d.png -vcodec libx264 -preset slow -crf 22 -r 15 -threads 0 video.mp4
-i out_%04d.png are your rendered frames with four digids and leading zeros. vcodec libx264 uses x264 for H.264 encoding (which has wide spread compatibility). -preset slow controls the encoding speed and results in better quality/compression ratio during encoding. -crf 22 is a measure of quality; 23 is an accepted default, with smaller values resulting in less loss of information and 51 decreases the image quality to a artifact riddled pulp. -r 15 controls the framerate; if you do not want to produce too many frames due to space or computing time reduce this value, to slow down your video. -threads 0 gives you all the CPUs you have available. And video.mp4 is evidently the generated output file.
How to clean Text from HTML tags and UTF-8 symbols
The mailing list I use for the bike project I'm involved in, is set to forward text mode only and in some cases spits out unintelligible HTML/UTF gibberish. To clean up the mess, I found a quite robust converter under MailChimp.