Category Archives: opensource

cure cancer with leftover azure credits

1 Reply

An azure window grid, reflecting clouds. Perfect fit isn't it?Maybe you have some Azure credits lying from a Visual Studio subscription you have from work, waiting to be spent on cool and nifty experiments, but don’t end up actually using them.
How about spending some of those dollars on cancer research? Or help find a cure against Zika? Fighting AIDS maybe?

Enter the World Community Grid, a vast grid computing network running on the Open Source BOINC client software, started as a philanthropic initiative by IBM.
Sounds good right?

All you need to do is create a WCG account, spin up a Linux machine on Azure, install the BOINC headless client on it and link it to your account. In about half an hour you’ll be computing cancer markers, folding genes or fighting some horrible disease. Well, the software will be doing that really, which is even better.

Here we go, step by step.

1. Setup your Azure Linux machine, I chose an Ubuntu 16 LTS machine. I pick the classic VM because its way easier to setup.
Depending on the type of machine you’ll have more compute power and thus turning out more results. Try a few out and see how you can maximize your Azure credits.

2. Once provisioned, log in using PUTTY or your favorite SSH client. Now it’s time to update the Linux packages and then install the BOINC client:

sudo apt-get update
sudo apt-get install boinc-client

3. Setup auto startup of the BOINC client, so if your machine reboots, you don’t have to go in and start it up yourself (automate all the things remember):

sudo /etc/init.d/boinc-client restart

4. Get your BOINC authentication key so you can hook up the client to your account:

boinccmd --lookup_account https://www.worldcommunitygrid.org

5. Now use the key to attach your selected projects from WCG:

boinccmd --project_attach https://www.worldcommunitygrid.org

Of course you want to check what’s going on, so you can check the BOINC client’s state like this:

boinccmd --get_state

This should give you a list of the tasks and their state. It might take a while before they start kicking in, but you’ll see results coming in after a day or so on the WGC website under your contribution history.

That’s about it. Your client is all set up. All you need to do is keep those VM’s running in the cloud, which normally takes non effort at all. Neat.

There are more boinccmd command line switches in the documentation if you’d need to troubleshoot or find out more.

What’s next? Well, you can set up more than 1 machine if you like, or a heavier one and see what gives you more bang for your buck.  You can also join my World Computing Grid team called “Team Azure” and see how many cloud bucks we can burn. It’ll be effortless fun, I promise! ;)

Credit goes to Joel Christian’s headless Ubuntu installation guide. His guide made my quest to setup BOINC on an Azure Ubuntu box a lot easier.

Photo by Dan, cc-licensed.

make vim awesome with plugins

Leave a reply
Artistic closeup of a .vimrc file

Vim is a great lightweight editor as it is. But after setting it all up on your windows box and tweaking your _vimrc it still might lack that bit of awesome you’re looking for in a modern text editor.

Time to spice things up with plugins!

Vim plugins are written in viml or vimscript, an internal script language in the vim editor, and are plain .vim files containing scripting code which extend vim in all sorts of wonderful ways. There are tons of vim scripts out there so finding the right ones for your needs takes a bit of time. There are however some helpful guides out there and blog posts like this one to help you on your way. I’ll list some of those and links to more plugins at the end.

Installing those scripts and plugins can be tedious though. Download a zip, unpack, copy files, yada-yada-yada. Since we’re into package managers these days we want things to go automatically with a few keystrokes.

Enter Vundle.

You’ve probably guessed by now this is a vim plugin manager (and a plugin by itself). It allows you to install, update and search for available vim scripts among other things. I like this one in particular because it does this all from vim itself with a number of specific commands.
To get started you’ll have to install this one manually though, but it only takes a few command line statements and some .vimrc edits. Once you have this up and running, you’ll be able to install most plugins using it so it’s worth the hassle.

Check out the info on the Vundle github page on how to install and then come back here. ;)

Alright. Now, what plugins should we get?
Well it depends on what you want to do of course, but here are some general purpose ones you might like.

vim-airline

A pretty looking status bar you’ll see in a lot of vim screenshots. It’s tweakable so you choose what kind of info you want it to show.

vim airline status bar plugin

ervandew/supertab

Adds tab completion to vim using the tab-key. That might sound odd but the default use a bunch of control keys so this just comes more “natural”.

The-NERD-tree

This is a directory browsing plugin which is just better than Netrw which comes out of the box. Visually you can fold/unfold folders, search (use any vim command in the window), manipulate files etc. Very handy to keep track of a project when programming or just see what other files are in a folder without having to exit vim.

vim nerdtree plugin

CtrlP

A fuzzy file search plugin. Press Ctrl-P and you’ll get list at the bottom of files in your current directory. Type in some characters of the filename you are looking for and it will filter the files matching those characters. So you don’t need to know the full name, and you can skip parts. Check out this video to get an idea of how it works.

One note on this. If you have a folder with a lot of files in the sub-folder tree (like a C# application with build files in the sub-folder) be sure to exclude any non-relevant types like object & dll files. CtrlP has a maximum file limit and those irrelevant files can stop you from finding those you actually want to see.

In my _vimrc I use this to exclude the .NET build artifacts and some more irrelevant file types:

set wildignore+=*\\obj\\*,*\\bin\\*,*.swp,*.zip,*.exe,*.dll

vim-fontzoom

vim-fontzoom is a simple plugin that allows you to increase or decrease your vim font size using the plus or minus key when you are in command mode. Note that this doesn’t work with the +/- on your numeric pad, just with the regular keys on your keyboard. But you can remap the keys if you want to change this.

chriskempson/base16-vim themes

Not really a functional plugin but hey, you want your editor to look pretty right? I’ve tried a ton of themes already but lately I’m sticking with the chriskempson/base16-vim set. In this package you get a bunch of nicely crafted and balanced color themes which will definitely have something you like. Dark and light themes, monokai, solarized and other classics, it has it all. The last theme plugin you’ll ever need.

Moar!

Depending on your workload there are plenty of more specific plugins out there. Google is your friend, but here’s a few places to get started:

  • The easiest to use and most awesome Vim plugin directory is called VimAwesome. Great to find new plugins, or great to find old ones and how to install them. Each plugin lists how to install it with Vundle or another plugin manager, which is super handy.
  • The 15 best vim plugins according to Steve Francia who made a vim distro called The Ultimate Vim Distribution, so I guess he knows what he’s talking about. :)

warmup your site or wordpress blog with a single command line statement

2 Replies
The Mother huge spider statue contrasting against the sky.

GNU Wget is a powerful tool when it comes to downloading files from the web or mirroring sites. It’s command line features can be daunting and not very obvious. With some experimentation, reading the (f..) manual and some Googling you can get it to do some pretty neat tricks for you.
All of that is from the command line too, which is great if you want to schedule this kind of magic or use it in a script.

For example, you might want to warm-up your site or WordPress blog so your homepage and all posts linked from it are present in your cache when a visitor arrives. I’m assuming you are using a caching on your site otherwise this is pretty pointless. For WordPress you can use a caching plugin like W3 Total Cache for example.

With Wget, it goes like this:

wget.exe http://n3wjack.net --spider --no-directories --level=1 --recursive 
         --accept-regex=n3wjack.net/20[1..9].*

The command line parameters (in order) mean something like:

  1. Crawl n3wjack.net.
  2. Crawl it like a spider (follow the links).
  3. Don’t create directories for downloads.
  4. Crawl 1 level deep (so anything linked on the homepage is OK, but don’t go deeper).
  5. Do this recursively (so it actually goes 1 level deep).
  6. Follow only links that start with "/201..." to "/209..." (it’s a regular expression).
    This one is a trick to have it only follow links to blog-posts because my URL scheme begins with the year of the post (2015, 2016, …). It’s good until 2099, which should do the trick I guess. :)
    This way I’m also avoiding it loading all tag, category or page links.

If your site has a different URL scheme you’ll have to change the accept regex pattern to fit your scheme.

You can download Wget from the GNU site. It’s Open Source and is available for Windows, Mac and various Unix systems.
For Chocolatey users, there is a wget package available to install it on your system.

a vim http log file syntax plugin

Leave a reply

Log files are dull to look at. Lines and lines of text and no pretty colors to make it nicer to look at and easier to spot those weird errors you can’t simulate on your machine.

Vim rocks and writing a syntax file is supposed to be a breeze judging from the vast amount of syntax plugins out there. I didn’t quite find one I liked for syntax highlighting HTTP log files, so I thought I’d get down and dirty with some vimscript myself and see if I could hack something together.

It turned out alright I think. So to share the fun I’m hosting the logsyntax.vim plugin on Github and the vim.org scripts library for all to use. It highlights dates, HTTP verbs, URLs, IP addresses etc for IIS, W3C extended, NCSA and probably a bunch more typical log formats.

May your logs be pretty and colorful from now on.

The vim log file syntax highlighting plugin screenshot. Look at those pretty colors.

a wordpress full site spell checker tool

Leave a reply

A while ago I noticed that some of my older posts had some silly misspellings in it, so I was looking for a way to spell check all my posts in one shot. I couldn’t really find anything that was free, so I figured I’d try to write something myself to do this for me.

I knew about the free and open source Hunspell spell checker and that you can use it from the command line. So I thought using that together with the WordPress export XML file which has all your post’s content it should be possible to spell check the whole lot.

The end result is a PowerShell script which reads out the XML export file and runs it through Hunspell, parses the spelling errors found and finally bundling it all into a simple HTML report.

It worked nicely for me, even though it’s pretty crude and simple. I only had to use this once, so I don’t see the point of fine-tuning it a lot further.

However this could be handy for others who want to do the same thing, so I cleaned it up a bit, slapped a readme file on it and posted it on Github as the WordPress full site spell checker.
Check it out if you want to spell check your WordPress blog in a single run and maybe this will be good enough to get your job done. You find more info on how to set up and use it on the Github page.

That very basic report I was talking about.

configuring vim: tweaking your _vimrc file

Leave a reply
Artistic closeup of a .vimrc file

In my earlier post on setting up Vim on Windows I already talked about a few vimrc changes to make vim behave on windows and look a bit better.
I learned that checking out other people’s vimrc files is a great way to learn how you can configure vim to become the kick-ass text editor you always wanted. In this post I’m going to run over the settings I have in my _vimrc file to make vim work better for me. Feel free to copy bits you find useful.
If you don’t know what that vimrc file is, or how to edit it, check out my previous post on how to set things up.

I tend to document my vimrc changes with some comments. That way I still know what I was trying to do months later and or what lines can be removed when a wicked plugin makes them obsolete. Most of it is self-explanatory, so I’ll just paste them right here.

First some system settings.

" Activate all the handy Windows key-bindings we're used to.
source $VIMRUNTIME/mswin.vim

" Have gvim behave properly on Windows.
behave mswin

" Use unicode/utf-8 encoding by default for keyboard, display and files.
set encoding=utf-8

" Set a more convenient leader key on an AZERTY layout than the default backslash
let mapleader = ","

Now let’s make things prettier.
First, we need to set a nicer fixed-width font, Consolas in this case. Then I hide the toolbar which I never use. Then, of course, a nice theme.

if has("gui_running")
  " Set a nicer font.
  set guifont=Consolas:h11:cDEFAULT
  " Hide the toolbar.
  set guioptions-=T
endif

" Set theme.
color badwolf

Then a number of visual things to make programming, editing and searching text easier.

" Display line and column number in bottom ruler.
set ruler
" Display the line numbers.
set number
" Turn sounds off.
set visualbell
" Shows a horizontal highlight on the line with the cursor.
set cursorline

" Activate highlighting search pattern matches & incremental search.
" Incremental search means your cursor will jump to the first match as you
" type.
set hlsearch
set incsearch
" Allow using  to kill the current search highlighting.
nnoremap   :nohlsearch

" Activate case-insensitive & smart case search (if a capital letter is used
" in your search query, Vim will search case-sensitive).
set ignorecase 
set smartcase

" Set wildchar visual completion awesomeness.
" This is enhanced command line completion and it rocks.
set wildmenu 
set wildmode=full

" Turning on line wrapping and line-break for easy text-file editing.
" Line-break wraps full words at the end of a sentence for readability.
set wrap
set linebreak

Programming settings. Who would have thought!
Tweak tab settings according to your religious preference of course. I can’t say I have used the folding a lot, but these settings come from another .vimrc somewhere, and they seem sensible.

" Activate syntax highlighting.
syntax enable

" Set tabs to 4 characters and expand to spaces, activate smart indentation.
" See tabstop help for more info.
" Setting tabstop & softtabstop to the same value to avoid messy layout with mixed tabs & spaces.
set tabstop=4
set shiftwidth=4
set softtabstop=4
set expandtab
set smartindent

" Enabled folding on indent level. That way it works on any code & html, xml
" etc. 
" Setting foldlevelstart ensures that for newly opened files folds are open
" unless they are 10 levels deep.
set foldmethod=indent
set foldenable
set foldlevelstart=10
set foldnestmax=10      " no more than 10 fold levels please

At the end of my vimrc I add my custom tweaks. They might not be useful for everybody, but they work well for me.

First, some syntax highlighting for file types vim doesn’t know about by default for .NET development.

" Set syntax highlighting for some .NET file types as XML files, cause that's what they are really.
autocmd BufNewFile,BufReadPost *.config set filetype=xml
autocmd BufNewFile,BufReadPost *.csproj set filetype=xml
autocmd BufNewFile,BufReadPost *.sln set filetype=xml

Then, some key mappings to access plugin features or plain vim commands quicker.

" Add a mapping to open a new tab with CTRL-T.
map  :tabe 
" F11 toggles menu visibility
nnoremap  :if &go=~#'m'set go-=melseset go+=mendif

" cd sets path to the path of the file in the current buffer.
nnoremap cd :cd %:p:h
" Open the NERDTree on the path of the file in the current buffer.
nnoremap t :NERDTree %:p:h

That’s about it. I hope this helps to get your own .vimrc set up the way you want it to. I’m not a vim expert so there are probably better ways to do some of these things. I you know how, feel free to let me know in the comments. I’m eager to learn some nice vim hacks.

As I said before, don’t just copy paste stuff in your vimrc you don’t understand. Take a look at the vim help first. Also, check out other people’s vimrc and see how they do things. There are plenty of good blog posts around on the subject and a github search turns up some good stuff too.