Category Archives: opensource

warmup your site or wordpress blog with a single command line statement

mother

GNU Wget is a powerful tool when it comes to downloading files from the web or mirroring sites. It’s command line features can be daunting and not very obvious. With some experimentation, reading the (f..) manual and some Googling you can get it to do some pretty neat tricks for you.
All of that is from the command line too, which is great if you want to schedule this kind of magic or use it in a script.

For example, you might want to warm-up your site or WordPress blog so your homepage and all posts linked from it are present in your cache when a visitor arrives. I’m assuming you are using a caching on your site otherwise this is pretty pointless. For WordPress you can use a caching plugin like W3 Total Cache for example.

With Wget, it goes like this:

wget.exe http://n3wjack.net --spider --no-directories --level=1 --recursive 
         --accept-regex=n3wjack.net/20[1..9].*

The command line parameters (in order) mean something like:

  1. Crawl n3wjack.net.
  2. Crawl it like a spider (follow the links).
  3. Don’t create directories for downloads.
  4. Crawl 1 level deep (so anything linked on the homepage is OK, but don’t go deeper).
  5. Do this recursively (so it actually goes 1 level deep).
  6. Follow only links that start with "/201..." to "/209..." (it’s a regular expression).
    This one is a trick to have it only follow links to blog-posts because my URL scheme begins with the year of the post (2015, 2016, …). It’s good until 2099, which should do the trick I guess. :)
    This way I’m also avoiding it loading all tag, category or page links.

If your site has a different URL scheme you’ll have to change the accept regex pattern to fit your scheme.

You can download Wget from the GNU site. It’s Open Source and is available for Windows, Mac and various Unix systems.
For Chocolatey users, there is a wget package available to install it on your system.

a vim http log file syntax plugin

Log files are dull to look at. Lines and lines of text and no pretty colors to make it nicer to look at and easier to spot those weird errors you can’t simulate on your machine.

Vim rocks and writing a syntax file is supposed to be a breeze judging from the vast amount of syntax plugins out there. I didn’t quite find one I liked for syntax highlighting HTTP log files, so I thought I’d get down and dirty with some vimscript myself and see if I could hack something together.

It turned out alright I think. So to share the fun I’m hosting the logsyntax.vim plugin on Github and the vim.org scripts library for all to use. It highlights dates, HTTP verbs, URLs, IP addresses etc for IIS, W3C extended, NCSA and probably a bunch more typical log formats.

May your logs be pretty and colorful from now on.

The vim log file syntax highlighting plugin screenshot. Look at those pretty colors.

a wordpress full site spell checker tool

A while ago I noticed that some of my older posts had some silly misspellings in it, so I was looking for a way to spell check all my posts in one shot. I couldn’t really find anything that was free, so I figured I’d try to write something myself to do this for me.

I knew about the free and open source Hunspell spell checker and that you can use it from the command line. So I thought using that together with the WordPress export XML file which has all your post’s content it should be possible to spell check the whole lot.

The end result is a PowerShell script which reads out the XML export file and runs it through Hunspell, parses the spelling errors found and finally bundling it all into a simple HTML report.

It worked nicely for me, even though it’s pretty crude and simple. I only had to use this once, so I don’t see the point of fine-tuning it a lot further.

However this could be handy for others who want to do the same thing, so I cleaned it up a bit, slapped a readme file on it and posted it on Github as the WordPress full site spell checker.
Check it out if you want to spell check your WordPress blog in a single run and maybe this will be good enough to get your job done. You find more info on how to set up and use it on the Github page.

That very basic report I was talking about.

configuring vim: tweaking your _vimrc file

.vimrc

In my earlier post on setting up Vim on Windows I already talked about a few vimrc changes to make vim behave on windows and look a bit better.
I learned that checking out other people’s vimrc files is a great way to learn how you can configure vim to become the kick-ass text editor you always wanted. In this post I’m going to run over the settings I have in my _vimrc file to make vim work better for me. Feel free to copy bits your find useful.
If you don’t know what that vimrc file is, or how to edit it, check out my previous post on how to set things up.

I tend to document my vimrc changes with some comments. That way I still know what I was trying to do months later and or what lines can be removed when a wicked plugin makes them obsolete. Most of it is self-explanatory, so I’ll just paste them right here.

First some system settings.

" Activate all the handy Windows key-bindings we're used to.
source $VIMRUNTIME/mswin.vim

" Have gvim behave properly on Windows.
behave mswin

" Use unicode/utf-8 encoding by default for keyboard, display and files.
set encoding=utf-8

" Set a more convenient leader key on an AZERTY layout than the default backslash
let mapleader = ","

Now let’s make things prettier.
First of all, we need to set a nicer fixed-width font, Consolas in this case. Then I hide the toolbar which I never use. Then, of course, a nice theme.

if has("gui_running")
  " Set a nicer font.
  set guifont=Consolas:h11:cDEFAULT
  " Hide the toolbar.
  set guioptions-=T
endif

" Set theme.
color badwolf

Then a number of visual things to make programming, editing and searching text easier.

" Display line and column number in bottom ruler.
set ruler
" Display the line numbers.
set number
" Turn sounds off.
set visualbell
" Shows a horizontal highlight on the line with the cursor.
set cursorline

" Activate highlighting search pattern matches & incremental search.
" Incremental search means your cursor will jump to the first match as you
" type.
set hlsearch
set incsearch
" Allow using  to kill the current search highlighting.
nnoremap   :nohlsearch

" Activate case-insensitive & smart case search (if a capital letter is used
" in your search query, Vim will search case-sensitive).
set ignorecase 
set smartcase

" Set wildchar visual completion awesomeness.
" This is enhanced command line completion and it rocks.
set wildmenu 
set wildmode=full

" Turning on line wrapping and line-break for easy text-file editing.
" Line-break wraps full words at the end of a sentence for readability.
set wrap
set linebreak

Programming settings. Who would have thought!
Tweak tab settings according to your religious preference of course. I can’t say I have used the folding a lot, but these settings come from another .vimrc somewhere and they seem sensible.

" Activate syntax highlighting.
syntax enable

" Set tabs to 4 characters and expand to spaces, activate smart indentation.
" See tabstop help for more info.
" Setting tabstop & softtabstop to the same value to avoid messy layout with mixed tabs & spaces.
set tabstop=4
set shiftwidth=4
set softtabstop=4
set expandtab
set smartindent

" Enabled folding on indent level. That way it works on any code & html, xml
" etc. 
" Setting foldlevelstart ensures that for newly opened files folds are open
" unless they are 10 levels deep.
set foldmethod=indent
set foldenable
set foldlevelstart=10
set foldnestmax=10      " no more than 10 fold levels please

At the end of my vimrc I add my custom tweaks. They might not be useful for everybody, but they work well for me.

First, some syntax highlighting for file types vim doesn’t know about by default for .NET development.

" Set syntax highlighting for some .NET file types as XML files, cause that's what they are really.
autocmd BufNewFile,BufReadPost *.config set filetype=xml
autocmd BufNewFile,BufReadPost *.csproj set filetype=xml
autocmd BufNewFile,BufReadPost *.sln set filetype=xml

Then, some key mappings to access plugin features or plain vim commands quicker.

" Add a mapping to open a new tab with CTRL-T.
map  :tabe 
" F11 toggles menu visibility
nnoremap  :if &go=~#'m'set go-=melseset go+=mendif

" cd sets path to the path of the file in the current buffer.
nnoremap cd :cd %:p:h
" Open the NERDTree on the path of the file in the current buffer.
nnoremap t :NERDTree %:p:h

That’s about it. I hope this helps getting your own .vimrc set up the way you want it to. I’m not a vim expert so there are probably better ways to do some of these things. I you know how, feel free to let me know in the comments. I’m eager to learn some nice vim hacks.

As I said before, don’t just copy paste stuff in your vimrc you don’t understand. Take a look at the vim help first. Also check out other people’s vimrc and see how they do things. There are plenty of good blog posts around on the subject and a github search turns up some good stuff too.

why an open sourced .NET framework is huge

free

I was already excited about the recent ASP.NET vNext developments. Things like the fact that you can get the ASP.NET source code on Github, that it’s completely FOSS and that it’s disconnected from the rest of the .NET stack are just plain awesome.

A huge step for ASP.NET vNext is that you don’t need Visual Studio to write software with it. You can use your favourite text editor like Vim, Sublime, Emacs or whatever you like, together with a number of open source command line tools.

A second huge thing is that ASP.NET 5 can now run cross-platform using Mono on Linux and Mac. Not only can you use your development tools of choice to write and build your C# code, you can also do it on the OS of your choice. .NET everywhere. Think about it. *mind blown*

Yesterday however, things got even sweeter as Microsoft is now releasing more of the v5 .NET Framework as open source. This means more and easier cross-platform development and Mono compatibility (as the source can be easily integrated in Mono) for .NET code.

On top of that there is now a new Visual Studio Community edition of Visual Studio available for free. This is equal to the Pro version, so you can ditch those crippled free “Express” versions and write code in the tools you’re used to professionally. I love this one. I’ve messed around with SharpDevelop and the Express version, but if you’re used to the “real thing” it feels like having to work with your hands tied behind your back.

As if this wasn’t enough there’s a bunch of other cool improvements too, like getting only the .NET framework components you need for your project and pull them in using NuGet. Scott Hanselman sums them up nicely.

So if you’ve always wanted to check out .NET or C# but didn’t want to because you had to run it on Windows and in Visual Studio, there’s nothing holding you back any more. For .NET developers this is great. It gives us more freedom than ever without having to learn a new language and framework. For people hacking away on OS X and Ubuntu with Ruby, Python etc. because they want to use FOSS, this is an opportunity to dip into the wealth of .NET resources out there and try something new.

The strategy is clear. They want everyone to use the .NET framework, they want everybody to run that code on Azure (even if you’re not using .NET) and they see that making it open is the only way to get there. Great times are ahead.

setting up vim on windows

So you want to make the plunge and check out what that awesome Vim editor is all about on a Windows machine? I did about a year ago, and this time I actually stuck with it. I tried it 2 times before already, but I gave up every time after a few hours or days because of the steep learning curve and the fact that nothing seemed to work the way I’m used to.

The goal here is to get Vim set up on a Windows box, make it work like a Windows app and make it look like something you aren’t afraid of being seen with (that default layout, euhm, seriously…).

installing Vim on Windows

The easiest way to install gVim (the non-console version) on Windows is by using the Vim Chocolatey package. It installs the Cream Windows flavored Vim for you from Sourceforge and sets up your system path so you can run “vim” and “gvim” to start the editor from the command line if you want to.

Chocolatey itself is like apt-get for Windows. It’s a NuGet based software package manager and if you’re a software developer or have to install systems a lot, I’m sure you’ll love it.

To install using Chocolatey, type this from a command shell (run in administrator mode):

    choco install vim

Easy isn’t it?
If you really don’t want to setup Chocolatey (but you should really) you can also get the Windows binaries from vim.org, or the Cream ones. You’ll get the Cream editor with that based on Vim, but I didn’t want that cause it’s not quite pure Vim anymore (not modal for example).

first steps

Before you start configuring Vim you should start with getting to know it. Whatever comes next will be a lot easier once you know how to move around, save files etc. You know, those typical things you do in a text editor anyway.
So basically you have to RTFM. Or at least, the short version of it.

Find the Vim tutor in your start menu and run it. Once you’ve completed that, you know enough to get around in Vim and find out how to get help for other stuff. Also, don’t bother with doing it the hardcore way. Just learn Vim at your own pace, use the cursor keys and the mouse if you want. You’ll pick up more advanced stuff as you go along.
Just like this guy said.

configuring Vim

Vim uses vimrc configuration files to store settings. There is a default vimrc file in your Vim installation folder, but you shouldn’t touch that one because it can be overwritten when you install a Vim update. Your own settings go into a new _vimrc file containing only the things you want. You can “source” or include other vimrc files, to avoid duplication or organize your configuration.

On a Windows machine this goes into your user folder (c:\users\username\_vimrc).

To see and edit your current vimrc (the one use by vim when it starts) type:

    :edit $MYVIMRC

Once you’ve created your own config file, the above command will automatically open it. So let’s do just that, like this. “e” is short for “edit” btw:

    :e ~\_vimrc

That tilde character (~) stands for the path to you home folder, which on Windows is normally set to c:\users\<username>.

On Windows the folder  ~\vimfiles is used to store custom plugins. I put my vimrc in ~\vimfiles and source it from the ~\_vimrc file. That way I easily sync my whole setup to another machine by simply copying the whole vimfiles folder. So my ~\vimrc file contains only this line.

source ~\vimfiles\nj-vimrc.vim

You can keep everything in _vimrc of course if you want.

You can test Vim statements from the command bar before you put them in your vimrc file. A handy way to test and see if it really does what you want it to without having to restart Vim and reload your configuration.

making Vim behave like a Windows editor

If you install the Windows gVim version this will normally be setup in your vimrc configuration file automatically, but it’s always handy to know what settings are making the magic happen.
Vim doesn’t use the typical Windows key binding like CTRL-S to save, CTRL-C,V,X to copy, paste and cut. As a hardcore Windows user you expect these to work for all applications, no matter how geeky they are. If you check out the original vimrc in the Vim installation folder you’ll see the same statements as below.
You can copy these into your own vimrc file.

" Activate all the handy Windows key-bindings we're used to.
source $VIMRUNTIME/mswin.vim

behave mswin

Or you can source the original config in yours:

source $VIM/_vimrc

Now your Vim should (still) behave pretty much like a regular Windows editor, making things a lot easier to use it for all your basic editing.

making it look good

gvim_fugly

Man, it does look ugly doesn’t it? All white and with that terrible font? Now let’s make it look pretty so you don’t have to be ashamed of all those hipsters sporting their fancy Sublime.

This depends on your personal preference of course, but I like it when my editor doesn’t look like it’s from the 80’s. I pasted some of my esthetic config changes below. Each has some comments on it explaining what they do. Use the help feature (:h ) if you want to find out more, and remember to only put things in your vimrc that you understand.

" Display line and column number in bottom ruler.
set ruler

" Display the line numbers.
set number

" Activate syntax highlighting.
syntax enable

" Set a nice theme.
color slate

Check out all the themes you have installed by using CTRL-D (auto-complete) after you typed in :color on the command bar.

if has("gui_running")
  " Set a nicer font.
  set guifont=Consolas:h11:cDEFAULT
  " Hide the toolbar.
  set guioptions-=T
endif

gvim looking pretty

You can exclude hiding the toolbar, but I just find it ugly and never use those buttons anyway. This is very minimal, and there are plenty of other themes and plugins available to change Vim’s appearance, but that’ll be for another post. For more vim customization tips, check out what I have in my _vimrc file.

Hopefully this gets you going in the wonderful world of the Vim editor on Windows. Just keep at it I’d say and feel free to drop any questions or remarks in the comments.

Edit: replaced the KicksassVim Chocolatey package with the Vim one because the first doesn’t seem to be updated any longer.

cool geek (dev) stuff I ran into lately

... "Mr. Droopy Eyes!"

I’ve had this list around for a while and though that most people would probably have heard of this by now so I didn’t see the point in posting about it.
Until last weekend someone on twitter was happy to find out about Chocolatey. So I guess not everybody knows these little gems yet, hence this blog post!

  • Chocolatey: a Windows packages manager of sorts. A bit like apt-get on Debian. It allows you to install a bunch of Windows software and tools from the command line. It’s pretty cool and is super handy to get a (developer) box up and running in no time. It’s also handy to keep your installed package up-to-date with the “cup all” statement. Sweet.
    There’s lot’s of good stuff in the gallery already, so you’ll probably find your favorite tool in there. If not, you can add it yourself because it’s built on the NuGet package manager system, or browse what’s available and find some new gems you didn’t know about yet.
  • I haven’t really used Boxstarter myself yet, but if you’re planning on using Chocolatey for some serious VM Windows installer magic, it might come in handy. It builds on top of Chocolatey and allows 100% uninterrupted Windows installs. Thought it was worth mentioning.
  • ScriptCS: one of Glenn Block & co little open source coding adventures. He thought it would be cool to use C# and the .NET framework to run scripts on Windows using the Roslyn compiler API. No need for Visual Studio, project files, compilers or anything like that. Just the scriptcs executable and a text file with your C# script code. Much like Node.js or Python for example. You know, scripting languages.
    Turns out this idea took off like a rocket in the community and has all sorts of cool features by now, like Nuget integration and script packs for reusability. It’s awesome.
  • dotnetfiddle.net : It’s jsfiddle for C# code. It’s a web site where you can type some C# code in a console application, run it and see your output instantly. Great of small bits of test code. It even has intellisense support so it’s easier if to use than LinqPad for this kind of tests apps if you don’t know all the statements by heart.
  • devdocs.io: all web dev docs in one place and easily searchable. Contains docs for thing like the HTML5 spec, JS, HTTP, HTML DOM and the most popular frameworks like Ember, Backbone, Angular, Knockout and Underscore. Also language like Python, Node, Ruby etc. In short, useful stuff for any web developer working with a modern stack.

Image by James Vaughan, cc-licensed.