Category Archives: programming

a vim http log file syntax plugin

Log files are dull to look at. Lines and lines of text and no pretty colors to make it nicer to look at and easier to spot those weird errors you can’t simulate on your machine.

Vim rocks and writing a syntax file is supposed to be a breeze judging from the vast amount of syntax plugins out there. I didn’t quite find one I liked for syntax highlighting HTTP log files, so I thought I’d get down and dirty with some vimscript myself and see if I could hack something together.

It turned out alright I think. So to share the fun I’m hosting the logsyntax.vim plugin on Github and the vim.org scripts library for all to use. It highlights dates, HTTP verbs, URLs, IP addresses etc for IIS, W3C extended, NCSA and probably a bunch more typical log formats.

May your logs be pretty and colorful from now on.

The vim log file syntax highlighting plugin screenshot. Look at those pretty colors.

a wordpress full site spell checker tool

A while ago I noticed that some of my older posts had some silly misspellings in it, so I was looking for a way to spell check all my posts in one shot. I couldn’t really find anything that was free, so I figured I’d try to write something myself to do this for me.

I knew about the free and open source Hunspell spell checker and that you can use it from the command line. So I thought using that together with the WordPress export XML file which has all your post’s content it should be possible to spell check the whole lot.

The end result is a PowerShell script which reads out the XML export file and runs it through Hunspell, parses the spelling errors found and finally bundling it all into a simple HTML report.

It worked nicely for me, even though it’s pretty crude and simple. I only had to use this once, so I don’t see the point of fine-tuning it a lot further.

However this could be handy for others who want to do the same thing, so I cleaned it up a bit, slapped a readme file on it and posted it on Github as the WordPress full site spell checker.
Check it out if you want to spell check your WordPress blog in a single run and maybe this will be good enough to get your job done. You find more info on how to set up and use it on the Github page.

That very basic report I was talking about.

find and delete duplicate files with just Powershell

... analog computer!

Consider this. You have the same files with different file names spread out over a bunch of folders. If you are on a recent Windows machine, Powershell is all you need to get out of that mess and delete the duplicates.
This also means you get to do this from the command line which makes it extra l33t.

Cool. Let’s get started.

ls *.* -recurse | get-filehash | group -property hash | where { $_.count -gt 1 } 
| % { $_.group | select -skip 1 } | del

Bam!
You’re done.

Alright. Here’s what going on in detail:

ls *.* -recurse             # get all files in the current folder and all subfolders
get-filehash                # calculate the file hash for all files
group -property hash        # group them by the hash value
where { $_.count -gt 1 }    # select those groups with more than 1 item
% {                         # for all groups
    $_.group |              # output the group content, which are the files
    select -skip 1          # select all but the first file 
   }                        # (we don't want to delete all of them right?)
del                         # delete the file

If you want to experiment with this I’d recommend you to change the last del command with something safer, like echo which just prints out the file.

Oh yeah, **DISCLAIMER**. Don’t just randomly copy past Powershell code and execute it on your machine if you don’t know what you are doing. Certainly if it’s deleting files like the example above. You might end up deleting more than you bargained for.  :)

Photo by James Vaughan, cc-licensed.

yo dawg I heard you like package managers

Let’s say you want to do this little web project, a SPA for example, and you want to settle for Angular as your JS framework. But downloading those scripts manually is so oldskool, so you need yourself a package manager to shoot those into your still empty project folder.

For example, with something like Bower, “the package manager or the web”:

bower install angular

Cool, so how do I get Bower?

npm install -g bower

Node Package Manager huh. Hmm. So I’ll need Node first. OK. I bet there’s a Chocolatey package for that.

choco install nodejs

Awesome! So how do I get Chocolatey (we’re clearly on Windows here, use your fav *nix distro package manager otherwise)?

@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

Alright, now we’re talking!
What was I doing again?

As a Windows .NET dev you could also use Visual Studio Community 2013 of course, open up the Nuget Package Manager Console and run:

Install-Package angularjs

But how do you get Visual Studio (*)?
Oh, there’s Chocolatey again.

choco install visualstudiocommunity2013

Or maybe you can use WebPi…, or… argh. Never mind.

(Post inspiration by @mattiasgeniar)

(*) As a .NET developer you have this installed already of course, but to setup a new machine, Choco is the bomb.

configuring vim: tweaking your _vimrc file

.vimrc

In my earlier post on setting up Vim on Windows I already talked about a few vimrc changes to make vim behave on windows and look a bit better.
I learned that checking out other people’s vimrc files is a great way to learn how you can configure vim to become the kick-ass text editor you always wanted. In this post I’m going to run over the settings I have in my _vimrc file to make vim work better for me. Feel free to copy bits your find useful.
If you don’t know what that vimrc file is, or how to edit it, check out my previous post on how to set things up.

I tend to document my vimrc changes with some comments. That way I still know what I was trying to do months later and or what lines can be removed when a wicked plugin makes them obsolete. Most of it is self-explanatory, so I’ll just paste them right here.

First some system settings.

" Activate all the handy Windows key-bindings we're used to.
source $VIMRUNTIME/mswin.vim

" Have gvim behave properly on Windows.
behave mswin

" Use unicode/utf-8 encoding by default for keyboard, display and files.
set encoding=utf-8

" Set a more convenient leader key on an AZERTY layout than the default backslash
let mapleader = ","

Now let’s make things prettier.
First of all, we need to set a nicer fixed-width font, Consolas in this case. Then I hide the toolbar which I never use. Then, of course, a nice theme.

if has("gui_running")
  " Set a nicer font.
  set guifont=Consolas:h11:cDEFAULT
  " Hide the toolbar.
  set guioptions-=T
endif

" Set theme.
color badwolf

Then a number of visual things to make programming, editing and searching text easier.

" Display line and column number in bottom ruler.
set ruler
" Display the line numbers.
set number
" Turn sounds off.
set visualbell
" Shows a horizontal highlight on the line with the cursor.
set cursorline

" Activate highlighting search pattern matches & incremental search.
" Incremental search means your cursor will jump to the first match as you
" type.
set hlsearch
set incsearch
" Allow using  to kill the current search highlighting.
nnoremap   :nohlsearch

" Activate case-insensitive & smart case search (if a capital letter is used
" in your search query, Vim will search case-sensitive).
set ignorecase 
set smartcase

" Set wildchar visual completion awesomeness.
" This is enhanced command line completion and it rocks.
set wildmenu 
set wildmode=full

" Turning on line wrapping and line-break for easy text-file editing.
" Line-break wraps full words at the end of a sentence for readability.
set wrap
set linebreak

Programming settings. Who would have thought!
Tweak tab settings according to your religious preference of course. I can’t say I have used the folding a lot, but these settings come from another .vimrc somewhere and they seem sensible.

" Activate syntax highlighting.
syntax enable

" Set tabs to 4 characters and expand to spaces, activate smart indentation.
" See tabstop help for more info.
" Setting tabstop & softtabstop to the same value to avoid messy layout with mixed tabs & spaces.
set tabstop=4
set shiftwidth=4
set softtabstop=4
set expandtab
set smartindent

" Enabled folding on indent level. That way it works on any code & html, xml
" etc. 
" Setting foldlevelstart ensures that for newly opened files folds are open
" unless they are 10 levels deep.
set foldmethod=indent
set foldenable
set foldlevelstart=10
set foldnestmax=10      " no more than 10 fold levels please

At the end of my vimrc I add my custom tweaks. They might not be useful for everybody, but they work well for me.

First, some syntax highlighting for file types vim doesn’t know about by default for .NET development.

" Set syntax highlighting for some .NET file types as XML files, cause that's what they are really.
autocmd BufNewFile,BufReadPost *.config set filetype=xml
autocmd BufNewFile,BufReadPost *.csproj set filetype=xml
autocmd BufNewFile,BufReadPost *.sln set filetype=xml

Then, some key mappings to access plugin features or plain vim commands quicker.

" Add a mapping to open a new tab with CTRL-T.
map  :tabe 
" F11 toggles menu visibility
nnoremap  :if &go=~#'m'set go-=melseset go+=mendif

" cd sets path to the path of the file in the current buffer.
nnoremap cd :cd %:p:h
" Open the NERDTree on the path of the file in the current buffer.
nnoremap t :NERDTree %:p:h

That’s about it. I hope this helps getting your own .vimrc set up the way you want it to. I’m not a vim expert so there are probably better ways to do some of these things. I you know how, feel free to let me know in the comments. I’m eager to learn some nice vim hacks.

As I said before, don’t just copy paste stuff in your vimrc you don’t understand. Take a look at the vim help first. Also check out other people’s vimrc and see how they do things. There are plenty of good blog posts around on the subject and a github search turns up some good stuff too.

why an open sourced .NET framework is huge

free

I was already excited about the recent ASP.NET vNext developments. Things like the fact that you can get the ASP.NET source code on Github, that it’s completely FOSS and that it’s disconnected from the rest of the .NET stack are just plain awesome.

A huge step for ASP.NET vNext is that you don’t need Visual Studio to write software with it. You can use your favourite text editor like Vim, Sublime, Emacs or whatever you like, together with a number of open source command line tools.

A second huge thing is that ASP.NET 5 can now run cross-platform using Mono on Linux and Mac. Not only can you use your development tools of choice to write and build your C# code, you can also do it on the OS of your choice. .NET everywhere. Think about it. *mind blown*

Yesterday however, things got even sweeter as Microsoft is now releasing more of the v5 .NET Framework as open source. This means more and easier cross-platform development and Mono compatibility (as the source can be easily integrated in Mono) for .NET code.

On top of that there is now a new Visual Studio Community edition of Visual Studio available for free. This is equal to the Pro version, so you can ditch those crippled free “Express” versions and write code in the tools you’re used to professionally. I love this one. I’ve messed around with SharpDevelop and the Express version, but if you’re used to the “real thing” it feels like having to work with your hands tied behind your back.

As if this wasn’t enough there’s a bunch of other cool improvements too, like getting only the .NET framework components you need for your project and pull them in using NuGet. Scott Hanselman sums them up nicely.

So if you’ve always wanted to check out .NET or C# but didn’t want to because you had to run it on Windows and in Visual Studio, there’s nothing holding you back any more. For .NET developers this is great. It gives us more freedom than ever without having to learn a new language and framework. For people hacking away on OS X and Ubuntu with Ruby, Python etc. because they want to use FOSS, this is an opportunity to dip into the wealth of .NET resources out there and try something new.

The strategy is clear. They want everyone to use the .NET framework, they want everybody to run that code on Azure (even if you’re not using .NET) and they see that making it open is the only way to get there. Great times are ahead.

speeding up your TFS CI builds

Entering Hyperspace

If you’re using TFS as your build server for CI builds of .NET projects, you want your CI build to be bleeding fast so devs don’t sit around waiting until their build completes. This is even more the case if you use a gated check-in on that CI build.

The ideal time frame is less than 5 minutes, but as a solution grows and more projects are added it gets hard to stay below that time limit without tweaking the build. With a CI build in this case I mean a build used to check if all code compiles, is integrated with other code and all unit tests pass on it.
This build is not used for deployment. There should be a separate build for that.

So what are the time sinks here in a standard TFS build?

It helps if you know what’s it’s doing, even though it’s pretty straightforward. First your code is fetched from TFS. A local workspace is used on the build server and all files are retrieved. Then the VS solutions build with MSBuild. Code analysis is performed if requested, unit tests run, outputs are copied to your drop folder and build reports are published.

Here are a number of things you can do to speed this up:

  • Get only the latest changes, not the full code base every time by not completely cleaning up the workspace. If you only clean up the outputs, it takes a lot less time to update the workspace. The more code you have, the more this counts.
  • Code analysis is CPU intensive and slows down your build significantly. Make sure you set this to “as configured” in your build configuration, so it doesn’t run on projects that you don’t want analysed. Then configure your projects in Visual Studio that need CA, and disable it for the rest.
  • In the Source and symbol server settings, set “Index sources” to false for the CI build. This feature includes source information into your PDB files which makes things easier to debug later on, but since we’re building strictly for a CI build we don’t need this data afterwards.
  • Disable copying output files to the drop folder. The output of the CI build shouldn’t be used for any deployments, so this can all be skipped to speed up the build.
  • Look for slow tests. Often these are integration-like tests which will be causing disk I/O by manipulating files or accessing a database etc. You can disable them for the CI build only by adding a TestCategory attribute called “Integration” (or whatever you like) and exclude those from the build in the test settings. Another trick is to put all your integration tests in a separate assembly and exclude the tests from running using the test assembly file pattern.  E.g. With *.Test.dll project Foo’s Foo.Test.dll is included, but Foo.IntegrationTest.dll isn’t.
  • Avoid lots of small projects in your solution. Every project will take a few extra seconds to build, so merging them into a single larger project will build faster. Make good use of namespaces instead. For example one assembly for unit tests (or 2 if you take that integration test project in mind) is better than a test project for each regular project.

Pick and choose as needed, but these should give you a considerable boost.
Remember to analyse your build before starting to tweak it. There’s no point in optimizing what’s fast as hell already. A good tool to check out what’s taking so long is the open source TFS Build Explorer. It shows your build steps in a tree-view, including timing info and is a hell of a lot quicker than the default build output from TFS for big builds. Depending on your TFS server version you might have to use an older executable, but they are all available in the download section.

Photo by Éole Wind, cc-licensed.