Category Archives: programming

find and delete duplicate files with just Powershell

... analog computer!

Consider this. You have the same files with different file names spread out over a bunch of folders. If you are on a recent Windows machine, Powershell is all you need to get out of that mess and delete the duplicates.
This also means you get to do this from the command line which makes it extra l33t.

Cool. Let’s get started.

ls *.* -recurse | get-filehash | group -property hash | where { $_.count -gt 1 } 
| % { $ | select -skip 1 } | del

You’re done.

Alright. Here’s what going on in detail:

ls *.* -recurse             # get all files in the current folder and all subfolders
get-filehash                # calculate the file hash for all files
group -property hash        # group them by the hash value
where { $_.count -gt 1 }    # select those groups with more than 1 item
% {                         # for all groups
    $ |              # output the group content, which are the files
    select -skip 1          # select all but the first file 
   }                        # (we don't want to delete all of them right?)
del                         # delete the file

If you want to experiment with this I’d recommend you to change the last del command with something safer, like echo which just prints out the file.

Oh yeah, **DISCLAIMER**. Don’t just randomly copy past Powershell code and execute it on your machine if you don’t know what you are doing. Certainly if it’s deleting files like the example above. You might end up deleting more than you bargained for.  :)

Photo by James Vaughan, cc-licensed.

yo dawg I heard you like package managers

Let’s say you want to do this little web project, a SPA for example, and you want to settle for Angular as your JS framework. But downloading those scripts manually is so oldskool, so you need yourself a package manager to shoot those into your still empty project folder.

For example, with something like Bower, “the package manager or the web”:

bower install angular

Cool, so how do I get Bower?

npm install -g bower

Node Package Manager huh. Hmm. So I’ll need Node first. OK. I bet there’s a Chocolatey package for that.

choco install nodejs

Awesome! So how do I get Chocolatey (we’re clearly on Windows here, use your fav *nix distro package manager otherwise)?

@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString(''))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

Alright, now we’re talking!
What was I doing again?

As a Windows .NET dev you could also use Visual Studio Community 2013 of course, open up the Nuget Package Manager Console and run:

Install-Package angularjs

But how do you get Visual Studio (*)?
Oh, there’s Chocolatey again.

choco install visualstudiocommunity2013

Or maybe you can use WebPi…, or… argh. Never mind.

(Post inspiration by @mattiasgeniar)

(*) As a .NET developer you have this installed already of course, but to setup a new machine, Choco is the bomb.

configuring vim: tweaking your _vimrc file


In my earlier post on setting up Vim on Windows I already talked about a few vimrc changes to make vim behave on windows and look a bit better.
I learned that checking out other people’s vimrc files is a great way to learn how you can configure vim to become the kick-ass text editor you always wanted. In this post I’m going to run over the settings I have in my _vimrc file to make vim work better for me. Feel free to copy bits your find useful.
If you don’t know what that vimrc file is, or how to edit it, check out my previous post on how to set things up.

I tend to document my vimrc changes with some comments. That way I still know what I was trying to do months later and or what lines can be removed when a wicked plugin makes them obsolete. Most of it is self-explanatory, so I’ll just paste them right here.

First some system settings.

" Activate all the handy Windows key-bindings we're used to.
source $VIMRUNTIME/mswin.vim

" Have gvim behave properly on Windows.
behave mswin

" Use unicode/utf-8 encoding by default for keyboard, display and files.
set encoding=utf-8

" Set a more convenient leader key on an AZERTY layout than the default backslash
let mapleader = ","

Now let’s make things prettier.
First of all, we need to set a nicer fixed-width font, Consolas in this case. Then I hide the toolbar which I never use. Then, of course, a nice theme.

if has("gui_running")
  " Set a nicer font.
  set guifont=Consolas:h11:cDEFAULT
  " Hide the toolbar.
  set guioptions-=T

" Set theme.
color badwolf

Then a number of visual things to make programming, editing and searching text easier.

" Display line and column number in bottom ruler.
set ruler
" Display the line numbers.
set number
" Turn sounds off.
set visualbell
" Shows a horizontal highlight on the line with the cursor.
set cursorline

" Activate highlighting search pattern matches & incremental search.
" Incremental search means your cursor will jump to the first match as you
" type.
set hlsearch
set incsearch
" Allow using  to kill the current search highlighting.
nnoremap   :nohlsearch

" Activate case-insensitive & smart case search (if a capital letter is used
" in your search query, Vim will search case-sensitive).
set ignorecase 
set smartcase

" Set wildchar visual completion awesomeness.
" This is enhanced command line completion and it rocks.
set wildmenu 
set wildmode=full

" Turning on line wrapping and line-break for easy text-file editing.
" Line-break wraps full words at the end of a sentence for readability.
set wrap
set linebreak

Programming settings. Who would have thought!
Tweak tab settings according to your religious preference of course. I can’t say I have used the folding a lot, but these settings come from another .vimrc somewhere and they seem sensible.

" Activate syntax highlighting.
syntax enable

" Set tabs to 4 characters and expand to spaces, activate smart indentation.
" See tabstop help for more info.
" Setting tabstop & softtabstop to the same value to avoid messy layout with mixed tabs & spaces.
set tabstop=4
set shiftwidth=4
set softtabstop=4
set expandtab
set smartindent

" Enabled folding on indent level. That way it works on any code & html, xml
" etc. 
" Setting foldlevelstart ensures that for newly opened files folds are open
" unless they are 10 levels deep.
set foldmethod=indent
set foldenable
set foldlevelstart=10
set foldnestmax=10      " no more than 10 fold levels please

At the end of my vimrc I add my custom tweaks. They might not be useful for everybody, but they work well for me.

First, some syntax highlighting for file types vim doesn’t know about by default for .NET development.

" Set syntax highlighting for some .NET file types as XML files, cause that's what they are really.
autocmd BufNewFile,BufReadPost *.config set filetype=xml
autocmd BufNewFile,BufReadPost *.csproj set filetype=xml
autocmd BufNewFile,BufReadPost *.sln set filetype=xml

Then, some key mappings to access plugin features or plain vim commands quicker.

" Add a mapping to open a new tab with CTRL-T.
map  :tabe 
" F11 toggles menu visibility
nnoremap  :if &go=~#'m'set go-=melseset go+=mendif

" cd sets path to the path of the file in the current buffer.
nnoremap cd :cd %:p:h
" Open the NERDTree on the path of the file in the current buffer.
nnoremap t :NERDTree %:p:h

That’s about it. I hope this helps getting your own .vimrc set up the way you want it to. I’m not a vim expert so there are probably better ways to do some of these things. I you know how, feel free to let me know in the comments. I’m eager to learn some nice vim hacks.

As I said before, don’t just copy paste stuff in your vimrc you don’t understand. Take a look at the vim help first. Also check out other people’s vimrc and see how they do things. There are plenty of good blog posts around on the subject and a github search turns up some good stuff too.

why an open sourced .NET framework is huge


I was already excited about the recent ASP.NET vNext developments. Things like the fact that you can get the ASP.NET source code on Github, that it’s completely FOSS and that it’s disconnected from the rest of the .NET stack are just plain awesome.

A huge step for ASP.NET vNext is that you don’t need Visual Studio to write software with it. You can use your favourite text editor like Vim, Sublime, Emacs or whatever you like, together with a number of open source command line tools.

A second huge thing is that ASP.NET 5 can now run cross-platform using Mono on Linux and Mac. Not only can you use your development tools of choice to write and build your C# code, you can also do it on the OS of your choice. .NET everywhere. Think about it. *mind blown*

Yesterday however, things got even sweeter as Microsoft is now releasing more of the v5 .NET Framework as open source. This means more and easier cross-platform development and Mono compatibility (as the source can be easily integrated in Mono) for .NET code.

On top of that there is now a new Visual Studio Community edition of Visual Studio available for free. This is equal to the Pro version, so you can ditch those crippled free “Express” versions and write code in the tools you’re used to professionally. I love this one. I’ve messed around with SharpDevelop and the Express version, but if you’re used to the “real thing” it feels like having to work with your hands tied behind your back.

As if this wasn’t enough there’s a bunch of other cool improvements too, like getting only the .NET framework components you need for your project and pull them in using NuGet. Scott Hanselman sums them up nicely.

So if you’ve always wanted to check out .NET or C# but didn’t want to because you had to run it on Windows and in Visual Studio, there’s nothing holding you back any more. For .NET developers this is great. It gives us more freedom than ever without having to learn a new language and framework. For people hacking away on OS X and Ubuntu with Ruby, Python etc. because they want to use FOSS, this is an opportunity to dip into the wealth of .NET resources out there and try something new.

The strategy is clear. They want everyone to use the .NET framework, they want everybody to run that code on Azure (even if you’re not using .NET) and they see that making it open is the only way to get there. Great times are ahead.

speeding up your TFS CI builds

Entering Hyperspace

If you’re using TFS as your build server for CI builds of .NET projects, you want your CI build to be bleeding fast so devs don’t sit around waiting until their build completes. This is even more the case if you use a gated check-in on that CI build.

The ideal time frame is less than 5 minutes, but as a solution grows and more projects are added it gets hard to stay below that time limit without tweaking the build. With a CI build in this case I mean a build used to check if all code compiles, is integrated with other code and all unit tests pass on it.
This build is not used for deployment. There should be a separate build for that.

So what are the time sinks here in a standard TFS build?

It helps if you know what’s it’s doing, even though it’s pretty straightforward. First your code is fetched from TFS. A local workspace is used on the build server and all files are retrieved. Then the VS solutions build with MSBuild. Code analysis is performed if requested, unit tests run, outputs are copied to your drop folder and build reports are published.

Here are a number of things you can do to speed this up:

  • Get only the latest changes, not the full code base every time by not completely cleaning up the workspace. If you only clean up the outputs, it takes a lot less time to update the workspace. The more code you have, the more this counts.
  • Code analysis is CPU intensive and slows down your build significantly. Make sure you set this to “as configured” in your build configuration, so it doesn’t run on projects that you don’t want analysed. Then configure your projects in Visual Studio that need CA, and disable it for the rest.
  • In the Source and symbol server settings, set “Index sources” to false for the CI build. This feature includes source information into your PDB files which makes things easier to debug later on, but since we’re building strictly for a CI build we don’t need this data afterwards.
  • Disable copying output files to the drop folder. The output of the CI build shouldn’t be used for any deployments, so this can all be skipped to speed up the build.
  • Look for slow tests. Often these are integration-like tests which will be causing disk I/O by manipulating files or accessing a database etc. You can disable them for the CI build only by adding a TestCategory attribute called “Integration” (or whatever you like) and exclude those from the build in the test settings. Another trick is to put all your integration tests in a separate assembly and exclude the tests from running using the test assembly file pattern.  E.g. With *.Test.dll project Foo’s Foo.Test.dll is included, but Foo.IntegrationTest.dll isn’t.
  • Avoid lots of small projects in your solution. Every project will take a few extra seconds to build, so merging them into a single larger project will build faster. Make good use of namespaces instead. For example one assembly for unit tests (or 2 if you take that integration test project in mind) is better than a test project for each regular project.

Pick and choose as needed, but these should give you a considerable boost.
Remember to analyse your build before starting to tweak it. There’s no point in optimizing what’s fast as hell already. A good tool to check out what’s taking so long is the open source TFS Build Explorer. It shows your build steps in a tree-view, including timing info and is a hell of a lot quicker than the default build output from TFS for big builds. Depending on your TFS server version you might have to use an older executable, but they are all available in the download section.

Photo by Éole Wind, cc-licensed.

cool geek (dev) stuff I ran into lately

... "Mr. Droopy Eyes!"

I’ve had this list around for a while and though that most people would probably have heard of this by now so I didn’t see the point in posting about it.
Until last weekend someone on twitter was happy to find out about Chocolatey. So I guess not everybody knows these little gems yet, hence this blog post!

  • Chocolatey: a Windows packages manager of sorts. A bit like apt-get on Debian. It allows you to install a bunch of Windows software and tools from the command line. It’s pretty cool and is super handy to get a (developer) box up and running in no time. It’s also handy to keep your installed package up-to-date with the “cup all” statement. Sweet.
    There’s lot’s of good stuff in the gallery already, so you’ll probably find your favorite tool in there. If not, you can add it yourself because it’s built on the NuGet package manager system, or browse what’s available and find some new gems you didn’t know about yet.
  • I haven’t really used Boxstarter myself yet, but if you’re planning on using Chocolatey for some serious VM Windows installer magic, it might come in handy. It builds on top of Chocolatey and allows 100% uninterrupted Windows installs. Thought it was worth mentioning.
  • ScriptCS: one of Glenn Block & co little open source coding adventures. He thought it would be cool to use C# and the .NET framework to run scripts on Windows using the Roslyn compiler API. No need for Visual Studio, project files, compilers or anything like that. Just the scriptcs executable and a text file with your C# script code. Much like Node.js or Python for example. You know, scripting languages.
    Turns out this idea took off like a rocket in the community and has all sorts of cool features by now, like Nuget integration and script packs for reusability. It’s awesome.
  • : It’s jsfiddle for C# code. It’s a web site where you can type some C# code in a console application, run it and see your output instantly. Great of small bits of test code. It even has intellisense support so it’s easier if to use than LinqPad for this kind of tests apps if you don’t know all the statements by heart.
  • all web dev docs in one place and easily searchable. Contains docs for thing like the HTML5 spec, JS, HTTP, HTML DOM and the most popular frameworks like Ember, Backbone, Angular, Knockout and Underscore. Also language like Python, Node, Ruby etc. In short, useful stuff for any web developer working with a modern stack.

Image by James Vaughan, cc-licensed.

how to get code from tfs from the command line


Ever wanted to pull an update from the TFS server for all workspaces you have without having to start up the Visual Studio? Well I did. Visual Studio is rather slow in starting up and then you have to navigate to the TFS explorer (clicky clicky) and manually update the workspaces (clicky clicky) and then wait till the whole thing is done and VS becomes responsive again (waity waity).

You have the tf.exe command which you can use from a command prompt, but that requires you to use the VS command line because the right paths are set there. You could set the paths so they are available in all your shells, but that’s not very handy either.

I just wanted a batch script that I could run from any shell, that would just update my workspaces for me. Nice and easy.
Like when I press CTRL-R in Windows (Run) and type getfoo.cmd<enter>, it just makes it all happen for me.

So here’s how I did that:

@echo off


:: Importing VS 2012 command line variables so we can run TF.exe
if exist "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\VsDevCmd.bat" (
echo Importing VS 2012 environment variables...
call "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\VsDevCmd.bat"

echo *** Updating TFS workspaces ***

:: Repeat this for all workspaces you want to update.
cd D:\WS\pathtosolutionfiles\

tf get . /version:T /recursive

echo *** Workspace updated. ***