Category Archives: tips

publish a static website to Azure using GitHub actions

Last post I talked about setting up a serverless website on Azure using BLOB storage. What I didn’t go into is how to publish files to that site automatically. Yes, you can use the BLOB explorer to manually upload your files but seriously, who wants to do that kind of boring task if you can let computers do that for you.

Instead, what I do to publish this excellent programming guidelines website is the following:

  • I make my changes locally and test it.
  • I commit & push my changes to the master branch of my git repository.
  • A GitHub action kicks in and published the site to the Azure BLOB container.

How sweet is that? Pretty sweet, I know. How do you set this up? Well let me take you through the steps my friend, and automated Git deployment will soon be yours to enjoy as well.

  • You need to create a Git repository on GitHub.
    Now that you can create unlimited private repositories, you don’t even have to expose it to the public, which is nice.
  • Clone the repo locally, and create a source/ directory in it. This is where the source code will go, and that’s what we’ll push to the Azure BLOB container. Any other files you don’t want published go in the root, or in other folders in the root of your repository.
  • Copy your source code into the source/ folder, or create a simple index.html file for testing the publishing action.
  • Go to your repository page on the GitHub site, and click the Actions tab at the top.
  • Click New Workflow, choose “set up a workflow yourself”.
  • It will now create a YAML file for you containing your workflow code.
  • Paste the content for your YAML file listed below. Notice the “source” folder in there? That indicates what folder will be copied to Azure.
    In case you run into trouble, you can dig in to the Azure Storage Action setup yourself, but it should do the trick.
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    steps: 
    - uses: actions/checkout@v1
    - uses: actions/setup-dotnet@v1
      with:
        dotnet-version: '3.0.100'
    - uses: lauchacarro/Azure-Storage-Action@v1.0
      with:
        enabled-static-website: 'true'
        folder: 'source'
        index-document: 'index.html'
        error-document: '404.html' 
        connection-string: ${{ secrets.CONNECTION_STRING }}
  • Last step is to set up that CONNECTION_STRING secret. This is the connection string to your Azure storage container. You can set the secret from your GitHub repository Settings tab, under Secrets.
    Click New Secret, then use the name CONNECTION_STRING and paste the access key value from your Azure storage account.

You’re all set up!
To test your publishing flow, all you need to do now is push a commit to your master branch, and see the GitHub action kick in and do its thing.
You should see your site appear in a few seconds. Sweet!

Update: recently I found out the workflow broke because of a bug in the latest version of the action. To bypass this I now fixed the version in my workflow YAML file to v1.0, which still works. It’s probably a good idea to avoid this kind of breaking changes by fixing the version of any action you use in your GitHub actions anyway. It will avoid those annoying issues where things work one day, and don’t the next.

programming guidelines, sort of

Years ago I ran into a website offering crude design advice. I thought it would be funny to make something similar for programming advice or guidelines. I started with a one-page website with a bunch of tips and then after a while forgot about it. Recently I ran into that project again and figured I might as well put it out here for the heck of it.

So here it is, some good fucking programming guidelines for you developers out there to have a laugh with, or perhaps even find a few useful tips and links in there. I swear, most of those tips are actually valid, even though they are presented in a tongue in cheek way.

So have fun with it. I know I did when I built the damn thing.

use a PowerShell script as a Vim filter to encode a url

Artistic closeup of a .vimrc file

Vim filters are cool. They let you run the content of your current buffer through a command and have its output returned into the bugger. That means that you can use filters to edit text using any command your operating system has available.

Now on a Linux machine that’s quite handy. On a Windows machine that isn’t so handy, because the default shell is CMD and that doesn’t have all those handy Unix text manipulating utilities. But what about a PowerShell script that would encode or decode a URL for example?

It turns out that wasn’t as easy as I expected, so I’ll spill it here in case you’re looking to do something similar.

First calling the script from Vim means you have to call powershell.exe because by default Vim uses cmd.exe as its shell. That isn’t so hard:

nnoremap <Leader>dc :%!powershell.exe -noprofile -nologo -file c:\tools\decode-string.ps1<CR>
nnoremap <Leader>ec :%!powershell.exe -noprofile -nologo -file c:\tools\encode-string.ps1<CR>

I create 2 shortcuts here to encode (ec) and decode (dc) trigger using the leader key.
These will pipe the content of your buffer through the script, and get whatever it spits out through the standard output back into your buffer.
The extra command line parameters I pass into powershell.exe are to make it as fast as possible and not do any unnecessary junk. The -noprofile is the most important one as this skips loading any special modules you have set up in your PowerShell profile.

Reading from the pipe in a PowerShell script wasn’t as easy as I thought. Using the ValueFromPipeline attribute on the input parameter didn’t work for some reason. After some searching I came across a magical $input parameter that gets anything you receive from standard input for use in your script. Handy.
So the script was as simple as doing this:

[Reflection.Assembly]::LoadWithPartialName("System.Web") | Out-Null
[System.Web.HttpUtility]::UrlEncode($input)

What do we do when PowerShell can’t help us out of the box? That’s right. We summon the powers of .NET assemblies to get the job done. In this case, we’re summoning System.Web and using the HttpUtility class to encode the incoming data.
We do exactly the same to decode text by the way:

[Reflection.Assembly]::LoadWithPartialName("System.Web") | Out-Null
[System.Web.HttpUtility]::UrlDecode($input)

This is pretty powerful. By using the $input variable in PowerShell together with the Vim filter function, you can do all sorts of text transformations. This can make your developer life a lot easier.

testing CORS headers with PowerShell

CORS can be a pain in the ass to test for your backend service. This because you have to emulate a cross domain request from a browser and sometimes you just want to quickly confirm your CORS configuration or code is working. Can this be done using PowerShell, so you don’t have to fire up a test site to do that request, or use special tooling like Postman or whatever?

Well yes! Using the statement below you can see all the headers that are returned from the request to the test URL on your API. If CORS is correctly configured, you will see the Access-Control-Allow-Origin header in that list. It contains the allowed origin site domain or the asterisk to indicate any domain is allowed.
If not, something is still wrong.


(Invoke-WebRequest https://bar.com/api/stuff/ -UseBasicParsing -Headers @{ "Origin" = "http://www.foo.com/" }).Headers

The trick is passing in the Origin header when you make the request. Without it, you won’t get any CORS headers back in most cases. I tested this using an ASP.NET Core backend by the way, but most backend systems should behave in the same way, as this is how browsers do it.

a practical guide to using KeePass password manager

Thinking about using a password manager that is free, secure and you have your doubts about the online ones? Well lucky you, this is just the post you are looking for.
With all these hacks and breaches going around you shouldn’t be reusing passwords and you know it. Instead, you can let a password manager generate long and gibberish-like random passwords for all your logins. That way hackers have to throw a thousand cores and millions of years at it before they can crack them. If they crack one anyway, it won’t matter much because it will only work on that one site.
Trusting all your passwords to a piece of software? Is that a good idea? What about if I need my passwords on another machine, or my phone? What if I’m on vacation?
I’ve been storing all my passwords in KeePass for many years now, so I’ll share my setup. You can use this as inspiration to set up your own KeePass flow.

Why KeePass

There are a few cloud-based alternatives out there but when I started with KeePass those weren’t around yet or I didn’t know about them.
I thought about switching to one but eventually didn’t because:

  1. They are not free or have limited free-plans.
  2. They are using proprietary software, so you can’t tell how they work and if they really do store your passwords safely. KeePass however is open source and has been audited for security in the past.
  3. Storing all your passwords on a server owned by someone else without a local backup sounds like a bad idea to me.
  4. Some can’t be used for things other than websites. Like desktop app credentials. Or even SSH logins and other weird and geeky stuff you need random secrets for.

Yes they are slightly more convenient and look a bit more polished. But for me that doesn’t weigh up against the extra control I get with KeePass.

Installing KeePass

KeePass exists for Windows, Linux, macOS and Android. It’s a typical installation. If you’re as geeky and paranoid as me you download it from the main site and you check the md5 hash of the installation files. That way you’re 100% sure you didn’t download some altered or hacked version. It hasn’t happened with KeePass before, but it did happen to the Linux Mint ISO’s at one point so you can never be sure.

There is a getting started guide on the KeePass website that guides you through setting up and creating a first database. This Lifehacker post does the same thing and also has some nice screenshots for guidance.

Securing your password database

When it comes to securing your password database you have to make sure your master password to unlock it is of course a pretty damn good one. It has to be as long as possible (at least 10 characters, but more is better), higher case, lower case, number, special characters, the whole shebang. On top of that, you’ll have to be able to remember it too. So I guess this is one of the hardest bits.
There are tricks to make this easy though. Think of a good phrase you can easily remember. Or any list of words. Take the first or first few letters of each word, mix it up with some special characters and you end up with something hard to crack and easy to remember.
Or just come up with a good passphrase of random words you can remember. Don’t use Correct Horse Battery Staple or a popular lyrics phrase because they are probably in some password list database already. You can use a word list to generate a random password using the EFF word lists and some dice, or use one of the many generators online.

Just be original. Or try anyway.

Small steps

When I started out I didn’t trust KeePass enough to dump and change all my passwords from day 1. I started out simple, by adding new sites I registered to and use randomly generated passwords from the built in password generator. Later I added sites I frequently used and changed their passwords to more complex ones. Now everything is in there. But not every password is random though. Really important accounts I have in my head too, using a unique, complex password that I can still remember. Really important accounts also have 2-factor authentication activated so even if a hacker finds the password, they still won’t get in.
Knowing those key passwords is also a fallback in case I don’t have access to my KeePass DB for some reason.

Syncing the DB

Now you want to use this on more machines than just your laptop I guess.
There are a few options:

  1. You put the DB on a thumb drive you always have on you. This is a good backup too. You can use PortableApps or a portable KeePass version on the thumb drive and use it anywhere like that.
  2. You sync the DB to your favorite cloud drive and sync it to every machine you want to use it on.

I use Dropbox myself which is great for this to sync between home, work and my phone. OneDrive would also work as it works pretty much the same way.
If you want to get your own Dropbox drive (2 GB free), use this link. Use that to get 500 MB bonus space, and so do I ;).
There are also a number of plugins for KeePass to sync to Google Drive, FTP, and other online providers, so I’m sure you’ll find something you like.

On your phone

Phone closeup with simcard and micro SD card.

If you want access to your passwords on your phone, you’ll need some extra apps. I use Android myself, but I’m sure the same apps exist for iOS.
You will need 2 apps, one to be able to open and use the database, and then something to sync the file to your phone. Unless you do that manually, but I wouldn’t advise it.

To use the database there are plenty of options when you search for KeePass, but the best one I’ve used so far is Keepass2Android.

For syncing the file to my phone I use Dropsync. This syncs a Dropbox folder to a folder your phone. You can use the free version if you’re only setting up 2 folders.
You can also use the Dropbox app itself and mark the file to be available offline, but I’ve noticed this doesn’t always work. I often ended up with an old version of the database when I needed it.
Maybe in the future this’ll get better, but until then, Dropsync is what I’m using.

Extensions

KeePass has a ton of plugins allowing you to customize it for all sorts of things. There are plugins to have it integrate in your browser, synchronize files over all sorts of protocols and services, export, import, add visual features and whatever.

I use as little plugins as possible though, as each plugin has access to your database and can be a possible vulnerability. Yes. Tin-foil hat here. But LastPass’ Chrome plugin leaked your login credentials a while ago, so there you go.

By using the standard keyboard shortcuts on PC you can get a long way already. Be sure to check out the Auto-Type override documentation if you have a website which isn’t playing nice with the defaults. You can find a way to get it to work for 99% of the websites out there. The other 1% just have really shitty UX.

download cover art for all your albums, with powershell

Album covers are nice eye candy when you’re using a media player like Foobar2000 which automatically picks up a cover.jpg file in an album folder. The problem is that I have a lot of mp3 albums I ripped from my CD’s from way back and those don’t have any fancy cover art files.

I looked around for some tools that could automagically download covers for my albums but didn’t find anything handy. Since my music is structured in sub-folders like \ I thought this should be easy enough to parse and get pictures for.
If only there was a service that could easily provide those…

I tried the Musicbrainz API’s but that turned out to be hard to use and didn’t give me any covers for some test albums either. Then I thought of Last.fm. They have a lot of cover art, and their URL structure is the same as my folder structure… hmmm.

And here it is, a Powershell script which runs over your folder structure, tries to get the album page from Last.fm and then saves a cover.jpg image from the album page metadata.

A few things to know:

  • Your mp3’s are expected to be in a folder structure like (artist)\(album)\*.mp3
    E.g. The Prodigy\The Fat of the Land
  • If a folder contains any JPG or PNG image, it will be skipped. So that means you can run the script multiple times, and it will only download images once.
  • The “Various artists” folder is skipped by default because it didn’t fit the search pattern. If you store these type of albums in another folder, you might want to update that line with the correct folder name. If it does happen to process that folder in your case because of a different name, nothing will go wrong. It simply won’t find any album matches.

To use it, copy the code below in a file called get-albumart.ps1, or whatever name you fancy. Then run it as follows to get those pretty cover albums:

.\get-albumart.ps1 d:\music

And as always, this script comes as is, without any kind of warranty and you’re free to try it at your own risk. I wrote and used it and it worked great for me. I hope it works for you too. If Last.fm sues you because you’re downloading every image they have on the site because of your huge album collection? You didn’t get this script from me OK. Nope. Not me. ;-)

param ([Parameter(Mandatory=$true)][string]$path)

$progressPreference = 'silentlyContinue'
pushd
cd $path
$artistFolders = ls -directory | where { $_.name -ne "Various artists"}

foreach ($artistFolder in $artistFolders)
{
    $artist = $artistFolder.name
    write-host "::: $artist :::" -foregroundcolor green

    cd -Literalpath $artistFolder
    $releaseFolders = ls -directory
    
    foreach ($releaseFolder in $releaseFolders)
    {
        $release = $releaseFolder.name
        write-host "$release" -foregroundcolor cyan
        cd -literalpath "$releaseFolder"

        if ((test-path *.png) -or (test-path *.jpg))
        {
            write-host "- Images found, skipping."
        }
        else
        {
            $url = "https://www.last.fm/music/$($artist)/$($release)"
            $r = $null

            try 
            {
                $r = invoke-webrequest $url -usebasicparsing
            }
            catch 
            {
                write-host "- Release not found, skipping: $artist - $release" -foregroundcolor red
            }

            if ($r -ne $null)
            {
                $s = $r.content -split "`n" | where { $_ -like "*`"og:image`"*"} 
                $img = ($s -split '"') | where { $_ -like "*https*.jpg*" }

                if ($img -ne $null)
                {
                    write-host "- Downloading image for $artist - $release from $url"
                    invoke-webrequest $img -outfile cover.jpg
                }
                else
                {
                    write-host "- No image for $artist - $release from $url" -foregroundcolor yellow
                }
            }
        }
        cd ..
    }
    cd ..
}

popd
$progressPreference = 'Continue'