Using microk8s with system kubectl

I was already using kubectl with some plugins, and the nice completion already set up, to access remote Kubernetes clusters from my Ubuntu desktop. So when I decided to try microk8s, I wanted to keep the same convenience, without maintaining two configs, or having two different commands. It took a little bit of research to figure out how to make it work, so here it is.

First, find the admin password for your microk8s:

microk8s.kubectl config view | grep password

Now you can create a new context in kubectl, adding in your password on line 2:

kubectl config set-cluster microk8s --server= --certificate-authority=/var/snap/microk8s/current/certs/ca.crt

kubectl config set-credentials microk8s-admin --username=admin --password={password from before}

kubectl config set-context microk8s --cluster=microk8s --namespace=default --user=microk8s-admin

Now you can switch to the new context with kubectl config use-context microk8s and work as before. kubectl config get-contexts will list the other available contexts so you can switch back.


TIL: Putty “Server refused our key”

I’ve had this strange problem for about a year or more, where on my main Windows system, I can’t log in to local Linux servers using SSH Public Key authentication. I get this “Server refused our key” message. The same key files on a different computer work fine though.

Tonight I finally got annoyed enough to investigate deeper. Googling did not help me, so here’s a blog post so that it might help you. The final straw was installing the RancherOS ISO, which exclusively uses SSH keys, and not being able to get into it from my normal desktop. I suspect this is not the most common cause of “Server refused our key” errors, especially if you are getting them from multiple client to the same server. Look at the server’s SSH config in that case.

I use Pageant, and it has my usual keys loaded at boot time.

So I enabled debug logging in Putty and SSHed to my Linux desktop.

With that enabled, I was able to see this in the logs:

Event Log: Reading key file "C:/xampp/htdocs/cacti-1.0.6/plugins/weathermap/dev/Vagrant/.vagrant/machines/develop/virtualbox/private_key.ppk"
Event Log: Pageant is running. Requesting keys.
Event Log: Pageant has 4 SSH-2 keys
Event Log: Configured key file not in Pageant

So it turns out that at some point I had a Vagrant box running, and saved its private key into my DEFAULT putty profile. From there, it got saved into other profiles. Further, if putty had a specific key in this box, and it no longer exists, it never gets as far as the Pageant-supplied keys, and just gives up!

Removing that entry (at the bottom of the SSH auth section) has cured all my problems, and RancherOS is back on the menu!


Reusing DHCP config in ISC dhcpd

This is quite a niche thing, but since it took me ages to piece it together, here it is.

I run a PXE build environment for bare-metal install of new systems. I use PuppetLabs’ Razor to do the installation bit, and that works well. I do this for multiple environments, and I wanted to have a single server serve all of these related subnets (they come and go as projects complete). The single server acts as TFTP, DHCP, & Razor server for each subnet, as well as a NAT for external access, so it’s possible to use any IP addressing inside the build environment (except the network the razor box is on, of course).

In each subnet, the DHCP config is somewhat complicated. Aside from the normal DHCP stuff (IPs, gateway etc), the boot file to be used varies depending on the client-id of the host. EFI devices get a difference iPXE boot file from BIOS devices. Also, iPXE then makes its own DHCP request, which needs to be given a script instead. The script filename is different per subnet. The naive way to do all that results in an enormous tangle to update as projects change.

So my goals:

  1. Single copy of the boot file switchy config
  2. Simple to update/edit config for new subnets

For the switching stuff, I started with this Gist from robinsmidsrod, and chopped out the cases I don’t care about. That leaves me with this include file:

option 1;

if exists user-class and option user-class = "iPXE" {
    filename "bootstrap-";
elsif option arch = 00:06 {
    # EFI 32-bit
    filename "ipxe-x86.efi";
elsif option arch = 00:07 {
    # EFI 64-bit    
    filename "ipxe-x64.efi";
elsif option arch = 00:00 {
    # Legacy BIOS x86 mode
    filename "ipxe.pxe";

and scopes like:

# Network 1
subnet netmask {
   option routers;

   include "/etc/dhcp/ipxe-common-";

So that works, but has a lot of duplication! Being able to define parameters would be useful.

So that looks like this:

option bootstrap_file code 254 = string;

# Network 1
subnet netmask {
   option routers;

   option bootstrap_file "bootstrap-";

   include "/etc/dhcp/ipxe-common.conf";

and then:

if exists user-class and option user-class = "iPXE" {
    option bootfile-name = concat("",config-option bootstrap_file);

in the included file means that nothing needs to change in there anymore. I can just add a new scope in a few lines, drop in the appropriate TFTP file and be up and running.


Getting GNU Smalltalk to work on Ubuntu 16

I thought I’d play with Smalltalk. GNU Smalltalk is free, and also has the classic Smalltalk browser application. Except out of the box, it fails on Ubuntu 16 with:

Object: CFunctionDescriptor new: 1 "<0x7f1ffaacf690>" error: Invalid C call-out g_date_get_type

So here’s the packages you need to install so that gst-browser actually starts up on Xenial:

sudo apt install gnu-smalltalk-browser gnu-smalltalk gnu-smalltalk-doc libgtk2.0-dev libcairo2-dev libfontconfig1-dev libfreetype6-dev libpng-dev libpango1.0-dev libgdk-pixbuf2.0-dev

Now I can read the nice Adele Goldberg books I bought from thriftbooks, and play along 🙂


Modernising the Windows Console

My work laptop is stuck in Windows-land really, because of some required corporate applications. Lately I’ve been writing a fair bit of Python, and using the coloredlogs extension to the logging module to produce pretty logging from my code. Except on Windows, where the standard CMD prompt doesn’t understand ANSI colour codes, which is kind of a pain.

There are a few solutions that patch the default command prompt, to add back in ANSI support (DOS always had it available, way back when), but they all seemed a bit hacky.

So I switched to ConEmu, which provides a huge list of features, including ANSI colour support, tabs, all kinds of display modes, transparency, xterm-style cut&paste, and a canned list of contained applications, including CMD, Powershell, Chocolatey, Cygwin and that funny shell with all the environment set up that Visual Studio uses. Of course, you can also add your own.

Now I have a nicer-looking and nicer-acting console, and my Python logging colours work properly!

After playing with that for a while, I noticed the option to enable Clink inside ConEmu. Clink adds a large range of bash-style features to Cmd itself, like saved history, incremental history search and fancy completion, as well as simple things like being able to press Ctrl-V to paste, and readline editing (emacs keys to edit the input).


IOS XR SSH: “Server refused to allocate pty”

Just another SSH note, since it took me a while to work out what was going on: if you putty into an IOS XR router (ASR9001 in my case) with SSH Agent Forwarding enabled then it just drops the connection with this message: “Server refused to allocate pty”

Solution: don’t do that. Turn off Agent Forwarding in the SSH options (or stop using ssh -A for the commandline client).

Monitoring Network Tech Uncategorized

RANCID, ssh, Cisco MDS and “too many authentication failures”

I just ran into this, and it took a little while to figure out, so here’s my quick note. If you have a Cisco MDS being backed up by RANCID, then you can get the following odd message, even if it’s the first time you tried to log in with this user:

Received disconnect from 2: Too many authentication failures for confbackup

What is happening is that the ssh client tries with whatever public keys it has configured first, and then the password-based auth that you thought it was doing all along. With a few keys, that’s enough to annoy the MDS into closing the connection.

The solution is to disable public-key auth for this connection. To do that with RANCID requires a little bit of extra work. First, create a shellscript (I call mine /opt/rancid/local/ssh-no-pubkey):


ssh -o PubkeyAuthentication=no $*

Then for the devices that are suffering, tell RANCID to use this new SSH command instead of just ‘ssh’. In .cloginrc:

add sshcmd mds01 {/opt/rancid/local/ssh-no-pubkey}

Now RANCID can login and backup the config fine.

Additional tip – the ‘cisco’ device type seems to work better than the (theoretically correct) ‘cisco-nx’ device type for MDS switches.


KIO’s zasm on Ubuntu

kio makes a nice-looking z80 assembler with lots of interesting output options for cross-development to a ZX Spectrum target, like writing .SNA files straight from the assembler. However, it only comes with binaries for a few platforms (or rather not all versions on all platforms), and source-code comes with just an xcode build file. The instructions then just say ‘create a Makefile’ to get it to compile! Here’s how I compiled it on Ubuntu 14.04…

First, you’ll need a c++ compiler:

sudo apt-get install g++

Then, you’ll need the config.h for linux:

cd {the directory you unzipped the source into}
cp config.h config.h-dist
cp config.h.LINUX config.h

Then compile it all together in a single line:

g++ -o zasm -I. -I kio *.cpp kio/*.cpp -lpthread

Which should leave you with a zasm binary in the current directory.

To install:

sudo cp zasm /usr/local/bin

Finally, to get an emulator to feed your project to:

sudo apt-get install fuse-emulator-gtk

Personalising your Apple product is the door to a world of pain…

Apple do this nice service when you order directly from the Apple Online Store where you can get an engraving on the back of your iPod, iPad etc for free. It looks really nice, and it adds about a day onto the delivery time. What could go wrong? Well, let me tell you a story…

I recently bought an iPad Mini for my girlfriend’s birthday. Got her a nice designer case for it, a data nanosim and a cute little private joke engraved on the back (along with her e-mail address – I’m still fairly practical). She loved it, and started “moving in” straight away. The next day though, it was failing already – the display backlight wouldn’t light and in just the right light you could make out a very rough-looking display. So we wiped it clean (iTunes could still talk to it) and went to the Apple store.

First of all, one does not simply walk into an Apple store. Obviously you need an appointment with a “genius” to actually get customer service. Although it turns out if you stand in the middle of a ipad-buying crowd with a dead unit and a grumpy face, someone will help. Except they can’t, because it’s personalised, so it has to go back to the mothership.

The one super-impressive part of the customer service experiences comes next – Apple’s phone system. If you call from the phone that they have recorded against your name for your account, the phone system just says (literally) “Oh, is it about that ipad?”, and everyone knows what is going on. Neat!

They send out a courier box for the unit, and you send it back. Then they send out the repaired one. Unlike the Apple store itself, they use TNT, and they don’t let you know the tracking number for the repaired item.

So my repair is turned around in a day or so, and then the item is just marked “shipped” on the Apple site. I wait, and wait some more. After speaking to Apple customer service, they want to wait until TNT have had their “allow for delivery” time, which is something like 10 days. I did get the tracking number from them though, so I could play along at home – it just says “out for delivery” for several days. Eventually TNT admit that actually they’ve lost it, so Apple organise another replacement, with personalisation. They confirm the details, and get it sent. It arrives a few days later, and it’s the wrong spec. We go back two steps and repeat one more time. Finally, 6 weeks after her birthday, we have a working unit!

In their defence, I did get personal contact details from several folks at Apple who followed the case through for me, but it shouldn’t really come to that. Obviously, if I’d have just bought the unit without personalisation, I could have had it swapped out at the local physical store, and been done in a day or two.

Think twice before engraving!


Something is wrong somewhere

I was sitting at my desk watching John Allspaw’s Alert Design presentation in the corner of my screen, while doing some ‘real’ work, and I heard this odd noise. It was familiar, it was an alert sound, but it wasn’t in the right context. It wasn’t my phone, or my PC… eventually I realised it was the noise the our car makes when you are low on fuel (a soft ‘bong’ on recent BMWs). Looking out the window, way down the street was someone loading their BMW with stuff, setting off an alarm.

Anyway, context is important for alarms. The presentation is interesting.