Rack diagrams with Weathermap

This is a response to a github issue for Weathermap. This way, everyone might benefit! (for those that don’t know, I write an open source tool to generate diagrams with live data from your monitoring system)

rack-weathermap

This is a demo I did years ago for the datacentre I was working in then. We were working on an asset register system, and I was hoping to be able to generate these maps from that data. You should be able to do the same kind of thing with racktables or OpenDCIM.

The key parts are:

  • A column of ‘notch’ nodes. All but the first one are positioned relative to the one above. These are a simple gif image.
  • The actual servers and equipment. Those are positioned relative to the notch nodes. We used a combination of rack-diagram images from Visio, and actual photos taken square-on to the front of racks. Pick a size for 1U, and that will dictate what 19 inches across is. For us, it was 400 pixels. Use that as a scale on all your icons. The vertical will fall into place, but set the Y scale on your icons to something large, so that it will never be the ‘important’ scaling dimension.
  • Status indicators. Those are positioned relative to the servers.

The original version also had some nice fonts defined, but distributing fonts is much more complicated so I changed the downloadable example to use the built-in fonts.

“Notches”

So first, we define a template node for how to draw a ‘notch’. This contains the icon, and the label, so they don’t need to be repeated.

NODE notch
    LABEL {node:this:name}
    LABELOFFSET C
    LABELFONT 15
    LABELFONTCOLOR 255 255 255
    ICON 1u-notch.png

Then we define all the notches.

NODE U1
    TEMPLATE notch
    POSITION 150 100

NODE U2
    TEMPLATE notch
    POSITION U1 0 39

NODE U3
    TEMPLATE notch
    POSITION U2 0 39

    [etc etc]

You can generate this code in Excel or a scripting language very easily. Or just stick with my dimensions and use the template in the zip file.

Servers

Next comes the server. For a 1U server, things are simple. We just use relative positioning from the appropriate U notch. For larger servers we need to do a little maths: the node is positioned from its centre, so we need to know that a 2U server is 19 pixels down from the first U position it occupies.

NODE server2
    LABEL Web Server 3
    ICON 400 300 dell2u.png
    POSITION U6 225 19

NODE server3
    LABEL Web Server 3
    ICON 400 300 dell1u.png
    POSITION U9 225 0

Status

Finally, if you need to show the current state from somewhere, you can easily add indicators:

NODE server3_state
    ICON 20 20 red-ball-64.png
    POSITION server3 -270 0

(obviously you would add some TARGET information to get the current state)

Note that everything is positioned relatively. This means you can move the whole rack by moving one thing (the U1 notch). The status indicator will follow the server if you move it, too.

Example

I’ve zipped up an example map, some server images, and a blank 42U rack config file that can be used as a template for your own racks:

weathermap-rack-diagrams.zip

1 Comment

Filed under Weathermap

Modernising the Windows Console

My work laptop is stuck in Windows-land really, because of some required corporate applications. Lately I’ve been writing a fair bit of Python, and using the coloredlogs extension to the logging module to produce pretty logging from my code. Except on Windows, where the standard CMD prompt doesn’t understand ANSI colour codes, which is kind of a pain.

There are a few solutions that patch the default command prompt, to add back in ANSI support (DOS always had it available, way back when), but they all seemed a bit hacky.

So I switched to ConEmu, which provides a huge list of features, including ANSI colour support, tabs, all kinds of display modes, transparency, xterm-style cut&paste, and a canned list of contained applications, including CMD, Powershell, Chocolatey, Cygwin and that funny shell with all the environment set up that Visual Studio uses. Of course, you can also add your own.

Now I have a nicer-looking and nicer-acting console, and my Python logging colours work properly!

After playing with that for a while, I noticed the option to enable Clink inside ConEmu. Clink adds a large range of bash-style features to Cmd itself, like saved history, incremental history search and fancy completion, as well as simple things like being able to press Ctrl-V to paste, and readline editing (emacs keys to edit the input).

Leave a Comment

Filed under Uncategorized

Making old joysticks from new ones…?

I have a reasonable collection of 80s home computers and consoles, many of which use the “standard Atari” joystick port. Those Atari CX40 joysticks, which is where this standard comes from, are particularly crappy though, so most gamers at that time would use something else with the same wiring, like a classic Competition Pro. You can get Competition Pros on ebay, but they are upwards of £30 – so what alternatives are there?

2015-09-29 16.10.59-sized

I have a Speedlink USB Competition Pro that I bought from Amazon last year (for much less than £30!) with all-microswitch switching but a USB port – so I opened it up. Inside it’s actually a not-very integrated circuit, which is good news for un-USBing it. There’s a little wiring harness for the microswitches, all going back to a tiny PCB where the autofire switch is. A new small wiring harness, a DE-9 socket and a 8-core shielded cable, and we’re in business, with a brand-new moulding and switches! (or the cable from a dead CX40 or similar). People like Digikey will sell you a 6ft ribbon cable with a DE-9 connector already attached at one end for a few pounds, so then it’s just the wiring.

Here’s some pictures of the insides, in case anyone else was wondering – just a small blob (presumably a PIC or similar) and the wiring connector, really:

2015-09-29 12.17.22

2015-09-29 12.17.41

Leave a Comment

Filed under Projects & Hacking, Toys

From DVD ISO to mkv

I spent a while ripping DVDs to ISO files using DVD Decrypter a while ago, which allows me to use the Kodi (XBMC) menus to select DVDs to watch from anywhere in the house. However, lately I’ve been interested in Plex Media Server, which, although it’s a fork of XBMC, doesn’t support the ability to play DVD ISOs. Plex is much more geared to streaming, with apps for phones, tablets, game consoles and various IP TV boxes like the Roku. It also has this neat cloud feature where you can watch your media from anywhere with a fast enough internet connection, and share with friends and family. I think it’d be cool to get my parents a Roku box, and be able to ‘curate’ a movie collection for them with no effort on their part and without them having to have a server/NAS running, by ripping their DVDs onto my home NAS.

Another feature of Kodi that has been taken out of Plex is the concept of a ‘stub’ file, where you just keep a 0 byte file on the NAS that says “out in the physical world, there’s a DVD with ‘The Godfather (1972)’ on it”, so that Kodi can be your librarian for offline movies too. So now, to use Plex, I definitely need to have some kind of online copy of each DVD (or maybe a 2 second video to take its place, that says “look on the shelf behind you”… hmmm)

Anyway, I now have a lot of ISO files that need to become MKV or avi or something else, ideally with no input from me. It took me a while to work out how to get a title extracted from a DVD ISO (not a physical DVD) as an MPEG2 stream, so here’s another little post about that. ‘title’ is what the DVD standard calls the individual video streams on the DVD – for example, one for the main movie, one for the trailer, and others for special features. They are all stored in MPEG-2 video format, which is getting old now, and not so great for compression. The cool kids are using H.264 these days.

I did this on my Windows desktop, but the tools are natively Linux tools so it should work on Linux or OS X too. You need mplayer and ffmpeg. Actually, you probably could do this with just mplayer/mencoder, but this worked for me and so I stopped looking. I specifically wanted a command-line method to do it. If you don’t care about that, just use Handbrake, which is very nice. I may yet do that too, since it also has a job queue.

First, to extract Title 1 from the DVD as MPEG-2 video:

mplayer dvdnav://1 -dvd-device DVDVolume.iso -dumpstream -dumpfile title1.m2v

If you don’t know which title you need, mplayer can help there too:

mplayer dvdnav:// -dvd-device DVDVolume.iso -frames 0 -identify -vo null -ao null -nocache

which will spit out a LOT of information, including this:

ID_DVD_TITLE_1_LENGTH=6714.000
ID_DVD_TITLE_1_CHAPTERS=25
TITLE 1, CHAPTERS: 00:01:37,00:10:51,00:14:10,00:23:21,00:24:09,00:27:44 {etc}
ID_DVD_TITLE_2_LENGTH=130.033
ID_DVD_TITLE_2_CHAPTERS=1
TITLE 2, CHAPTERS: 00:02:10,

which tells me that title 1 is the movie, and title 2 is the trailer (only 2 minutes long).

Then to make an MKV file of the resulting MPEG-2 file:

ffmpeg.exe -i title1.m2v ./title1.mkv

There are loads of settings for x264 that I don’t understand, so I just ignored them. The results look pretty good so far, and I get files of about 1.5GB instead of the 4GB original. Each one takes a while (about an hour? I haven’t timed it), but there’s nothing to watch or poke, so it doesn’t really matter. I can leave it to do its thing.

Leave a Comment

Filed under Tech

wraprancid and RANCID 3.x

Jethro R Binks’ excellent wraprancid script allows you to bring in configurations (and pretty much anything else that can be text) without having to get involved in writing a new ?rancid/?login combination for your device. That avoids some pretty hairy perl and Tcl code, so it’s definitely a Good Thing! It’s also useful for devices that don’t even have a command-line, but might allow you to fetch their config from a web page, or TFTP.

The trouble is, RANCID changed the way it deals with device types between RANCID 2.x and RANCID 3. It changed in a good way, so that the patches to rancid-fe that tools like wraprancid required are no longer necessary. What was previously hard-coded in the source of rancid-fe is now a proper configuration file, with a second config file for you to add your own types to. Here’s how to get wraprancid working with RANCID 3.x

First, I’m assuming you have a working wrapplugin script. Here’s one I use to fetch the config from Asterisk servers.

#!/opt/perl/bin/perl -w
#
#######################################################
# Modules
#######################################################

# Load any modules needed
use strict;
use Getopt::Std;
use Net::SSH::Perl;

#######################################################
# Variables
#######################################################

# Initialize variables used in this script

my $debug = 0;

my %options = ();
getopts('df:', \%options);
my $file = $options{'f'};
my $fh;
my $host = $ARGV[0];

$debug = $options{'d'};

print STDERR "to host: $host\n" if $debug;

my $ssh = Net::SSH::Perl->new($host, protocol => '2,1', debug => $debug );

print STDERR "made ssh obj\n" if $debug;
$ssh->login("root");

print STDERR "login\n" if $debug;
my ($stdout, $stderr, $exit) = $ssh->cmd("true");
print STDERR "got output\n" if $debug;

# Open the output file.
open($fh, ">", $file) or die "Cannot open output file\n";
print $fh "#RANCID-CONTENT-TYPE: wrapper.asterisk\n#\n";

print $fh $stdout;
print STDERR "wrote output of ". length($stdout)." bytes\n" if $debug;

#######
# End #
#######
close($fh);
print STDERR "done\n" if $debug;

That lives in ~rancid/bin/asterisk.wrapplugin, just as it did in version 2.

Then, in ~rancid/etc/rancid.types.conf, we’ll define a new device type called wrapper-asterisk:

wrapper-asterisk;script;wraprancid -s asterisk.wrapplugin
wrapper-asterisk;login;clogin

(I don’t think the login script matters, as it’s never used, but it must be specified to keep RANCID happy)

And finally in the router.db, you can put your actual device:

asterisk-sipgateway;wrapper-asterisk;up;

That’s it. You can repeat for whichever other scripts you need to do this for.

Bonus Tip

The asterisk end of the script above works like this: we use SSH public key authentication to connect to the server, and then in ~root/.ssh/authorized_keys, there is a line like this:

command="/usr/sbin/asterisk -V; echo 'extensions.conf'; cat /etc/asterisk/extensions.conf; echo 'sip.conf'; cat /etc/asterisk/sip.conf; echo 'iax.conf';cat  /etc/asterisk/iax.conf",from="myrancidhost" ssh-dss AAAAB3NzaC174ENozlUVBe5hH32Wy/duAJt1b4nWbVPoW1GP/koSZNv3888s3fx23nEpLMJxispulA== rancid@myrancidhost

So that the user authenticating with that particular key doesn’t get a shell, they just get the output from a series of cat commands, and then disconnected. They must also be connecting from the RANCID server.

So, now we have Asterisk in the same version control system as our network gear. You can use a similar setup for things like BSD ipfw-based firewalls, or Quagga routers.

Leave a Comment

Filed under Monitoring, Network, Projects & Hacking

IOS XR SSH: “Server refused to allocate pty”

Just another SSH note, since it took me a while to work out what was going on: if you putty into an IOS XR router (ASR9001 in my case) with SSH Agent Forwarding enabled then it just drops the connection with this message: “Server refused to allocate pty”

Solution: don’t do that. Turn off Agent Forwarding in the SSH options (or stop using ssh -A for the commandline client).

Leave a Comment

Filed under Uncategorized

RANCID, ssh, Cisco MDS and “too many authentication failures”

I just ran into this, and it took a little while to figure out, so here’s my quick note. If you have a Cisco MDS being backed up by RANCID, then you can get the following odd message, even if it’s the first time you tried to log in with this user:

Received disconnect from 10.0.7.5: 2: Too many authentication failures for confbackup

What is happening is that the ssh client tries with whatever public keys it has configured first, and then the password-based auth that you thought it was doing all along. With a few keys, that’s enough to annoy the MDS into closing the connection.

The solution is to disable public-key auth for this connection. To do that with RANCID requires a little bit of extra work. First, create a shellscript (I call mine /opt/rancid/local/ssh-no-pubkey):

#!/bin/sh

ssh -o PubkeyAuthentication=no $*

Then for the devices that are suffering, tell RANCID to use this new SSH command instead of just ‘ssh’. In .cloginrc:

add sshcmd mds01 {/opt/rancid/local/ssh-no-pubkey}

Now RANCID can login and backup the config fine.

Additional tip – the ‘cisco’ device type seems to work better than the (theoretically correct) ‘cisco-nx’ device type for MDS switches.

Leave a Comment

Filed under Monitoring, Network, Tech, Uncategorized

KIO’s zasm on Ubuntu

kio makes a nice-looking z80 assembler with lots of interesting output options for cross-development to a ZX Spectrum target, like writing .SNA files straight from the assembler. However, it only comes with binaries for a few platforms (or rather not all versions on all platforms), and source-code comes with just an xcode build file. The instructions then just say ‘create a Makefile’ to get it to compile! Here’s how I compiled it on Ubuntu 14.04…

First, you’ll need a c++ compiler:

sudo apt-get install g++

Then, you’ll need the config.h for linux:

cd {the directory you unzipped the source into}
cp config.h config.h-dist
cp config.h.LINUX config.h

Then compile it all together in a single line:

g++ -o zasm -I. -I kio *.cpp kio/*.cpp -lpthread

Which should leave you with a zasm binary in the current directory.

To install:

sudo cp zasm /usr/local/bin

Finally, to get an emulator to feed your project to:

sudo apt-get install fuse-emulator-gtk

1 Comment

Filed under Uncategorized

Virtual serial ports in Windows VM hosts (for IOS XRv)

I’ve been trying to play around with Cisco IOS XRv this morning – the virtual machine version of the IOS XR software used on the ASR-9000 and CRS-1 series routers. Having expensive hardware like that for a simple test environment is tricky, but XRv means you can have one on your desktop.

The VM boots fine, but then leaves you with a nearly-blank screen that says “Booting IOS XRv”. What happened? All the action is happening on the serial port of the device, just like on a real router. So you need a serial port in the VM, and some way to talk to that.

I went through quite a few different versions of this, using NPTP, and VMwareGateway. I tried with VirtualBox and VMware Workstation. In each case, I would get a login prompt that didn’t work. I would see my password echoed back on the screen, and the router would generally act strangely – showing a password prompt and immediately saying that login failed, for example.

Then I found out that PuTTY can talk directly to a named pipe. So all you actually need is PuTTY. Fire it up, select Serial, then paste in the named pipe name you used when configuring the VM. It works! And login works, too!

So the full process: import the OVA. Then add a serial port to the VM (if it’s in VMware Workstation, VirtualBox already has one). Set the serial port type to Named Pipe, and add something like “.\pipe\my_xrv” to the name for the pipe. It MUST start with “.\pipe\”. In VMware, you want to say “This is the server”. In VBox, you say “Create Pipe”.

Then start the VM, and use Putty to connect (with Serial connection selected) to “.\pipe\my_xrv” where it usually says “COM1”.

Obviously the same technique works for anything else that needs a serial port to talk to the world.

Leave a Comment

Filed under Network, Tech

See inside long-running Unix pipelines

So you are restoring a database on your Linux system, and you have the 3GiB SQL file ready to go, and so

mysql -uroot -p mydb < backup-2013-10-17.sql

and then… wait. But for how long? It’s probably broken right? It should never take this long! Your service desk needs to know what to tell customers.

Pipe Viewer is a handy little tool to use in place of cat(1), which gives you a progress bar and throughput figures for long-running processes like this.

pv -cN source < backup-2013-10-17.sql | mysql -uroot -p mydb

It also gives an ETA, which is about as good as Windows file copy ETAs, but knowing that something is moving, and at what kind of pace is very reassuring.

Leave a Comment

Filed under Tech