Modernising the Windows Console

My work laptop is stuck in Windows-land really, because of some required corporate applications. Lately I’ve been writing a fair bit of Python, and using the coloredlogs extension to the logging module to produce pretty logging from my code. Except on Windows, where the standard CMD prompt doesn’t understand ANSI colour codes, which is kind of a pain.

There are a few solutions that patch the default command prompt, to add back in ANSI support (DOS always had it available, way back when), but they all seemed a bit hacky.

So I switched to ConEmu, which provides a huge list of features, including ANSI colour support, tabs, all kinds of display modes, transparency, xterm-style cut&paste, and a canned list of contained applications, including CMD, Powershell, Chocolatey, Cygwin and that funny shell with all the environment set up that Visual Studio uses. Of course, you can also add your own.

Now I have a nicer-looking and nicer-acting console, and my Python logging colours work properly!

After playing with that for a while, I noticed the option to enable Clink inside ConEmu. Clink adds a large range of bash-style features to Cmd itself, like saved history, incremental history search and fancy completion, as well as simple things like being able to press Ctrl-V to paste, and readline editing (emacs keys to edit the input).

Leave a Comment

Filed under Uncategorized

Making old joysticks from new ones…?

I have a reasonable collection of 80s home computers and consoles, many of which use the “standard Atari” joystick port. Those Atari CX40 joysticks, which is where this standard comes from, are particularly crappy though, so most gamers at that time would use something else with the same wiring, like a classic Competition Pro. You can get Competition Pros on ebay, but they are upwards of £30 – so what alternatives are there?

2015-09-29 16.10.59-sized

I have a Speedlink USB Competition Pro that I bought from Amazon last year (for much less than £30!) with all-microswitch switching but a USB port – so I opened it up. Inside it’s actually a not-very integrated circuit, which is good news for un-USBing it. There’s a little wiring harness for the microswitches, all going back to a tiny PCB where the autofire switch is. A new small wiring harness, a DE-9 socket and a 8-core shielded cable, and we’re in business, with a brand-new moulding and switches! (or the cable from a dead CX40 or similar). People like Digikey will sell you a 6ft ribbon cable with a DE-9 connector already attached at one end for a few pounds, so then it’s just the wiring.

Here’s some pictures of the insides, in case anyone else was wondering – just a small blob (presumably a PIC or similar) and the wiring connector, really:

2015-09-29 12.17.22

2015-09-29 12.17.41

Leave a Comment

Filed under Projects & Hacking, Toys

From DVD ISO to mkv

I spent a while ripping DVDs to ISO files using DVD Decrypter a while ago, which allows me to use the Kodi (XBMC) menus to select DVDs to watch from anywhere in the house. However, lately I’ve been interested in Plex Media Server, which, although it’s a fork of XBMC, doesn’t support the ability to play DVD ISOs. Plex is much more geared to streaming, with apps for phones, tablets, game consoles and various IP TV boxes like the Roku. It also has this neat cloud feature where you can watch your media from anywhere with a fast enough internet connection, and share with friends and family. I think it’d be cool to get my parents a Roku box, and be able to ‘curate’ a movie collection for them with no effort on their part and without them having to have a server/NAS running, by ripping their DVDs onto my home NAS.

Another feature of Kodi that has been taken out of Plex is the concept of a ‘stub’ file, where you just keep a 0 byte file on the NAS that says “out in the physical world, there’s a DVD with ‘The Godfather (1972)’ on it”, so that Kodi can be your librarian for offline movies too. So now, to use Plex, I definitely need to have some kind of online copy of each DVD (or maybe a 2 second video to take its place, that says “look on the shelf behind you”… hmmm)

Anyway, I now have a lot of ISO files that need to become MKV or avi or something else, ideally with no input from me. It took me a while to work out how to get a title extracted from a DVD ISO (not a physical DVD) as an MPEG2 stream, so here’s another little post about that. ‘title’ is what the DVD standard calls the individual video streams on the DVD – for example, one for the main movie, one for the trailer, and others for special features. They are all stored in MPEG-2 video format, which is getting old now, and not so great for compression. The cool kids are using H.264 these days.

I did this on my Windows desktop, but the tools are natively Linux tools so it should work on Linux or OS X too. You need mplayer and ffmpeg. Actually, you probably could do this with just mplayer/mencoder, but this worked for me and so I stopped looking. I specifically wanted a command-line method to do it. If you don’t care about that, just use Handbrake, which is very nice. I may yet do that too, since it also has a job queue.

First, to extract Title 1 from the DVD as MPEG-2 video:

mplayer dvdnav://1 -dvd-device DVDVolume.iso -dumpstream -dumpfile title1.m2v

If you don’t know which title you need, mplayer can help there too:

mplayer dvdnav:// -dvd-device DVDVolume.iso -frames 0 -identify -vo null -ao null -nocache

which will spit out a LOT of information, including this:

TITLE 1, CHAPTERS: 00:01:37,00:10:51,00:14:10,00:23:21,00:24:09,00:27:44 {etc}
TITLE 2, CHAPTERS: 00:02:10,

which tells me that title 1 is the movie, and title 2 is the trailer (only 2 minutes long).

Then to make an MKV file of the resulting MPEG-2 file:

ffmpeg.exe -i title1.m2v ./title1.mkv

There are loads of settings for x264 that I don’t understand, so I just ignored them. The results look pretty good so far, and I get files of about 1.5GB instead of the 4GB original. Each one takes a while (about an hour? I haven’t timed it), but there’s nothing to watch or poke, so it doesn’t really matter. I can leave it to do its thing.

Leave a Comment

Filed under Tech

wraprancid and RANCID 3.x

Jethro R Binks’ excellent wraprancid script allows you to bring in configurations (and pretty much anything else that can be text) without having to get involved in writing a new ?rancid/?login combination for your device. That avoids some pretty hairy perl and Tcl code, so it’s definitely a Good Thing! It’s also useful for devices that don’t even have a command-line, but might allow you to fetch their config from a web page, or TFTP.

The trouble is, RANCID changed the way it deals with device types between RANCID 2.x and RANCID 3. It changed in a good way, so that the patches to rancid-fe that tools like wraprancid required are no longer necessary. What was previously hard-coded in the source of rancid-fe is now a proper configuration file, with a second config file for you to add your own types to. Here’s how to get wraprancid working with RANCID 3.x

First, I’m assuming you have a working wrapplugin script. Here’s one I use to fetch the config from Asterisk servers.

#!/opt/perl/bin/perl -w
# Modules

# Load any modules needed
use strict;
use Getopt::Std;
use Net::SSH::Perl;

# Variables

# Initialize variables used in this script

my $debug = 0;

my %options = ();
getopts('df:', \%options);
my $file = $options{'f'};
my $fh;
my $host = $ARGV[0];

$debug = $options{'d'};

print STDERR "to host: $host\n" if $debug;

my $ssh = Net::SSH::Perl->new($host, protocol => '2,1', debug => $debug );

print STDERR "made ssh obj\n" if $debug;

print STDERR "login\n" if $debug;
my ($stdout, $stderr, $exit) = $ssh->cmd("true");
print STDERR "got output\n" if $debug;

# Open the output file.
open($fh, ">", $file) or die "Cannot open output file\n";
print $fh "#RANCID-CONTENT-TYPE: wrapper.asterisk\n#\n";

print $fh $stdout;
print STDERR "wrote output of ". length($stdout)." bytes\n" if $debug;

# End #
print STDERR "done\n" if $debug;

That lives in ~rancid/bin/asterisk.wrapplugin, just as it did in version 2.

Then, in ~rancid/etc/rancid.types.conf, we’ll define a new device type called wrapper-asterisk:

wrapper-asterisk;script;wraprancid -s asterisk.wrapplugin

(I don’t think the login script matters, as it’s never used, but it must be specified to keep RANCID happy)

And finally in the router.db, you can put your actual device:


That’s it. You can repeat for whichever other scripts you need to do this for.

Bonus Tip

The asterisk end of the script above works like this: we use SSH public key authentication to connect to the server, and then in ~root/.ssh/authorized_keys, there is a line like this:

command="/usr/sbin/asterisk -V; echo 'extensions.conf'; cat /etc/asterisk/extensions.conf; echo 'sip.conf'; cat /etc/asterisk/sip.conf; echo 'iax.conf';cat  /etc/asterisk/iax.conf",from="myrancidhost" ssh-dss AAAAB3NzaC174ENozlUVBe5hH32Wy/duAJt1b4nWbVPoW1GP/koSZNv3888s3fx23nEpLMJxispulA== rancid@myrancidhost

So that the user authenticating with that particular key doesn’t get a shell, they just get the output from a series of cat commands, and then disconnected. They must also be connecting from the RANCID server.

So, now we have Asterisk in the same version control system as our network gear. You can use a similar setup for things like BSD ipfw-based firewalls, or Quagga routers.

Leave a Comment

Filed under Monitoring, Network, Projects & Hacking

IOS XR SSH: “Server refused to allocate pty”

Just another SSH note, since it took me a while to work out what was going on: if you putty into an IOS XR router (ASR9001 in my case) with SSH Agent Forwarding enabled then it just drops the connection with this message: “Server refused to allocate pty”

Solution: don’t do that. Turn off Agent Forwarding in the SSH options (or stop using ssh -A for the commandline client).

Leave a Comment

Filed under Uncategorized

RANCID, ssh, Cisco MDS and “too many authentication failures”

I just ran into this, and it took a little while to figure out, so here’s my quick note. If you have a Cisco MDS being backed up by RANCID, then you can get the following odd message, even if it’s the first time you tried to log in with this user:

Received disconnect from 2: Too many authentication failures for confbackup

What is happening is that the ssh client tries with whatever public keys it has configured first, and then the password-based auth that you thought it was doing all along. With a few keys, that’s enough to annoy the MDS into closing the connection.

The solution is to disable public-key auth for this connection. To do that with RANCID requires a little bit of extra work. First, create a shellscript (I call mine /opt/rancid/local/ssh-no-pubkey):


ssh -o PubkeyAuthentication=no $*

Then for the devices that are suffering, tell RANCID to use this new SSH command instead of just ‘ssh’. In .cloginrc:

add sshcmd mds01 {/opt/rancid/local/ssh-no-pubkey}

Now RANCID can login and backup the config fine.

Additional tip – the ‘cisco’ device type seems to work better than the (theoretically correct) ‘cisco-nx’ device type for MDS switches.

Leave a Comment

Filed under Monitoring, Network, Tech, Uncategorized

KIO’s zasm on Ubuntu

kio makes a nice-looking z80 assembler with lots of interesting output options for cross-development to a ZX Spectrum target, like writing .SNA files straight from the assembler. However, it only comes with binaries for a few platforms (or rather not all versions on all platforms), and source-code comes with just an xcode build file. The instructions then just say ‘create a Makefile’ to get it to compile! Here’s how I compiled it on Ubuntu 14.04…

First, you’ll need a c++ compiler:

sudo apt-get install g++

Then, you’ll need the config.h for linux:

cd {the directory you unzipped the source into}
cp config.h config.h-dist
cp config.h.LINUX config.h

Then compile it all together in a single line:

g++ -o zasm -I. -I kio *.cpp kio/*.cpp -lpthread

Which should leave you with a zasm binary in the current directory.

To install:

sudo cp zasm /usr/local/bin

Finally, to get an emulator to feed your project to:

sudo apt-get install fuse-emulator-gtk

1 Comment

Filed under Uncategorized

Virtual serial ports in Windows VM hosts (for IOS XRv)

I’ve been trying to play around with Cisco IOS XRv this morning – the virtual machine version of the IOS XR software used on the ASR-9000 and CRS-1 series routers. Having expensive hardware like that for a simple test environment is tricky, but XRv means you can have one on your desktop.

The VM boots fine, but then leaves you with a nearly-blank screen that says “Booting IOS XRv”. What happened? All the action is happening on the serial port of the device, just like on a real router. So you need a serial port in the VM, and some way to talk to that.

I went through quite a few different versions of this, using NPTP, and VMwareGateway. I tried with VirtualBox and VMware Workstation. In each case, I would get a login prompt that didn’t work. I would see my password echoed back on the screen, and the router would generally act strangely – showing a password prompt and immediately saying that login failed, for example.

Then I found out that PuTTY can talk directly to a named pipe. So all you actually need is PuTTY. Fire it up, select Serial, then paste in the named pipe name you used when configuring the VM. It works! And login works, too!

So the full process: import the OVA. Then add a serial port to the VM (if it’s in VMware Workstation, VirtualBox already has one). Set the serial port type to Named Pipe, and add something like “.\pipe\my_xrv” to the name for the pipe. It MUST start with “.\pipe\”. In VMware, you want to say “This is the server”. In VBox, you say “Create Pipe”.

Then start the VM, and use Putty to connect (with Serial connection selected) to “.\pipe\my_xrv” where it usually says “COM1”.

Obviously the same technique works for anything else that needs a serial port to talk to the world.

Leave a Comment

Filed under Network, Tech

See inside long-running Unix pipelines

So you are restoring a database on your Linux system, and you have the 3GiB SQL file ready to go, and so

mysql -uroot -p mydb < backup-2013-10-17.sql

and then… wait. But for how long? It’s probably broken right? It should never take this long! Your service desk needs to know what to tell customers.

Pipe Viewer is a handy little tool to use in place of cat(1), which gives you a progress bar and throughput figures for long-running processes like this.

pv -cN source < backup-2013-10-17.sql | mysql -uroot -p mydb

It also gives an ETA, which is about as good as Windows file copy ETAs, but knowing that something is moving, and at what kind of pace is very reassuring.

Leave a Comment

Filed under Tech

Personalising your Apple product is the door to a world of pain…

Apple do this nice service when you order directly from the Apple Online Store where you can get an engraving on the back of your iPod, iPad etc for free. It looks really nice, and it adds about a day onto the delivery time. What could go wrong? Well, let me tell you a story…

I recently bought an iPad Mini for my girlfriend’s birthday. Got her a nice designer case for it, a data nanosim and a cute little private joke engraved on the back (along with her e-mail address – I’m still fairly practical). She loved it, and started “moving in” straight away. The next day though, it was failing already – the display backlight wouldn’t light and in just the right light you could make out a very rough-looking display. So we wiped it clean (iTunes could still talk to it) and went to the Apple store.

First of all, one does not simply walk into an Apple store. Obviously you need an appointment with a “genius” to actually get customer service. Although it turns out if you stand in the middle of a ipad-buying crowd with a dead unit and a grumpy face, someone will help. Except they can’t, because it’s personalised, so it has to go back to the mothership.

The one super-impressive part of the customer service experiences comes next – Apple’s phone system. If you call from the phone that they have recorded against your name for your account, the phone system just says (literally) “Oh, is it about that ipad?”, and everyone knows what is going on. Neat!

They send out a courier box for the unit, and you send it back. Then they send out the repaired one. Unlike the Apple store itself, they use TNT, and they don’t let you know the tracking number for the repaired item.

So my repair is turned around in a day or so, and then the item is just marked “shipped” on the Apple site. I wait, and wait some more. After speaking to Apple customer service, they want to wait until TNT have had their “allow for delivery” time, which is something like 10 days. I did get the tracking number from them though, so I could play along at home – it just says “out for delivery” for several days. Eventually TNT admit that actually they’ve lost it, so Apple organise another replacement, with personalisation. They confirm the details, and get it sent. It arrives a few days later, and it’s the wrong spec. We go back two steps and repeat one more time. Finally, 6 weeks after her birthday, we have a working unit!

In their defence, I did get personal contact details from several folks at Apple who followed the case through for me, but it shouldn’t really come to that. Obviously, if I’d have just bought the unit without personalisation, I could have had it swapped out at the local physical store, and been done in a day or two.

Think twice before engraving!

Leave a Comment

Filed under Uncategorized