Paul Rayner -- pr0.uk

liarsping - Faking ping times for fun and (no) profit

Sun, 19 Apr 2026 13:00:00 +0100

In my ongoing quest to resurrect and finish abandoned projects, I present liarsping…

According to man ping, ping first appeared in 4.3BSD which was released in 1986. It’s main use (for me at least) is to see if a remote machine is reachable, with the secondary goal of seeing how far away that machine is in ters of latency. It’s the second goal I’m going to mess with here.

There seem to be two different ways used to calculate the round trip latency in a ping. The first (used by linux and Mac OS) is to encode the current timestamp into the outgoing ICMP ECHO packet, then when it comes back, calculate the difference between the current system time and the timestamp in the returned packet. The second (used by Windows) is to store locally the timestamp of the outgoing ping message against an ID, send the sequence number and then look up the send timestamp when the reply comes back. Windows uses a monotonically increasing sequence number for the ID.

Both of these systems can be gamed by a naughty ICMP ECHO server to return not just bigger ping times (add a delay in responding, not very interesting), but also a smaller ping time.

Linux

This is pretty trivial - add some time onto the timestamp before echoing it back. The client doesn’t validate anything, so if you add 10ms, the client will report a 10ms shorter ping time. Most linux servers are synced to an NTP server, so the clock time on the client and server will likely be very close. This additionally allows us to make a good guess at the real latency from the server side, and knock off the right ammount of latency to achieve whatever number we want the client to see.

Windows

Because Windows sends a predictable sequence of pings, we can send the reply before we get the message. By default, ping sends a message per second, so if we want to shave 10ms from the ping time, we send the next response 990ms after the last one. This will work for everything except the first packet which we can either drop, or answer honestly.

Liarsping

I wrote a small test script in rust about 2 years ago which proved to me that this works, but didn’t have the time to polish it and publish it.

Enter Claude..

I gave claude my proof of concept and asked it to wrap it in config options and generally polish the code. The result is liarsping.

Try it out

Send ping requests to ping.pr0.uk. If you are sending from Windows, I’ve shaved off 20ms, and if you’re sending from linux, I’ve shaved off 150% of the one way trip time according to the timestamp on my server.

Written by a human

For the avoidance of doubt, this post was not written with any help from any AI, even though the code was.

Thanks for reading

pgfs - a FUSE filesystem on top of a postgres database

Tue, 18 Nov 2025 12:00:00 +0000

pgfs is a FUSE based filesystem which is backed by a postgres table, with a bytea field as the file. The filesystem is configured with a simple toml file:

mountpoint = "/home/paul/pgfs"

[database]

database = "127.0.0.1/pgfstest"
user = "user"
pass = "pass"

[filestest]

table_name = "files"
data_type = "bytea"
id_field = "id"
length_field = "length"
data_field = "file"
created_date_field = "created"
modified_date_field = "modified"
data_query = "select id, name,  octet_length(file) as length, created, modified from files"
readonly = false

if you then create a table files:

create table files (
    id serial primary key,
    name varchar,
    created timestamp default now(),
    modified timestamp,
    file bytea
);

Running pgfs with the specified config will present you with a filesystem at /home/paul/pgfs which contains a single folder “files”. Inside that folder, there will be one file per record in the files table. You can modify files, add them (creates a new record), or delete them. It’s not very fast (FUSE uses 4k writes and postgres isn’t optimised for seeking inside a bytea field), but it works. On my linux box, vim works on text files just fine, and drag and drop, renaming, thumbnails, copy & paste etc all work fine in nautilus.

I built this mostly for fun (to see if I could get it to work), but for some small projects I store files as bytea fields for convenience so it’s useful.

I plan to add a few niceties, one which would make it a lot more useful would be to allow paths to show up as links, so if you don’t store bytea files (which I wouldn’t typically do either), but store paths to files outside the database, you can have a filesystem created with those links in. Presenting a table as a delimited text file you can edit as a file would also be neat.

The code is on Github, and it’s published to crates.io, so you can install with

cargo install pgfs

I wrote most of this code back in 2021, but am trying to tidy up and publish a few utilities I’ve created over the years in the coming months.

Thanks for reading

Print (only) my public IP

Sun, 29 Oct 2023 11:10:53 +0000

Every now and then, I need to know my public IP. The easiest way to find it is to visit one of the sites which will display it for you, such as https://whatismyip.com. Whilst useful, all of the ones I know (including that one) are chock full of adverts, and can’t easily be scraped as part of any automated scripting.

This has been a minor irritation for years, so the other night I decided to fix it.

http://ip.pr0.uk is my answer. It’s 50 lines of rust, and is accessible via tcp on port 11111, and via http on port 8080.

use std::io::Write;

use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr, TcpListener, TcpStream};
use chrono::Utc;
use threadpool::ThreadPool;

fn main() {
    let worker_count = 4;
    let pool = ThreadPool::new(worker_count);
    let tcp_port = 11111;
    let socket_v4_tcp = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), tcp_port);

    let http_port = 8080;
    let socket_v4_http = SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), http_port);

    let socket_addrs = vec![socket_v4_tcp, socket_v4_http];
    let listener = TcpListener::bind(&socket_addrs[..]);
    if let Ok(listener) = listener {
        println!("Listening on {}:{}", listener.local_addr().unwrap().ip(), listener.local_addr().unwrap().port());
        for stream in listener.incoming() {
            let stream = stream.unwrap();
            let addr =stream.peer_addr().unwrap().ip().to_string();
            if stream.local_addr().unwrap_or(socket_v4_http).port() == tcp_port {
                pool.execute(move||send_tcp_response(stream, addr));
            } else {
                //http might be proxied via https so let anything which is not the tcp port be http
                pool.execute(move||send_http_response(stream, addr));
            }
        }
    } else {
        println!("Unable to bind to port")
    }
}

fn send_tcp_response(mut stream:TcpStream, addr:String) {
    stream.write_all(addr.as_bytes()).unwrap();
}

fn send_http_response(mut stream:TcpStream, addr:String) {

    let html = format!("<html><head><title>{}</title></head><body><h1>{}</h1></body></html>", addr, addr);
    let length = html.len();
    let response = format!("HTTP/1.1 200 OK\r\nContent-Length: {length}\r\n\r\n{html}" );
    stream.write_all(response.as_bytes()).unwrap();
    println!("{}\tHTTP\t{}",Utc::now().to_rfc2822(),addr)
}

A little explanation is needed on the array of SocketAddr. This came from an initial misreading of the docs, but I liked the result and decided to keep it that way. Calls to listen() will only listen on one port - the first one in the array which is free. The result is that when you run this program, it listens on port 11111. If you keep it running and start another copy, that one listens on port 80 (because it can’t bind to port 11111). So to run this on my server, I just have systemd keep 2 copies alive at any time.

The code and binaries for Linux and Windows are available on Github.

Next steps

I might well leave it there. It works for me, so it’s done. Here are some things I could do though:

1) Don’t hard code the ports 2) Proxy https 3) make a client 4) make it available as a binary for anyone to run on crates.io 5) Optionally print the ttl. This would be mostly useful to people running their own instance.

Boring Details

Logging

I log the IP, port, and time of each connection. This is just in case it ever gets flooded and I need to block an IP/range. The code you see above is the code I run. No browser detection, user agent or anythign like that is read or logged. Any data you send with the connection is discarded. If I proxied https via nginx, that might log a bit of extra data as a side effect.

Systemd setup

There’s not much to this either. I have a template file:

[Unit]
Description=Run the whatip binary. Instance %i
After=network.target

[Service]
ExecStart=/path/to/whatip
Restart=on-failure

StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=whatip%i

[Install]
WantedBy=multi-user.target

stored at /etc/systemd/system/whatip@.service and then set up two instances to run:

systemctl enable whatip@1
systemctl enable whatip@2

Thanks for reading

Putting dc in (chroot) jail

Sun, 06 Feb 2022 21:43:53 +0000

A little over 4 years ago, I set up a VM and configured it to offer dc over a network connection using xinetd. I set it up at http://dc.pr0.uk and made it available via a socket connection on port 1312.

Yesterday morning I woke to read a nice email from Sylvan Butler pointing out that users could run shell commands from dc…

I had set up the dc command to run as a user “dc”, but still, if someone could run a shell command they could, for example, put a key in the dc user’s .ssh config, run sendmail (if it was set up), try for privelidge escalations to get root etc.

I’m not sure what the 2017 version of me was thinking (or wasn’t), but the 2022 version of me is not happy to leave it like this. So here’s how I put dc in jail.

Firstly, how do you run shell commands from dc? It’s very easy. Just prefix with a bang:

$ dc
!echo "I was here" > /tmp/foo
!cat /tmp/foo
I was here

So, really easy. Even if it was hard, it would still be bad.

This needed to be fixed. Firstly I thought about what else was on the VM - nothing that matters. This is a good thing because the helpful Sylvan might not have been the first person to spot the issue (although network dc is pretty niche). I still don’t want this vulnerability though as someone else getting access to this box could still use it to send spam, host malware or anything else they wanted to do to a cheap tiny vm.

I looked at restricting the dc user further (it had no login shell, and no home directory already), but it felt like I would always be missing something, so I turned to chroot jails.

A chroot jail lets you run a command, specifying a directory which is used as / for that command. The command (In theory) can’t escape that directory, so can’t see or touch anything outside it. Chroot is a kernel feature, and forms a basic security feature of linux, so should be good enough to protect network dc if set up correctly, even if it’s not perfect.

Firstly, let’s set up the directory for the jail. We need the programs to run inside the jail, and their dependent libraries. The script to run a networked dc instance looks like this:

#!/bin/bash
dc --version
sed -u -e 's/\r/\n/g' | dc

Firstly, I’ve used bash here, but this script is trivial, so it can use sh instead. We also need to keep the sed (I’m sure there are plenty of ways to do the replace not using sed, but it’s working fine as it is). For each of the 3 programs we need to run the script, I ran ldd to get their dependencies:

$ ldd /usr/bin/dc
	linux-vdso.so.1 =>  (0x00007fffc85d1000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc816f8d000)
	/lib64/ld-linux-x86-64.so.2 (0x0000555cd93c8000)
$ ldd /bin/sh
	linux-vdso.so.1 =>  (0x00007ffdd80e0000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa3c4855000)
	/lib64/ld-linux-x86-64.so.2 (0x0000556443a1e000)
$ ldd /bin/sed
	linux-vdso.so.1 =>  (0x00007ffd7d38e000)
	libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007faf5337f000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007faf52fb8000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007faf52d45000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007faf52b41000)
	/lib64/ld-linux-x86-64.so.2 (0x0000562e5eabc000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007faf52923000)
$

So we copy those files to the exact directory structure inside the jail directory. Afterwards it looks like this:

$ ls -alR
.:
total 292
drwxr-xr-x 4 root root   4096 Feb  5 10:13 .
drwxr-xr-x 4 root root   4096 Feb  5 09:42 ..
-rwxr-xr-x 1 root root  47200 Feb  5 09:50 dc
-rwxr-xr-x 1 root root     72 Feb  5 10:13 dctelnet
drwxr-xr-x 3 root root   4096 Feb  5 09:49 lib
drwxr-xr-x 2 root root   4096 Feb  5 09:50 lib64
-rwxr-xr-x 1 root root  72504 Feb  5 09:58 sed
-rwxr-xr-x 1 root root 154072 Feb  5 10:06 sh

./lib:
total 12
drwxr-xr-x 3 root root 4096 Feb  5 09:49 .
drwxr-xr-x 4 root root 4096 Feb  5 10:13 ..
drwxr-xr-x 2 root root 4096 Feb  5 10:01 x86_64-linux-gnu

./lib/x86_64-linux-gnu:
total 2584
drwxr-xr-x 2 root root    4096 Feb  5 10:01 .
drwxr-xr-x 3 root root    4096 Feb  5 09:49 ..
-rwxr-xr-x 1 root root 1856752 Feb  5 09:49 libc.so.6
-rw-r--r-- 1 root root   14608 Feb  5 10:00 libdl.so.2
-rw-r--r-- 1 root root  468920 Feb  5 10:00 libpcre.so.3
-rwxr-xr-x 1 root root  142400 Feb  5 10:01 libpthread.so.0
-rw-r--r-- 1 root root  146672 Feb  5 09:59 libselinux.so.1

./lib64:
total 168
drwxr-xr-x 2 root root   4096 Feb  5 09:50 .
drwxr-xr-x 4 root root   4096 Feb  5 10:13 ..
-rwxr-xr-x 1 root root 162608 Feb  5 10:01 ld-linux-x86-64.so.2
$

and here is the modified dctelnet command:

#!/sh
#dc | dos2unix 2>&1
./dc --version
./sed -u -e 's/\r/\n/g' | ./dc

I’ve switched to using sh instead of bash, and all of the commands are now relative paths, as they are just in the root directory.

First attempt

Now I have a directory that I can use for a chrooted dc network dc. I need to set up the xinetdconfig to use chroot and the jail I have set up:

service dc
{
	disable		= no
	type		= UNLISTED
	id		= dc-stream
	socket_type	= stream
	protocol	= tcp
	server		= /usr/sbin/chroot
	server_args	= /home/dc/ ./dctelnet
	user		= root
	wait		= no
	port		= 1312
	rlimit_cpu	= 60
	env		= HOME=/ PATH=/
}

I needed to set the HOME and PATH environment variables otherwise (not sure whether it was sh,sed or dc causing it) I got a segfault, and to run chroot, you need to be root, so I could no longer run the service as the user dc. This shouldn’t be a problem because the resulting process is constrained.

A bit more security

Chroot jails have a reputation for being easy to get wrong, and they are not something I have done a lot of work with, so I want to take a bit of time to think about whether I’ve left any glaring holes, and also try to improve on the simple option above a bit if I can.

Firstly, can dc still execute commands with the ! operation?

 ~> nc -v dc.pr0.uk 1312
Connection to dc.pr0.uk 1312 port [tcp/*] succeeded!
dc (GNU bc 1.06.95) 1.3.95

Copyright 1994, 1997, 1998, 2000, 2001, 2004, 2005, 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.
!ls
^C⏎

Nope. Ok, that’s good. The chroot jail has sh though, and has it in the PATH, so can it still get a shell and call dc, sh and sed?

 ~> nc -v dc.pr0.uk 1312
Connection to dc.pr0.uk 1312 port [tcp/*] succeeded!
dc (GNU bc 1.06.95) 1.3.95

Copyright 1994, 1997, 1998, 2000, 2001, 2004, 2005, 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.
!pwd
^C⏎

pwd is a builtin, so it looks like the answer is no, but why? Running strings on my version of dc, there is no mention of sh or exec, but there is a mention of system. From the man page of system:

The system() library function uses fork(2) to create a child process that executes the shell  command  specified in command using execl(3) as follows:

           execl("/bin/sh", "sh", "-c", command, (char *) 0);

So dc calls system() when you use !, which makes sense. system() calls /bin/sh, which does not exist in the jail, breaking the ! call.

For a system that I don’t care about, that is of little value to anyone else, that sees very little traffic, that’s probably good enough, but I want to make it a bit better - if there was a problem with the dc program, or you could get it to pass something to sed, and trigger an issue with that, you could mess with the jail file system, overwrite the dc application, and likely break out of jail as the whole thing is running as root.

So I want to do two things. Firstly, I don’t want dc running as root in the jail. Secondly, I want to throw away the environment after each use, so if you figure out how to mess with it you don’t affect anyone else’s fun.

Here’s a bash script which I think does both of these things:

#!/bin/bash
set -e
DCDIR="$(mktemp -d /tmp/dc_XXXX)"
trap '/bin/rm -rf -- "$DCDIR"' EXIT
cp -R /home/dc/ $DCDIR/
cd $DCDIR/dc
PATH=/
HOME=/
export PATH
export HOME
/usr/sbin/chroot --userspec=1001:1001 . ./dctelnet

Line 2 - set -e causes the script to exit on the first error
Lines 3 & 4 - make a temporary directory to run in, then set a trap to clean it up when the script exits.
I then copy the required files for the jail to the new temp directory, set $HOME and SPATH and run the jail as an unprivileged user (uid 1001).

Now to make some changes to the xinetd file:

service dc
{
        disable         = no
        type            = UNLISTED
        id              = dc-stream
        socket_type     = stream
        protocol        = tcp
        server          = /usr/local/bin/dcinjail
        user            = root
        wait            = no
        port            = 1312
        rlimit_cpu      = 60
        log_type        = FILE /var/log/dctelnet.log
        log_on_success  = HOST PID DURATION
        log_on_failure  = HOST
}

The new version just runs the script from above. It still needs to run as root to be able to chroot.

I’ve also added some logging as this has piqued my interest and I want to see how many people (other than me) ever connect, and for how long.

As always, I’m interested in feedback or questions. I’m no expert in this setup so may not be able to answer questions, but if you see something that looks wrong (or that you know is wrong), please let me know. I’m also interested to hear other ways of process isolation - I know I could have used containers, and think I could have used systemd or SELinux features (or both) to further lock down the dc user and achive a similar result.

Thanks for reading.

Snakes and Ladders, Estimation and Stats (or 'Sometimes It Takes Ages')

Sun, 09 Jan 2022 15:23:53 +0000

Snakes And Ladders

Simple kids game, roll a dice and move along a board. Up ladders, down snakes. Not much to it?

We’ve been playing snakes and ladders a bit (lot) as a family because my 5 year old loves it. Our board looks like this:

Some games on this board take a really long time. My son likes to play games till the end, so until all players have finished. It’s apparently really funny when everyone else has finished and I keep finding the snakes over and over. Sometimes one player finishes really quickly - they hit some good ladders, few or no snakes and they are done in no time.

This got me thinking. What’s the distribution of game lengths for snakes and ladders? How long should we expect a game to take? How long before we typically have a winner?

Fortunately for me, snakes and ladders is a very simple game to model with a bit of python code.

Firstly, here are the rules we play:

1) Each player rolls a normal 6 sided dice and moves their token that number of squares forward. 2) If a player lands on the head of a snake, they go down the snake 3) If a player lands on the bottom of a ladder, they go up to the top of the ladder. 4) If a player rolls a 6, they get another roll 5) On this board, some ladders and snakes interconnect - the bottom of a snake is the head of another, or the top of a ladder is also the head of a snake. When this happens, you do all of the actions in turn, so down both snakes or up the ladder, down the snake. 6) You don’t need an exact roll to finish, once you get 100 or more, you are done.

To model the board in python, all we really need are the coordinates of the snakes and the ladders - their starting and ending squares.

def get_snakes_and_ladders():

   snakes = [
        (96,27),
        (88,66),
        (89,46),
        (79,44),
        (76,19),
        (74,52),
        (57,3),
        (60,39),
        (52,17),
        (50,7),
        (32,15),
        (30,9)
    ]
    ladders = [
        (6,28),
        (10,12),
        (18,37),
        (40,42),
        (49,67),
        (55,92),
        (63,76),
        (61,81),
        (86,94)
    ]
    return snakes + ladders

Since snakes and ladders are both mappings from one point to another, we can combine them in one array as above.

The game is moddeled with a few lines of python:

class Game:

    def __init__(self) -> None:
        self.token = 1
        snakes_and_ladders_list = get_snakes_and_ladders()
        self.sl = {}
        for entry in snakes_and_ladders_list:
            self.sl[entry[0]] = entry[1]

    def move(self, howmany):
        self.token += howmany
        while (self.token in self.sl):
            self.token = self.sl[self.token]
        return self.token

    def turn(self):
        num = self.roll()
        self.move(num)
        if num == 6:
            self.turn()
        if self.token>=100:
            return True
        return False

    def roll(self):
        return randint(1,6)

A turn consists of all the actions taken by a player before the next player gets their turn. This can consist of multiple moves if the player rolles one or more sixes, as rolling a six gives you another move.

With this, we can run some games and plot them. Here’s what a sample looks like.

The Y axis is the position on the board, and the X axis is the number of turns. This small graphical representation of the game shows how variable it can be. The red player finishes in under 20 moves, whereas the orange player takes over 80.

To see how variable it is, we can run the simulation a large number of times and look at the results. Running for 10,000 games we get the following:

function	result
min	5
max	918
mean	90.32
median	65

So the fastest finish in 10,000 games was just 5 turns, and the slowest was an awful (if you were rolling the dice) 918 turns.

Here are some histograms for the distribution of game lengths, the distribution of number of turns for a player to win in a 3 person game, and the number of turns for all players to finish in a 3 person game.

The python code for this post is at snakes.py

Valentines Gift - a Tidy Computer Cupboard

Sun, 14 Feb 2021 20:43:53 +0000

Today, my lovely wife (who is far more practical than me) gave me this as a valentines present (along with a nice new pair of Nano X1 trainers).

This is my nice new home server rack. It’s constructed from the finest pallet wood and repurposed chipboard, and has 8 caster wheels (cheaper than the apple ones) on the bottom.

After three and a half years living in our house, the cupboard under the stairs was a mess of jumbled cables and computer bits. It all worked, but with things balanced on other things, held up by their cables, and three years of dust everywhere it really needed an overhall. We’ve recently had a new fibre connection go in (yay - 1Gbps at home!), so yet another cable, and yet another box to balance on top of other boxes.

This was the sorry mess that I called a home network this morning:

And thanks to my lovely gift, and some time to rewire everything (make new cables), it now looks like this:

and a close up:

In there I have my server, UPS, NAS, phone system, lighting system, FTTC broadband, Fibre broadband, router, main switch, and a cable going out and round the house into my office. Lovely and neat, and because it’s on wheels, I can pull it out to get round the back :-)

I am very happy with my new setup.

Helper functions for prototyping with Rocket

Thu, 31 Dec 2020 17:05:00 +0000

Over the holidays I have enjoyed playing a little with Rocket. Here are a couple of things I’ve written which might be useful to others when prototyping a new site using rocket.

Firstly, the examples show you how to create an instance of a struct from either a Query or a Form, but when using a template (I am using a rust implementation of Handlebars) it can be useful to just pass all of the fields through as a map. Here are two simple methods (one for Form, another for Query) which populate a HashMap with the incoming data.

struct RequestMap {
    map: HashMap<String,String>
}

impl<'f> FromForm<'f> for RequestMap {
    type Error = ();

    fn from_form(items: &mut FormItems<'f>, _strict: bool) -> Result<RequestMap, ()> {
        let mut map = HashMap::new();

        for item in items {
            let k = item.key.url_decode().map_err(|_| ())?;
            let v = item.value.url_decode().map_err(|_| ())?;
            map.insert(k, v );

        }

        Ok(RequestMap { map})
    }
}

impl FromQuery<'_> for RequestMap {
    type Error = ();

    fn from_query(items: Query) -> Result<RequestMap, ()> {
        let mut map = HashMap::new();
        for item in items {
            let k = item.key.url_decode().map_err(|_| ())?;
            let v = item.value.url_decode().map_err(|_| ())?;
            map.insert(k, v );

        }

        Ok(RequestMap { map})
    }
}

We create a struct RequestMap which contains just a HashMap, then implement the methods FromForm and FromQuery on the map.

Now these maps can be used in routes as follows:

#[get("/example_get/<name>?<params..>")]
fn example_get_route(name: String, params: RequestMap) -> Template {
    Template::render(name, &params.map)
}

#[post("/example_post/<name>", data="<params>")]
fn example_post_route(name: String, params: Form<RequestMap>) -> Template {
    Template::render(name, &params.into_inner().map)
}

In these examples I have also set up a name parameter which maps to the template name, so you can copy and paste templates around and try them out with different parameters easily.

The second Thing I have found useful in prototyping with Rocket is to set up a Handlebars helper to print out information from the provided context. You can put this in to render as a comment in your template so that you can easily see what context is being provided to your template.

Here is the helper definition:

#[derive(Clone,Copy)]
struct DebugInfo;

impl HelperDef for DebugInfo {
    fn call<'reg:'rc,'rc> (&self, h: &Helper, _: &Handlebars, ctxt: &Context, rc: &mut RenderContext, out: &mut dyn Output) -> HelperResult {
        out.write(&format!("Context:{:?}",ctxt.data()))?;
        out.write(&format!("Current Template:{:?}", rc.get_current_template_name()))?;
        out.write(&format!("Root Template:{:?}", rc.get_root_template_name()))?;
        for (key, value) in h.hash() {
            out.write(&format!("HashKey:{:?}, HashValue:{:?}",key, value))?;
        }
        Ok(())
    }
}

and you set it up like this in the Rocket initialisation:

rocket::ignite().mount("/", routes![index,example_get_route,example_post_route])
        .attach(Template::custom(|engines|{
            engines.handlebars.register_helper("debug", Box::new(DebugInfo));
        }))

To use this as a helper, put something like this inside your Handlebars template:

<!--{{debug nameparam=username}}-->

The output should look something like this:

<!--Context:Object({"username": String("aa")})Current Template:Some("testroute")Root Template:Some("testroute")HashKey:"nameparam", HashValue:PathAndJson { relative_path: Some("username"), value: Context(String("paul"), ["username"]) }-->

The above is for a get request where the URL was http:///testroute?username=paul

Thanks for reading. I hope the above proves useful - I am still experimenting with Rocket (also with writing Handlebars helpers and combining all this with htmx ), so there may be simpler or better ways of achieving the above. If you know of one, or if you have any questions, suggestions, or to point out any mistakes, please contact me at the email address below. I’d love to hear from you.

What is an Err?

Mon, 17 Aug 2020 11:05:00 +0100

In rust, we don’t throw exceptions, we return Results. When coming from a language which throws exceptions, it can be easy to think of the Err type in a result as being just like an exception, but sometimes a little more thought can provide something more elegant.

In this example, I am looking at a simple function

fn add_user(username:&str, email:&str, password:&str) -> ???

For this function, we want to add a user to a system. The return type for this could take a number of forms.

First Attempt: Return a struct representing the user if we added the user, and an error if we didn’t

Result<User,Err>

This would be the goto function signature for me when starting out with rust. In my head (coming from Java) it maps nicely to a function signature along the lines of

public User add_user(String username, String email, String password)
	 throws SomeException, SomeOtherException {
...
}

So lets think about what those exceptional circumstances would be.

You might be trying to write to a database, and the connection fails. That’s an exceptional circumstance, but one we need to handle gracefully. You probably don’t want your real world app to crash when your database is unavailable briefly for some maintenance. You could internalise this in the function if this was a public API, so your return value in this case might be saying something like Something went wrong. This is probably temporary, so please try again in a bit, or in a function internal to a service, you might want to explicitly handle database errors (email to DBA, log something etc.).
The data you got might be no good. The username or email might already exist, the password might be blank, etc. Here you want a return value which informs the caller of what happened explicitly so that they can take corrective action within normal program flow.

Mixing these together in an Err return is inelegant for the caller to handle. They have to handle the Ok case and the other program flow cases, but will probably want to pass the exceptional cases back up the stack.

Better: Return an enum representing all events in normal program flow, and an Error for exceptions

Here we add an enum that looks something like this:

type Reason =  String;

enum AddUser {
	Added(User),
    	UsernameExists,
    	EmailExists,
    	UsernameInvalid(Reason),
    	EmailInvalid(Reason),
    	PaswordInvalid(Reason)
}

The function definition would now look like this:

fn add_user(username:&str, email:&str, password:&str) -> Result<AddUser,Err>

This looks much better. Err is reserved for genuine errors which we would likely propogate upwards, and our normal program flow is captured in one match statement. The calling function now looks something like this:

...

match add_user(name,email,pass)? {
	Added(u) => {...},
     	UsernameExists => {...}
    	...
}

...

The Err part can now be propogated unchanged up to a level where we can handle genuinely exceptional cases consistently.

Thanks for reading. If you have any questions, suggestions, or to point out any mistakes, please contact me at the email address below. I’d love to hear from you.

The rust type system - elegant and simple

Mon, 20 Apr 2020 21:05:53 +0100

Rust has an elegeant type system. To declare a type is as easy as this:

type MyType = u32;

Now that we have MyType, we can use it as an argument to a function, a return type, or anywhere else that you would use a primitive or struct.

The big advantage this confers is safety. I’d like to illustrate this with an aside into programming history, based on a very worthwile read.

Apps Hungarian was a useful way to help prevent common programming errors. Programmers would include a short description of what a variable’s type should be in the name, so an integer representing a relative coordinate in the x-axis might have a name like BoxMinSz_RelX. On seeing the statement

Dim relx_MaxWidth as Integer
set relx_MaxWidth = max(relx_BoxMinSz, absx_LineMinSz)

It is clear that you are comparing a relative and absolute coordinate, and that this is likely an error.

Hungarian notation got a terrible reputation because the useful technique above is not what people learned as hungarian notation. They learned to write this:

Dim iMaxWidth as Integer
iMaxWidth = max(iBoxMinSz, iLineMinSz)

Which doesn’t add any new and useful semantic information, just bloat.

Defining a type in rust enforces the first case. You can’t pass an u32 where a MyType is expected. This eliminates a whole class of bugs, and in rust it is a zero cost abstraction, so it’s not the wasteful object wrapping you would get in something like Java where one would typically write

class MyType {
	int inner;
}

which turns a primitive into an object, adding references, indirection and increasing storage size.

Thanks for reading. If you have any questions, suggestions, or to point out any mistakes, please contact me at the email address below. I’d love to hear from you.

Nested loops - beat the borrow checker

Mon, 02 Sep 2019 08:05:53 +0100

When coming to rust from other languages, the borrow checker can be frustrating. Common, ingrained ways of doing things just don’t work, and you have to find the “idiomatic rust” way. I guess this is a good thing, and helps prevent the “all my python code looks like bash scripts” problem, whereby you take your patterns for one language and shoehorn them into whatever language you are actually using, but it doesn’t stop it being frustrating…

One pattern I get into quite a bit is the nested loop, where you loop over a collection, and within that loop, loop over it (or a subset) again.

Here is a trivial example you might see in a turn based game. Each player takes their turn (outer loop). When a player acts, the other players see the move, so get a call with the move the other player just made (inner loop). The simple implementation in rust doesn’t compile:

Common code:

trait Player {
    fn get_id(&self)->u32;
    fn act(&mut self)->i32;
    fn see_action(&mut self, player_id:u32, action:i32);
}

struct SimplePlayer {
    id:u32,
    other_player_actions:Vec<i32>
}

impl SimplePlayer {
    pub fn new(player_id:u32) -> SimplePlayer {
        SimplePlayer {
            id:player_id,
            other_player_actions:vec![]
        }
    }
}

impl Player for SimplePlayer {
    fn get_id(&self) ->u32 {
        self.id
    }
    fn act(&mut self) ->i32 {
        if self.other_player_actions.len() == 0 {
            42
        } else {
            self.other_player_actions.iter().sum()
        }
    }
    fn see_action(&mut self, _player_id:u32, action:i32) {
        self.other_player_actions.push(action);
    }
}

Broken loop:

fn main() {
    
    let mut players:Vec<Box<dyn Player>> = Vec::new();
    players.push(Box::new(SimplePlayer::new(1)));
    players.push(Box::new(SimplePlayer::new(2)));
    players.push(Box::new(SimplePlayer::new(3)));
    
    for player in players.iter() {
        let action = player.act();
        for p in players.iter() {
            p.see_action(player.get_id(), action);
        }
    }
    
}

There are several ways around this:

Iterate over a range for the outer loop.
Restructure
Use interior mutability

Let’s look at each of these in turn.

1) Iterate over a range for the outer loop

We can change the outer loop to iterate over a range representing the indices into the Players Vec, and just get a mutable reference to a player for the small scope in which it is needed.

    for player_num in 0..players.len() {
        let action = players.get_mut(player_num).unwrap().act();
        let player_id = players.get(player_num).unwrap().get_id();
        for p in players.iter_mut() {
            p.see_action(player_id, action);
        }
    }

This is the easiest way out here - we don’t need to create a mutable iterator at the top level. If it wasn’t for the inner loop, a mutable iterator would be a bit less code, but with nested loops, itersting over a range allows us to only borrow each player mutably for the short time we are mutating it. When we run the second loop, we have nothing borrowed, so can create a mutable iterator.

2) Restructure

Many times, a nested loop can be replaced with a different program structure. In this case, instead of calling each player to tell it about each event from another player, we can create a game state which is visible to each player.

In our trivial game simulation, the players don’t need to know the moves of the other players until it is thier turn to act. This allows us to refactor the player trait to take in the list of moves as an argument to act.

trait Player {
    fn get_id(&self)->u32;
    fn act(&mut self, moves:&Vec<(u32,i32)>)->i32;
}

struct SimplePlayer {
    id:u32,
}

impl SimplePlayer {
    pub fn new(player_id:u32) -> SimplePlayer {
        SimplePlayer {
            id:player_id
        }
    }
}

impl Player for SimplePlayer {
    fn get_id(&self) ->u32 {
        self.id
    }
    fn act(&mut self, moves:&Vec<(u32,i32)>) ->i32 {
        if moves.len() == 0 {
            42
        } else {
            moves.iter().map(|(_,x)| x).sum()
        }
    }
}

This negates the need to have an inner loop.

 	let mut moves:Vec<(u32,i32)> = vec![];
    
    for player in players.iter_mut() {
        let action = player.act(&moves);
        moves.push((player.get_id(), action));
    }

3. Use interior mutability

Interior mutability moves the borrow checking from compile time to runtime. When we know we won’t have multiple mutable borrows at the same time, but are not able to prove it to the compiler, we can use this to get our program to work. We don’t need to do this here, because as the examples above show, the desired goal can be achieved without it. The rust book has a better example for the use of RefCell with the MockMessenger

For reference, here is an example using RefCell to achieve this.

We change the SimplePlayer to hold a *RefCell<Vec>* rather than a *Vec*

struct SimplePlayer {
    id:u32,
    other_player_actions:RefCell<Vec<i32>>
}

impl Player for SimplePlayer {
    fn get_id(&self) ->u32 {
        self.id
    }
    fn act(&self) ->i32 {
        if self.other_player_actions.borrow().len() == 0 {
            42
        } else {
            self.other_player_actions.borrow().iter().sum()
        }
    }
    fn see_action(&self, _player_id:u32, action:i32) {
        self.other_player_actions.borrow_mut().push(action);
    }
}

Now the original loop works as expected. Here is the full code:

use std::cell::RefCell;

trait Player {
    fn get_id(&self)->u32;
    fn act(&self)->i32;
    fn see_action(&self, player_id:u32, action:i32);
}

struct SimplePlayer {
    id:u32,
    other_player_actions:RefCell<Vec<i32>>
}

impl SimplePlayer {
    pub fn new(player_id:u32) -> SimplePlayer {
        SimplePlayer {
            id:player_id,
            other_player_actions:RefCell::new(vec![])
        }
    }
}

impl Player for SimplePlayer {
    fn get_id(&self) ->u32 {
        self.id
    }
    fn act(&self) ->i32 {
        if self.other_player_actions.borrow().len() == 0 {
            42
        } else {
            self.other_player_actions.borrow().iter().sum()
        }
    }
    fn see_action(&self, _player_id:u32, action:i32) {
        self.other_player_actions.borrow_mut().push(action);
    }
}

fn main() {
    
    let mut players:Vec<Box<dyn Player>> = Vec::new();
    players.push(Box::new(SimplePlayer::new(1)));
    players.push(Box::new(SimplePlayer::new(2)));
    players.push(Box::new(SimplePlayer::new(3)));
    
    for player in players.iter() {
        let action = player.act();
        for p in players.iter() {
            p.see_action(player.get_id(), action);
        }
    }
    
}

Thanks for reading. If you have any questions, suggestions, or to point out any mistakes, please contact me at the email address below. I’d love to hear from you.