Performance in OpenSim

Jun 17, 2020

Yay, I have a new job! I’m now an Open-Source Software Developer at TU Delft, where I’m going to be working on biomechanical simulation software.

Why do I mention this? Because I’m initially tasked with trying to make OpenSim faster, which is something that beautifully ties together a few of my loves (research software, systems development, and low-level perf optimizations) with a few of my hates (software written by researchers, C++, and diagnosing cache misses) and I’ve been wanting to learn+write about performance for a while.

TextAdventurer: Rust Edition

May 9, 2020

Just as a little fun, I decided to rewrite textadventurer in Rust, source).

The server was initially written in ~600 LOC of Java with a basic websocket library. The Java implementation worked fine–it was essentially just a tiny tech demo to demo a jobson feature–but I decided to rewrite it in Rust so I could understand where the pain-points are in an application like this.

Jobson 1.0.0

Nov 5, 2018

After many weekends and evenings of fixing little bugs, cleaning up the codebase, and polishing the build, I’ve finally managed to publish v1.0.0 of jobson.

I open-sourced jobson late November 2017. The version I demoed here was already close to release-grade in terms of implementation (the server had >200 tests, was used in prod, etc.). However, the deployment, installation, documentation, and maintenance needed work.

Side Project: Rust: fo2dat

Jun 26, 2018

tl;dr: I used Rust to make this CLI utility for extracting Fallout 1+2 DAT files.

I love the occasional playthrough of Fallout 1 and 2. They were some of the the first “serious” games I played. Sure, DOOM/Quake/Command & Conquer were also “mature”—I played them around the age of ~10, which marked me as doomed (heh) by the kind of adults that would also think eating sweets is a surefire path to heroin addition or something—but F1+2 included prostitutes, drugs, slavery, and infanticide: irresistibly entertaining topics for a teenager.

Integrating Software

Apr 26, 2018

tl;dr: If you find you’re spending a lot of time integrating various pieces of software across multiple computers and are currently using a mixture of scripts, build systems, and manual methods to do that, look into configuration managers. They’re easy to pick up and automate the most common tasks. I’m using ansible, because it’s standard, simple, and written in python.

Research software typically requires integrating clusters, high-performance numerical libraries, 30-year-old Fortran applications by geniuses, and 30-minute-old python scripts written by PhD students.

(Not so) Fancy-Pants new Website

Mar 10, 2018

So, I just spent an evening + morning refactoring the site into a, uh, “new” design.

I only ocassionally work on this site these days—I now see it as the sparse journal of a madman that also likes to distribute mobile-friendly versions of his CV—but I thought it would be a nice and easy blog post to reflect on how the site has changed over the last 3 years.

The original version of adamkewley.com was launched in May 2015 and was the fanciest design:

State Machines in ReactJS

Jan 6, 2018

I’m currently implementing job resubmission in Jobson UI and found that state machines greatly simplify the code needed to render a user workflow.

Cheeky Hackers

Dec 15, 2017

I’ve been running several webservers behind custom domains for a while now (plateyplatey.com, textadventurer.tk, and this site) and it never ceases to amaze me how cheeky bots are getting.

For example, certbot recently started complaining that textadventurer.tk’s TLS certificate is about to expire. That shouldn’t happen because there’s a nightly cronjob for certbot renew.

On SSHing into the server I found an immediate problem: the disk was full. Why? Because some bot, listed as from contabotserver.net decided to spam the server with >10 million requests one afternoon and fill the HTTP logs. Great. Looks like I’m finally going to implement some log compression+rotation.

Then there’s the almost hourly attempts to find a PHPMyAdmin panel on my sites. That one always surprised me: surely only a small percentage of PHP sites are misconfigured that badly? Lets look at the stats:

Percentage of websites using PHP

Even if 1 % of them are misconfigured, we’re doomed.

Jobson: Now in 2D

Nov 25, 2017

I recently made screencasts that explain Jobson in more detail. The first explains what Jobson is and how to install it. Overall, Jobson seems well-recieved. The first video seems to be leveling off at around 2700 views and Jobson’s github repo has seen a spike in attention.

Will other teams start adopting it or not? Only time will tell.

Jobson: Webify CLI Applications

Oct 15, 2017

This is a post about Jobson, an application I developed and recently got permission to open-source along with its UI.

Do any of these problems sound familiar to you?:

I’d like my application to have a UI.

I want to trace my application’s usage.

I want to share my application.

They’re very common problems to have, and are almost always solved by building a UI in a framework (e.g. Qt, WPF), sprinkling on usage metrics, and packaging everything into a fat executable. In this post, I’d like to present an alternative.

Text Adventurer

Jul 30, 2017

One of the first things people learn when they start programming is how to write a text prompt. It’s a decades-old exercise that teaches new programmers input-output.

In order to inject a little excitement, learners are normally encouraged to write interactive games using standard IO. This helps them learn programming by interactively - they will need to learn conditional logic to handle the “Will you stab the monster with your sword or run away?” prompt.

This creative learning process is great but, unfortunately, it’s hard to show the creations to other people. Any players will have to undergo the nuisance of installing an interpreter, libraries, and the game in order to play. These deployment problems are alleviated on the web: javascript can be distributed and executed remotely in a browser with no effort required from the client.

Keep Side Projects Small

Jun 26, 2017

I’ve officially thrown a deployment of PlateyPlatey (PP) up onto plateyplatey.com (galley) along with its server and frontend source code. I previously explained PP in a blog post.

PP is missing many features I would of liked to have. However, a few weeks ago I took a long look in the mirror and decided to just tidy it then throw it up on the net. This let me start sharing it on sites such as reddit to gauge if anyone’s interested. After about a week of PP being up, I concluded that no, no one is interested and yes, I spent way too much time coming to that conclusion.

Low Level Programming

May 21, 2017

I haven’t posted in a while because I have been busy with a few random studies and projects.

Low level programming has always, to a high-level programmer like myself, felt like a mysterious playground where all the real hackers write graphics engines, password crackers, and operating systems. I have always had a fascination with people who can write something hardcore like an emulator. Sure, I can open a window, draw some graphics, and update the UI to reflect state changes. However, an application that does all that and some low-level wizardry is in an entirely different league of complexity.

Engineering Flexibility

Mar 14, 2017

It’s tempting to believe that having a flexible file format for a system’s configuration is enough to ensure forward-compatibility and flexibility as the system grows. The thought process is that with a suitable data format—for example, XML or JSON—extra information can be added without breaking existing systems. For adding new fields into the data, this is true. However, if you don’t engineer additional flexibility into the fields themselves then you may end up needing to make breaking changes anyway or, worse, end up with a much more complicated data file.

Imagine you are implementing an ecommerce server that exposes products. The spec for the server stipulates that there will only be a very small number of products. It also stipulates that rebooting the server to refresh the system with new products is fine. With that in mind, you add a products field to the server’s configuration:

port: 8080
products:
 - name: Acme Explosive
 - name: Fake Backdrop

The Cycle of Platform Maturity

Mar 4, 2017

In programming, “mature” is a funny word. When I first started programming, it had negative connotations. A “mature” programming language was boring, had complicated tools, over-complicated design patterns, and would likely die off soon. By contrast, non-mature languages were exciting, free-form, and carving out a new frontier in software.

My opinion has evolved since working as a programmer rather than studying as one. Mature now implies that a language will have have good debuggers and profilers. It will have an established build and testing systems for the language. Its core libraries won’t change midway through a critical project. Most importantly, though, mature languages (tend to) have moved past the same boring old problems that every language ecosystem encounters in its early days.

Just Make Stuff

Dec 17, 2016

Software development can be a bottomless well of languages, build environments, platforms, coding standards, and cat analysis algorithms. For someone who likes to study and learn new techniques, this is great. A bottomless well of stuff to better at. However, after almost ten years of doing just that, I’m willing to make a bold statement: curiosity can be maddening and knowledge can be hindrance.

Curiosity is maddening because the more I learn about assembler, linux, garbage collection, compiler theory, engineering techniques, software principles, functional programming, networking, etc. The more I’m convinced that I know almost nothing about computers.

PlateyPlatey

Oct 18, 2016

This blog post is about PlateyPlatey, a webapp I am developing to solve the (quite specific) problem I outline in this blog post. PlateyPlatey is in the very early prototype stages of development. I am having fun seeing how far I can develop it. If it saves even one researcher from the complete ballache of reorganizing research data, I will be ecstatic.

Once upon a time, I was a lab researcher. Boy, was it fun. This one time, I spent a month trying to crystallize a particularly annoying-to-crystallize molecule only to find that a decent proportion of the batch I was working with was actually silicone grease which, consequently, made crystallization impossible. Another time, my wrist twitched midway through adding LiAlH powder to a reaction and I managed to fill an entire schlenk line and vacuum pump with a mixture of corpse-scented phosphine and hazardous LiAlH powder. I now can’t smell phosphines without vomiting - fun.

Going Full Circle

Oct 3, 2016

Write code to control your application. Configure the code’s behavior using standard configuration files. After that, go full circle: write code to control the code that runs your application. In this blog post I’ll explain when you might want do that.

Free Online Courses

Sep 3, 2016

With the rise of free online courses, it’s becoming much easier to learn programming. Over time, more people will learn programming through those courses. Overall, this is a good thing. It means I’ll be able to buy a singing fridge sooner. However, beware of the dragons.

Big IT companies are suspiciously keen to provide free online software courses. Take bigdatauniversity.com for example. It’s a very slick site with lots of content. However, that content is tainted by the site’s ulterior motive: it doesn’t just want you to learn big data, it wants you to learn how IBM is the company to use if you’re working with big data.

So, instead of explaining abstract big data concepts, some of the material descends into the corporate agenda: how big data makes corporations’ wheels turn 15 % faster, improves customer turnover by a factor of 3, and how IBM could’ve gotten you all of that yesterday - for a small fee.

Projects That Break You

Aug 24, 2016

Everyone has had one: a project that’s so hard that it breaks you.

I was recently flicking through my old project folders and found that very project. It was an innocuous form-generating WPF application (source) that I thought I had thoroughly buried in my projects-dropped/ folder. However, while browsing through that folder tonight, the project managed to catch my eye.

You can see more details about the application on the Github page. It was an application that was trying to solve every problem that COSHH-form-generating software just needed to solve. It needed composable document models. It needed a variety of IO methods. It needed to be functionally pure. It needed to be architecturally easy to understand. It also needed to be fast, run on any PC, and sell nationwide on day one.

Don't Spare the Low-Level Details

Aug 6, 2016

Abstractions are wonderful things, that is until they leak. At that point, I tend to wish someone didn’t spare me the low-level details.

Recently, I was tasked with developing a system that continually logs temperature readings from 12 hotplates. The plates’ use an RS232 communication interface, which is very easy to negotiate.

With only those high-level details available, I declared that the logging software would be an “afternoon in n’ out job” and created an appropriate design for the timescale / effort:

  • Hard-code the 12 plates’ COM/ttyS port identifiers and output paths
  • Loop through each port/path pair
  • Send the GET_TEMPERATURE request
  • Write the response to the output file

Job done.

Port identifiers can change.

Wait, who are you?