Welcome
Graduate Computer Science student specialising on robotics and machine learning. I write about projects I worked on and things I find worth sharing.
What happens when you let an LLM write robot programs?
In 2022 I was tasked with designing an interactive robot system that takes a natural language instruction and outputs a working robot trajectory. This is a write-up of the design decisions and a showcase of a web demo using MuJoCo that. The code in this demo is a rebuild of the original code "Prompt: Put all the cubes on the plate" How can you make robots listen to what a user says? Human-Robot-Interaction (HRI) has come a long way over the last couple of years. Vision-Language-Action Models are the de facto state of the art in robotics research. PhAIL is a leaderboard aimed at measuring real-world performance against humans, which makes it clear that general physical AI is not there yet. ...
Unsuccessfully training AI to play my favorite childhood game nobody ever heard of
I trained 1,500 parallel bots to play DDNet, a cooperative 2D platformer with grappling hooks, freeze mechanics, and maps that take actual players hours to complete. This is what it looks like: In this video you can see my army encroaching agents running through the initial section of a Tutorial map that teaches basic movements and adds difficulty incrementally. It’s designed for someone who’s never played before, lends itself well for reinforcement learning. ...