Doodle Grid

Hello! I had a lot of fun at my first DEF CON in several years with free time and no responsibilities. Turns out running a CTF is hard work, and while it's also a wonderful experience, I'm also happy that our friends at Order of the Overflow get that experience now :)

I've enjoyed many of the visual art on display at DEF CON and other events around the world for over a decade now. In particular, interactive ones fascinate me, and ones at hacker events that have some kind of network connectivity seem especially fitting. At SHA-2017, there was a massive LED grid behind a well-staffed and well-stocked bar, all controllable over TCP to a public IP. This worked well, with lots of little visuals of animated characters slipping and sliding around the screen, until my friend Shadghost ransomware'd it with rented servers, crashing the machine hosting it, and requiring the staff to reboot it.

LED displays showing images and a cryptolocker ransom screen behind a bar

So I started thinking about how I'd do it better. Do I want IPv6, where addresses have enough bits to put the whole message in a destination address for a near-empty packet? Do I want to use a projector so the setup isn't a huge wall? Raspberry Pi? I dithered on this for most of a year, until piesocks and I started a quick project for Toorcamp, a portable battery-powewred Doom setup. Piesocks's portable WiFi AP used a little WiFi SoC, which basically made the whole thing super-easy. Also at the event were a bunch of microcontroller-powered LED displays, both in semi-official installs and also just taped together in varaious campsites.

fifteen LED grids taped into a rectangle, held up by someone wearing pyjama pants, in front of a cluttered table, resting on grass

On the journey home from Toorcamp, I started narrowing down to something I could actually build in the month before DEF CON. Without any contests I had to be at, I could just roam around with a backpack, loaded with drinking water and other important supplies, with the LED grid stuck on the back.

My first order was: an ESP-8266 prototype board, a 16x16 LED grid, and a small OLED to show the device status. Once it arrived, I used a breadboard I had lying around to wire it all together, and, over the course of a few evenings, started playing with some small ESP-8266 programs. One to connect to WiFi, one to control the LED matrix, and finally one that would accept UDP and change LEDs.

an LED grid showing an octothorpe logo connected to a breadboard

The third program didn't work: the LED grid required a ton of CPU time for a 256x256 grid, enough that it couldn't do Wifi stuff at the same time. I tried a few different libraries, but the most promising thing was to switch to an ESP-32 microcontroller; that one's got two cores and an additional peripheral that'd handle the LED grid on its own. In addition, while the LED grid appeared to work fine on 3.3V for both signalling and power, I decided to run it off 5V, which required a level shifter. A third problem was that ramping up the LED matrix to full brightness would cause a voltage drop that would brown out the microcontroller. I watched a few videos and talked to a few people and learned about bypass capacitors, and that I should run with 'em.

I ordered an ESP-32, a handful of level shifters, and a big ol' box of ceramic capacitors. They showed up, and I set down to rewire things. The ESP-32 had almost the same pinout as the ESP-8266, the level shifter provided a convenient place to wire in all the power pins for everything, and to jam in a capacitor. I also had to rewrite some of the software to use a special LED library for ESP-32.

an ESP32 prototype board and a small red level-shifter board on a breadboard, with jumper wires running all over the place

Changing parts and updating software to match got almost everything working. The capacitor didn't really fix the brownout problem. My big capacitor box only had small ones. Since DEF CON was getting closer and I didn't feel like ordering more stuff online, I tried to rip a few out of an old Xbox 360 I had lying around. This was a pain, since Xboxes use RoHS-safe solder that doesn't want to melt at the temperature my cheap soldering iron tops out at.

an Xbox 360 main board, with a soldering iron on top of it and several capacitors next to it

Once I recovered the caps, I didn't feel like janking them into the breadboard, so I just disregarded them, right-shifted the brightness down a bunch, and also tested the setup off the battery pack I'd use at DEF CON. It worked fine.

The next task was to set up the microcontroller as a WiFi access point. This was remarkably easy: I changed the function call from connecting to an existing network to run an AP, commented out the loop that waits for the connection to be established, and that was basically it.

Once the circuits were all in order, I decided to put them all together in a box that would survive backpack life. I started with an amenity kit from a flight in a tough plastic box, and did a bit of customizing. I cut out the fabric lining, Naomi Klein'd the logo off the outside, cut a hole for the OLED status display, and drilled a hole for the cables vowed to leave the zipper open a bit for cables.

an airline amenity kit box with a poorly-cut hole cut in the front for an OLED display

Mounting the OLED was tricky. I couldn't find screws that fit the OLED, so instead I just kind of taped it in. The tape didn't work, so I cut up an old hotel key card and hot-glued it together to bolster the display. By this point, the display didn't really do anything, so honestly I should've skipped it, but whatever. It's not gonna move now, heh.

cut up pieces of plastic hot glued down, holding something covered in electrical tape down

My first attempt at mounting the LED involved using Velcro to stick it on the outside of the backpack. It worked really well!

The night before I left for DEF CON, I realized that instead of the Ruby scripts I was using on my desktop to draw to the display, I should be able to use my phone. Building an iOS client for the doodle grid used the sum total of my iOS experience (half a Mastodon client), which wasn't nearly enough, so I learned a lot more. I got a lot of experience with graphics-oriented parts of the iOS environment, including the CG* family of classes. Additionally, I wrote a page of instructions and printed out some Ruby code.

a piece of paper, with a message in calligraphy and normal print handwriting
Vito Leds
SSID: twitter @vito_lbs
Security: none lmao go nuts
DHCP: ya
192.168.4.1
UDP 27420
Message: <PixelCount:u8> [<Pixel>]
PixelCount: 8-bit unsigned integer
Pixel: [x,y,r,g,b]: all 8-bit unsigned
it's 16x16 and iirc it bounds-checks
[1,0,0,255,0,0] make some corner red

I threw the LED grid in the original box for it, put it in the backpack, and put it in my checked luggage. The big battery pack and circuitry went in my briefcase. TSA didn't even flinch (in either direction), so that's cool. During my flights to Vegas, I spent some time on the iOS app, and then once I got Vegas, I put the setup together, and spent a bit of time fixing the app.

On Thursday, I talked with ffe4 over breakfast, and he was interested in messing with the backpack.

Later that day, while in the badge/registration line, I was wearing the backpack, showing it off, and it was cool, until I got hacked. A literal baby pulled on the wires for the LED grid until they disconnected, which was hilarious and educational. I got a few pieces of gaff tape to control the bare wires, and ran with that for the rest of DEF CON. I also ran into sigtrap and counterflow, who were really hyped about the project! They both took a couple pictures of the instruction page, since none of us were toting computers.

a twitter post 'dude, your backpack's lit'

Over the weekend, I wore the backpack and had it showing my name most of the time. It barely hit the big USB battery pack I brought, and the backpack was also useful for keeping beverages and stickers handy.

CTF ends Sunday afternoon, shortly before closing ceremonies, and I went in their room to see the final countdown. Without coördination, ffe4 and sigtrap were also there, and showed off some of the progress they made in the chaos of the last few minutes of CTF.

For 2019, I'll probably work on the software a bit more (it got crashy when misused) and figure out how to put it on the DEF CON network instead of its own thing.

source code

The microcontroller source is at https://github.com/vito-lbs/doodle_grid . I used the Arduino IDE for it, so the interesting stuff is in the doodle_grid.ino file.

The iOS client is at https://github.com/vito-lbs/doodle-grid-client . The drawing mechanics are mostly in DoodleView.swift, the network stuff is in ViewController.swift, and of course there's a storyboard that's optimized for iphone x (deal with it).

greets and shout-outs

Thanks pronto and shadghost for fun times and reference backfills for SHA-2017. Huge thanks to piesocks for getting me thinking about low-power hardware for toorcamp. Thank you rager & quails for just messing around with the taped together display at toorcamp and also for the rum-mune. Thanks for the picture, crowell. And thanks for messing with my display, ffe4, sigtrap, and counterflow :)