In 2016, DEF CON CTF will use the same game format as the DARPA Cyber Grand Challenge (CGC). It will not be the same attack-defense format it has been in previous years. The challenges will all be DECREE x86 binaries, instead of a wide sampling of challenge formats. Grab a beverage and some index cards, it's complicated.
What is the Cyber Grand Challenge game format?
The CGC game format is notionally similar to attack-defense. Teams receive vulnerable software, field patched versions, and launch exploits against other teams. The big differences are what aspects of the game teams control, and the role the scoring system plays in running the game.
We Hope You Like APIs
Because DEF CON CTF 2016 will feature the winning autonomous Cyber Reasoning System (CRS) from the Cyber Grand Challenge, our scoring system will provide the same APIs as CGC to all teams.
There will be a human-friendly interface that supports every feature of the CRS APIs. The CGC API is available today, and you can get a leg up on competitors by building tooling around it in advance. You did build baby’s first CRS for the thousand cuts quals challenges, right?
Virtual Competition is a CGC game API simulator that can be run locally. The virtual-competition source is also available.
Teams Don’t Control a Server
Attack-defense teams are usually given a privileged account (although maybe not root) on servers, which allows them to replace binaries, kill errant processes, and other activities.
In CGC-format games, teams don’t have privileged access to the servers that run their software; instead, they upload software as a DECREE Replacement Challenge Binary (RCB) directly to the scoring system. Finally, teams don’t launch their own exploits; instead, teams build exploits using one of the DECREE Proof-of-Vulnerability (PoV) formats, and upload them to the scoring system.
Understanding Proofs of Vulnerability is a high-level overview of how PoVs work. The sample challenge sets NRFIN_00073, CROMU_00070, and CROMU_00071 have working examples of PoVs that can be tested locally.
The virtual-competition system can accept and validate uploads of RCBs, firewall rules, and PoVs, but only stores them, and doesn't run them.
Teams Get a Programmable Firewall
Unlike previous Legitimate Business Syndicate games, defending teams have the option of writing firewall rules for the CGC network appliance. Just like RCBs and PoVs, these are uploaded to the scoring system.
Additionally, the firewall provides dumps of both poller and PoV traffic to teams over UDP.
Using the Network Appliance from the CGC documentation is a good starting point for how to make firewall rules. The network appliance implementation provided by DARPA is open-source. cb-packet-log is a tool for receiving packet dumps from the firewall.
Patched Challenges and Firewall Rules Are Shared
There’s very limited security-by-obscurity in commercial, industrial, and open-source software. Patches are analyzed to understand the vulnerability they fix, new software can be tested in lab conditions, third-party patches for major vulnerabilities aren’t unheard-of, and there’s little to be done about it.
Replacement challenge binaries and firewall rules will be shared among teams as they’re fielded. This means you can figure out how PPP’s patching progressed, or just field their binary yourself #yolo
How Do We Play It?
Just like DEF CON CTF years past, the game is divided into rounds. During each round, the functionality of teams’ challenges is evaluated, exploits fire, teams upload new stuff, and scores change. There is a bit of lag, and that’s where things get interesting.
Game Start
At game start, teams will be able to grab a list of currently-enabled challenge sets, and download the challenge binaries for them. As the game progresses, more challenge sets will be come available, and teams will download fresh binaries for them.
Analysis
The challenge binaries are DECREE executables. Analyze them how you wish.
In addition to the CBs, traffic tested against binaries fielded by your team is sent to you. This includes both poller and exploit traffic.
- Debugging a Challenge Binary in the CGC documentation
- 2000 cuts with Binary Ninja by Ryan Stortz of Trail of Bits
‘Sploits
In round 16, team “ShadyTel” has got an exploit that they’d like to field against “Milliways.” They create a C-based PoV for the exploit, and upload it the the scoring system. It’s an unreliable exploit, so they want to run it ten times per round.
curl --digest -u shadytel:loud \-F throws=10 \-F csid=LEGIT_00006 \-F file=@pwn_milliways \-F team=2 \https://ti.defcon.legitbs.net/pov
Starting in round 17, the scoring system runs the ShadyTel PoV against the Milliways challenge set ten times, mixed in with regular poller traffic. If it negotiates and successfully proves a register control (Type 1) or private memory disclosure (Type 2) vulnerability, ShadyTel gets offense points and Milliways loses them.
Patching
Milliways sees the ShadyTel PoV traffic (it ran ten times a round, super noisy), confirm that they’re vulnerable to it, and produce a patched Replacement Challenge Binary. They upload it in round 20.
curl --digest -u milliways:dome \-F csid=LEGIT_00006 \-F LEGIT_00006=@patched-6 \https://ti.defcon.legitbs.net/rcb
In round 21, Milliways fails all their pollers by design. Patching has a cost. Any PoVs against it automatically fail too. Milliways' new RCB also becomes available to other teams. ShadyTel downloads it. So does team “Psychoholics.”
curl -f --digest -u shadytel:loud \https://ti.defcon.legitbs.net/round/21/evaluation/cb/2
In round 22, Milliways’ new RCB passes pollers.
Scoring
During their analysis, the Psychoholics notice that their in-progress replacement has already fixed the vulnerability Milliways patched, but that Milliways also tuned up some code to require less space, use less memory, and run faster. They gank it for their patch, since in addition to being graded on security and availability, teams are graded on efficiency as well.
curl -f --digest -u psychoholics:pdp11 \https://ti.defcon.legitbs.net/status
But You Don’t Have To Take My Word For It
- Cyber Grand Challenge documentation republished by Legitimate Business Syndicate
- Cyber Grand Challenge Frequently Asked Questions as provided by DARPA
CGC is an unprecedented investment in the future of CTF, and while it feels extremely academic today, we're confident that CGC-derived technologies are going to be a fixture in CTF from now on. We’re extremely excited to see what the future of CTF will be like, and we think you will be too.