Introduction

Its been a few weeks since the GSoC coding period started. Seeing all the other contributors create similar blog posts definitely pressured me to start this, but let’s get to it.

I also considered making yet another “HOW I CRACKED GSOC 3 LAKH STIPEND” post but there’s nothing I need to say thats unique and isn’t shown in my public proposal already. You can see its PDF here.

I’ll assume the reader has a few prerequisites such as basic reverse engineering and malware knowledge, Git workflows, and the structure of executables across platforms such as PE executables, ELF and MachO.

A few basics:

  • The org: I’m contributing to the FLARE organization. They’re the reverse engineering arm of Mandiant - the cybersecurity division of Google Cloud.

  • The tool: I’m working on their tool FLOSS. Think of it as a fancier strings.exe for the first stage of malware analysis. Normally strings has a simple algorithm - find all the readable ASCII inside a file. But malware authors employ many tricks to avoid detection of these strings that might reveal the behaviour of their code, such as

    • Encrypting the strings through techniques as simple as XORing them using a key, to complex algorithms such as AES-256.
    • Constructing strings dynamically on the stack through pushing each character on at a time, so a string may look like a garbled mess to static analyzers.

  • The project: Now having all the strings is great, but what do we do with them? The output for FLOSS is extremely bare-bones and doesn’t give any relevant metadata about the string such as its location, associated library, type, etc.

    This is where QUANTUMSTRAND (QS from now on) comes in: its a feature branch with tons of metadata about the strings that is of great help to malware analysts. My GSoC project is to integrate the best of these features from QUANTUMSTRAND into FLOSS.

Example FLOSS output
Example FLOSS output

Example QS output
Example QS output

The first few weeks of this along with community bonding were for doing a few things:

  1. Fixing existing features in QS
  2. Adding new features to QS that would be useful to have before merging
  3. Adding an automated build step in the CI that keeps string databases updated (more on this later)

Also for future reference, my mentor is Moritz Raabe (@mr-tz), the main maintainer of FLOSS, a FLARE engineer and the GSoC org admin.

Community Bonding

This phase started with me fixing an issue I was asked on a while ago, the dealing with CI hygiene.

The GitHub Actions workflows had no SHA pinning, which is a supply chain risk (as if we’ve not had enough of those already). If a workflow references uses: some-action@v2 without a commit hash, a compromised tag could silently swap out the action. I added a pinact workflow to verify all action pins going forward.

The other community bonding item was adding an AGENTS.md. We had seen a few AI-generated contributions come through the repo that were clearly written without context, with people letting agents loose on the codebase and submitting whatever they produced. The idea behind AGENTS.md is to give AI tools the necessary prerequisite knowledge about the repo before they start generating code, so they don’t just produce slop. My first attempt was far too detailed; after discussion with Moritz and Willi, we simplified it down to a few focused bullets and merged that instead.

The ELF parser

The first real deliverable was ELF support in QuantumStrand. QS already knew how to parse PE and Mach-O binaries structurally by walking their section tables, deduplicating overlapping ranges, tagging relocations. But ELF was missing entirely.

I started with a draft and discussed scope with Moritz:

me: for ELF what features do you think we should add?

Moritz: generic section parsing and rudimentary code detection are a good start. I’d like to then focus on Windows PE files with the signatures, databases, etc.

So the priorities were set. Just the structural parsing right first, don’t over-engineer it. The prototype (#1312) covers:

  • Parsing ELF section headers into a Layout tree like QS does
  • Skipping SHT_NOBITS sections (.bss etc.) that have no actual content. For .bss, this happens because its where the uninitialized variables are stored in an ELF, so no strings will be there.
  • Deduplicating overlapping section ranges before scanning
  • Tagging relocation sections so QS doesn’t misclassify relocation data as strings
  • Detecting executable sections (via the SHF_EXECINSTR flag) for code range exclusion

This went through a few review cycles.

Gemini’s code review bot flagged a few things: a bug in the deduplication logic where cursor was advancing even when a section was skipped, and inefficient per-byte relocation range construction.

Moritz caught more: missing type annotations, a misleading docstring in Slice.contains_range, too-broad except Exception catching, and slice overriding the Python builtin in function signatures.

One discussion I want to document because it came up and got resolved cleanly: Moritz asked whether we should reuse capa’s ELF extractor (capa/features/extractors/elf.py) rather than calling pyelftools directly. My reasoning against it:

  1. QS is doing much simpler parsing than capa’s extractor, so we only need section headers, not the full feature extraction.
  2. We’ve already taken on machofile and pefile as direct dependencies so using pyelftools directly follows that pattern and keeps us from introducing a capa-shaped coupling.

To unblock the ELF parser tests I also raised two PRs in the testfiles submodule: #23 to fix black formatting (the pre-commit hooks in flare-floss were picking up the unformatted submodule scripts and failing) and #24 to add three ELF test binaries:

  1. An x86-64 PIE ls
  2. An ARM aarch64 shared object
  3. One from the capa test set

To have complete edge case testing over the possible features that pyelftools may have extracted.

In this period I also migrated the QS string databases to Git LFS. The .gz database files were committed as plain binaries, which adds ~24 MB to every shallow clone. LFS was the right move here especially since the automated rebuild pipeline would be adding more of these.

Also a lot of the initial prototype work on the ELF parsing goes to @FredCoast. We had to upgrade to a newer version of the lancelot disassembly library, and his fork with the ELF prototype was of great help to us.

OSS string databases and a lancelot bug

With ELF in and the LFS migration done, the next few weeks focused on the automated database construction pipeline. A string database is essentially a collection of strings known to come from a particular library, which would be of great help being shown in the QS output.

But these libraries change occasionally with new strings and need to be freshly created to be updated and relevant. My work in this phase is towards automating this.

The pipeline (PR #5 in my fork) does the following:

  1. Uses vcpkg to build each library against a target system configuration (for now its x64-windows-static/msvc143).
  2. Runs jh (the string extractor from lancelot) to pull strings from the resulting .lib files
  3. Converts output to QS’s jsonl format, deduplicates, and merges into the existing database
  4. Stores build metrics (entry_counts, timing, version) in a build_metrics.json file alongside the databases
  5. Opens a PR automatically (the mechanism that produced PR #6 in the fork)

On the library selection front, we’re still working through it. I ran a prevalence scan across the capa testfiles corpus and ranked by hit count. The usual suspects came up like zlib, openssl, brotli.

Moritz told me to pull random new files from VT and found the capa testfile results were highly skewed, which is not something I expected.

The FLARE team weighed in with their real-world experience and told us to target these:

Moritz: libev, libevent, libhv

Moritz: another one: boost and poco

Boost turned out to be a metapackage that chews significant build time, so I’m looking at whether it’s actually showing up in samples before committing to it. The others are being added to the matrix and this is WIP for now.

The bug is upstream!

While building the databases, I hit a crash in jh on two libraries: cryptopp and jsoncpp. Both are C++ libraries that use weak external symbols. These are essentially labels to functions/variables from an object file that can be found as duplicate and valid, hence the “weak” nomenclature. Lancelot’s COFF relocation resolver had no handler for symbols with an Unknown section, so it fell through to an unimplemented!() branch and panicked.

I filed issue #235 against lancelot with a full reproduction case (vcpkg build, exact jh invocation, VirusTotal hash of the triggering binary), then opened PR #237 with the fix — treating these symbols as weak externals rather than crashing. Willi merged it same day.

flowchart TD
    A[Build static library<br>cryptopp.lib] --> B[Run jh string extractor]
    B --> C{COFF relocation<br>resolver}
    C -->|Broken<br>Text symbol<br>section: Unknown| D[unimplemented! panic]
    C -->|Fixed<br>Text/Data + Unknown<br>treated as weak external| F[Resolve via externs table]
    F --> G[Continue extraction]

    classDef start fill:#3a3a3a,stroke:#555,color:#e0e0e0
    classDef decision fill:#4a4030,stroke:#6d5d48,color:#e0e0e0
    classDef error fill:#4a2e2e,stroke:#7a4a4a,color:#e0e0e0
    classDef fix fill:#2e4a2e,stroke:#4a7a4a,color:#e0e0e0
    classDef ok fill:#2e3a4a,stroke:#4a5d7a,color:#e0e0e0

    class A,B start
    class C decision
    class D error
    class F fix
    class G ok

Side note: I’d been patching jh in CI as a temporary workaround to get jsonl output directly instead of CSV (which had been causing downstream parsing issues). Moritz flagged it as too hacky, so a proper PR to add jsonl export to lancelot itself is in the works too.

There was also a useful conversation about the longer-term role of jh in this pipeline. Currently it only finds statically referenced strings. No unreferenced strings, no UTF-16, no dynamically constructed ones. I floated the idea of using FLOSS itself for string extraction from these libraries (COFF support would be a TODO), since it covers more surface area.

Moritz: yeah the less we can depend on lancelot the better, it’s not maintained

That’s not an immediate change but it’s the direction. I definitely want all the string extraction code to be native FLOSS without any dependencies on external libraries, and its something I’lll look into post-GSoC.

Moritz had also been doing his own benchmarking in parallel, running the QS pipeline against a dataset of samples and looking at library match rates. Some databases had very few strings with a lot of garbled data (brotli was the example).

Another thing I noticed looking at the raw output: function names were showing up alongside literal strings, which makes sense since unstripped binaries export symbol names. So along with extracted strings, function names are also first-class elements in the string database now.

I also threw up a small Vercel site to visualize the string DB output and browse what the databases actually contain without grepping through jsonl files.

What’s next?

Most likely finishing the final list of libraries to create databases from, and start the integration of FLOSS with QS (y’know, the main part of the project!)

I think we’ll be able to get this done before the mid-term evaluation in the second week of July. After that, we have some non-glamorous frontend work for the GUI that needs to be taken care of.

Planning for another blog post some time after this one before the end-term eval. See ya!