# FLOSS Migration Development Spec

## Purpose and scope

This document is the development specification for the migration of FLOSS to a layout and tag aware tool.

It has three work streams:

1. The string display decisions, which define how results are ordered and rendered.
2. The deferred items from [**\#1347**](https://github.com/mandiant/flare-floss/pull/1347), which are now in scope.
3. The new command line interface, which replaces the current arguments, adds rendering and filtering, and adds result caching.

## 1\. String display decisions

### 1.1 Scope for now

Layout rendering and tag features apply to static strings only. Do not add this context to recovered strings.

Technical notes:

- The layout tree type `ResultLayout` and the serializable string type `ResultString` already carry tags, section, structure, and encoding fields.
- The pipeline in `floss/pipeline.py` already routes static strings through the layout path when a structured layout parses. Keep that routing. Do not extend it to emulated strings.

### 1.2 Position of recovered strings

Render order is fixed:

1. Static strings, layout aware when a layout is available
2. Language strings
3. Stack strings
4. Tight strings
5. Decoded strings

Static strings get the rich layout render. Recovered strings are shown after static and language strings, in their own sections, still in a rich tabular format. Each recovered section has a heading that names the string type, in the same style as the layout section headings.

The heading is a centered section name on its own line, with a horizontal line above and below. The heading text for each section is lowercase and matches the type name, for example "stack strings", "tight strings", and "decoded strings".

Technical notes:

- `floss/render/default.py` is the render coordinator. It already emits sections in this exact order. Keep the ordering and the rule that a section is shown whenever its mode is enabled, including a zero count.
- Reuse the existing rich table renderers for stack, tight, and decoded strings (function, offset, string columns) from `floss/render/default.py`.
- The classic metadata table (file path, language, runtime, version) is not shown on the layout path. See section 2.4 for where this metadata moves.

## 2\. Deferred items from \#1347

### 2.1 Runtime timing fields

The `Runtime` dataclass in `floss/results.py` gains two new fields for metrics: `layout` and `tags`. Both are float seconds, defaulting to zero.

Populate the fields in `floss/pipeline.py`:

- Record the elapsed time around the layout and tag step and write it to `runtime.layout`.
- Record the elapsed time of the tag database matching step and write it to `runtime.tags`.

### 2.2 Load flag ignores layout flags

The `--no-layout` flag and the `--load` flag are removed (see sections 3.1 and 3.4.7). Loading a saved results document is now automatic and is detected from the input file content.

### 2.3 Render function location

The `render_strings` function is currently in `floss/render/layout_text.py` Move `render_strings` into default. Consolidate the render modules and remove the classic view and the obsolete classic helpers.

### 2.4 Render meta not shown with layout

When the layout view renders, the classic metadata table is not shown \--- intentional.

The sample metadata moves to the summary output. The new `--summary` option (section 3.5) prints sample metadata: file path, hashes, identified language, runtime, version, imagebase, minimum length, and per-type string counts. This gives users the context that the classic table used to provide.

### 2.5 Full-buffer read for static extraction

The pipeline reads the whole sample into memory with a single `read_bytes` call in `floss/pipeline.py`, then reuses that buffer for both classic static extraction and layout extraction. Keep this.

### 2.6 Outdated DB locations

Two scripts still point at the old DB location `floss/qs/db/data`. Fix both so they use the current database location under `floss/tags/data`:

- `scripts/tags/query_string.py:27`
- `scripts/tags/build_oss_db.py:1102`

## 3\. CLI specification and design

### 3.1 Removals, simplifications, and anti-bloat rules

Flags removed:

- `--no-layout`, because layout is automatic and on by default.
- `--no-tags`, because tag display is controlled by `--columns` and filtering is controlled by the tag flags.
- `--view`, replaced by `--plain` for the classic list.
- `-l` / `--load`, replaced by automatic detection of a JSON input file.
- `-H`, folded into `--help`. `--help` now shows the full option list.
- `--no` and `--only`, replaced by the symmetrical `--string-type` and `--no-string-type` flags.
- `--functions`, renamed to `--analyze-functions`.

#### 3.1.2 Symmetrical filtering flags

- `--string-type <type...>` and `--no-string-type <type...>` select or exclude string types. Valid values: static, stack, tight, decoded, language, all.
- `--section <name...>` and `--no-section <name...>` filter or exclude by binary section name, for example `.rdata` or `.text`.
- `--structure <name...>` and `--no-structure <name...>` filter or exclude by binary structure, for example `import_table` or `pe_header`.
- `--tag <tag...>` and `--no-tag <tag...>` filter or exclude by semantic classification tag. Tag families include winapi, crypto, zlib or openssl. Optionally, meta tags like oss allow to filter on all oss strings.
- The include and exclude form of a pair are mutually exclusive. Passing both, for example `--section .text --no-section .text`, should be an argument error.
- Multi-value matching uses OR logic. Passing several values to one flag includes or excludes strings that match any of the values.
- Flags accept repeated occurrences. Repeated `--tag` flags accumulate values.
- `--interesting` is a shortcut that automatically excludes the noisy tags `#common`, `#duplicate`, `#code`, `#reloc`, and `#code-junk`.

### 3.2 Output rendering and terminal views

There are two terminal views:

- Tree: hierarchical layout tree. The default when a layout is present.
- Flat: a plain table of strings.

`--plain` switches to the classic flat list without layout or tags.

#### 3.2.1 In-tree search and filtering

- `--query <string>` filters strings by content while preserving the parent tree and structure headings. Empty branches are pruned but the headers of branches that still contain matches are kept. Accepts multiple expressions \--- the patterns are ORed.
- `--max-strings <n>` caps the emitted strings per section to the top N highest-relevance strings.

Relevance order within a section is deterministic:

1. Strings with a highlighted tag first.
2. Then untagged strings.
3. Then strings with any non-noisy tag.
4. Within each group, ascending by offset.

Take the first N strings of that ordering. Document this properly in code and to users.

`--query` is repeatable. Multiple patterns are ORed: a string is kept if it matches any pattern.

#### 3.2.2 Column display option

`--columns <col...>` controls column visibility in the flat and grouped views. It is a space separated list. Available columns: tags, offset, structure, encoding. Default: tags, offset. Example: omitting `tags` from the list disables the tag column.

### 3.3 Output formats

- `-j`, `--json`: emit the full structured ResultDocument as JSON on STDOUT, with sorted keys. This is the single machine-readable interface.
- `--summary`: emit a concise summary containing sample metadata, section counts, tag histograms, and strings matching high-value tags. High-value means any tag other than the noisy set listed in section 3.1.3. The summary is built from the same ResultDocument as the JSON output, so it is programmatic and consistent. This is meant for human consumers and agents so it should be pretty printed with tables and other nice-to-haves.
- Structured JSON errors: whenever a JSON output mode is active, fatal CLI or parsing errors emit a single JSON object on STDERR, for example `{"error": "...", "code": 1}`.

### 3.4 Complete command line interface reference

#### 3.3.1 Positional arguments

| Argument | Behavior |
| :---- | :---- |
| sample | Path to the target binary to analyze, or to a saved FLOSS results JSON document, which is detected automatically. |

#### 3.4.2 Core options

| Option | Behavior |
| :---- | :---- |
| no arguments | Print the full option list, including advanced options, exit with code 1\. |
| \--help | Print the full option list, including advanced options, exit with code 0\. |
| \--version | Print the program version and exit with code 0\. |
| \-n, \--minimum-length | Minimum string length threshold. Default: 4\. |

#### 3.4.3 Analysis and extraction options

| Option | Behavior |
| :---- | :---- |
| \--string-type \<type...\> | Select the string types to extract. Valid: static, stack, tight, decoded, language, all. Default: all. |
| \--no-string-type \<type...\> | Exclude the given string types from extraction. Conflicts with \--string-type. |

#### 3.4.4 Layout and structure options

| Option | Behavior |
| :---- | :---- |
| \--section \<name...\> | Restrict static strings to the given binary sections. |
| \--no-section \<name...\> | Exclude static strings in the given sections. |
| \--structure \<name...\> | Restrict static strings to the given binary structures. |
| \--no-structure \<name...\> | Exclude static strings in the given structures. |

Section names match the layout node names, for example `.text`. Structure names match the structure field, for example `import_table`. User to the section should be a slug, for example `--structure import-table`.

Structure names should be given in the output of the `--help` command.

#### 3.4.5 Tagging, searching, and noise filtering options

| Option | Behavior |
| :---- | :---- |
| \--query  | Filter strings by regular expression while preserving the tree and structure headers. Repeatable, ORed. |
| \--tag \<tag...\> | Emit only strings that match the given tags. |
| \--no-tag \<tag...\> | Suppress strings that match the given tags. |
| \--interesting | Shortcut that excludes the noisy tags: \#common, \#duplicate, \#code, \#reloc, \#code-junk. |
| \--max-strings  | Cap emitted strings per section to the top N by relevance. |

#### 3.4.6 Output and rendering options

| Option | Behavior |
| :---- | :---- |
| \--plain | Render the classic flat list of strings without layout and tags. |
| \--columns \<col...\> | Columns to show: tags, offset, structure, encoding. Default: tags, offset. |
| \-j, \--json | Emit the full structured JSON document. |
| \--summary | Emit the token-efficient summary (metadata, counts, tag histograms, high-value strings). |
| \--color  | ANSI color mode: auto, always, never. Default: auto. |

#### 3.4.7 Advanced and decoder control options

| Option | Behavior |
| :---- | :---- |
| \-f, \--format  | Sample file format: auto, pe, sc32, sc64. Default: auto. |
| \--language  | Language-specific string extraction: auto, go, rust, none. Default: auto. |
| \--analyze-functions \<addr...\> | Restrict emulation analysis to the given function addresses. Addresses are hexadecimal, for example 0x401000, space separated. When used, static strings are not shown. Renames the old \--functions flag. |
| \--signatures  | Path to a custom FLIRT .sig or .pat file, or a directory of them. Default: embedded signatures. |

#### 3.4.8 Logging and debugging options

| Option | Behavior |
| :---- | :---- |
| \-d, \--debug | Enable debug output on STDERR. Repeatable for more detail. |
| \-q, \--quiet | Disable all status messages on STDOUT except fatal errors. |

### 3.5 Analysis caching

Automatic caching makes repeated analyses fast. The full ResultDocument is cached on first execution and reused on later runs.

Cache directory: the platform cache directory, overridable with the `FLOSS_CACHE_DIR` environment variable. Defaults: `$XDG_CACHE_HOME/floss` on Linux, `~/Library/Caches/floss` on macOS, `%LOCALAPPDATA%\floss\Cache` on Windows.

Cache enable switch: `FLOSS_CACHE_ENABLE=0` disables caching. The default is enabled. Document this envar in `--help` along with `FLOSS_CACHE_DIR`.

Cache key: a content-addressed SHA-256 of the sample bytes, appended with the FLOSS version.

Layout: store entries under `{cache_dir}/{sha256}-{floss-version}.json`.

Stored payload: the full ResultDocument JSON, the same schema as `--json`.

First run: after a successful analysis, serialize the ResultDocument and write it atomically. Write to a temporary file in the cache directory, then rename it into place. Guard the write with a lock file so concurrent first runs do not corrupt the entry. If the lock cannot be acquired, skip caching, throw a warning and continue normally.

Subsequent runs: compute the key at startup. On a valid hit, load the cached ResultDocument, skip extraction, layout, and tagging entirely, and render from it. Surface hits through `--debug`.

Validation: on a parse failure or a checksum or version mismatch, drop the entry and re-analyze.

Post-load filtering: rendering flags such as `--query`, `--columns`, `--max-strings`, and the tag and section filters apply at render time, so they work unchanged on cached results.

A new module `floss/cache.py` provides: the cache directory resolution, the key computation, the load function, and the atomic store function.

Integration points: the pipeline computes the key and checks for a hit before analysis, and stores the document after a successful run. The load and store steps wrap the analysis flow.

Caching applies to binary sample analysis only. Loading a user-supplied JSON document does not write to the cache.