Chef

chef is a Go CLI for batch transformations of files and database rows, driven by Lua scripts in regex, JSON, XML or SQL mode. It runs commands from cook files (.toml or .yml/.yaml), supports glob-based file selection, and keeps snapshots so changes can be reset.

Every mode is built on one rule: chef never reads its target to decide what to do. It reads the copy it saved, edits that, and writes the result out. So a command produces the same result on its thousandth run as on its first, no matter what happened to the file or the table in between — and can always be undone.

What it does

• Applies regex-based edits using Lua on capture groups (v1, s1, named groups, etc.).
• Applies JSON edits by exposing parsed JSON as data in Lua (json = true command mode).
• Applies XML edits by exposing parsed XML nodes as data in Lua (xml = true command mode).
• Applies edits to rows in MySQL, PostgreSQL or SQLite via sql.* in Lua (sql = true command mode).
• Every command gets a ctx table with source_file, target_file, command_name, and (in regex mode) capture entries.
• Supports multi-command pipelines, command filtering, per-command log levels, isolate passes, and reset behavior.
• Saves original file snapshots in data.sqlite, and original database rows in COOK_ shadow tables on the target database, so you can restore either later.

Installation

go build -o chef .

Quick start

1. Generate an example config:

chef --example

This writes example_cook.toml in the current directory.

2. Run it:

chef example_cook.toml

3. Run only selected commands:

chef -f "UpdateAmounts,Prefix" example_cook.toml

4. Undo everything — restore files from their snapshots and database rows from their shadow tables:

chef reset

5. Forget everything — delete the snapshots and drop the shadow tables. Irreversible; there is nothing left to reset from afterwards:

chef dump

6. Open the interactive TUI:

chef tui example_cook.toml

CLI flags

• -P, --parallel number of files to process concurrently (default 100).
• -f, --filter comma-separated command name filters.
• -c, --conv convert YAML cook files to TOML (skips output if target .toml already exists).
• -e, --example generate example_cook.toml and exit.
• -m, --meta generate meta.lua (LuaLS autocomplete helper) and exit.
• -l, --loglevel global log level (ERROR, WARNING, INFO, DEBUG, TRACE).

TUI mode

Launch with:

chef tui <cook-files...>

Behavior:

• Displays files as top-level rows and commands indented under each file.
• Toggle a command row to select that command for next run.
• Toggle a file row to select/deselect all commands under that file.
• Shows per-command status lines after runs:
  • green (modified, no errors)
  • yellow (no modifications, no errors)
  • red (errors)
• Watched commands are marked with a blue watch indicator.

Hotkeys:

• j / k or arrow keys: move cursor
• Space: toggle focused row
• a: select all commands
• c: clear all selections
• s: run selected
• S: run all
• w: toggle watch on focused command
• W: toggle watch for all commands under focused file
• e: escape clipboard text (regex quote)
• E: escape clipboard text with whitespace minimization
• q: quit

Watch mode details:

• Watch source: cook/config files only.
• Trigger engine: fsnotify only.
• Debounce: 500ms.
• On cook file change, TUI reloads commands and reruns only watched commands whose definitions changed.

Cook file format

chef executes commands from cook files. Prefer TOML (shown below), YAML is also supported.

[[commands]]
name = "MultiplyValues"
regex = "value = !num"
lua = "v1 = v1 * 1.5"
files = ["data/**/*.txt"]
loglevel = "INFO"

[[commands]]
name = "JSONUpdate"
json = true
lua = "data.version = '2.0.0'"
files = ["config/**/*.json"]

[[commands]]
name = "XMLUpdate"
xml = true
lua = "data.attr.version = '2'"
files = ["config/**/*.xml"]

Command fields

• name: display/log/filter name.
• regex: single regex pattern (regex mode).
• regexes: multiple regex patterns for one command.
• regex_pred: optional prefilter regex; command runs only when file content matches.
• regex_preds: optional list of prefilter regex patterns (OR semantics).
• lua: Lua snippet or external file reference via @path/to/script.lua.
• files: glob patterns to match target files.
• json: command-level JSON mode.
• xml: command-level XML mode.
• sql: command-level SQL mode. Needs dsn; needs no files.
• dsn: database connection URL for sql = true. Its scheme names the engine — mysql://, postgres://, sqlite://. Cascades from file level.
• noreset: skip reset for target files before this run (false by default, so files are reset unless true).
• git: reset target files with git checkout -- <file> instead of from the stored snapshot. Has no effect with noreset = true.
• isolate: run command in an isolated pass before regular commands.
• nodedup: disable overlap deduplication for capture groups.
• disable: skip command.
• raw: when used alone (no regex/json/xml), passes entire file content as the data string in Lua — equivalent to regex = '(?s).*' without regex. Has no effect when combined with json/xml (values are already plain).
• loglevel: per-command log level override.

Processing modes

Regex mode (default)

• Use regex or regexes in a command.
• Pattern helpers:
  • !num -> numeric capture pattern ((-?\d*\.?\d+))
  • !any -> non-greedy wildcard (.*?)
  • !rep(pattern, n) -> repeats pattern n times
• Unnamed capture groups are available as s1..sN (string) and v1..vN (number when parseable).
• Named capture groups are available as Lua variables by name.
• Set replacement in Lua to replace the full match; otherwise updates are applied to changed captures.

JSON mode

• Enable per-command with json = true.
• Parsed JSON is exposed as data in Lua.
• Values are plain Lua values: numbers, strings, booleans, and null. Use N(x) (an alias for tonumber) to convert a string to a number.
• JSONPath helper is available: json.jpath(node, expr) (use data for root).
• JSONPath remover is available: json.jpathrm(node, expr) (returns removed count).
• Changes are applied surgically where possible to preserve unrelated formatting.

XML mode

• Enable per-command with xml = true.
• Parsed XML is exposed as data in Lua.
• XML nodes expose:
  • data.tag: element name
  • data.text: plain string text
  • data.attr: attribute table of plain string values
  • data.children: child node array
  • data.name.local: local name
  • data.name.uri: namespace URI
  • data.name.prefix: namespace prefix
• XPath helper is available: xml.xpath(node, expr) (use data for root).
• XPath remover is available: xml.xpathrm(node, expr) (returns removed count).
• Add/remove child nodes by mutating data.children with normal Lua table operations.
• Existing XML is edited surgically where possible; newly added nodes are serialized from the Lua node table.

SQL mode

• Enable per-command with sql = true.
• The command edits rows in an external database (MySQL, PostgreSQL or SQLite) instead of files, with the same "run it as often as you like, get the same result, and be able to put it back" guarantee the file modes have.
• A sql = true command needs no files. Instead it needs a dsn, either at file level (cascading to every SQL command that omits its own, exactly like files) or on the command. There is no separate driver field: the DSN's scheme already says which engine it is.

dsn = 'mysql://chef:chef@localhost:3306/acore_world'

[[commands]]
name = 'Bigger stacks'
sql = true
lua = '''
local rows = sql.select("creature_loot_template",
  "Reference = 0 AND Item IN (SELECT entry FROM item_template WHERE stackable > 1)",
  { "MaxCount" })
for _, row in ipairs(rows) do
  row.MaxCount = math.min(row.MaxCount * 50, 255)
end
sql.update("creature_loot_template", rows)
'''

# Per-command override, same mechanism as files = [...]
[[commands]]
name = 'Purge legacy region on staging'
sql = true
dsn = 'mysql://chef:chef@localhost:3306/acore_world_staging'
lua = 'sql.delete("creature_loot_template", "Reference = 999")'

The dsn is the whole connection, and its scheme is what picks the engine:

scheme	DSN
mysql://	mysql://user:pass@host:3306/dbname?parseTime=true
postgres:// (or postgresql://)	postgres://chef:chef@localhost:5432/game?sslmode=disable
sqlite:// (or sqlite3://, file://)	sqlite://./game.sqlite, or sqlite://:memory:

Everything after the scheme is passed to the engine's own driver, so query parameters like ?sslmode=disable or ?parseTime=true work as they always did.

How it stays repeatable

The file modes never read the file on disk to decide what to do — they read chef's stored snapshot, edit that, and write the result out. SQL mode does the same thing with rows:

1. sql.select copies the matching rows into a COOK_<table> shadow table on the target database, once, and then reads them back out of the shadow.
2. Lua edits those row objects.
3. sql.update writes them back as absolute values.

Because step 1 always hands Lua the originals, a relative edit like row.amount = row.amount * 2 gives exactly 2x on the first run and on the thousandth — and it converges even if somebody edited the table in between. This is why sql.update takes rows rather than a set/where pair: a relative SQL expression could never be repeated safely.

An original is written to the shadow once and never overwritten until you chef dump.

Requirements and limits

• Every tracked table needs a primary key. It is how a row is identified in order to undo what was done to it. Composite keys are fine. A table without one fails the command rather than falling back to something fragile like a row hash.
• where clauses are raw SQL, passed to the engine untouched — subqueries included. chef does not parse or escape them, on the same trust model as re() running arbitrary Go regexes.
• Every row must carry its primary key when handed to sql.update or sql.insert.
• sql.delete refuses an empty where rather than emptying a table.

Reset and dump

• chef reset replays the originals out of every COOK_ shadow table: updated rows are set back, deleted rows are inserted back, and rows chef inserted are removed. It leaves the shadow rows in place, so a later re-run still lands on the same result.
• chef dump drops the COOK_ tables outright. That is irreversible — after a dump there is nothing left to reset from.

chef remembers which shadow tables exist, and on which connection, in its own data.sqlite, so reset and dump work in a later run with no cook file in hand.

Lua runtime reference

Variables available in Lua

• @script.lua in lua loads an external script relative to the cook file directory.
• file: current file path being processed.
• modified: boolean gate for applying changes.
  • Defaults to false before script execution.
  • Set to true when your script returns nil or truthy.
  • Set modified = false (or return false) to skip writing changes.
• ctx: table with contextual metadata about the current command execution:
  • ctx.source_file: path to the cook file where this command is defined.
  • ctx.target_file: the file being processed in this iteration (same as file).
  • ctx.command_name: the command's name (same as command_name global).
  • ctx.capture: in regex mode, a table mirroring all capture group variables (s1, v1, named groups, etc.) as fields.

Regex mode adds:

• s1..sN: string value for unnamed capture groups.
• v1..vN: numeric value for unnamed capture groups when parseable as number.
• Named captures as variables, for example (?P<amount>!num) -> amount.
• replacement: optional full-match replacement string.

Notes:

• If you prefer fixed positions, treat captures as s1..s9 and v1..v9 (and beyond when needed).
• vX only exists when capture sX parses as a number.

JSON mode adds:

• data: parsed JSON document as a mutable Lua table.
• json.jpath(node, expr): evaluate JSONPath relative to node.
• json.jpathrm(node, expr): remove JSON nodes matched by JSONPath.

XML mode adds:

• data: parsed XML node tree as a mutable Lua table.
• Node shape:
  • tag
  • text
  • attr
  • children
  • name.local
  • name.uri
  • name.prefix
• xml.xpath(node, expr): evaluate XPath relative to node.
• xml.xpathrm(node, expr): remove XML nodes matched by XPath.

Example:

for _, item in ipairs(xml.xpath(data, "//Item")) do
  if item.attr.Weight then
    item.attr.Weight = N(item.attr.Weight) * 2
  end
end

for _, group in ipairs(xml.xpath(data, "//Group")) do
  for _, entry in ipairs(xml.xpath(group, ".//Entry")) do
    entry.attr.Enabled = "true"
  end
end

CSV helpers

• csv.read(text, options) parses CSV/TSV into rows.
• csv.write(rows, options) converts rows back into CSV/TSV text.

Supported options keys:

• delimiter (default ",")
• hasheader (default false)
• hascomments (default false, skips lines starting with #)

Example:

local rows = csv.read(csvText, {
  delimiter = "\t",
  hasheader = true,
  hascomments = true,
})

rows[1].Price = util.num(rows[1].Price) * 1.1
replacement = csv.write(rows, { delimiter = "\t", hasheader = true })

Useful Lua helpers

Built-in helpers:

• CSV: csv.read(text, opts), csv.write(rows, opts)
• JSON: json.read(path), json.visit(data, cb), json.find(data, pred), json.modifyNumbers(data, pred, mod), json.jpath(node, expr), json.jpathrm(node, expr)
• XML: xml.read(path), xml.find(root, tag), xml.visit(root, cb), xml.filter(root, pred), xml.getAttr/xml.setAttr/xml.hasAttr, xml.getNumAttr/xml.setNumAttr/xml.modifyNumAttr, xml.getText/xml.setText, xml.findFirst, xml.addChild/xml.removeChildren/xml.getChildren/xml.countChildren, xml.xpath(node, expr), xml.xpathrm(node, expr)
• SQL (sql = true commands only): sql.select(table, where, cols), sql.update(table, rows), sql.insert(table, rows), sql.delete(table, where)
• Math: math.round(x, n)
• Strings: string.trim(s), string.split(s, sep)
• Util: util.printf(fmt, ...), util.num(str), util.str(n), util.isNumber(s), util.isArray(t), util.dump(t)
• HTTP: fetch(url, options) returns {status, statusText, ok, body, headers}
• Regex: re(pattern, input) returns match table ([1] full match, [2+] groups)

Run chef --help for the full generated helper reference, and chef --meta to generate meta.lua for editor autocomplete.

Notes about example_cook.toml

example_cook.toml is intentionally broad and demonstrates many patterns (regexes, isolate, nodedup, JSON commands, multiline regex, external systems). Treat it as a feature showcase and copy only the parts you need into your own cook files.