36 Commits

Author SHA1 Message Date
bb14087598 Fix oopsie 2025-03-26 02:52:28 +01:00
66a522aa12 Try to include xml node children in lua table 2025-03-26 02:50:33 +01:00
1a4b4f76f2 Map the weird numeric escapes to textual ones 2025-03-26 02:34:21 +01:00
2bfd9f951e Fix up some more xml tests and other small bugs 2025-03-26 02:17:42 +01:00
e5092edf53 Implement parsing xml to and from lua
A lot more complex than json.........
2025-03-26 01:36:49 +01:00
e31c0e4e8f Implement xpath (by calling library) 2025-03-26 01:19:41 +01:00
73d93367a0 Refactor some things around a little 2025-03-25 23:16:03 +01:00
64f690f6b4 Fix usage to reflect recent flag changes 2025-03-25 23:02:32 +01:00
34477b2c34 Make readme and rework the flags a little 2025-03-25 22:55:01 +01:00
d5c08d86f5 Code polish 2025-03-25 19:22:44 +01:00
68127fe453 Add more json tests
To bring it in line with the xml ones
2025-03-25 19:22:07 +01:00
872f2dd46d Fix another changed test for json 2025-03-25 19:16:09 +01:00
4eed05c7c2 Fix some more minor bugs and tests 2025-03-25 19:14:21 +01:00
4640281fbf Enable root modifications
Though I can not see why you would want to.....
But there's no reason you would not be able to
2025-03-25 18:57:32 +01:00
aba10267d1 Fix more tests 2025-03-25 18:47:55 +01:00
fed140254b Fix some json tests 2025-03-25 18:32:51 +01:00
db92033642 Rework rounding and building lua script
To allow user script to specify what was modified where
2025-03-25 18:28:33 +01:00
1b0b198297 Add xpath tests 2025-03-25 17:56:48 +01:00
15ae116447 Fix up some miscellaneous shit around the project regarding lua conversions 2025-03-25 17:46:21 +01:00
1bcc6735ab Improve error handling across the board 2025-03-25 17:27:20 +01:00
20bab894e3 Add path data to the selected nodes for reconstruction via set 2025-03-25 17:27:20 +01:00
396992b3d0 Implement Set and SetAll (and Get) 2025-03-25 17:27:20 +01:00
054dcbf835 Fix up the recursive descent
Again I guess?
2025-03-25 17:27:20 +01:00
88887b9a12 Fix up the recursive descent 2025-03-25 17:27:20 +01:00
533a563dc5 Fix up parser 2025-03-25 17:27:20 +01:00
7fc2956b6d Begin to implement jsonpath 2025-03-25 17:27:20 +01:00
6f9f3f5eae Begin to rework the json parsing 2025-03-25 17:27:20 +01:00
d2419b761e Implement a generic translator between lua and go tables (maps) 2025-03-25 17:27:20 +01:00
ab98800ca0 Rework regex processor to be more betterer 2025-03-25 17:27:20 +01:00
0f7ee521ac Fix up lua variable writing and reading for regex 2025-03-25 17:27:20 +01:00
fd81861a64 Fix reading lua variables 2025-03-25 17:27:20 +01:00
430234dd3b Clean up after claude 2025-03-25 17:27:20 +01:00
17bb3d4f71 Refactor everything to processors and implement json and xml processors such as they are 2025-03-25 17:27:20 +01:00
84e0a8bed6 Add xpath dependencies 2025-03-25 17:27:20 +01:00
bdcf096fdd Fix up the oopsies 2025-03-25 17:27:15 +01:00
1a90046c89 Add release script 2025-03-25 17:26:23 +01:00
19 changed files with 2598 additions and 574 deletions

9
.vscode/launch.json vendored
View File

@@ -9,8 +9,13 @@
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${fileDirname}",
"args": []
"program": "${workspaceFolder}",
"args": [
"-mode=json",
"$..name",
"v='pero'",
"test.json"
]
}
]
}

116
README.md Normal file
View File

@@ -0,0 +1,116 @@
# Big Chef
A Go-based tool for modifying XML, JSON, and text documents using XPath/JSONPath/Regex expressions and Lua transformations.
## Features
- **Multi-Format Processing**:
- XML (XPath)
- JSON (JSONPath)
- Text (Regex)
- **Node Value Modification**: Update text values in XML elements, JSON properties or text matches
- **Attribute Manipulation**: Modify XML attributes, JSON object keys or regex capture groups
- **Conditional Logic**: Apply transformations based on document content
- **Complex Operations**:
- Mathematical calculations
- String manipulations
- Date conversions
- Structural changes
- Whole ass Lua environment
- **Error Handling**: Comprehensive error detection for:
- Invalid XML/JSON
- Malformed XPath/JSONPath
- Lua syntax errors
## Usage Examples
### 1. Basic field modification
```xml
<!-- Input -->
<price>44.95</price>
<!-- Command -->
chef -xml "//price" "v=v*2" input.xml
<!-- Output -->
<price>89.9</price>
```
### 2. Supports glob patterns
```xml
chef -xml "//price" "v=v*2" data/**.xml
```
### 3. Attribute Update
```xml
<!-- Input -->
<item price="10.50"/>
<!-- Command -->
chef -xml "//item/@price" "v=v*2" input.xml
<!-- Output -->
<item price="21"/>
```
### 3. JSONPath Transformation
```json
// Input
{
"products": [
{"name": "Widget", "price": 19.99},
{"name": "Gadget", "price": 29.99}
]
}
// Command
chef -json "$.products[*].price" "v=v*0.75" input.json
// Output
{
"products": [
{"name": "Widget", "price": 14.99},
{"name": "Gadget", "price": 22.49}
]
}
```
### 4. Regex Text Replacement
Regex works slightly differently, up to 12 match groups are provided as v1..v12 and s1..s12 for numbers and strings respectively.
A special shorthand "!num" is also provided that simply expands to `(\d*\.?\d+)`.
```xml
<!-- Input -->
<description>Price: $15.00 Special Offer</description>
<!-- Command -->
chef "Price: $!num Special Offer" "v1 = v1 * 0.92" input.xml
<!-- Output -->
<description>Price: $13.80 Special Offer</description>
```
### 5. Conditional Transformation
```xml
<!-- Input -->
<item stock="5" price="10.00"/>
<!-- Command -->
chef -xml "//item" "if tonumber(v.stock) > 0 then v.price = v.price * 0.8 end" input.xml
<!-- Output -->
<item stock="5" price="8.00"/>
```
## Installation
```bash
go build -o chef main.go
```
```bash
# Process XML file
./chef -xml "//price" "v=v*1.2" input.xml
# Process JSON file
./chef -json "$.prices[*]" "v=v*0.9" input.json
```

4
go.mod
View File

@@ -3,6 +3,7 @@ module modify
go 1.24.1
require (
github.com/PaesslerAG/jsonpath v0.1.1
github.com/antchfx/xmlquery v1.4.4
github.com/bmatcuk/doublestar/v4 v4.8.1
github.com/yuin/gopher-lua v1.1.1
@@ -10,11 +11,8 @@ require (
require (
github.com/PaesslerAG/gval v1.0.0 // indirect
github.com/PaesslerAG/jsonpath v0.1.1 // indirect
github.com/antchfx/xpath v1.3.3 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/sergi/go-diff v1.3.1 // indirect
github.com/stretchr/testify v1.10.0 // indirect
golang.org/x/net v0.33.0 // indirect
golang.org/x/text v0.21.0 // indirect
)

16
go.sum
View File

@@ -9,21 +9,9 @@ github.com/antchfx/xpath v1.3.3 h1:tmuPQa1Uye0Ym1Zn65vxPgfltWb/Lxu2jeqIGteJSRs=
github.com/antchfx/xpath v1.3.3/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/bmatcuk/doublestar/v4 v4.8.1 h1:54Bopc5c2cAvhLRAzqOGCYHYyhcDHsFF4wWIR5wKP38=
github.com/bmatcuk/doublestar/v4 v4.8.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
@@ -92,7 +80,3 @@ golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=

78
main.go
View File

@@ -19,20 +19,12 @@ type GlobalStats struct {
FailedFiles int
}
type FileMode string
const (
ModeRegex FileMode = "regex"
ModeXML FileMode = "xml"
ModeJSON FileMode = "json"
)
var stats GlobalStats
var logger *log.Logger
var (
fileModeFlag = flag.String("mode", "regex", "Processing mode: regex, xml, json")
verboseFlag = flag.Bool("verbose", false, "Enable verbose output")
jsonFlag = flag.Bool("json", false, "Process JSON files")
xmlFlag = flag.Bool("xml", false, "Process XML files")
)
func init() {
@@ -43,28 +35,39 @@ func init() {
}
func main() {
// TODO: Implement some sort of git integration
// Maybe use go-git
// Specify a -git flag
// If we are operating with git then:
// Inmitialize a repo if one doesn't exist (try to open right?)
// For each file matched by glob first figure out if it's already tracked
// If not tracked then track it and commit (either it alone or maybe multiple together somehow)
// Then reset the file (to undo previous modifications)
// THEN change the file
// In addition add a -undo flag that will ONLY reset the files without changing them
// Only for the ones matched by glob
// ^ important because binary files would fuck us up
flag.Usage = func() {
fmt.Fprintf(os.Stderr, "Usage: %s [options] <pattern> <lua_expression> <...files_or_globs>\n", os.Args[0])
fmt.Fprintf(os.Stderr, "\nOptions:\n")
fmt.Fprintf(os.Stderr, " -json\n")
fmt.Fprintf(os.Stderr, " Process JSON files\n")
fmt.Fprintf(os.Stderr, " -xml\n")
fmt.Fprintf(os.Stderr, " Process XML files\n")
fmt.Fprintf(os.Stderr, " -mode string\n")
fmt.Fprintf(os.Stderr, " Processing mode: regex, xml, json (default \"regex\")\n")
fmt.Fprintf(os.Stderr, " -xpath string\n")
fmt.Fprintf(os.Stderr, " XPath expression (for XML mode)\n")
fmt.Fprintf(os.Stderr, " -jsonpath string\n")
fmt.Fprintf(os.Stderr, " JSONPath expression (for JSON mode)\n")
fmt.Fprintf(os.Stderr, " -verbose\n")
fmt.Fprintf(os.Stderr, " Enable verbose output\n")
fmt.Fprintf(os.Stderr, "\nExamples:\n")
fmt.Fprintf(os.Stderr, " Regex mode (default):\n")
fmt.Fprintf(os.Stderr, " %s \"<value>(\\d+)</value>\" \"*1.5\" data.xml\n", os.Args[0])
fmt.Fprintf(os.Stderr, " XML mode:\n")
fmt.Fprintf(os.Stderr, " %s -mode=xml -xpath=\"//value\" \"*1.5\" data.xml\n", os.Args[0])
fmt.Fprintf(os.Stderr, " %s -xml \"//value\" \"*1.5\" data.xml\n", os.Args[0])
fmt.Fprintf(os.Stderr, " JSON mode:\n")
fmt.Fprintf(os.Stderr, " %s -mode=json -jsonpath=\"$.items[*].value\" \"*1.5\" data.json\n", os.Args[0])
fmt.Fprintf(os.Stderr, " %s -json \"$.items[*].value\" \"*1.5\" data.json\n", os.Args[0])
fmt.Fprintf(os.Stderr, "\nNote: v1, v2, etc. are used to refer to capture groups as numbers.\n")
fmt.Fprintf(os.Stderr, " s1, s2, etc. are used to refer to capture groups as strings.\n")
fmt.Fprintf(os.Stderr, " Helper functions: num(str) converts string to number, str(num) converts number to string\n")
fmt.Fprintf(os.Stderr, " is_number(str) checks if a string is numeric\n")
fmt.Fprintf(os.Stderr, " For XML and JSON, the captured values are exposed as 'v', which can be of any type we capture (string, number, table).\n")
fmt.Fprintf(os.Stderr, " If expression starts with an operator like *, /, +, -, =, etc., v1 is automatically prepended\n")
fmt.Fprintf(os.Stderr, " You can use any valid Lua code, including if statements, loops, etc.\n")
fmt.Fprintf(os.Stderr, " Glob patterns are supported for file selection (*.xml, data/**.xml, etc.)\n")
@@ -74,7 +77,7 @@ func main() {
args := flag.Args()
if len(args) < 3 {
fmt.Fprintf(os.Stderr, "%s mode requires %d arguments minimum\n", *fileModeFlag, 3)
log.Printf("At least %d arguments are required", 3)
flag.Usage()
return
}
@@ -83,15 +86,9 @@ func main() {
var pattern, luaExpr string
var filePatterns []string
if *fileModeFlag == "regex" {
pattern = args[0]
luaExpr = args[1]
filePatterns = args[2:]
} else {
// For XML/JSON modes, pattern comes from flags
luaExpr = args[0]
filePatterns = args[1:]
}
pattern = args[0]
luaExpr = args[1]
filePatterns = args[2:]
// Prepare the Lua expression
originalLuaExpr := luaExpr
@@ -114,21 +111,19 @@ func main() {
// Create the processor based on mode
var proc processor.Processor
switch *fileModeFlag {
case "regex":
switch {
case *xmlFlag:
proc = &processor.XMLProcessor{}
logger.Printf("Starting XML modifier with XPath %q, expression %q on %d files",
pattern, luaExpr, len(files))
case *jsonFlag:
proc = &processor.JSONProcessor{}
logger.Printf("Starting JSON modifier with JSONPath %q, expression %q on %d files",
pattern, luaExpr, len(files))
default:
proc = &processor.RegexProcessor{}
logger.Printf("Starting regex modifier with pattern %q, expression %q on %d files",
pattern, luaExpr, len(files))
// case "xml":
// proc = &processor.XMLProcessor{}
// pattern = *xpathFlag
// logger.Printf("Starting XML modifier with XPath %q, expression %q on %d files",
// pattern, luaExpr, len(files))
// case "json":
// proc = &processor.JSONProcessor{}
// pattern = *jsonpathFlag
// logger.Printf("Starting JSON modifier with JSONPath %q, expression %q on %d files",
// pattern, luaExpr, len(files))
}
var wg sync.WaitGroup
@@ -139,7 +134,8 @@ func main() {
defer wg.Done()
logger.Printf("Processing file: %s", file)
modCount, matchCount, err := proc.Process(file, pattern, luaExpr)
// It's a bit fucked, maybe I could do better to call it from proc... But it'll do for now
modCount, matchCount, err := processor.Process(proc, file, pattern, luaExpr)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to process file %s: %v\n", file, err)
stats.FailedFiles++

View File

@@ -3,10 +3,8 @@ package processor
import (
"encoding/json"
"fmt"
"log"
"modify/processor/jsonpath"
"os"
"path/filepath"
"strings"
lua "github.com/yuin/gopher-lua"
)
@@ -14,39 +12,6 @@ import (
// JSONProcessor implements the Processor interface for JSON documents
type JSONProcessor struct{}
// Process implements the Processor interface for JSONProcessor
func (p *JSONProcessor) Process(filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
cwd, err := os.Getwd()
if err != nil {
return 0, 0, fmt.Errorf("error getting current working directory: %v", err)
}
fullPath := filepath.Join(cwd, filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ProcessContent implements the Processor interface for JSONProcessor
func (p *JSONProcessor) ProcessContent(content string, pattern string, luaExpr string) (string, int, int, error) {
// Parse JSON document
@@ -67,99 +32,125 @@ func (p *JSONProcessor) ProcessContent(content string, pattern string, luaExpr s
return content, 0, 0, nil
}
// Initialize Lua
L, err := NewLuaState()
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error creating Lua state: %v", err)
}
defer L.Close()
modCount := 0
for _, node := range nodes {
log.Printf("Processing node at path: %s with value: %v", node.Path, node.Value)
err = p.ToLua(L, nodes)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error converting to Lua: %v", err)
}
// Initialize Lua
L, err := NewLuaState()
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error creating Lua state: %v", err)
}
defer L.Close()
log.Println("Lua state initialized successfully.")
// Execute Lua script
if err := L.DoString(luaExpr); err != nil {
return content, len(nodes), 0, fmt.Errorf("error executing Lua %s: %v", luaExpr, err)
}
err = p.ToLua(L, node.Value)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error converting to Lua: %v", err)
}
log.Printf("Converted node value to Lua: %v", node.Value)
// Get modified value
result, err := p.FromLua(L)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error getting result from Lua: %v", err)
}
originalScript := luaExpr
fullScript := BuildLuaScript(luaExpr)
log.Printf("Original script: %q, Full script: %q", originalScript, fullScript)
// Apply the modification to the JSON data
err = p.updateJSONValue(jsonData, pattern, result)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error updating JSON: %v", err)
// Execute Lua script
log.Printf("Executing Lua script: %q", fullScript)
if err := L.DoString(fullScript); err != nil {
return content, len(nodes), 0, fmt.Errorf("error executing Lua %q: %v", fullScript, err)
}
log.Println("Lua script executed successfully.")
// Get modified value
result, err := p.FromLua(L)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error getting result from Lua: %v", err)
}
log.Printf("Retrieved modified value from Lua: %v", result)
modified := false
modified = L.GetGlobal("modified").String() == "true"
if !modified {
log.Printf("No changes made to node at path: %s", node.Path)
continue
}
// Apply the modification to the JSON data
err = p.updateJSONValue(jsonData, node.Path, result)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error updating JSON: %v", err)
}
log.Printf("Updated JSON at path: %s with new value: %v", node.Path, result)
modCount++
}
// Convert the modified JSON back to a string with same formatting
var jsonBytes []byte
if indent, err := detectJsonIndentation(content); err == nil && indent != "" {
// Use detected indentation for output formatting
jsonBytes, err = json.MarshalIndent(jsonData, "", indent)
} else {
// Fall back to standard 2-space indent
jsonBytes, err = json.MarshalIndent(jsonData, "", " ")
jsonBytes, err = json.MarshalIndent(jsonData, "", " ")
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error marshalling JSON: %v", err)
}
// We changed all the nodes trust me bro
return string(jsonBytes), len(nodes), len(nodes), nil
return string(jsonBytes), modCount, matchCount, nil
}
// detectJsonIndentation tries to determine the indentation used in the original JSON
func detectJsonIndentation(content string) (string, error) {
lines := strings.Split(content, "\n")
if len(lines) < 2 {
return "", fmt.Errorf("not enough lines to detect indentation")
}
// Look for the first indented line
for i := 1; i < len(lines); i++ {
line := lines[i]
trimmed := strings.TrimSpace(line)
if trimmed == "" {
continue
}
// Calculate leading whitespace
indent := line[:len(line)-len(trimmed)]
if len(indent) > 0 {
return indent, nil
}
}
return "", fmt.Errorf("no indentation detected")
}
// / Selects from the root node
// // Selects nodes in the document from the current node that match the selection no matter where they are
// . Selects the current node
// @ Selects attributes
// /bookstore/* Selects all the child element nodes of the bookstore element
// //* Selects all elements in the document
// /bookstore/book[1] Selects the first book element that is the child of the bookstore element.
// /bookstore/book[last()] Selects the last book element that is the child of the bookstore element
// /bookstore/book[last()-1] Selects the last but one book element that is the child of the bookstore element
// /bookstore/book[position()<3] Selects the first two book elements that are children of the bookstore element
// //title[@lang] Selects all the title elements that have an attribute named lang
// //title[@lang='en'] Selects all the title elements that have a "lang" attribute with a value of "en"
// /bookstore/book[price>35.00] Selects all the book elements of the bookstore element that have a price element with a value greater than 35.00
// /bookstore/book[price>35.00]/title Selects all the title elements of the book elements of the bookstore element that have a price element with a value greater than 35.00
// updateJSONValue updates a value in the JSON structure based on its JSONPath
func (p *JSONProcessor) updateJSONValue(jsonData interface{}, path string, newValue interface{}) error {
// Special handling for root node
if path == "$" {
// For the root node, we'll copy the value to the jsonData reference
// This is a special case since we can't directly replace the interface{} variable
// We need to handle different types of root elements
switch rootValue := newValue.(type) {
case map[string]interface{}:
// For objects, we need to copy over all keys
rootMap, ok := jsonData.(map[string]interface{})
if !ok {
// If the original wasn't a map, completely replace it with the new map
// This is handled by the jsonpath.Set function
return jsonpath.Set(jsonData, path, newValue)
}
// Clear the original map
for k := range rootMap {
delete(rootMap, k)
}
// Copy all keys from the new map
for k, v := range rootValue {
rootMap[k] = v
}
return nil
case []interface{}:
// For arrays, we need to handle similarly
rootArray, ok := jsonData.([]interface{})
if !ok {
// If the original wasn't an array, use jsonpath.Set
return jsonpath.Set(jsonData, path, newValue)
}
// Clear and recreate the array
*&rootArray = rootValue
return nil
default:
// For other types, use jsonpath.Set
return jsonpath.Set(jsonData, path, newValue)
}
}
// For non-root paths, use the regular Set method
err := jsonpath.Set(jsonData, path, newValue)
if err != nil {
return fmt.Errorf("failed to update JSON value at path '%s': %w", path, err)
}
return nil
}
// ToLua converts JSON values to Lua variables
func (p *JSONProcessor) ToLua(L *lua.LState, data interface{}) error {
table, err := ToLuaTable(L, data)
table, err := ToLua(L, data)
if err != nil {
return err
}
@@ -170,5 +161,5 @@ func (p *JSONProcessor) ToLua(L *lua.LState, data interface{}) error {
// FromLua retrieves values from Lua
func (p *JSONProcessor) FromLua(L *lua.LState) (interface{}, error) {
luaValue := L.GetGlobal("v")
return FromLuaTable(L, luaValue.(*lua.LTable))
return FromLua(L, luaValue)
}

File diff suppressed because it is too large Load Diff

View File

@@ -138,10 +138,6 @@ func Set(data interface{}, path string, value interface{}) error {
return fmt.Errorf("failed to parse JSONPath %q: %w", path, err)
}
if len(steps) <= 1 {
return fmt.Errorf("cannot set root node; the provided path %q is invalid", path)
}
success := false
err = setWithPath(data, steps, &success, value, "$", ModifyFirstMode)
if err != nil {
@@ -157,10 +153,6 @@ func SetAll(data interface{}, path string, value interface{}) error {
return fmt.Errorf("failed to parse JSONPath %q: %w", path, err)
}
if len(steps) <= 1 {
return fmt.Errorf("cannot set root node; the provided path %q is invalid", path)
}
success := false
err = setWithPath(data, steps, &success, value, "$", ModifyAllMode)
if err != nil {
@@ -178,17 +170,20 @@ func setWithPath(node interface{}, steps []JSONStep, success *bool, value interf
// Skip root step
actualSteps := steps
if len(steps) > 0 && steps[0].Type == RootStep {
if len(steps) == 1 {
return fmt.Errorf("cannot set root node; the provided path %q is invalid", currentPath)
}
actualSteps = steps[1:]
}
// Process the first step
// If we have no steps left, we're setting the root value
if len(actualSteps) == 0 {
return fmt.Errorf("cannot set root node; no steps provided for path %q", currentPath)
// For the root node, we need to handle it differently depending on what's passed in
// since we can't directly replace the interface{} variable
// We'll signal success and let the JSONProcessor handle updating the root
*success = true
return nil
}
// Process the first step
step := actualSteps[0]
remainingSteps := actualSteps[1:]
isLastStep := len(remainingSteps) == 0

View File

@@ -355,14 +355,14 @@ func TestSet(t *testing.T) {
}
})
t.Run("setting on root should fail", func(t *testing.T) {
t.Run("setting on root should not fail (anymore)", func(t *testing.T) {
data := map[string]interface{}{
"name": "John",
}
err := Set(data, "$", "Jane")
if err == nil {
t.Errorf("Set() returned no error, expected error for setting on root")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}

View File

@@ -2,16 +2,19 @@ package processor
import (
"fmt"
"reflect"
"os"
"path/filepath"
"strings"
"github.com/antchfx/xmlquery"
lua "github.com/yuin/gopher-lua"
)
// Processor defines the interface for all file processors
type Processor interface {
// Process handles processing a file with the given pattern and Lua expression
Process(filename string, pattern string, luaExpr string) (int, int, error)
// Now implemented as a base function in processor.go
// Process(filename string, pattern string, luaExpr string) (int, int, error)
// ProcessContent handles processing a string content directly with the given pattern and Lua expression
// Returns the modified content, modification count, match count, and any error
@@ -52,65 +55,139 @@ func NewLuaState() (*lua.LState, error) {
return L, nil
}
// ToLuaTable converts a struct or map to a Lua table recursively
func ToLuaTable(L *lua.LState, data interface{}) (*lua.LTable, error) {
luaTable := L.NewTable()
func Process(p Processor, filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
cwd, err := os.Getwd()
if err != nil {
return 0, 0, fmt.Errorf("error getting current working directory: %v", err)
}
fullPath := filepath.Join(cwd, filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ToLua converts a struct or map to a Lua table recursively
func ToLua(L *lua.LState, data interface{}) (lua.LValue, error) {
switch v := data.(type) {
case *xmlquery.Node:
luaTable := L.NewTable()
luaTable.RawSetString("text", lua.LString(v.Data))
// Should be a map, simple key value pairs
attr, err := ToLua(L, v.Attr)
if err != nil {
return nil, err
}
luaTable.RawSetString("attr", attr)
return luaTable, nil
case map[string]interface{}:
luaTable := L.NewTable()
for key, value := range v {
luaValue, err := ToLuaTable(L, value)
luaValue, err := ToLua(L, value)
if err != nil {
return nil, err
}
luaTable.RawSetString(key, luaValue)
}
case struct{}:
val := reflect.ValueOf(v)
for i := 0; i < val.NumField(); i++ {
field := val.Type().Field(i)
luaValue, err := ToLuaTable(L, val.Field(i).Interface())
return luaTable, nil
case []interface{}:
luaTable := L.NewTable()
for i, value := range v {
luaValue, err := ToLua(L, value)
if err != nil {
return nil, err
}
luaTable.RawSetString(field.Name, luaValue)
luaTable.RawSetInt(i+1, luaValue) // Lua arrays are 1-indexed
}
return luaTable, nil
case string:
luaTable.RawSetString("v", lua.LString(v))
return lua.LString(v), nil
case bool:
luaTable.RawSetString("v", lua.LBool(v))
return lua.LBool(v), nil
case float64:
luaTable.RawSetString("v", lua.LNumber(v))
return lua.LNumber(v), nil
case nil:
return lua.LNil, nil
default:
return nil, fmt.Errorf("unsupported data type: %T", data)
}
return luaTable, nil
}
// FromLuaTable converts a Lua table to a struct or map recursively
func FromLuaTable(L *lua.LState, luaTable *lua.LTable) (map[string]interface{}, error) {
result := make(map[string]interface{})
luaTable.ForEach(func(key lua.LValue, value lua.LValue) {
switch v := value.(type) {
case *lua.LTable:
nestedMap, err := FromLuaTable(L, v)
if err != nil {
return
}
result[key.String()] = nestedMap
case lua.LString:
result[key.String()] = string(v)
case lua.LBool:
result[key.String()] = bool(v)
case lua.LNumber:
result[key.String()] = float64(v)
default:
result[key.String()] = nil
// FromLua converts a Lua table to a struct or map recursively
func FromLua(L *lua.LState, luaValue lua.LValue) (interface{}, error) {
switch v := luaValue.(type) {
// Well shit...
// Tables in lua are both maps and arrays
// As arrays they are ordered and as maps, obviously, not
// So when we parse them to a go map we fuck up the order for arrays
// We have to find a better way....
case *lua.LTable:
isArray, err := IsLuaTableArray(L, v)
if err != nil {
return nil, err
}
})
if isArray {
result := make([]interface{}, 0)
v.ForEach(func(key lua.LValue, value lua.LValue) {
converted, _ := FromLua(L, value)
result = append(result, converted)
})
return result, nil
} else {
result := make(map[string]interface{})
v.ForEach(func(key lua.LValue, value lua.LValue) {
converted, _ := FromLua(L, value)
result[key.String()] = converted
})
return result, nil
}
case lua.LString:
return string(v), nil
case lua.LBool:
return bool(v), nil
case lua.LNumber:
return float64(v), nil
default:
return nil, nil
}
}
return result, nil
func IsLuaTableArray(L *lua.LState, v *lua.LTable) (bool, error) {
L.SetGlobal("table_to_check", v)
// Use our predefined helper function from InitLuaHelpers
err := L.DoString(`is_array = isArray(table_to_check)`)
if err != nil {
return false, fmt.Errorf("error determining if table is array: %w", err)
}
// Check the result of our Lua function
isArray := L.GetGlobal("is_array")
// LVIsFalse returns true if a given LValue is a nil or false otherwise false.
if !lua.LVIsFalse(isArray) {
return true, nil
}
return false, nil
}
// InitLuaHelpers initializes common Lua helper functions
@@ -119,7 +196,10 @@ func InitLuaHelpers(L *lua.LState) error {
-- Custom Lua helpers for math operations
function min(a, b) return math.min(a, b) end
function max(a, b) return math.max(a, b) end
function round(x) return math.floor(x + 0.5) end
function round(x, n)
if n == nil then n = 0 end
return math.floor(x * 10^n + 0.5) / 10^n
end
function floor(x) return math.floor(x) end
function ceil(x) return math.ceil(x) end
function upper(s) return string.upper(s) end
@@ -139,6 +219,22 @@ end
function is_number(str)
return tonumber(str) ~= nil
end
function isArray(t)
if type(t) ~= "table" then return false end
local max = 0
local count = 0
for k, _ in pairs(t) do
if type(k) ~= "number" or k < 1 or math.floor(k) ~= k then
return false
end
max = math.max(max, k)
count = count + 1
end
return max == count
end
modified = false
`
if err := L.DoString(helperScript); err != nil {
return fmt.Errorf("error loading helper functions: %v", err)
@@ -157,8 +253,7 @@ func LimitString(s string, maxLen int) string {
return s[:maxLen-3] + "..."
}
// BuildLuaScript prepares a Lua expression from shorthand notation
func BuildLuaScript(luaExpr string) string {
func PrependLuaAssignment(luaExpr string) string {
// Auto-prepend v1 for expressions starting with operators
if strings.HasPrefix(luaExpr, "*") ||
strings.HasPrefix(luaExpr, "/") ||
@@ -176,10 +271,30 @@ func BuildLuaScript(luaExpr string) string {
if !strings.Contains(luaExpr, "=") {
luaExpr = "v1 = " + luaExpr
}
return luaExpr
}
// BuildLuaScript prepares a Lua expression from shorthand notation
func BuildLuaScript(luaExpr string) string {
luaExpr = PrependLuaAssignment(luaExpr)
// This allows the user to specify whether or not they modified a value
// If they do nothing we assume they did modify (no return at all)
// If they return before our return then they themselves specify what they did
// If nothing is returned lua assumes nil
// So we can say our value was modified if the return value is either nil or true
// If the return value is false then the user wants to keep the original
fullScript := fmt.Sprintf(`
function run()
%s
end
local res = run()
modified = res == nil or res
`, luaExpr)
return fullScript
}
// Max returns the maximum of two integers
func Max(a, b int) int {
if a > b {

View File

@@ -2,8 +2,6 @@ package processor
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
@@ -14,34 +12,6 @@ import (
// RegexProcessor implements the Processor interface using regex patterns
type RegexProcessor struct{}
// Process implements the Processor interface for RegexProcessor
func (p *RegexProcessor) Process(filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
fullPath := filepath.Join(".", filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ToLua sets capture groups as Lua variables (v1, v2, etc. for numeric values and s1, s2, etc. for strings)
func (p *RegexProcessor) ToLua(L *lua.LState, data interface{}) error {
captures, ok := data.([]string)

View File

@@ -35,10 +35,11 @@ func TestBuildLuaScript(t *testing.T) {
{"v1 * 2", "v1 = v1 * 2"},
{"v1 * v2", "v1 = v1 * v2"},
{"v1 / v2", "v1 = v1 / v2"},
{"12", "v1 = 12"},
}
for _, c := range cases {
result := BuildLuaScript(c.input)
result := PrependLuaAssignment(c.input)
if result != c.expected {
t.Errorf("BuildLuaScript(%q): expected %q, got %q", c.input, c.expected, result)
}

View File

@@ -2,8 +2,8 @@ package processor
import (
"fmt"
"os"
"path/filepath"
"log"
"modify/processor/xpath"
"strings"
"github.com/antchfx/xmlquery"
@@ -13,44 +13,18 @@ import (
// XMLProcessor implements the Processor interface for XML documents
type XMLProcessor struct{}
// Process implements the Processor interface for XMLProcessor
func (p *XMLProcessor) Process(filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
fullPath := filepath.Join(".", filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ProcessContent implements the Processor interface for XMLProcessor
func (p *XMLProcessor) ProcessContent(content string, pattern string, luaExpr string) (string, int, int, error) {
func (p *XMLProcessor) ProcessContent(content string, path string, luaExpr string) (string, int, int, error) {
// Parse XML document
// We can't really use encoding/xml here because it requires a pre defined struct
// And we HAVE TO parse dynamic unknown XML
doc, err := xmlquery.Parse(strings.NewReader(content))
if err != nil {
return content, 0, 0, fmt.Errorf("error parsing XML: %v", err)
}
// Find nodes matching the XPath pattern
nodes, err := xmlquery.QueryAll(doc, pattern)
nodes, err := xpath.Get(doc, path)
if err != nil {
return content, 0, 0, fmt.Errorf("error executing XPath: %v", err)
}
@@ -60,104 +34,45 @@ func (p *XMLProcessor) ProcessContent(content string, pattern string, luaExpr st
return content, 0, 0, nil
}
// Initialize Lua
L := lua.NewState()
defer L.Close()
// Load math library
L.Push(L.GetGlobal("require"))
L.Push(lua.LString("math"))
if err := L.PCall(1, 1, nil); err != nil {
return content, 0, 0, fmt.Errorf("error loading Lua math library: %v", err)
}
// Load helper functions
if err := InitLuaHelpers(L); err != nil {
return content, 0, 0, err
}
// Apply modifications to each node
modCount := 0
for _, node := range nodes {
// Reset Lua state for each node
L.SetGlobal("v1", lua.LNil)
L.SetGlobal("s1", lua.LNil)
// Get the node value
var originalValue string
if node.Type == xmlquery.AttributeNode {
originalValue = node.InnerText()
} else if node.Type == xmlquery.TextNode {
originalValue = node.Data
} else {
originalValue = node.InnerText()
L, err := NewLuaState()
if err != nil {
return content, 0, 0, fmt.Errorf("error creating Lua state: %v", err)
}
defer L.Close()
// Convert to Lua variables
err = p.ToLua(L, originalValue)
err = p.ToLua(L, node)
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error converting to Lua: %v", err)
}
// Execute Lua script
if err := L.DoString(luaExpr); err != nil {
err = L.DoString(BuildLuaScript(luaExpr))
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error executing Lua: %v", err)
}
// Get modified value
result, err := p.FromLua(L)
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error getting result from Lua: %v", err)
}
log.Printf("%#v", result)
newValue, ok := result.(string)
if !ok {
return content, modCount, matchCount, fmt.Errorf("expected string result from Lua, got %T", result)
}
// Skip if no change
if newValue == originalValue {
modified := false
modified = L.GetGlobal("modified").String() == "true"
if !modified {
log.Printf("No changes made to node at path: %s", node.Data)
continue
}
// Apply modification
if node.Type == xmlquery.AttributeNode {
// For attribute nodes, update the attribute value
node.Parent.Attr = append([]xmlquery.Attr{}, node.Parent.Attr...)
for i, attr := range node.Parent.Attr {
if attr.Name.Local == node.Data {
node.Parent.Attr[i].Value = newValue
break
}
}
} else if node.Type == xmlquery.TextNode {
// For text nodes, update the text content
node.Data = newValue
} else {
// For element nodes, replace inner text
// Simple approach: set the InnerText directly if there are no child elements
if node.FirstChild == nil || (node.FirstChild != nil && node.FirstChild.Type == xmlquery.TextNode && node.FirstChild.NextSibling == nil) {
if node.FirstChild != nil {
node.FirstChild.Data = newValue
} else {
// Create a new text node and add it as the first child
textNode := &xmlquery.Node{
Type: xmlquery.TextNode,
Data: newValue,
}
node.FirstChild = textNode
}
} else {
// Complex case: node has mixed content or child elements
// Replace just the text content while preserving child elements
// This is a simplified approach - more complex XML may need more robust handling
for child := node.FirstChild; child != nil; child = child.NextSibling {
if child.Type == xmlquery.TextNode {
child.Data = newValue
break // Update only the first text node
}
}
}
// Apply modification based on the result
if updatedValue, ok := result.(string); ok {
// If the result is a simple string, update the node value directly
xpath.Set(doc, path, updatedValue)
} else if nodeData, ok := result.(map[string]interface{}); ok {
// If the result is a map, apply more complex updates
updateNodeFromMap(node, nodeData)
}
modCount++
@@ -169,49 +84,329 @@ func (p *XMLProcessor) ProcessContent(content string, pattern string, luaExpr st
declaration := doc.FirstChild.OutputXML(true)
// Remove the firstChild (declaration) before serializing the rest of the document
doc.FirstChild = doc.FirstChild.NextSibling
return declaration + doc.OutputXML(true), modCount, matchCount, nil
return ConvertToNamedEntities(declaration + doc.OutputXML(true)), modCount, matchCount, nil
}
return doc.OutputXML(true), modCount, matchCount, nil
// Convert numeric entities to named entities for better readability
return ConvertToNamedEntities(doc.OutputXML(true)), modCount, matchCount, nil
}
func (p *XMLProcessor) ToLua(L *lua.LState, data interface{}) error {
table, err := p.ToLuaTable(L, data)
if err != nil {
return err
}
L.SetGlobal("v", table)
return nil
}
// ToLua converts XML node values to Lua variables
func (p *XMLProcessor) ToLua(L *lua.LState, data interface{}) error {
value, ok := data.(string)
func (p *XMLProcessor) ToLuaTable(L *lua.LState, data interface{}) (lua.LValue, error) {
// Check if data is an xmlquery.Node
node, ok := data.(*xmlquery.Node)
if !ok {
return fmt.Errorf("expected string value, got %T", data)
return nil, fmt.Errorf("expected xmlquery.Node, got %T", data)
}
// Set as string variable
L.SetGlobal("s1", lua.LString(value))
// Create a simple table with essential data
table := L.NewTable()
// Try to convert to number if possible
L.SetGlobal("v1", lua.LNumber(0)) // Default to 0
if err := L.DoString(fmt.Sprintf("v1 = tonumber(%q) or 0", value)); err != nil {
return fmt.Errorf("error converting value to number: %v", err)
// For element nodes, just provide basic info
L.SetField(table, "type", lua.LString(nodeTypeToString(node.Type)))
L.SetField(table, "name", lua.LString(node.Data))
L.SetField(table, "value", lua.LString(node.InnerText()))
// Add children if any
children := L.NewTable()
for child := node.FirstChild; child != nil; child = child.NextSibling {
childTable, err := p.ToLuaTable(L, child)
if err == nil {
children.Append(childTable)
}
}
L.SetField(table, "children", children)
return nil
attrs := L.NewTable()
if len(node.Attr) > 0 {
for _, attr := range node.Attr {
L.SetField(attrs, attr.Name.Local, lua.LString(attr.Value))
}
}
L.SetField(table, "attr", attrs)
return table, nil
}
// FromLua gets modified values from Lua
func (p *XMLProcessor) FromLua(L *lua.LState) (interface{}, error) {
// Check if string variable was modified
s1 := L.GetGlobal("s1")
if s1 != lua.LNil {
if s1Str, ok := s1.(lua.LString); ok {
return string(s1Str), nil
}
luaValue := L.GetGlobal("v")
// Handle string values directly
if luaValue.Type() == lua.LTString {
return luaValue.String(), nil
}
// Check if numeric variable was modified
v1 := L.GetGlobal("v1")
if v1 != lua.LNil {
if v1Num, ok := v1.(lua.LNumber); ok {
return fmt.Sprintf("%v", v1Num), nil
}
// Handle tables (for attributes and more complex updates)
if luaValue.Type() == lua.LTTable {
return luaTableToMap(L, luaValue.(*lua.LTable)), nil
}
// Default return empty string
return "", nil
return luaValue.String(), nil
}
// Simple helper to convert a Lua table to a Go map
func luaTableToMap(L *lua.LState, table *lua.LTable) map[string]interface{} {
result := make(map[string]interface{})
table.ForEach(func(k, v lua.LValue) {
if k.Type() == lua.LTString {
key := k.String()
if v.Type() == lua.LTTable {
result[key] = luaTableToMap(L, v.(*lua.LTable))
} else {
result[key] = v.String()
}
}
})
return result
}
// Simple helper to convert node type to string
func nodeTypeToString(nodeType xmlquery.NodeType) string {
switch nodeType {
case xmlquery.ElementNode:
return "element"
case xmlquery.TextNode:
return "text"
case xmlquery.AttributeNode:
return "attribute"
default:
return "other"
}
}
// Helper function to update an XML node from a map
func updateNodeFromMap(node *xmlquery.Node, data map[string]interface{}) {
// Update node value if present
if value, ok := data["value"]; ok {
if strValue, ok := value.(string); ok {
// For element nodes, replace text content
if node.Type == xmlquery.ElementNode {
// Find the first text child if it exists
var textNode *xmlquery.Node
for child := node.FirstChild; child != nil; child = child.NextSibling {
if child.Type == xmlquery.TextNode {
textNode = child
break
}
}
if textNode != nil {
// Update existing text node
textNode.Data = strValue
} else {
// Create new text node
newText := &xmlquery.Node{
Type: xmlquery.TextNode,
Data: strValue,
Parent: node,
}
// Insert at beginning of children
if node.FirstChild != nil {
newText.NextSibling = node.FirstChild
node.FirstChild.PrevSibling = newText
node.FirstChild = newText
} else {
node.FirstChild = newText
node.LastChild = newText
}
}
} else if node.Type == xmlquery.TextNode {
// Directly update text node
node.Data = strValue
} else if node.Type == xmlquery.AttributeNode {
// Update attribute value
if node.Parent != nil {
for i, attr := range node.Parent.Attr {
if attr.Name.Local == node.Data {
node.Parent.Attr[i].Value = strValue
break
}
}
}
}
}
}
// Update attributes if present
if attrs, ok := data["attr"].(map[string]interface{}); ok && node.Type == xmlquery.ElementNode {
for name, value := range attrs {
if strValue, ok := value.(string); ok {
// Look for existing attribute
found := false
for i, attr := range node.Attr {
if attr.Name.Local == name {
node.Attr[i].Value = strValue
found = true
break
}
}
// Add new attribute if not found
if !found {
node.Attr = append(node.Attr, xmlquery.Attr{
Name: struct {
Space, Local string
}{Local: name},
Value: strValue,
})
}
}
}
}
}
// Helper function to get a string representation of node type
func nodeTypeName(nodeType xmlquery.NodeType) string {
switch nodeType {
case xmlquery.ElementNode:
return "element"
case xmlquery.TextNode:
return "text"
case xmlquery.AttributeNode:
return "attribute"
case xmlquery.CommentNode:
return "comment"
case xmlquery.DeclarationNode:
return "declaration"
default:
return "unknown"
}
}
// ConvertToNamedEntities replaces numeric XML entities with their named counterparts
func ConvertToNamedEntities(xml string) string {
// Basic XML entities
replacements := map[string]string{
// Basic XML entities
"&#34;": "&quot;", // double quote
"&#39;": "&apos;", // single quote
"&#60;": "&lt;", // less than
"&#62;": "&gt;", // greater than
"&#38;": "&amp;", // ampersand
// Common symbols
"&#160;": "&nbsp;", // non-breaking space
"&#169;": "&copy;", // copyright
"&#174;": "&reg;", // registered trademark
"&#8364;": "&euro;", // euro
"&#163;": "&pound;", // pound
"&#165;": "&yen;", // yen
"&#162;": "&cent;", // cent
"&#167;": "&sect;", // section
"&#8482;": "&trade;", // trademark
"&#9824;": "&spades;", // spade
"&#9827;": "&clubs;", // club
"&#9829;": "&hearts;", // heart
"&#9830;": "&diams;", // diamond
// Special characters
"&#161;": "&iexcl;", // inverted exclamation
"&#191;": "&iquest;", // inverted question
"&#171;": "&laquo;", // left angle quotes
"&#187;": "&raquo;", // right angle quotes
"&#183;": "&middot;", // middle dot
"&#8226;": "&bull;", // bullet
"&#8230;": "&hellip;", // horizontal ellipsis
"&#8242;": "&prime;", // prime
"&#8243;": "&Prime;", // double prime
"&#8254;": "&oline;", // overline
"&#8260;": "&frasl;", // fraction slash
// Math symbols
"&#177;": "&plusmn;", // plus-minus
"&#215;": "&times;", // multiplication
"&#247;": "&divide;", // division
"&#8734;": "&infin;", // infinity
"&#8776;": "&asymp;", // almost equal
"&#8800;": "&ne;", // not equal
"&#8804;": "&le;", // less than or equal
"&#8805;": "&ge;", // greater than or equal
"&#8721;": "&sum;", // summation
"&#8730;": "&radic;", // square root
"&#8747;": "&int;", // integral
// Accented characters
"&#192;": "&Agrave;", // A grave
"&#193;": "&Aacute;", // A acute
"&#194;": "&Acirc;", // A circumflex
"&#195;": "&Atilde;", // A tilde
"&#196;": "&Auml;", // A umlaut
"&#197;": "&Aring;", // A ring
"&#198;": "&AElig;", // AE ligature
"&#199;": "&Ccedil;", // C cedilla
"&#200;": "&Egrave;", // E grave
"&#201;": "&Eacute;", // E acute
"&#202;": "&Ecirc;", // E circumflex
"&#203;": "&Euml;", // E umlaut
"&#204;": "&Igrave;", // I grave
"&#205;": "&Iacute;", // I acute
"&#206;": "&Icirc;", // I circumflex
"&#207;": "&Iuml;", // I umlaut
"&#208;": "&ETH;", // Eth
"&#209;": "&Ntilde;", // N tilde
"&#210;": "&Ograve;", // O grave
"&#211;": "&Oacute;", // O acute
"&#212;": "&Ocirc;", // O circumflex
"&#213;": "&Otilde;", // O tilde
"&#214;": "&Ouml;", // O umlaut
"&#216;": "&Oslash;", // O slash
"&#217;": "&Ugrave;", // U grave
"&#218;": "&Uacute;", // U acute
"&#219;": "&Ucirc;", // U circumflex
"&#220;": "&Uuml;", // U umlaut
"&#221;": "&Yacute;", // Y acute
"&#222;": "&THORN;", // Thorn
"&#223;": "&szlig;", // Sharp s
"&#224;": "&agrave;", // a grave
"&#225;": "&aacute;", // a acute
"&#226;": "&acirc;", // a circumflex
"&#227;": "&atilde;", // a tilde
"&#228;": "&auml;", // a umlaut
"&#229;": "&aring;", // a ring
"&#230;": "&aelig;", // ae ligature
"&#231;": "&ccedil;", // c cedilla
"&#232;": "&egrave;", // e grave
"&#233;": "&eacute;", // e acute
"&#234;": "&ecirc;", // e circumflex
"&#235;": "&euml;", // e umlaut
"&#236;": "&igrave;", // i grave
"&#237;": "&iacute;", // i acute
"&#238;": "&icirc;", // i circumflex
"&#239;": "&iuml;", // i umlaut
"&#240;": "&eth;", // eth
"&#241;": "&ntilde;", // n tilde
"&#242;": "&ograve;", // o grave
"&#243;": "&oacute;", // o acute
"&#244;": "&ocirc;", // o circumflex
"&#245;": "&otilde;", // o tilde
"&#246;": "&ouml;", // o umlaut
"&#248;": "&oslash;", // o slash
"&#249;": "&ugrave;", // u grave
"&#250;": "&uacute;", // u acute
"&#251;": "&ucirc;", // u circumflex
"&#252;": "&uuml;", // u umlaut
"&#253;": "&yacute;", // y acute
"&#254;": "&thorn;", // thorn
"&#255;": "&yuml;", // y umlaut
}
result := xml
for numeric, named := range replacements {
result = strings.ReplaceAll(result, numeric, named)
}
return result
}

View File

@@ -5,13 +5,21 @@ import (
"testing"
"regexp"
"github.com/antchfx/xmlquery"
lua "github.com/yuin/gopher-lua"
)
// Helper function to normalize whitespace for comparison
func normalizeXMLWhitespace(s string) string {
// Replace all whitespace sequences with a single space
re := regexp.MustCompile(`\s+`)
return re.ReplaceAllString(strings.TrimSpace(s), " ")
s = re.ReplaceAllString(strings.TrimSpace(s), " ")
// Normalize XML entities for comparison
s = ConvertToNamedEntities(s)
return s
}
func TestXMLProcessor_Process_NodeValues(t *testing.T) {
@@ -39,7 +47,7 @@ func TestXMLProcessor_Process_NodeValues(t *testing.T) {
<catalog>
<book id="bk101">
<author>Gambardella, Matthew</author>
<title>XML Developer's Guide</title>
<title>XML Developer&apos;s Guide</title>
<genre>Computer</genre>
<price>89.9</price>
<publish_date>2000-10-01</publish_date>
@@ -56,7 +64,7 @@ func TestXMLProcessor_Process_NodeValues(t *testing.T) {
</catalog>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -93,7 +101,7 @@ func TestXMLProcessor_Process_Attributes(t *testing.T) {
</items>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//item/@price", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//item/@price", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -130,7 +138,7 @@ func TestXMLProcessor_Process_ElementText(t *testing.T) {
</names>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//n/text()", "v = string.upper(v)")
result, modCount, matchCount, err := p.ProcessContent(content, "//n/text()", "v.value = string.upper(v.value)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -171,7 +179,7 @@ func TestXMLProcessor_Process_ElementAddition(t *testing.T) {
</config>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//settings/*", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//settings/*", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -236,7 +244,7 @@ func TestXMLProcessor_Process_ComplexXML(t *testing.T) {
</store>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v = v * 1.2")
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v.value = round(v.value * 1.2, 3)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -271,19 +279,21 @@ func TestXMLProcessor_ConditionalModification(t *testing.T) {
expected := `<?xml version="1.0" encoding="UTF-8"?>
<inventory>
<item id="1" stock="5" price="8.00"/>
<item id="2" stock="15" price="16.00"/>
<item id="3" stock="0" price="15.00"/>
<item id="1" stock="5" price="8.00"></item>
<item id="2" stock="15" price="16.00"></item>
<item id="3" stock="0" price="15.00"></item>
</inventory>`
p := &XMLProcessor{}
// Apply 20% discount but only for items with stock > 0
luaExpr := `
-- In the table-based approach, attributes are accessible directly
if v.stock and tonumber(v.stock) > 0 then
v.price = tonumber(v.price) * 0.8
if v.attr.stock and tonumber(v.attr.stock) > 0 then
v.attr.price = tonumber(v.attr.price) * 0.8
-- Format to 2 decimal places
v.price = string.format("%.2f", v.price)
v.attr.price = string.format("%.2f", v.attr.price)
else
return false
end
`
result, modCount, matchCount, err := p.ProcessContent(content, "//item", luaExpr)
@@ -327,7 +337,7 @@ func TestXMLProcessor_Process_SpecialCharacters(t *testing.T) {
</data>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//entry", "v = string.upper(v)")
result, modCount, matchCount, err := p.ProcessContent(content, "//entry", "v.value = string.upper(v.value)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -362,15 +372,14 @@ func TestXMLProcessor_Process_ChainedOperations(t *testing.T) {
// Apply multiple operations to the price: add tax, apply discount, round
luaExpr := `
-- When v is a numeric string, we can perform math operations directly
local price = v
local price = v.value
-- Add 15% tax
price = price * 1.15
-- Apply 10% discount
price = price * 0.9
-- Round to 2 decimal places
price = math.floor(price * 100 + 0.5) / 100
v = price
price = round(price, 2)
v.value = price
`
expected := `<?xml version="1.0" encoding="UTF-8"?>
@@ -422,7 +431,7 @@ func TestXMLProcessor_Process_MathFunctions(t *testing.T) {
</measurements>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//measurement", "v = round(v)")
result, modCount, matchCount, err := p.ProcessContent(content, "//measurement", "v.value = round(v.value)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -468,9 +477,9 @@ func TestXMLProcessor_Process_StringOperations(t *testing.T) {
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//email", `
-- With the table approach, v contains the text content directly
v = string.gsub(v, "@.+", "@anon.com")
local username = string.match(v, "(.+)@")
v = string.gsub(username, "%.", "") .. "@anon.com"
v.value = string.gsub(v.value, "@.+", "@anon.com")
local username = string.match(v.value, "(.+)@")
v.value = string.gsub(username, "%.", "") .. "@anon.com"
`)
if err != nil {
@@ -479,7 +488,7 @@ func TestXMLProcessor_Process_StringOperations(t *testing.T) {
// Test phone number masking
result, modCount2, matchCount2, err := p.ProcessContent(result, "//phone", `
v = string.gsub(v, "%d%d%d%-%d%d%d%-%d%d%d%d", function(match)
v.value = string.gsub(v.value, "%d%d%d%-%d%d%d%-%d%d%d%d", function(match)
return string.sub(match, 1, 3) .. "-XXX-XXXX"
end)
`)
@@ -536,14 +545,14 @@ func TestXMLProcessor_Process_DateManipulation(t *testing.T) {
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//date", `
local year, month, day = string.match(v, "(%d%d%d%d)-(%d%d)-(%d%d)")
local year, month, day = string.match(v.value, "(%d%d%d%d)-(%d%d)-(%d%d)")
-- Postpone events by 1 month
month = tonumber(month) + 1
if month > 12 then
month = 1
year = tonumber(year) + 1
end
v = string.format("%04d-%02d-%s", tonumber(year), month, day)
v.value = string.format("%04d-%02d-%s", tonumber(year), month, day)
`)
if err != nil {
@@ -609,36 +618,6 @@ func TestXMLProcessor_Process_Error_InvalidLua(t *testing.T) {
}
}
func TestXMLProcessor_Process_NoChanges(t *testing.T) {
content := `<?xml version="1.0" encoding="UTF-8"?>
<root>
<element>123</element>
</root>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//element", "v1 = v1")
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matchCount != 1 {
t.Errorf("Expected 1 match, got %d", matchCount)
}
if modCount != 0 {
t.Errorf("Expected 0 modifications, got %d", modCount)
}
// Normalize whitespace for comparison
normalizedResult := normalizeXMLWhitespace(result)
normalizedContent := normalizeXMLWhitespace(content)
if normalizedResult != normalizedContent {
t.Errorf("Expected content to be unchanged")
}
}
func TestXMLProcessor_Process_ComplexXPathSelectors(t *testing.T) {
content := `<?xml version="1.0" encoding="UTF-8"?>
<library>
@@ -684,7 +663,7 @@ func TestXMLProcessor_Process_ComplexXPathSelectors(t *testing.T) {
p := &XMLProcessor{}
// Target only fiction books and apply 20% discount to price
result, modCount, matchCount, err := p.ProcessContent(content, "//book[@category='fiction']/price", "v = v * 0.8")
result, modCount, matchCount, err := p.ProcessContent(content, "//book[@category='fiction']/price", "v.value = round(v.value * 0.8, 2)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -767,13 +746,13 @@ func TestXMLProcessor_Process_NestedStructureModification(t *testing.T) {
p := &XMLProcessor{}
// Boost hero stats by 20%
result, modCount, matchCount, err := p.ProcessContent(content, "//character[@id='hero']/stats/*", "v = math.floor(v * 1.2)")
result, modCount, matchCount, err := p.ProcessContent(content, "//character[@id='hero']/stats/*", "v.value = round(v.value * 1.2)")
if err != nil {
t.Fatalf("Error processing stats content: %v", err)
}
// Also upgrade hero equipment
result, modCount2, matchCount2, err := p.ProcessContent(result, "//character[@id='hero']/equipment/*/@damage|//character[@id='hero']/equipment/*/@defense", "v = v + 2")
result, modCount2, matchCount2, err := p.ProcessContent(result, "//character[@id='hero']/equipment/*/@damage|//character[@id='hero']/equipment/*/@defense", "v.value = v.value + 2")
if err != nil {
t.Fatalf("Error processing equipment content: %v", err)
}
@@ -835,8 +814,8 @@ func TestXMLProcessor_Process_ElementReplacement(t *testing.T) {
luaExpr := `
-- With a proper table approach, this becomes much simpler
local price = tonumber(v.price)
local quantity = tonumber(v.quantity)
local price = tonumber(v.attr.price)
local quantity = tonumber(v.attr.quantity)
-- Add a new total element
v.total = string.format("%.2f", price * quantity)
@@ -902,11 +881,11 @@ func TestXMLProcessor_Process_AttributeAddition(t *testing.T) {
-- We can access the "inStock" element directly
if v.inStock == "true" then
-- Add a new attribute directly
v._attr = v._attr or {}
v._attr.status = "available"
v.attr = v.attr or {}
v.attr.status = "available"
else
v._attr = v._attr or {}
v._attr.status = "out-of-stock"
v.attr = v.attr or {}
v.attr.status = "out-of-stock"
end
`
@@ -1031,9 +1010,9 @@ func TestXMLProcessor_Process_ElementReordering(t *testing.T) {
luaExpr := `
-- With table approach, we can reorder elements by redefining the table
-- Store the values
local artist = v.artist
local title = v.title
local year = v.year
local artist = v.attr.artist
local title = v.attr.title
local year = v.attr.year
-- Clear the table
for k in pairs(v) do
@@ -1041,9 +1020,9 @@ func TestXMLProcessor_Process_ElementReordering(t *testing.T) {
end
-- Add elements in the desired order
v.title = title
v.artist = artist
v.year = year
v.attr.title = title
v.attr.artist = artist
v.attr.year = year
`
result, modCount, matchCount, err := p.ProcessContent(content, "//song", luaExpr)
@@ -1194,13 +1173,13 @@ func TestXMLProcessor_Process_DynamicXPath(t *testing.T) {
expected := `<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<settings>
<setting name="timeout" value="60" />
<setting name="retries" value="3" />
<setting name="backoff" value="exponential" />
<setting name="timeout" value="60"></setting>
<setting name="retries" value="3"></setting>
<setting name="backoff" value="exponential"></setting>
</settings>
<advanced>
<setting name="logging" value="debug" />
<setting name="timeout" value="120" />
<setting name="logging" value="debug"></setting>
<setting name="timeout" value="120"></setting>
</advanced>
</configuration>`
@@ -1208,7 +1187,7 @@ func TestXMLProcessor_Process_DynamicXPath(t *testing.T) {
p := &XMLProcessor{}
// Double all timeout values in the configuration
result, modCount, matchCount, err := p.ProcessContent(content, "//setting[@name='timeout']/@value", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//setting[@name='timeout']/@value", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -1263,34 +1242,34 @@ func TestXMLProcessor_Process_TableBasedStructureCreation(t *testing.T) {
local summary = ""
-- Process each child option
if v.settings and v.settings.option then
local options = v.settings.option
-- If there's just one option, wrap it in a table
if options._attr then
options = {options}
end
for i, opt in ipairs(options) do
count = count + 1
if opt._attr.name == "debug" then
summary = summary .. "Debug: " .. (opt._attr.value == "true" and "ON" or "OFF")
elseif opt._attr.name == "log_level" then
summary = summary .. "Logging: " .. opt._attr.value
end
if i < #options then
summary = summary .. ", "
end
end
end
local settings = v.children[1]
local options = settings.children
-- if settings and options then
-- if options.attr then
-- options = {options}
-- end
--
-- for i, opt in ipairs(options) do
-- count = count + 1
-- if opt.attr.name == "debug" then
-- summary = summary .. "Debug: " .. (opt.attr.value == "true" and "ON" or "OFF")
-- elseif opt.attr.name == "log_level" then
-- summary = summary .. "Logging: " .. opt.attr.value
-- end
--
-- if i < #options then
-- summary = summary .. ", "
-- end
-- end
-- end
-- Create a new calculated section
v.calculated = {
stats = {
count = tostring(count),
summary = summary
}
}
-- v.children[2] = {
-- stats = {
-- count = tostring(count),
-- summary = summary
-- }
-- }
`
result, modCount, matchCount, err := p.ProcessContent(content, "/data", luaExpr)
@@ -1530,3 +1509,267 @@ func TestXMLProcessor_Process_DeepPathNavigation(t *testing.T) {
// Add more test cases for specific XML manipulation scenarios below
// These tests would cover additional functionality as the implementation progresses
func TestXMLToLua(t *testing.T) {
// Sample XML to test with
xmlStr := `
<root id="1">
<person name="John" age="30">
<address type="home">
<street>123 Main St</street>
<city>Anytown</city>
<zip>12345</zip>
</address>
<contact type="email">john@example.com</contact>
</person>
<person name="Jane" age="28">
<address type="work">
<street>456 Business Ave</street>
<city>Worktown</city>
<zip>54321</zip>
</address>
<contact type="phone">555-1234</contact>
</person>
</root>
`
// Parse the XML
doc, err := xmlquery.Parse(strings.NewReader(xmlStr))
if err != nil {
t.Fatalf("Failed to parse XML: %v", err)
}
// Create a new Lua state
L := lua.NewState()
defer L.Close()
// Create an XML processor
processor := &XMLProcessor{}
// Test converting the root element to Lua
t.Run("RootElement", func(t *testing.T) {
// Find the root element
root := doc.SelectElement("root")
if root == nil {
t.Fatal("Failed to find root element")
}
// Convert to Lua
err := processor.ToLua(L, root)
if err != nil {
t.Fatalf("Failed to convert to Lua: %v", err)
}
// Verify the result
luaTable := L.GetGlobal("v")
if luaTable.Type() != lua.LTTable {
t.Fatalf("Expected table, got %s", luaTable.Type().String())
}
// Check element type
typeVal := L.GetField(luaTable, "type")
if typeVal.String() != "element" {
t.Errorf("Expected type 'element', got '%s'", typeVal.String())
}
// Check name
nameVal := L.GetField(luaTable, "name")
if nameVal.String() != "root" {
t.Errorf("Expected name 'root', got '%s'", nameVal.String())
}
// Check attributes
attrsTable := L.GetField(luaTable, "attributes")
if attrsTable.Type() != lua.LTTable {
t.Fatalf("Expected attributes table, got %s", attrsTable.Type().String())
}
idVal := L.GetField(attrsTable, "id")
if idVal.String() != "1" {
t.Errorf("Expected id '1', got '%s'", idVal.String())
}
// Check that we have children
childrenTable := L.GetField(luaTable, "children")
if childrenTable.Type() != lua.LTTable {
t.Fatalf("Expected children table, got %s", childrenTable.Type().String())
}
})
// Test converting a nested element to Lua
t.Run("NestedElement", func(t *testing.T) {
// Find a nested element
street := doc.SelectElement("//street")
if street == nil {
t.Fatal("Failed to find street element")
}
// Convert to Lua
err := processor.ToLua(L, street)
if err != nil {
t.Fatalf("Failed to convert to Lua: %v", err)
}
// Verify the result
luaTable := L.GetGlobal("v")
if luaTable.Type() != lua.LTTable {
t.Fatalf("Expected table, got %s", luaTable.Type().String())
}
// Check element type
typeVal := L.GetField(luaTable, "type")
if typeVal.String() != "element" {
t.Errorf("Expected type 'element', got '%s'", typeVal.String())
}
// Check name
nameVal := L.GetField(luaTable, "name")
if nameVal.String() != "street" {
t.Errorf("Expected name 'street', got '%s'", nameVal.String())
}
// Check value
valueVal := L.GetField(luaTable, "value")
if valueVal.String() != "123 Main St" {
t.Errorf("Expected value '123 Main St', got '%s'", valueVal.String())
}
})
// Test FromLua with a simple string update
t.Run("FromLuaString", func(t *testing.T) {
// Set up a Lua state with a string value
L := lua.NewState()
defer L.Close()
L.SetGlobal("v", lua.LString("New Value"))
// Convert from Lua
result, err := processor.FromLua(L)
if err != nil {
t.Fatalf("Failed to convert from Lua: %v", err)
}
// Verify the result
strResult, ok := result.(string)
if !ok {
t.Fatalf("Expected string result, got %T", result)
}
if strResult != "New Value" {
t.Errorf("Expected 'New Value', got '%s'", strResult)
}
})
// Test FromLua with a complex table update
t.Run("FromLuaTable", func(t *testing.T) {
// Set up a Lua state with a table value
L := lua.NewState()
defer L.Close()
table := L.NewTable()
L.SetField(table, "value", lua.LString("Updated Text"))
attrTable := L.NewTable()
L.SetField(attrTable, "id", lua.LString("new-id"))
L.SetField(attrTable, "class", lua.LString("highlight"))
L.SetField(table, "attributes", attrTable)
L.SetGlobal("v", table)
// Convert from Lua
result, err := processor.FromLua(L)
if err != nil {
t.Fatalf("Failed to convert from Lua: %v", err)
}
// Verify the result
mapResult, ok := result.(map[string]interface{})
if !ok {
t.Fatalf("Expected map result, got %T", result)
}
// Check value
if value, ok := mapResult["value"]; !ok || value != "Updated Text" {
t.Errorf("Expected value 'Updated Text', got '%v'", value)
}
// Check attributes
attrs, ok := mapResult["attributes"].(map[string]interface{})
if !ok {
t.Fatalf("Expected attributes map, got %T", mapResult["attributes"])
}
if id, ok := attrs["id"]; !ok || id != "new-id" {
t.Errorf("Expected id 'new-id', got '%v'", id)
}
if class, ok := attrs["class"]; !ok || class != "highlight" {
t.Errorf("Expected class 'highlight', got '%v'", class)
}
})
// Test updateNodeFromMap with a simple value update
t.Run("UpdateNodeValue", func(t *testing.T) {
// Create a simple element to update
xmlStr := `<test>Original Text</test>`
doc, _ := xmlquery.Parse(strings.NewReader(xmlStr))
node := doc.SelectElement("test")
// Create update data
updateData := map[string]interface{}{
"value": "Updated Text",
}
// Update the node
updateNodeFromMap(node, updateData)
// Verify the update
if node.InnerText() != "Updated Text" {
t.Errorf("Expected value 'Updated Text', got '%s'", node.InnerText())
}
})
// Test updateNodeFromMap with attribute updates
t.Run("UpdateNodeAttributes", func(t *testing.T) {
// Create an element with attributes
xmlStr := `<test id="old">Text</test>`
doc, _ := xmlquery.Parse(strings.NewReader(xmlStr))
node := doc.SelectElement("test")
// Create update data
updateData := map[string]interface{}{
"attributes": map[string]interface{}{
"id": "new",
"class": "added",
},
}
// Update the node
updateNodeFromMap(node, updateData)
// Verify the id attribute was updated
idFound := false
classFound := false
for _, attr := range node.Attr {
if attr.Name.Local == "id" {
idFound = true
if attr.Value != "new" {
t.Errorf("Expected id 'new', got '%s'", attr.Value)
}
}
if attr.Name.Local == "class" {
classFound = true
if attr.Value != "added" {
t.Errorf("Expected class 'added', got '%s'", attr.Value)
}
}
}
if !idFound {
t.Error("Expected to find 'id' attribute but didn't")
}
if !classFound {
t.Error("Expected to find 'class' attribute but didn't")
}
})
}

View File

@@ -0,0 +1,4 @@
// The package is now using github.com/antchfx/xmlquery for XPath parsing.
// The parsing functionality tests have been removed since we're now
// delegating XPath parsing to the xmlquery library.
package xpath

View File

@@ -0,0 +1,4 @@
// The package is now using github.com/antchfx/xmlquery for XPath parsing.
// The parsing functionality tests have been removed since we're now
// delegating XPath parsing to the xmlquery library.
package xpath

133
processor/xpath/xpath.go Normal file
View File

@@ -0,0 +1,133 @@
package xpath
import (
"errors"
"fmt"
"github.com/antchfx/xmlquery"
)
// Get retrieves nodes from XML data using an XPath expression
func Get(node *xmlquery.Node, path string) ([]*xmlquery.Node, error) {
if node == nil {
return nil, errors.New("nil node provided")
}
// Execute xpath query directly
nodes, err := xmlquery.QueryAll(node, path)
if err != nil {
return nil, fmt.Errorf("failed to execute XPath query: %v", err)
}
return nodes, nil
}
// Set updates a single node in the XML data using an XPath expression
func Set(node *xmlquery.Node, path string, value interface{}) error {
if node == nil {
return errors.New("nil node provided")
}
// Find the node to update
nodes, err := xmlquery.QueryAll(node, path)
if err != nil {
return fmt.Errorf("failed to execute XPath query: %v", err)
}
if len(nodes) == 0 {
return fmt.Errorf("no nodes found for path: %s", path)
}
// Update the first matching node
updateNodeValue(nodes[0], value)
return nil
}
// SetAll updates all nodes that match the XPath expression
func SetAll(node *xmlquery.Node, path string, value interface{}) error {
if node == nil {
return errors.New("nil node provided")
}
// Find all nodes to update
nodes, err := xmlquery.QueryAll(node, path)
if err != nil {
return fmt.Errorf("failed to execute XPath query: %v", err)
}
if len(nodes) == 0 {
return fmt.Errorf("no nodes found for path: %s", path)
}
// Update all matching nodes
for _, matchNode := range nodes {
updateNodeValue(matchNode, value)
}
return nil
}
// Helper function to update a node's value
func updateNodeValue(node *xmlquery.Node, value interface{}) {
strValue := fmt.Sprintf("%v", value)
// Handle different node types
switch node.Type {
case xmlquery.AttributeNode:
// For attribute nodes, update the attribute value
parent := node.Parent
if parent != nil {
for i, attr := range parent.Attr {
if attr.Name.Local == node.Data {
parent.Attr[i].Value = strValue
break
}
}
}
case xmlquery.TextNode:
// For text nodes, update the text content
node.Data = strValue
case xmlquery.ElementNode:
// For element nodes, clear existing text children and add a new text node
// First, remove all existing text children
var nonTextChildren []*xmlquery.Node
for child := node.FirstChild; child != nil; child = child.NextSibling {
if child.Type != xmlquery.TextNode {
nonTextChildren = append(nonTextChildren, child)
}
}
// Clear all children
node.FirstChild = nil
node.LastChild = nil
// Add a new text node
textNode := &xmlquery.Node{
Type: xmlquery.TextNode,
Data: strValue,
Parent: node,
}
// Set the text node as the first child
node.FirstChild = textNode
node.LastChild = textNode
// Add back non-text children
for _, child := range nonTextChildren {
child.Parent = node
// If this is the first child being added back
if node.FirstChild == textNode && node.LastChild == textNode {
node.FirstChild.NextSibling = child
child.PrevSibling = node.FirstChild
node.LastChild = child
} else {
// Add to the end of the chain
node.LastChild.NextSibling = child
child.PrevSibling = node.LastChild
node.LastChild = child
}
}
}
}

View File

@@ -0,0 +1,474 @@
package xpath
import (
"strings"
"testing"
"github.com/antchfx/xmlquery"
)
// Parse test XML data once at the beginning for use in multiple tests
func parseTestXML(t *testing.T, xmlData string) *xmlquery.Node {
doc, err := xmlquery.Parse(strings.NewReader(xmlData))
if err != nil {
t.Fatalf("Failed to parse test XML: %v", err)
}
return doc
}
// XML test data as a string for our tests
var testXML = `
<store>
<book category="fiction">
<title lang="en">The Fellowship of the Ring</title>
<author>J.R.R. Tolkien</author>
<year>1954</year>
<price>22.99</price>
</book>
<book category="fiction">
<title lang="en">The Two Towers</title>
<author>J.R.R. Tolkien</author>
<year>1954</year>
<price>23.45</price>
</book>
<book category="technical">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
<bicycle>
<color>red</color>
<price>199.95</price>
</bicycle>
</store>
`
func TestEvaluator(t *testing.T) {
// Parse the test XML data once for all test cases
doc := parseTestXML(t, testXML)
tests := []struct {
name string
path string
error bool
}{
{
name: "simple_element_access",
path: "/store/bicycle/color",
},
{
name: "recursive_element_access",
path: "//price",
},
{
name: "wildcard_element_access",
path: "/store/book/*",
},
{
name: "attribute_exists_predicate",
path: "//title[@lang]",
},
{
name: "attribute_equals_predicate",
path: "//title[@lang='en']",
},
{
name: "value_comparison_predicate",
path: "/store/book[price>35.00]/title",
error: true,
},
{
name: "last_predicate",
path: "/store/book[last()]/title",
error: true,
},
{
name: "last_minus_predicate",
path: "/store/book[last()-1]/title",
error: true,
},
{
name: "position_predicate",
path: "/store/book[position()<3]/title",
error: true,
},
{
name: "invalid_index",
path: "/store/book[10]/title",
error: true,
},
{
name: "nonexistent_element",
path: "/store/nonexistent",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := Get(doc, tt.path)
// Handle expected errors
if tt.error {
if err == nil && len(result) == 0 {
// If we expected an error but got empty results instead, that's okay
return
}
if err != nil {
// If we got an error as expected, that's okay
return
}
} else if err != nil {
// If we didn't expect an error but got one, that's a test failure
t.Errorf("Get(%q) returned unexpected error: %v", tt.path, err)
return
}
// Special cases where we don't care about exact matches
switch tt.name {
case "wildcard_element_access":
// Just check that we got some elements
if len(result) == 0 {
t.Errorf("Expected multiple elements for wildcard, got none")
}
return
case "attribute_exists_predicate", "attribute_equals_predicate":
// Just check that we got some titles
if len(result) == 0 {
t.Errorf("Expected titles with lang attribute, got none")
}
// Ensure all are title elements
for _, node := range result {
if node.Data != "title" {
t.Errorf("Expected title elements, got: %s", node.Data)
}
}
return
case "nonexistent_element":
// Just check that we got empty results
if len(result) != 0 {
t.Errorf("Expected empty results for nonexistent element, got %d items", len(result))
}
return
}
// For other cases, just verify we got results
if len(result) == 0 {
t.Errorf("Expected results for path %s, got none", tt.path)
}
})
}
}
func TestEdgeCases(t *testing.T) {
t.Run("nil_node", func(t *testing.T) {
result, err := Get(nil, "/store/book")
if err == nil {
t.Errorf("Expected error for nil node")
return
}
if len(result) > 0 {
t.Errorf("Expected empty result, got %v", result)
}
})
t.Run("invalid_xml", func(t *testing.T) {
invalidXML, err := xmlquery.Parse(strings.NewReader("<invalid>xml"))
if err != nil {
// If parsing fails, that's expected
return
}
_, err = Get(invalidXML, "/store")
if err == nil {
t.Error("Expected error for invalid XML structure")
}
})
// For these tests with the simple XML, we expect just one result
simpleXML := `<root><book><title lang="en">Test</title></book></root>`
doc := parseTestXML(t, simpleXML)
t.Run("current_node", func(t *testing.T) {
result, err := Get(doc, "/root/book/.")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) > 1 {
t.Errorf("Expected at most 1 result, got %d", len(result))
}
if len(result) > 0 {
// Verify it's the book node
if result[0].Data != "book" {
t.Errorf("Expected book node, got %v", result[0].Data)
}
}
})
t.Run("attributes", func(t *testing.T) {
result, err := Get(doc, "/root/book/title/@lang")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 1 || result[0].InnerText() != "en" {
t.Errorf("Expected 'en', got %v", result[0].InnerText())
}
})
}
func TestGetWithPaths(t *testing.T) {
// Use a simplified, well-formed XML document
simpleXML := `<store>
<book category="fiction">
<title lang="en">The Book Title</title>
<author>Author Name</author>
<price>19.99</price>
</book>
<bicycle>
<color>red</color>
<price>199.95</price>
</bicycle>
</store>`
// Parse the XML for testing
doc := parseTestXML(t, simpleXML)
// Debug: Print the test XML
t.Logf("Test XML:\n%s", simpleXML)
tests := []struct {
name string
path string
expectedValue string
}{
{
name: "simple_element_access",
path: "/store/bicycle/color",
expectedValue: "red",
},
{
name: "attribute_access",
path: "/store/book/title/@lang",
expectedValue: "en",
},
{
name: "recursive_with_attribute",
path: "//title[@lang='en']",
expectedValue: "The Book Title",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Debug: Print the path we're looking for
t.Logf("Looking for path: %s", tt.path)
result, err := Get(doc, tt.path)
if err != nil {
t.Errorf("Get(%q) returned error: %v", tt.path, err)
return
}
// Debug: Print the results
t.Logf("Got %d results", len(result))
for i, r := range result {
t.Logf("Result %d: Node=%s, Value=%v", i, r.Data, r.InnerText())
}
// Check that we got results
if len(result) == 0 {
t.Errorf("Get(%q) returned no results", tt.path)
return
}
// For attribute access test, do more specific checks
if tt.name == "attribute_access" {
// Check the first result's value matches expected
if result[0].InnerText() != tt.expectedValue {
t.Errorf("Attribute value: got %v, expected %s", result[0].InnerText(), tt.expectedValue)
}
}
// For simple element access, check the text content
if tt.name == "simple_element_access" {
if text := result[0].InnerText(); text != tt.expectedValue {
t.Errorf("Element text: got %s, expected %s", text, tt.expectedValue)
}
}
// For recursive with attribute test, check title elements with lang="en"
if tt.name == "recursive_with_attribute" {
for _, node := range result {
// Check the node is a title
if node.Data != "title" {
t.Errorf("Expected title element, got %s", node.Data)
}
// Check text content
if text := node.InnerText(); text != tt.expectedValue {
t.Errorf("Text content: got %s, expected %s", text, tt.expectedValue)
}
// Check attributes - find the lang attribute
hasLang := false
for _, attr := range node.Attr {
if attr.Name.Local == "lang" && attr.Value == "en" {
hasLang = true
break
}
}
if !hasLang {
t.Errorf("Expected lang=\"en\" attribute, but it was not found")
}
}
}
})
}
}
func TestSet(t *testing.T) {
t.Run("simple element", func(t *testing.T) {
xmlData := `<root><name>John</name></root>`
doc := parseTestXML(t, xmlData)
err := Set(doc, "/root/name", "Jane")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}
// Verify the change
result, err := Get(doc, "/root/name")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 1 {
t.Errorf("Expected 1 result, got %d", len(result))
return
}
// Check text content
if text := result[0].InnerText(); text != "Jane" {
t.Errorf("Expected text 'Jane', got '%s'", text)
}
})
t.Run("attribute", func(t *testing.T) {
xmlData := `<root><element id="123"></element></root>`
doc := parseTestXML(t, xmlData)
err := Set(doc, "/root/element/@id", "456")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}
// Verify the change
result, err := Get(doc, "/root/element/@id")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 1 {
t.Errorf("Expected 1 result, got %d", len(result))
return
}
// For attributes, check the inner text
if text := result[0].InnerText(); text != "456" {
t.Errorf("Expected attribute value '456', got '%s'", text)
}
})
t.Run("indexed element", func(t *testing.T) {
xmlData := `<root><items><item>first</item><item>second</item></items></root>`
doc := parseTestXML(t, xmlData)
err := Set(doc, "/root/items/item[1]", "changed")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}
// Verify the change using XPath that specifically targets the first item
result, err := Get(doc, "/root/items/item[1]")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
// Check if we have results
if len(result) == 0 {
t.Errorf("Expected at least one result for /root/items/item[1]")
return
}
// Check text content
if text := result[0].InnerText(); text != "changed" {
t.Errorf("Expected text 'changed', got '%s'", text)
}
})
}
func TestSetAll(t *testing.T) {
t.Run("multiple elements", func(t *testing.T) {
xmlData := `<root><items><item>first</item><item>second</item></items></root>`
doc := parseTestXML(t, xmlData)
err := SetAll(doc, "//item", "changed")
if err != nil {
t.Errorf("SetAll() returned error: %v", err)
return
}
// Verify all items are changed
result, err := Get(doc, "//item")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 2 {
t.Errorf("Expected 2 results, got %d", len(result))
return
}
// Check each node
for i, node := range result {
if text := node.InnerText(); text != "changed" {
t.Errorf("Item %d: expected text 'changed', got '%s'", i, text)
}
}
})
t.Run("attributes", func(t *testing.T) {
xmlData := `<root><item id="1"/><item id="2"/></root>`
doc := parseTestXML(t, xmlData)
err := SetAll(doc, "//item/@id", "new")
if err != nil {
t.Errorf("SetAll() returned error: %v", err)
return
}
// Verify all attributes are changed
result, err := Get(doc, "//item/@id")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 2 {
t.Errorf("Expected 2 results, got %d", len(result))
return
}
// For attributes, check inner text
for i, node := range result {
if text := node.InnerText(); text != "new" {
t.Errorf("Attribute %d: expected value 'new', got '%s'", i, text)
}
}
})
}

48
release.sh Normal file
View File

@@ -0,0 +1,48 @@
#!/bin/bash
echo "Figuring out the tag..."
TAG=$(git describe --tags --exact-match 2>/dev/null || echo "")
if [ -z "$TAG" ]; then
# Get the latest tag
LATEST_TAG=$(git describe --tags $(git rev-list --tags --max-count=1))
# Increment the patch version
IFS='.' read -r -a VERSION_PARTS <<< "$LATEST_TAG"
VERSION_PARTS[2]=$((VERSION_PARTS[2]+1))
TAG="${VERSION_PARTS[0]}.${VERSION_PARTS[1]}.${VERSION_PARTS[2]}"
# Create a new tag
git tag $TAG
git push origin $TAG
fi
echo "Tag: $TAG"
echo "Building the thing..."
go build -o chef.exe .
echo "Creating a release..."
TOKEN="$GITEA_API_KEY"
GITEA="https://git.site.quack-lab.dev"
REPO="dave/BigChef"
# Create a release
RELEASE_RESPONSE=$(curl -s -X POST \
-H "Authorization: token $TOKEN" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d '{
"tag_name": "'"$TAG"'",
"name": "'"$TAG"'",
"draft": false,
"prerelease": false
}' \
$GITEA/api/v1/repos/$REPO/releases)
# Extract the release ID
echo $RELEASE_RESPONSE
RELEASE_ID=$(echo $RELEASE_RESPONSE | awk -F'"id":' '{print $2+0; exit}')
echo "Release ID: $RELEASE_ID"
echo "Uploading the things..."
curl -X POST \
-H "Authorization: token $TOKEN" \
-F "attachment=@chef.exe" \
"$GITEA/api/v1/repos/$REPO/releases/${RELEASE_ID}/assets?name=chef.exe"
rm chef.exe