54 Commits

Author SHA1 Message Date
d3a1f1bd96 Rework regex grouping to avoid changing the same area twice 2025-03-26 22:24:19 +01:00
07a5f3f1a4 Add replaceCommands to avoid index suicide 2025-03-26 21:55:34 +01:00
e2257e082a Add a regression test 2025-03-26 21:55:17 +01:00
b3fce4244d Fix regex for numbers to support negative numbers 2025-03-26 18:30:21 +01:00
bd443067b6 Add support for !num inside of named capture groups 2025-03-26 14:04:39 +01:00
a9b6f7f984 Implement printing from lua 2025-03-26 13:37:39 +01:00
10c39b02a0 Fix some regex tests 2025-03-26 13:13:53 +01:00
7f4392b10e Implement "replacement" variable that simply replaces the match 2025-03-26 13:00:52 +01:00
7e19cf4e2c Rework named captures to be array
To comply with the whole reverse replacing
2025-03-26 12:50:55 +01:00
c5fb20e96a Implement named capture groups 2025-03-26 12:28:28 +01:00
a8c2257f20 Add named capture group tests 2025-03-26 12:17:01 +01:00
b63b4d1352 Add some more shorthands for regex 2025-03-26 12:06:04 +01:00
6a3d44ccd0 Redo what claude removed 2025-03-26 11:42:00 +01:00
c22e6ff41f Code polish 2025-03-26 11:40:23 +01:00
068c64d714 Fix up file reseting 2025-03-26 11:10:43 +01:00
2c7a4f5d97 Add a lot more logs to regex
What the fuck is going on?
2025-03-26 03:30:08 +01:00
0d7d251e76 Implement git reset 2025-03-26 03:23:16 +01:00
0d8c447ff6 Add support for git ie. automatically resetting changes to ensure clean slate 2025-03-26 03:22:05 +01:00
bb14087598 Fix oopsie 2025-03-26 02:52:28 +01:00
66a522aa12 Try to include xml node children in lua table 2025-03-26 02:50:33 +01:00
1a4b4f76f2 Map the weird numeric escapes to textual ones 2025-03-26 02:34:21 +01:00
2bfd9f951e Fix up some more xml tests and other small bugs 2025-03-26 02:17:42 +01:00
e5092edf53 Implement parsing xml to and from lua
A lot more complex than json.........
2025-03-26 01:36:49 +01:00
e31c0e4e8f Implement xpath (by calling library) 2025-03-26 01:19:41 +01:00
73d93367a0 Refactor some things around a little 2025-03-25 23:16:03 +01:00
64f690f6b4 Fix usage to reflect recent flag changes 2025-03-25 23:02:32 +01:00
34477b2c34 Make readme and rework the flags a little 2025-03-25 22:55:01 +01:00
d5c08d86f5 Code polish 2025-03-25 19:22:44 +01:00
68127fe453 Add more json tests
To bring it in line with the xml ones
2025-03-25 19:22:07 +01:00
872f2dd46d Fix another changed test for json 2025-03-25 19:16:09 +01:00
4eed05c7c2 Fix some more minor bugs and tests 2025-03-25 19:14:21 +01:00
4640281fbf Enable root modifications
Though I can not see why you would want to.....
But there's no reason you would not be able to
2025-03-25 18:57:32 +01:00
aba10267d1 Fix more tests 2025-03-25 18:47:55 +01:00
fed140254b Fix some json tests 2025-03-25 18:32:51 +01:00
db92033642 Rework rounding and building lua script
To allow user script to specify what was modified where
2025-03-25 18:28:33 +01:00
1b0b198297 Add xpath tests 2025-03-25 17:56:48 +01:00
15ae116447 Fix up some miscellaneous shit around the project regarding lua conversions 2025-03-25 17:46:21 +01:00
1bcc6735ab Improve error handling across the board 2025-03-25 17:27:20 +01:00
20bab894e3 Add path data to the selected nodes for reconstruction via set 2025-03-25 17:27:20 +01:00
396992b3d0 Implement Set and SetAll (and Get) 2025-03-25 17:27:20 +01:00
054dcbf835 Fix up the recursive descent
Again I guess?
2025-03-25 17:27:20 +01:00
88887b9a12 Fix up the recursive descent 2025-03-25 17:27:20 +01:00
533a563dc5 Fix up parser 2025-03-25 17:27:20 +01:00
7fc2956b6d Begin to implement jsonpath 2025-03-25 17:27:20 +01:00
6f9f3f5eae Begin to rework the json parsing 2025-03-25 17:27:20 +01:00
d2419b761e Implement a generic translator between lua and go tables (maps) 2025-03-25 17:27:20 +01:00
ab98800ca0 Rework regex processor to be more betterer 2025-03-25 17:27:20 +01:00
0f7ee521ac Fix up lua variable writing and reading for regex 2025-03-25 17:27:20 +01:00
fd81861a64 Fix reading lua variables 2025-03-25 17:27:20 +01:00
430234dd3b Clean up after claude 2025-03-25 17:27:20 +01:00
17bb3d4f71 Refactor everything to processors and implement json and xml processors such as they are 2025-03-25 17:27:20 +01:00
84e0a8bed6 Add xpath dependencies 2025-03-25 17:27:20 +01:00
bdcf096fdd Fix up the oopsies 2025-03-25 17:27:15 +01:00
1a90046c89 Add release script 2025-03-25 17:26:23 +01:00
20 changed files with 3899 additions and 638 deletions

9
.vscode/launch.json vendored
View File

@@ -9,8 +9,13 @@
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${fileDirname}",
"args": []
"program": "${workspaceFolder}",
"args": [
"-mode=json",
"$..name",
"v='pero'",
"test.json"
]
}
]
}

116
README.md Normal file
View File

@@ -0,0 +1,116 @@
# Big Chef
A Go-based tool for modifying XML, JSON, and text documents using XPath/JSONPath/Regex expressions and Lua transformations.
## Features
- **Multi-Format Processing**:
- XML (XPath)
- JSON (JSONPath)
- Text (Regex)
- **Node Value Modification**: Update text values in XML elements, JSON properties or text matches
- **Attribute Manipulation**: Modify XML attributes, JSON object keys or regex capture groups
- **Conditional Logic**: Apply transformations based on document content
- **Complex Operations**:
- Mathematical calculations
- String manipulations
- Date conversions
- Structural changes
- Whole ass Lua environment
- **Error Handling**: Comprehensive error detection for:
- Invalid XML/JSON
- Malformed XPath/JSONPath
- Lua syntax errors
## Usage Examples
### 1. Basic field modification
```xml
<!-- Input -->
<price>44.95</price>
<!-- Command -->
chef -xml "//price" "v=v*2" input.xml
<!-- Output -->
<price>89.9</price>
```
### 2. Supports glob patterns
```xml
chef -xml "//price" "v=v*2" data/**.xml
```
### 3. Attribute Update
```xml
<!-- Input -->
<item price="10.50"/>
<!-- Command -->
chef -xml "//item/@price" "v=v*2" input.xml
<!-- Output -->
<item price="21"/>
```
### 3. JSONPath Transformation
```json
// Input
{
"products": [
{"name": "Widget", "price": 19.99},
{"name": "Gadget", "price": 29.99}
]
}
// Command
chef -json "$.products[*].price" "v=v*0.75" input.json
// Output
{
"products": [
{"name": "Widget", "price": 14.99},
{"name": "Gadget", "price": 22.49}
]
}
```
### 4. Regex Text Replacement
Regex works slightly differently, up to 12 match groups are provided as v1..v12 and s1..s12 for numbers and strings respectively.
A special shorthand "!num" is also provided that simply expands to `(\d*\.?\d+)`.
```xml
<!-- Input -->
<description>Price: $15.00 Special Offer</description>
<!-- Command -->
chef "Price: $!num Special Offer" "v1 = v1 * 0.92" input.xml
<!-- Output -->
<description>Price: $13.80 Special Offer</description>
```
### 5. Conditional Transformation
```xml
<!-- Input -->
<item stock="5" price="10.00"/>
<!-- Command -->
chef -xml "//item" "if tonumber(v.stock) > 0 then v.price = v.price * 0.8 end" input.xml
<!-- Output -->
<item stock="5" price="8.00"/>
```
## Installation
```bash
go build -o chef main.go
```
```bash
# Process XML file
./chef -xml "//price" "v=v*1.2" input.xml
# Process JSON file
./chef -json "$.prices[*]" "v=v*0.9" input.json
```

35
go.mod
View File

@@ -3,18 +3,37 @@ module modify
go 1.24.1
require (
github.com/PaesslerAG/jsonpath v0.1.1
github.com/antchfx/xmlquery v1.4.4
github.com/bmatcuk/doublestar/v4 v4.8.1
github.com/yuin/gopher-lua v1.1.1
)
require (
github.com/PaesslerAG/gval v1.0.0 // indirect
github.com/PaesslerAG/jsonpath v0.1.1 // indirect
github.com/antchfx/xpath v1.3.3 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/sergi/go-diff v1.3.1 // indirect
github.com/stretchr/testify v1.10.0 // indirect
golang.org/x/net v0.33.0 // indirect
golang.org/x/text v0.21.0 // indirect
dario.cat/mergo v1.0.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/ProtonMail/go-crypto v1.1.5 // indirect
github.com/cloudflare/circl v1.6.0 // indirect
github.com/cyphar/filepath-securejoin v0.4.1 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.6.2 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/pjbgf/sha1cd v0.3.2 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
golang.org/x/crypto v0.35.0 // indirect
golang.org/x/sys v0.30.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
)
require (
github.com/PaesslerAG/gval v1.0.0 // indirect
github.com/antchfx/xpath v1.3.3 // indirect
github.com/go-git/go-git/v5 v5.14.0
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
golang.org/x/net v0.35.0 // indirect
golang.org/x/text v0.22.0 // indirect
)

89
go.sum
View File

@@ -1,38 +1,99 @@
dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/PaesslerAG/gval v1.0.0 h1:GEKnRwkWDdf9dOmKcNrar9EA1bz1z9DqPIO1+iLzhd8=
github.com/PaesslerAG/gval v1.0.0/go.mod h1:y/nm5yEyTeX6av0OfKJNp9rBNj2XrGhAf5+v24IBN1I=
github.com/PaesslerAG/jsonpath v0.1.0/go.mod h1:4BzmtoM/PI8fPO4aQGIusjGxGir2BzcV0grWtFzq1Y8=
github.com/PaesslerAG/jsonpath v0.1.1 h1:c1/AToHQMVsduPAa4Vh6xp2U0evy4t8SWp8imEsylIk=
github.com/PaesslerAG/jsonpath v0.1.1/go.mod h1:lVboNxFGal/VwW6d9JzIy56bUsYAP6tH/x80vjnCseY=
github.com/ProtonMail/go-crypto v1.1.5 h1:eoAQfK2dwL+tFSFpr7TbOaPNUbPiJj4fLYwwGE1FQO4=
github.com/ProtonMail/go-crypto v1.1.5/go.mod h1:rA3QumHc/FZ8pAHreoekgiAbzpNsfQAosU5td4SnOrE=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
github.com/antchfx/xmlquery v1.4.4 h1:mxMEkdYP3pjKSftxss4nUHfjBhnMk4imGoR96FRY2dg=
github.com/antchfx/xmlquery v1.4.4/go.mod h1:AEPEEPYE9GnA2mj5Ur2L5Q5/2PycJ0N9Fusrx9b12fc=
github.com/antchfx/xpath v1.3.3 h1:tmuPQa1Uye0Ym1Zn65vxPgfltWb/Lxu2jeqIGteJSRs=
github.com/antchfx/xpath v1.3.3/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/bmatcuk/doublestar/v4 v4.8.1 h1:54Bopc5c2cAvhLRAzqOGCYHYyhcDHsFF4wWIR5wKP38=
github.com/bmatcuk/doublestar/v4 v4.8.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/cloudflare/circl v1.6.0 h1:cr5JKic4HI+LkINy2lg3W2jF8sHCVTBncJr5gIIq7qk=
github.com/cloudflare/circl v1.6.0/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/cyphar/filepath-securejoin v0.4.1 h1:JyxxyPEaktOD+GAnqIqTf9A8tHyAG22rowi7HkoSU1s=
github.com/cyphar/filepath-securejoin v0.4.1/go.mod h1:Sdj7gXlvMcPZsbhwhQ33GguGLDGQL7h7bg04C/+u9jI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/elazarl/goproxy v1.7.2 h1:Y2o6urb7Eule09PjlhQRGNsqRfPmYI3KKQLFpCAV3+o=
github.com/elazarl/goproxy v1.7.2/go.mod h1:82vkLNir0ALaW14Rc399OTTjyNREgmdL2cVoIbS6XaE=
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
github.com/go-git/go-billy/v5 v5.6.2 h1:6Q86EsPXMa7c3YZ3aLAQsMA0VlWmy43r6FHqa/UNbRM=
github.com/go-git/go-billy/v5 v5.6.2/go.mod h1:rcFC2rAsp/erv7CMz9GczHcuD0D32fWzH+MJAU+jaUU=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMje31YglSBqCdIqdhKBW8lokaMrL3uTkpGYlE2OOT4=
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
github.com/go-git/go-git/v5 v5.14.0 h1:/MD3lCrGjCen5WfEAzKg00MJJffKhC8gzS80ycmCi60=
github.com/go-git/go-git/v5 v5.14.0/go.mod h1:Z5Xhoia5PcWA3NF8vRLURn9E5FRhSl7dGj9ItW3Wk5k=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ=
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/onsi/gomega v1.34.1 h1:EUMJIKUjM8sKjYbtxQI9A4z2o+rruxnzNvpknOXie6k=
github.com/onsi/gomega v1.34.1/go.mod h1:kU1QgUvBDLXBJq618Xvm2LUX6rSAfRaFRTcdOeDLwwY=
github.com/pjbgf/sha1cd v0.3.2 h1:a9wb0bp1oC2TGwStyn0Umc/IGKQnEgF0vVaZ8QF8eo4=
github.com/pjbgf/sha1cd v0.3.2/go.mod h1:zQWigSxVmsHEZow5qaLtPYxpcKMMQpa09ixqBxuCS6A=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/sergi/go-diff v1.3.1 h1:xkr+Oxo4BOQKmkn/B9eMK0g5Kg/983T9DqqPHwYqD+8=
github.com/sergi/go-diff v1.3.1/go.mod h1:aMJSSKb2lpPvRNec0+w3fl7LP9IOFzdc9Pa4NFbPK1I=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/skeema/knownhosts v1.3.1 h1:X2osQ+RAjK76shCbvhHHHVl3ZlgDm8apHEHFqRjnBY8=
github.com/skeema/knownhosts v1.3.1/go.mod h1:r7KTdC8l4uxWRyK2TpQZ/1o5HaSzh06ePQNxPwTcfiY=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.35.0 h1:b15kiHdrGCHrP6LvwaQ3c03kgNhhiMgvlhxHQhmg2Xs=
golang.org/x/crypto v0.35.0/go.mod h1:dy7dXNW32cAb/6/PRuTNsix8T+vJAqvuIy5Bli/x0YQ=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
@@ -40,14 +101,16 @@ golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.35.0 h1:T5GQRQb2y08kTAByq9L4/bz8cipCdA8FbRTXewonqY8=
golang.org/x/net v0.35.0/go.mod h1:EglIi67kWsHKlRzzVMUD93VMSWGFOMSZgxFjparz1Qk=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -56,9 +119,13 @@ golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -66,6 +133,8 @@ golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -75,16 +144,20 @@ golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/term v0.29.0 h1:L6pJp37ocefwRRtYPKSWOWzOtWSxVajvz2ldH/xi3iU=
golang.org/x/term v0.29.0/go.mod h1:6bl4lRlvVuDgSf3179VpIxBF0o10JUpXWOnI7nErv7s=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
@@ -94,5 +167,11 @@ golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxb
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

189
main.go
View File

@@ -5,9 +5,13 @@ import (
"fmt"
"log"
"os"
"path/filepath"
"sync"
"time"
"github.com/bmatcuk/doublestar/v4"
"github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/plumbing/object"
"modify/processor"
)
@@ -19,20 +23,16 @@ type GlobalStats struct {
FailedFiles int
}
type FileMode string
const (
ModeRegex FileMode = "regex"
ModeXML FileMode = "xml"
ModeJSON FileMode = "json"
)
var stats GlobalStats
var logger *log.Logger
var (
fileModeFlag = flag.String("mode", "regex", "Processing mode: regex, xml, json")
verboseFlag = flag.Bool("verbose", false, "Enable verbose output")
jsonFlag = flag.Bool("json", false, "Process JSON files")
xmlFlag = flag.Bool("xml", false, "Process XML files")
gitFlag = flag.Bool("git", false, "Use git to manage files")
resetFlag = flag.Bool("reset", false, "Reset files to their original state")
repo *git.Repository
worktree *git.Worktree
)
func init() {
@@ -43,28 +43,43 @@ func init() {
}
func main() {
// TODO: Implement some sort of git integration
// Maybe use go-git
// Specify a -git flag
// If we are operating with git then:
// Inmitialize a repo if one doesn't exist (try to open right?)
// For each file matched by glob first figure out if it's already tracked
// If not tracked then track it and commit (either it alone or maybe multiple together somehow)
// Then reset the file (to undo previous modifications)
// THEN change the file
// In addition add a -undo flag that will ONLY reset the files without changing them
// Only for the ones matched by glob
// ^ important because binary files would fuck us up
flag.Usage = func() {
fmt.Fprintf(os.Stderr, "Usage: %s [options] <pattern> <lua_expression> <...files_or_globs>\n", os.Args[0])
fmt.Fprintf(os.Stderr, "\nOptions:\n")
fmt.Fprintf(os.Stderr, " -json\n")
fmt.Fprintf(os.Stderr, " Process JSON files\n")
fmt.Fprintf(os.Stderr, " -xml\n")
fmt.Fprintf(os.Stderr, " Process XML files\n")
fmt.Fprintf(os.Stderr, " -git\n")
fmt.Fprintf(os.Stderr, " Use git to manage files\n")
fmt.Fprintf(os.Stderr, " -reset\n")
fmt.Fprintf(os.Stderr, " Reset files to their original state\n")
fmt.Fprintf(os.Stderr, " -mode string\n")
fmt.Fprintf(os.Stderr, " Processing mode: regex, xml, json (default \"regex\")\n")
fmt.Fprintf(os.Stderr, " -xpath string\n")
fmt.Fprintf(os.Stderr, " XPath expression (for XML mode)\n")
fmt.Fprintf(os.Stderr, " -jsonpath string\n")
fmt.Fprintf(os.Stderr, " JSONPath expression (for JSON mode)\n")
fmt.Fprintf(os.Stderr, " -verbose\n")
fmt.Fprintf(os.Stderr, " Enable verbose output\n")
fmt.Fprintf(os.Stderr, "\nExamples:\n")
fmt.Fprintf(os.Stderr, " Regex mode (default):\n")
fmt.Fprintf(os.Stderr, " %s \"<value>(\\d+)</value>\" \"*1.5\" data.xml\n", os.Args[0])
fmt.Fprintf(os.Stderr, " XML mode:\n")
fmt.Fprintf(os.Stderr, " %s -mode=xml -xpath=\"//value\" \"*1.5\" data.xml\n", os.Args[0])
fmt.Fprintf(os.Stderr, " %s -xml \"//value\" \"*1.5\" data.xml\n", os.Args[0])
fmt.Fprintf(os.Stderr, " JSON mode:\n")
fmt.Fprintf(os.Stderr, " %s -mode=json -jsonpath=\"$.items[*].value\" \"*1.5\" data.json\n", os.Args[0])
fmt.Fprintf(os.Stderr, " %s -json \"$.items[*].value\" \"*1.5\" data.json\n", os.Args[0])
fmt.Fprintf(os.Stderr, "\nNote: v1, v2, etc. are used to refer to capture groups as numbers.\n")
fmt.Fprintf(os.Stderr, " s1, s2, etc. are used to refer to capture groups as strings.\n")
fmt.Fprintf(os.Stderr, " Helper functions: num(str) converts string to number, str(num) converts number to string\n")
fmt.Fprintf(os.Stderr, " is_number(str) checks if a string is numeric\n")
fmt.Fprintf(os.Stderr, " For XML and JSON, the captured values are exposed as 'v', which can be of any type we capture (string, number, table).\n")
fmt.Fprintf(os.Stderr, " If expression starts with an operator like *, /, +, -, =, etc., v1 is automatically prepended\n")
fmt.Fprintf(os.Stderr, " You can use any valid Lua code, including if statements, loops, etc.\n")
fmt.Fprintf(os.Stderr, " Glob patterns are supported for file selection (*.xml, data/**.xml, etc.)\n")
@@ -72,9 +87,12 @@ func main() {
flag.Parse()
args := flag.Args()
if *resetFlag {
*gitFlag = true
}
if len(args) < 3 {
fmt.Fprintf(os.Stderr, "%s mode requires %d arguments minimum\n", *fileModeFlag, 3)
log.Printf("At least %d arguments are required", 3)
flag.Usage()
return
}
@@ -83,15 +101,9 @@ func main() {
var pattern, luaExpr string
var filePatterns []string
if *fileModeFlag == "regex" {
pattern = args[0]
luaExpr = args[1]
filePatterns = args[2:]
} else {
// For XML/JSON modes, pattern comes from flags
luaExpr = args[0]
filePatterns = args[1:]
}
pattern = args[0]
luaExpr = args[1]
filePatterns = args[2:]
// Prepare the Lua expression
originalLuaExpr := luaExpr
@@ -100,6 +112,14 @@ func main() {
logger.Printf("Transformed Lua expression from %q to %q", originalLuaExpr, luaExpr)
}
if *gitFlag {
err := setupGit()
if err != nil {
fmt.Fprintf(os.Stderr, "Error setting up git: %v\n", err)
return
}
}
// Expand file patterns with glob support
files, err := expandFilePatterns(filePatterns)
if err != nil {
@@ -112,23 +132,33 @@ func main() {
return
}
if *gitFlag {
err := cleanupGitFiles(files)
if err != nil {
fmt.Fprintf(os.Stderr, "Error cleaning up git files: %v\n", err)
return
}
}
if *resetFlag {
log.Printf("Files reset to their original state, nothing more to do")
return
}
// Create the processor based on mode
var proc processor.Processor
switch *fileModeFlag {
case "regex":
switch {
case *xmlFlag:
proc = &processor.XMLProcessor{}
logger.Printf("Starting XML modifier with XPath %q, expression %q on %d files",
pattern, luaExpr, len(files))
case *jsonFlag:
proc = &processor.JSONProcessor{}
logger.Printf("Starting JSON modifier with JSONPath %q, expression %q on %d files",
pattern, luaExpr, len(files))
default:
proc = &processor.RegexProcessor{}
logger.Printf("Starting regex modifier with pattern %q, expression %q on %d files",
pattern, luaExpr, len(files))
// case "xml":
// proc = &processor.XMLProcessor{}
// pattern = *xpathFlag
// logger.Printf("Starting XML modifier with XPath %q, expression %q on %d files",
// pattern, luaExpr, len(files))
// case "json":
// proc = &processor.JSONProcessor{}
// pattern = *jsonpathFlag
// logger.Printf("Starting JSON modifier with JSONPath %q, expression %q on %d files",
// pattern, luaExpr, len(files))
}
var wg sync.WaitGroup
@@ -139,7 +169,8 @@ func main() {
defer wg.Done()
logger.Printf("Processing file: %s", file)
modCount, matchCount, err := proc.Process(file, pattern, luaExpr)
// It's a bit fucked, maybe I could do better to call it from proc... But it'll do for now
modCount, matchCount, err := processor.Process(proc, file, pattern, luaExpr)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to process file %s: %v\n", file, err)
stats.FailedFiles++
@@ -162,6 +193,35 @@ func main() {
}
}
func setupGit() error {
cwd, err := os.Getwd()
if err != nil {
return fmt.Errorf("failed to get current working directory: %w", err)
}
logger.Printf("Current working directory obtained: %s", cwd)
logger.Printf("Attempting to open git repository at %s", cwd)
repo, err = git.PlainOpen(cwd)
if err != nil {
logger.Printf("No existing git repository found at %s, attempting to initialize a new git repository.", cwd)
repo, err = git.PlainInit(cwd, false)
if err != nil {
return fmt.Errorf("failed to initialize a new git repository at %s: %w", cwd, err)
}
logger.Printf("Successfully initialized a new git repository at %s", cwd)
} else {
logger.Printf("Successfully opened existing git repository at %s", cwd)
}
logger.Printf("Attempting to obtain worktree for repository at %s", cwd)
worktree, err = repo.Worktree()
if err != nil {
return fmt.Errorf("failed to obtain worktree for repository at %s: %w", cwd, err)
}
logger.Printf("Successfully obtained worktree for repository at %s", cwd)
return nil
}
func expandFilePatterns(patterns []string) ([]string, error) {
var files []string
filesMap := make(map[string]bool)
@@ -180,3 +240,50 @@ func expandFilePatterns(patterns []string) ([]string, error) {
}
return files, nil
}
func cleanupGitFiles(files []string) error {
for _, file := range files {
logger.Printf("Checking file: %s", file)
status, err := worktree.Status()
if err != nil {
fmt.Fprintf(os.Stderr, "Error getting worktree status: %v\n", err)
return fmt.Errorf("error getting worktree status: %w", err)
}
if status.IsUntracked(file) {
logger.Printf("Detected untracked file: %s. Attempting to add it to the git index.", file)
_, err = worktree.Add(file)
if err != nil {
fmt.Fprintf(os.Stderr, "Error adding file to git: %v\n", err)
return fmt.Errorf("error adding file to git: %w", err)
}
filename := filepath.Base(file)
logger.Printf("File %s added successfully. Now committing it with message: 'Track %s'", filename, filename)
_, err = worktree.Commit("Track "+filename, &git.CommitOptions{
Author: &object.Signature{
Name: "Big Chef",
Email: "bigchef@bigchef.com",
When: time.Now(),
},
})
if err != nil {
fmt.Fprintf(os.Stderr, "Error committing file: %v\n", err)
return fmt.Errorf("error committing file: %w", err)
}
logger.Printf("Successfully committed file: %s with message: 'Track %s'", filename, filename)
} else {
logger.Printf("File %s is already tracked. Restoring it to the working tree.", file)
err := worktree.Restore(&git.RestoreOptions{
Files: []string{file},
Staged: true,
Worktree: true,
})
if err != nil {
fmt.Fprintf(os.Stderr, "Error restoring file: %v\n", err)
return fmt.Errorf("error restoring file: %w", err)
}
logger.Printf("File %s restored successfully.", file)
}
}
return nil
}

View File

@@ -3,10 +3,8 @@ package processor
import (
"encoding/json"
"fmt"
"log"
"modify/processor/jsonpath"
"os"
"path/filepath"
"strings"
lua "github.com/yuin/gopher-lua"
)
@@ -14,39 +12,6 @@ import (
// JSONProcessor implements the Processor interface for JSON documents
type JSONProcessor struct{}
// Process implements the Processor interface for JSONProcessor
func (p *JSONProcessor) Process(filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
cwd, err := os.Getwd()
if err != nil {
return 0, 0, fmt.Errorf("error getting current working directory: %v", err)
}
fullPath := filepath.Join(cwd, filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ProcessContent implements the Processor interface for JSONProcessor
func (p *JSONProcessor) ProcessContent(content string, pattern string, luaExpr string) (string, int, int, error) {
// Parse JSON document
@@ -67,99 +32,125 @@ func (p *JSONProcessor) ProcessContent(content string, pattern string, luaExpr s
return content, 0, 0, nil
}
// Initialize Lua
L, err := NewLuaState()
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error creating Lua state: %v", err)
}
defer L.Close()
modCount := 0
for _, node := range nodes {
log.Printf("Processing node at path: %s with value: %v", node.Path, node.Value)
err = p.ToLua(L, nodes)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error converting to Lua: %v", err)
}
// Initialize Lua
L, err := NewLuaState()
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error creating Lua state: %v", err)
}
defer L.Close()
log.Println("Lua state initialized successfully.")
// Execute Lua script
if err := L.DoString(luaExpr); err != nil {
return content, len(nodes), 0, fmt.Errorf("error executing Lua %s: %v", luaExpr, err)
}
err = p.ToLua(L, node.Value)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error converting to Lua: %v", err)
}
log.Printf("Converted node value to Lua: %v", node.Value)
// Get modified value
result, err := p.FromLua(L)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error getting result from Lua: %v", err)
}
originalScript := luaExpr
fullScript := BuildLuaScript(luaExpr)
log.Printf("Original script: %q, Full script: %q", originalScript, fullScript)
// Apply the modification to the JSON data
err = p.updateJSONValue(jsonData, pattern, result)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error updating JSON: %v", err)
// Execute Lua script
log.Printf("Executing Lua script: %q", fullScript)
if err := L.DoString(fullScript); err != nil {
return content, len(nodes), 0, fmt.Errorf("error executing Lua %q: %v", fullScript, err)
}
log.Println("Lua script executed successfully.")
// Get modified value
result, err := p.FromLua(L)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error getting result from Lua: %v", err)
}
log.Printf("Retrieved modified value from Lua: %v", result)
modified := false
modified = L.GetGlobal("modified").String() == "true"
if !modified {
log.Printf("No changes made to node at path: %s", node.Path)
continue
}
// Apply the modification to the JSON data
err = p.updateJSONValue(jsonData, node.Path, result)
if err != nil {
return content, len(nodes), 0, fmt.Errorf("error updating JSON: %v", err)
}
log.Printf("Updated JSON at path: %s with new value: %v", node.Path, result)
modCount++
}
// Convert the modified JSON back to a string with same formatting
var jsonBytes []byte
if indent, err := detectJsonIndentation(content); err == nil && indent != "" {
// Use detected indentation for output formatting
jsonBytes, err = json.MarshalIndent(jsonData, "", indent)
} else {
// Fall back to standard 2-space indent
jsonBytes, err = json.MarshalIndent(jsonData, "", " ")
jsonBytes, err = json.MarshalIndent(jsonData, "", " ")
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error marshalling JSON: %v", err)
}
// We changed all the nodes trust me bro
return string(jsonBytes), len(nodes), len(nodes), nil
return string(jsonBytes), modCount, matchCount, nil
}
// detectJsonIndentation tries to determine the indentation used in the original JSON
func detectJsonIndentation(content string) (string, error) {
lines := strings.Split(content, "\n")
if len(lines) < 2 {
return "", fmt.Errorf("not enough lines to detect indentation")
}
// Look for the first indented line
for i := 1; i < len(lines); i++ {
line := lines[i]
trimmed := strings.TrimSpace(line)
if trimmed == "" {
continue
}
// Calculate leading whitespace
indent := line[:len(line)-len(trimmed)]
if len(indent) > 0 {
return indent, nil
}
}
return "", fmt.Errorf("no indentation detected")
}
// / Selects from the root node
// // Selects nodes in the document from the current node that match the selection no matter where they are
// . Selects the current node
// @ Selects attributes
// /bookstore/* Selects all the child element nodes of the bookstore element
// //* Selects all elements in the document
// /bookstore/book[1] Selects the first book element that is the child of the bookstore element.
// /bookstore/book[last()] Selects the last book element that is the child of the bookstore element
// /bookstore/book[last()-1] Selects the last but one book element that is the child of the bookstore element
// /bookstore/book[position()<3] Selects the first two book elements that are children of the bookstore element
// //title[@lang] Selects all the title elements that have an attribute named lang
// //title[@lang='en'] Selects all the title elements that have a "lang" attribute with a value of "en"
// /bookstore/book[price>35.00] Selects all the book elements of the bookstore element that have a price element with a value greater than 35.00
// /bookstore/book[price>35.00]/title Selects all the title elements of the book elements of the bookstore element that have a price element with a value greater than 35.00
// updateJSONValue updates a value in the JSON structure based on its JSONPath
func (p *JSONProcessor) updateJSONValue(jsonData interface{}, path string, newValue interface{}) error {
// Special handling for root node
if path == "$" {
// For the root node, we'll copy the value to the jsonData reference
// This is a special case since we can't directly replace the interface{} variable
// We need to handle different types of root elements
switch rootValue := newValue.(type) {
case map[string]interface{}:
// For objects, we need to copy over all keys
rootMap, ok := jsonData.(map[string]interface{})
if !ok {
// If the original wasn't a map, completely replace it with the new map
// This is handled by the jsonpath.Set function
return jsonpath.Set(jsonData, path, newValue)
}
// Clear the original map
for k := range rootMap {
delete(rootMap, k)
}
// Copy all keys from the new map
for k, v := range rootValue {
rootMap[k] = v
}
return nil
case []interface{}:
// For arrays, we need to handle similarly
rootArray, ok := jsonData.([]interface{})
if !ok {
// If the original wasn't an array, use jsonpath.Set
return jsonpath.Set(jsonData, path, newValue)
}
// Clear and recreate the array
*&rootArray = rootValue
return nil
default:
// For other types, use jsonpath.Set
return jsonpath.Set(jsonData, path, newValue)
}
}
// For non-root paths, use the regular Set method
err := jsonpath.Set(jsonData, path, newValue)
if err != nil {
return fmt.Errorf("failed to update JSON value at path '%s': %w", path, err)
}
return nil
}
// ToLua converts JSON values to Lua variables
func (p *JSONProcessor) ToLua(L *lua.LState, data interface{}) error {
table, err := ToLuaTable(L, data)
table, err := ToLua(L, data)
if err != nil {
return err
}
@@ -170,5 +161,5 @@ func (p *JSONProcessor) ToLua(L *lua.LState, data interface{}) error {
// FromLua retrieves values from Lua
func (p *JSONProcessor) FromLua(L *lua.LState) (interface{}, error) {
luaValue := L.GetGlobal("v")
return FromLuaTable(L, luaValue.(*lua.LTable))
return FromLua(L, luaValue)
}

File diff suppressed because it is too large Load Diff

View File

@@ -138,10 +138,6 @@ func Set(data interface{}, path string, value interface{}) error {
return fmt.Errorf("failed to parse JSONPath %q: %w", path, err)
}
if len(steps) <= 1 {
return fmt.Errorf("cannot set root node; the provided path %q is invalid", path)
}
success := false
err = setWithPath(data, steps, &success, value, "$", ModifyFirstMode)
if err != nil {
@@ -157,10 +153,6 @@ func SetAll(data interface{}, path string, value interface{}) error {
return fmt.Errorf("failed to parse JSONPath %q: %w", path, err)
}
if len(steps) <= 1 {
return fmt.Errorf("cannot set root node; the provided path %q is invalid", path)
}
success := false
err = setWithPath(data, steps, &success, value, "$", ModifyAllMode)
if err != nil {
@@ -178,17 +170,20 @@ func setWithPath(node interface{}, steps []JSONStep, success *bool, value interf
// Skip root step
actualSteps := steps
if len(steps) > 0 && steps[0].Type == RootStep {
if len(steps) == 1 {
return fmt.Errorf("cannot set root node; the provided path %q is invalid", currentPath)
}
actualSteps = steps[1:]
}
// Process the first step
// If we have no steps left, we're setting the root value
if len(actualSteps) == 0 {
return fmt.Errorf("cannot set root node; no steps provided for path %q", currentPath)
// For the root node, we need to handle it differently depending on what's passed in
// since we can't directly replace the interface{} variable
// We'll signal success and let the JSONProcessor handle updating the root
*success = true
return nil
}
// Process the first step
step := actualSteps[0]
remainingSteps := actualSteps[1:]
isLastStep := len(remainingSteps) == 0

View File

@@ -355,14 +355,14 @@ func TestSet(t *testing.T) {
}
})
t.Run("setting on root should fail", func(t *testing.T) {
t.Run("setting on root should not fail (anymore)", func(t *testing.T) {
data := map[string]interface{}{
"name": "John",
}
err := Set(data, "$", "Jane")
if err == nil {
t.Errorf("Set() returned no error, expected error for setting on root")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}

View File

@@ -2,16 +2,20 @@ package processor
import (
"fmt"
"reflect"
"log"
"os"
"path/filepath"
"strings"
"github.com/antchfx/xmlquery"
lua "github.com/yuin/gopher-lua"
)
// Processor defines the interface for all file processors
type Processor interface {
// Process handles processing a file with the given pattern and Lua expression
Process(filename string, pattern string, luaExpr string) (int, int, error)
// Now implemented as a base function in processor.go
// Process(filename string, pattern string, luaExpr string) (int, int, error)
// ProcessContent handles processing a string content directly with the given pattern and Lua expression
// Returns the modified content, modification count, match count, and any error
@@ -52,65 +56,139 @@ func NewLuaState() (*lua.LState, error) {
return L, nil
}
// ToLuaTable converts a struct or map to a Lua table recursively
func ToLuaTable(L *lua.LState, data interface{}) (*lua.LTable, error) {
luaTable := L.NewTable()
func Process(p Processor, filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
cwd, err := os.Getwd()
if err != nil {
return 0, 0, fmt.Errorf("error getting current working directory: %v", err)
}
fullPath := filepath.Join(cwd, filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ToLua converts a struct or map to a Lua table recursively
func ToLua(L *lua.LState, data interface{}) (lua.LValue, error) {
switch v := data.(type) {
case *xmlquery.Node:
luaTable := L.NewTable()
luaTable.RawSetString("text", lua.LString(v.Data))
// Should be a map, simple key value pairs
attr, err := ToLua(L, v.Attr)
if err != nil {
return nil, err
}
luaTable.RawSetString("attr", attr)
return luaTable, nil
case map[string]interface{}:
luaTable := L.NewTable()
for key, value := range v {
luaValue, err := ToLuaTable(L, value)
luaValue, err := ToLua(L, value)
if err != nil {
return nil, err
}
luaTable.RawSetString(key, luaValue)
}
case struct{}:
val := reflect.ValueOf(v)
for i := 0; i < val.NumField(); i++ {
field := val.Type().Field(i)
luaValue, err := ToLuaTable(L, val.Field(i).Interface())
return luaTable, nil
case []interface{}:
luaTable := L.NewTable()
for i, value := range v {
luaValue, err := ToLua(L, value)
if err != nil {
return nil, err
}
luaTable.RawSetString(field.Name, luaValue)
luaTable.RawSetInt(i+1, luaValue) // Lua arrays are 1-indexed
}
return luaTable, nil
case string:
luaTable.RawSetString("v", lua.LString(v))
return lua.LString(v), nil
case bool:
luaTable.RawSetString("v", lua.LBool(v))
return lua.LBool(v), nil
case float64:
luaTable.RawSetString("v", lua.LNumber(v))
return lua.LNumber(v), nil
case nil:
return lua.LNil, nil
default:
return nil, fmt.Errorf("unsupported data type: %T", data)
}
return luaTable, nil
}
// FromLuaTable converts a Lua table to a struct or map recursively
func FromLuaTable(L *lua.LState, luaTable *lua.LTable) (map[string]interface{}, error) {
result := make(map[string]interface{})
luaTable.ForEach(func(key lua.LValue, value lua.LValue) {
switch v := value.(type) {
case *lua.LTable:
nestedMap, err := FromLuaTable(L, v)
if err != nil {
return
}
result[key.String()] = nestedMap
case lua.LString:
result[key.String()] = string(v)
case lua.LBool:
result[key.String()] = bool(v)
case lua.LNumber:
result[key.String()] = float64(v)
default:
result[key.String()] = nil
// FromLua converts a Lua table to a struct or map recursively
func FromLua(L *lua.LState, luaValue lua.LValue) (interface{}, error) {
switch v := luaValue.(type) {
// Well shit...
// Tables in lua are both maps and arrays
// As arrays they are ordered and as maps, obviously, not
// So when we parse them to a go map we fuck up the order for arrays
// We have to find a better way....
case *lua.LTable:
isArray, err := IsLuaTableArray(L, v)
if err != nil {
return nil, err
}
})
if isArray {
result := make([]interface{}, 0)
v.ForEach(func(key lua.LValue, value lua.LValue) {
converted, _ := FromLua(L, value)
result = append(result, converted)
})
return result, nil
} else {
result := make(map[string]interface{})
v.ForEach(func(key lua.LValue, value lua.LValue) {
converted, _ := FromLua(L, value)
result[key.String()] = converted
})
return result, nil
}
case lua.LString:
return string(v), nil
case lua.LBool:
return bool(v), nil
case lua.LNumber:
return float64(v), nil
default:
return nil, nil
}
}
return result, nil
func IsLuaTableArray(L *lua.LState, v *lua.LTable) (bool, error) {
L.SetGlobal("table_to_check", v)
// Use our predefined helper function from InitLuaHelpers
err := L.DoString(`is_array = isArray(table_to_check)`)
if err != nil {
return false, fmt.Errorf("error determining if table is array: %w", err)
}
// Check the result of our Lua function
isArray := L.GetGlobal("is_array")
// LVIsFalse returns true if a given LValue is a nil or false otherwise false.
if !lua.LVIsFalse(isArray) {
return true, nil
}
return false, nil
}
// InitLuaHelpers initializes common Lua helper functions
@@ -119,7 +197,10 @@ func InitLuaHelpers(L *lua.LState) error {
-- Custom Lua helpers for math operations
function min(a, b) return math.min(a, b) end
function max(a, b) return math.max(a, b) end
function round(x) return math.floor(x + 0.5) end
function round(x, n)
if n == nil then n = 0 end
return math.floor(x * 10^n + 0.5) / 10^n
end
function floor(x) return math.floor(x) end
function ceil(x) return math.ceil(x) end
function upper(s) return string.upper(s) end
@@ -139,10 +220,28 @@ end
function is_number(str)
return tonumber(str) ~= nil
end
function isArray(t)
if type(t) ~= "table" then return false end
local max = 0
local count = 0
for k, _ in pairs(t) do
if type(k) ~= "number" or k < 1 or math.floor(k) ~= k then
return false
end
max = math.max(max, k)
count = count + 1
end
return max == count
end
modified = false
`
if err := L.DoString(helperScript); err != nil {
return fmt.Errorf("error loading helper functions: %v", err)
}
L.SetGlobal("print", L.NewFunction(printToGo))
return nil
}
@@ -157,8 +256,7 @@ func LimitString(s string, maxLen int) string {
return s[:maxLen-3] + "..."
}
// BuildLuaScript prepares a Lua expression from shorthand notation
func BuildLuaScript(luaExpr string) string {
func PrependLuaAssignment(luaExpr string) string {
// Auto-prepend v1 for expressions starting with operators
if strings.HasPrefix(luaExpr, "*") ||
strings.HasPrefix(luaExpr, "/") ||
@@ -176,10 +274,44 @@ func BuildLuaScript(luaExpr string) string {
if !strings.Contains(luaExpr, "=") {
luaExpr = "v1 = " + luaExpr
}
return luaExpr
}
// BuildLuaScript prepares a Lua expression from shorthand notation
func BuildLuaScript(luaExpr string) string {
luaExpr = PrependLuaAssignment(luaExpr)
// This allows the user to specify whether or not they modified a value
// If they do nothing we assume they did modify (no return at all)
// If they return before our return then they themselves specify what they did
// If nothing is returned lua assumes nil
// So we can say our value was modified if the return value is either nil or true
// If the return value is false then the user wants to keep the original
fullScript := fmt.Sprintf(`
function run()
%s
end
local res = run()
modified = res == nil or res
`, luaExpr)
return fullScript
}
func printToGo(L *lua.LState) int {
// Get the number of arguments passed to the Lua print function
n := L.GetTop()
// Create a slice to hold the arguments
args := make([]interface{}, n)
for i := 1; i <= n; i++ {
args[i-1] = L.Get(i) // Get the argument from Lua stack
}
// Print the arguments to Go's stdout
log.Print("Lua: ")
log.Println(args...)
return 0 // No return values
}
// Max returns the maximum of two integers
func Max(a, b int) int {
if a > b {

View File

@@ -2,9 +2,9 @@ package processor
import (
"fmt"
"os"
"path/filepath"
"log"
"regexp"
"sort"
"strconv"
"strings"
@@ -14,108 +14,100 @@ import (
// RegexProcessor implements the Processor interface using regex patterns
type RegexProcessor struct{}
// Process implements the Processor interface for RegexProcessor
func (p *RegexProcessor) Process(filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
fullPath := filepath.Join(".", filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ToLua sets capture groups as Lua variables (v1, v2, etc. for numeric values and s1, s2, etc. for strings)
func (p *RegexProcessor) ToLua(L *lua.LState, data interface{}) error {
captures, ok := data.([]string)
captureGroups, ok := data.([]*CaptureGroup)
if !ok {
return fmt.Errorf("expected []string for captures, got %T", data)
return fmt.Errorf("expected []*CaptureGroup for captures, got %T", data)
}
// Set variables for each capture group, starting from v1/s1 for the first capture
for i := 0; i < len(captures); i++ {
// Set string version (always available as s1, s2, etc.)
L.SetGlobal(fmt.Sprintf("s%d", i+1), lua.LString(captures[i]))
groupindex := 0
for _, capture := range captureGroups {
if capture.Name == "" {
// We don't want to change the name of the capture group
// Even if it's empty
tempName := fmt.Sprintf("%d", groupindex+1)
groupindex++
// Try to convert to number and set v1, v2, etc.
if val, err := strconv.ParseFloat(captures[i], 64); err == nil {
L.SetGlobal(fmt.Sprintf("v%d", i+1), lua.LNumber(val))
L.SetGlobal("s"+tempName, lua.LString(capture.Value))
val, err := strconv.ParseFloat(capture.Value, 64)
if err == nil {
L.SetGlobal("v"+tempName, lua.LNumber(val))
}
} else {
val, err := strconv.ParseFloat(capture.Value, 64)
if err == nil {
L.SetGlobal(capture.Name, lua.LNumber(val))
} else {
L.SetGlobal(capture.Name, lua.LString(capture.Value))
}
}
}
return nil
}
// FromLua implements the Processor interface for RegexProcessor
func (p *RegexProcessor) FromLua(L *lua.LState) (interface{}, error) {
// Get the modified values after Lua execution
modifications := make(map[int]string)
// Stub to satisfy interface
return nil, nil
}
// Check for modifications to v1-v12 and s1-s12
for i := 0; i < 12; i++ {
// Check both v and s variables to see if any were modified
vVarName := fmt.Sprintf("v%d", i+1)
sVarName := fmt.Sprintf("s%d", i+1)
// FromLua implements the Processor interface for RegexProcessor
func (p *RegexProcessor) FromLuaCustom(L *lua.LState, captureGroups []*CaptureGroup) ([]*CaptureGroup, error) {
captureIndex := 0
for _, capture := range captureGroups {
if capture.Name == "" {
capture.Name = fmt.Sprintf("%d", captureIndex+1)
vLuaVal := L.GetGlobal(vVarName)
sLuaVal := L.GetGlobal(sVarName)
vVarName := fmt.Sprintf("v%s", capture.Name)
sVarName := fmt.Sprintf("s%s", capture.Name)
captureIndex++
// If our value is a number then it's very likely we want it to be a number
// And not a string
// If we do want it to be a string we will cast it into a string in lua
// wait that wouldn't work... Casting v to a string would not load it here
if vLuaVal.Type() == lua.LTNumber {
modifications[i] = vLuaVal.String()
continue
vLuaVal := L.GetGlobal(vVarName)
sLuaVal := L.GetGlobal(sVarName)
if sLuaVal.Type() == lua.LTString {
capture.Updated = sLuaVal.String()
}
// Numbers have priority
if vLuaVal.Type() == lua.LTNumber {
capture.Updated = vLuaVal.String()
}
} else {
// Easy shit
capture.Updated = L.GetGlobal(capture.Name).String()
}
if sLuaVal.Type() == lua.LTString {
modifications[i] = sLuaVal.String()
continue
}
}
return modifications, nil
return captureGroups, nil
}
type CaptureGroup struct {
Name string
Value string
Updated string
Range [2]int
}
type ReplaceCommand struct {
From int
To int
With string
}
// ProcessContent applies regex replacement with Lua processing
func (p *RegexProcessor) ProcessContent(content string, pattern string, luaExpr string) (string, int, int, error) {
// Handle special pattern modifications
if !strings.HasPrefix(pattern, "(?s)") {
pattern = "(?s)" + pattern
}
pattern = ResolveRegexPlaceholders(pattern)
compiledPattern, err := regexp.Compile(pattern)
if err != nil {
log.Printf("Error compiling pattern: %v", err)
return "", 0, 0, fmt.Errorf("error compiling pattern: %v", err)
}
log.Printf("Compiled pattern successfully: %s", pattern)
previous := luaExpr
luaExpr = BuildLuaScript(luaExpr)
fmt.Printf("Changing Lua expression from: %s to: %s\n", previous, luaExpr)
L, err := NewLuaState()
if err != nil {
return "", 0, 0, fmt.Errorf("error creating Lua state: %v", err)
}
defer L.Close()
log.Printf("Changing Lua expression from: %s to: %s", previous, luaExpr)
// Initialize Lua environment
modificationCount := 0
@@ -123,12 +115,27 @@ func (p *RegexProcessor) ProcessContent(content string, pattern string, luaExpr
// Process all regex matches
result := content
indices := compiledPattern.FindAllStringSubmatchIndex(content, -1)
log.Printf("Found %d matches in the content", len(indices))
// We walk backwards because we're replacing something with something else that might be longer
// And in the case it is longer than the original all indicces past that change will be fucked up
// By going backwards we fuck up all the indices to the end of the file that we don't care about
// Because there either aren't any (last match) or they're already modified (subsequent matches)
for i := len(indices) - 1; i >= 0; i-- {
L, err := NewLuaState()
if err != nil {
log.Printf("Error creating Lua state: %v", err)
return "", 0, 0, fmt.Errorf("error creating Lua state: %v", err)
}
// Hmm... Maybe we don't want to defer this..
// Maybe we want to close them every iteration
// We'll leave it as is for now
defer L.Close()
log.Printf("Lua state created successfully")
matchIndices := indices[i]
log.Printf("Processing match indices: %v", matchIndices)
// Why we're doing this whole song and dance of indices is to properly handle empty matches
// Plus it's a little cleaner to surgically replace our matches
// If we were to use string.replace and encountered an empty match there'd be nothing to replace
@@ -137,60 +144,142 @@ func (p *RegexProcessor) ProcessContent(content string, pattern string, luaExpr
// As if concatenating in the middle of the array
// Plus it supports lookarounds
match := content[matchIndices[0]:matchIndices[1]]
log.Printf("Matched content: %s", match)
groups := matchIndices[2:]
if len(groups) <= 0 {
fmt.Println("No capture groups for lua to chew on")
log.Println("No capture groups for lua to chew on")
continue
}
if len(groups)%2 == 1 {
fmt.Println("Odd number of indices of groups, what the fuck?")
log.Println("Odd number of indices of groups, what the fuck?")
continue
}
captures := make([]string, 0, len(groups)/2)
for j := 0; j < len(groups); j += 2 {
captures = append(captures, content[groups[j]:groups[j+1]])
for _, index := range groups {
if index == -1 {
// return "", 0, 0, fmt.Errorf("negative indices encountered: %v. This indicates that there was an issue with the match indices, possibly due to an empty match or an unexpected pattern. Please check the regex pattern and input content.", matchIndices)
log.Printf("Negative indices encountered: %v. This indicates that there was an issue with the match indices, possibly due to an empty match or an unexpected pattern. This is not an error but it's possibly not what you want.", matchIndices)
continue
}
}
if err := p.ToLua(L, captures); err != nil {
fmt.Println("Error setting Lua variables:", err)
// We have to use array to preserve order
// Very important for the reconstruction step
// Because we must overwrite the values in reverse order
// See comments a few dozen lines above for more details
captureGroups := make([]*CaptureGroup, 0, len(groups)/2)
groupNames := compiledPattern.SubexpNames()[1:]
for i, name := range groupNames {
// if name == "" {
// continue
// }
start := groups[i*2]
end := groups[i*2+1]
if start == -1 || end == -1 {
continue
}
captureGroups = append(captureGroups, &CaptureGroup{
Name: name,
Value: content[start:end],
Range: [2]int{start, end},
})
}
log.Printf("Capture groups: %v", captureGroups)
if err := p.ToLua(L, captureGroups); err != nil {
log.Printf("Error setting Lua variables: %v", err)
continue
}
log.Println("Lua variables set successfully")
if err := L.DoString(luaExpr); err != nil {
fmt.Printf("Error executing Lua code %s for group %s: %v", luaExpr, captures, err)
log.Printf("Error executing Lua code %s for groups %+v: %v", luaExpr, captureGroups, err)
continue
}
log.Println("Lua code executed successfully")
// Get modifications from Lua
modResult, err := p.FromLua(L)
captureGroups, err = p.FromLuaCustom(L, captureGroups)
if err != nil {
fmt.Println("Error getting modifications:", err)
log.Printf("Error getting modifications: %v", err)
continue
}
// Apply modifications to the matched text
modsMap, ok := modResult.(map[int]string)
if !ok || len(modsMap) == 0 {
fmt.Println("No modifications to apply")
continue
replacement := ""
replacementVar := L.GetGlobal("replacement")
if replacementVar.Type() != lua.LTNil {
replacement = replacementVar.String()
}
if replacement == "" {
commands := make([]ReplaceCommand, 0, len(captureGroups))
// Apply the modifications to the original match
replacement = match
for _, capture := range captureGroups {
log.Printf("Applying modification: %s", capture.Updated)
// Indices of the group are relative to content
// To relate them to match we have to subtract the match start index
// replacement = replacement[:groupStart] + newVal + replacement[groupEnd:]
commands = append(commands, ReplaceCommand{
From: capture.Range[0] - matchIndices[0],
To: capture.Range[1] - matchIndices[0],
With: capture.Updated,
})
}
// Apply the modifications to the original match
replacement := match
for i := len(modsMap) - 1; i >= 0; i-- {
newVal := modsMap[i]
// Indices of the group are relative to content
// To relate them to match we have to subtract the match start index
groupStart := groups[i*2] - matchIndices[0]
groupEnd := groups[i*2+1] - matchIndices[0]
replacement = replacement[:groupStart] + newVal + replacement[groupEnd:]
sort.Slice(commands, func(i, j int) bool {
return commands[i].From > commands[j].From
})
for _, command := range commands {
replacement = replacement[:command.From] + command.With + replacement[command.To:]
}
}
modificationCount++
result = result[:matchIndices[0]] + replacement + result[matchIndices[1]:]
log.Printf("Modification count updated: %d", modificationCount)
}
log.Printf("Process completed with %d modifications", modificationCount)
return result, modificationCount, len(indices), nil
}
// The order of these replaces is important
// This one handles !num-s inside of named capture groups
// If it were not here our !num in a named capture group would
// Expand to another capture group in the capture group
// We really only want one (our named) capture group
func ResolveRegexPlaceholders(pattern string) string {
// Handle special pattern modifications
if !strings.HasPrefix(pattern, "(?s)") {
pattern = "(?s)" + pattern
log.Printf("Pattern modified to include (?s): %s", pattern)
}
namedGroupNum := regexp.MustCompile(`(?:(\?<[^>]+>)(!num))`)
pattern = namedGroupNum.ReplaceAllStringFunc(pattern, func(match string) string {
parts := namedGroupNum.FindStringSubmatch(match)
if len(parts) != 3 {
return match
}
replacement := `-?\d*\.?\d+`
return parts[1] + replacement
})
pattern = strings.ReplaceAll(pattern, "!num", `"?(-?\d*\.?\d+)"?`)
pattern = strings.ReplaceAll(pattern, "!any", `.*?`)
repPattern := regexp.MustCompile(`!rep\(([^,]+),\s*(\d+)\)`)
// !rep(pattern, count) repeats the pattern n times
// Inserting !any between each repetition
pattern = repPattern.ReplaceAllStringFunc(pattern, func(match string) string {
parts := repPattern.FindStringSubmatch(match)
if len(parts) != 3 {
return match
}
repeatedPattern := parts[1]
count := parts[2]
repetitions, _ := strconv.Atoi(count)
return strings.Repeat(repeatedPattern+".*?", repetitions-1) + repeatedPattern
})
return pattern
}

View File

@@ -35,10 +35,11 @@ func TestBuildLuaScript(t *testing.T) {
{"v1 * 2", "v1 = v1 * 2"},
{"v1 * v2", "v1 = v1 * v2"},
{"v1 / v2", "v1 = v1 / v2"},
{"12", "v1 = 12"},
}
for _, c := range cases {
result := BuildLuaScript(c.input)
result := PrependLuaAssignment(c.input)
if result != c.expected {
t.Errorf("BuildLuaScript(%q): expected %q, got %q", c.input, c.expected, result)
}
@@ -546,3 +547,793 @@ func TestEdgeCases(t *testing.T) {
})
}
}
func TestNamedCaptureGroups(t *testing.T) {
content := `<config>
<item>
<value>100</value>
</item>
</config>`
expected := `<config>
<item>
<value>200</value>
</item>
</config>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(content, `(?s)<value>(?<amount>\d+)</value>`, "amount = amount * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCaptureGroupsNum(t *testing.T) {
content := `<config>
<item>
<value>100</value>
</item>
</config>`
expected := `<config>
<item>
<value>200</value>
</item>
</config>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(content, `(?s)<value>(?<amount>!num)</value>`, "amount = amount * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestMultipleNamedCaptureGroups(t *testing.T) {
content := `<product>
<name>Widget</name>
<price>15.99</price>
<quantity>10</quantity>
</product>`
expected := `<product>
<name>WIDGET</name>
<price>23.99</price>
<quantity>15</quantity>
</product>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`(?s)<name>(?<prodName>[^<]+)</name>.*?<price>(?<prodPrice>\d+\.\d+)</price>.*?<quantity>(?<prodQty>\d+)</quantity>`,
`prodName = string.upper(prodName)
prodPrice = round(prodPrice + 8, 2)
prodQty = prodQty + 5`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestMixedIndexedAndNamedCaptures(t *testing.T) {
content := `<entry>
<id>12345</id>
<data>value</data>
</entry>`
expected := `<entry>
<id>24690</id>
<data>VALUE</data>
</entry>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`(?s)<id>(\d+)</id>.*?<data>(?<dataField>[^<]+)</data>`,
`v1 = v1 * 2
dataField = string.upper(dataField)`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestComplexNestedNamedCaptures(t *testing.T) {
content := `<person>
<details>
<name>John Smith</name>
<age>32</age>
</details>
<contact>
<email>john@example.com</email>
</contact>
</person>`
expected := `<person>
<details>
<name>JOHN SMITH (32)</name>
<age>32</age>
</details>
<contact>
<email>john@example.com</email>
</contact>
</person>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`(?s)<details>.*?<name>(?<fullName>[^<]+)</name>.*?<age>(?<age>\d+)</age>`,
`fullName = string.upper(fullName) .. " (" .. age .. ")"`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCaptureWithVariableReadback(t *testing.T) {
content := `<stats>
<health>100</health>
<mana>200</mana>
</stats>`
expected := `<stats>
<health>150</health>
<mana>300</mana>
</stats>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`(?s)<health>(?<hp>\d+)</health>.*?<mana>(?<mp>\d+)</mana>`,
`hp = hp * 1.5
mp = mp * 1.5`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCaptureWithSpecialCharsInName(t *testing.T) {
content := `<data value="42" min="10" max="100" />`
expected := `<data value="84" min="10" max="100" />`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<data value="(?<val_1>\d+)"`,
`val_1 = val_1 * 2`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestEmptyNamedCapture(t *testing.T) {
content := `<tag attr="" />`
expected := `<tag attr="default" />`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`attr="(?<value>.*?)"`,
`value = value == "" and "default" or value`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestMultipleNamedCapturesInSameLine(t *testing.T) {
content := `<rect x="10" y="20" width="100" height="50" />`
expected := `<rect x="20" y="40" width="200" height="100" />`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`x="(?<x>\d+)" y="(?<y>\d+)" width="(?<w>\d+)" height="(?<h>\d+)"`,
`x = x * 2
y = y * 2
w = w * 2
h = h * 2`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestConditionalNamedCapture(t *testing.T) {
content := `
<item status="active" count="5" />
<item status="inactive" count="10" />
`
expected := `
<item status="active" count="10" />
<item status="inactive" count="10" />
`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<item status="(?<status>[^"]+)" count="(?<count>\d+)"`,
`count = status == "active" and count * 2 or count`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 2 {
t.Errorf("Expected 2 matches, got %d", matches)
}
if mods != 2 {
t.Errorf("Expected 2 modifications, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestLuaFunctionsOnNamedCaptures(t *testing.T) {
content := `
<user name="john doe" role="user" />
<user name="jane smith" role="admin" />
`
expected := `
<user name="John Doe" role="user" />
<user name="JANE SMITH" role="admin" />
`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<user name="(?<name>[^"]+)" role="(?<role>[^"]+)"`,
`-- Capitalize first letters for regular users
if role == "user" then
name = name:gsub("(%w)(%w*)", function(first, rest) return first:upper()..rest end):gsub(" (%w)(%w*)", " %1%2")
else
-- Uppercase for admins
name = string.upper(name)
end`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 2 {
t.Errorf("Expected 2 matches, got %d", matches)
}
if mods != 2 {
t.Errorf("Expected 2 modifications, got %d", mods)
}
// For simpler tests, we can use this. More complex string modifications
// might need additional transformations before comparison
normalizedResult := normalizeWhitespace(result)
normalizedExpected := normalizeWhitespace(expected)
if normalizedResult != normalizedExpected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCaptureWithMath(t *testing.T) {
content := `
<item price="19.99" quantity="3" />
`
expected := `
<item price="19.99" quantity="3" total="59.97" />
`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<item price="(?<price>\d+\.\d+)" quantity="(?<qty>\d+)"!any$`,
`-- Calculate and add total
replacement = string.format('<item price="%s" quantity="%s" total="%.2f" />',
price, qty, price * qty)`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
result = normalizeWhitespace(result)
expected = normalizeWhitespace(expected)
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCaptureWithGlobals(t *testing.T) {
content := `<temp unit="C">25</temp>`
expected := `<temp unit="F">77</temp>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<temp unit="(?<unit>[CF]?)">(?<value>\d+)</temp>`,
`if unit == "C" then
value = value * 9/5 + 32
unit = "F"
elseif unit == "F" then
value = (value - 32) * 5/9
unit = "C"
end`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestMixedDynamicAndNamedCaptures(t *testing.T) {
content := `
<color rgb="255,0,0" name="red" />
<color rgb="0,255,0" name="green" />
`
expected := `
<color rgb="255,0,0" name="RED" hex="#FF0000" />
<color rgb="0,255,0" name="GREEN" hex="#00FF00" />
`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<color rgb="(?<r>\d+),(?<g>\d+),(?<b>\d+)" name="(?<colorName>[^"]+)" />`,
`-- Uppercase the name
colorName = string.upper(colorName)
-- Create hex color
local hex = string.format("#%02X%02X%02X", tonumber(r), tonumber(g), tonumber(b))
-- Replace the entire match
replacement = string.format('<color rgb="%s,%s,%s" name="%s" hex="%s" />',
r, g, b, colorName, hex)`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 2 {
t.Errorf("Expected 2 matches, got %d", matches)
}
if mods != 2 {
t.Errorf("Expected 2 modifications, got %d", mods)
}
result = normalizeWhitespace(result)
expected = normalizeWhitespace(expected)
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCapturesWithMultipleReferences(t *testing.T) {
content := `<text>Hello world</text>`
expected := `<text format="uppercase" length="11">HELLO WORLD</text>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<text>(?<content>[^<]+)</text>`,
`local uppercaseContent = string.upper(content)
local contentLength = string.len(content)
replacement = string.format('<text format="uppercase" length="%d">%s</text>',
contentLength, uppercaseContent)`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCaptureWithJsonData(t *testing.T) {
content := `<data>{"name":"John","age":30}</data>`
expected := `<data>{"name":"JOHN","age":30}</data>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<data>(?<json>\{.*?\})</data>`,
`-- Parse JSON (simplified, assumes valid JSON)
local name = json:match('"name":"([^"]+)"')
local upperName = string.upper(name)
json = json:gsub('"name":"([^"]+)"', '"name":"' .. upperName .. '"')`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestNamedCaptureInXML(t *testing.T) {
content := `
<product>
<sku>ABC-123</sku>
<price currency="USD">19.99</price>
<stock>25</stock>
</product>
`
expected := `
<product>
<sku>ABC-123</sku>
<price currency="USD">23.99</price>
<stock>20</stock>
</product>
`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`(?s)<price currency="(?<currency>[^"]+)">(?<price>\d+\.\d+)</price>.*?<stock>(?<stock>\d+)</stock>`,
`-- Add 20% to price if USD
if currency == "USD" then
price = round(price * 1.20, 2)
end
-- Reduce stock by 5
stock = stock - 5`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestComprehensiveNamedCaptures(t *testing.T) {
content := `
<products>
<product sku="AB-123" status="in-stock">
<name>Widget A</name>
<price currency="USD">19.99</price>
<quantity>15</quantity>
</product>
<product sku="CD-456" status="out-of-stock">
<name>Widget B</name>
<price currency="EUR">29.99</price>
<quantity>0</quantity>
</product>
<product sku="EF-789" status="in-stock">
<name>Widget C</name>
<price currency="GBP">39.99</price>
<quantity>5</quantity>
</product>
</products>
`
expected := `
<products>
<product sku="AB-123" status="in-stock" discounted="true">
<name>WIDGET A</name>
<price currency="USD">15.99</price>
<quantity>15</quantity>
</product>
<product sku="CD-456" status="out-of-stock" discounted="false">
<name>Widget B</name>
<price currency="EUR">29.99</price>
<quantity>0</quantity>
</product>
<product sku="EF-789" status="in-stock" discounted="true">
<name>WIDGET C</name>
<price currency="GBP">39.99</price>
<quantity>5</quantity>
</product>
</products>
`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`(?s)<product sku="(?<sku>[^"]+)" status="(?<status>[^"]+)"[^>]*>\s*<name>(?<product_name>[^<]+)</name>\s*<price currency="(?<currency>[^"]+)">(?<price>\d+\.\d+)</price>\s*<quantity>(?<qty>\d+)</quantity>`,
`-- Only process in-stock items
if status == "in-stock" then
-- Transform name to uppercase
product_name = string.upper(product_name)
-- Apply discount based on currency
local discounted = true
if currency == "USD" then
price = round(price * 0.8, 2) -- 20% discount for USD
elseif currency == "GBP" then
price = round(price * 0.8, 2) -- 20% discount for GBP
price = price + 8 -- Add shipping cost for GBP
else
discounted = false
end
-- Add discounted attribute
replacement = string.format('<product sku="%s" status="%s" discounted="%s">\n\t\t\t<name>%s</name>\n\t\t\t<price currency="%s">%.2f</price>\n\t\t\t<quantity>%s</quantity>',
sku, status, tostring(discounted), product_name, currency, price, qty)
else
-- Add discounted attribute for out-of-stock items (always false)
replacement = string.format('<product sku="%s" status="%s" discounted="false">\n\t\t\t<name>%s</name>\n\t\t\t<price currency="%s">%s</price>\n\t\t\t<quantity>%s</quantity>',
sku, status, product_name, currency, price, qty)
end`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 3 {
t.Errorf("Expected 3 matches, got %d", matches)
}
if mods != 3 {
t.Errorf("Expected 3 modifications, got %d", mods)
}
// Normalize whitespace for comparison
normalizedResult := normalizeWhitespace(result)
normalizedExpected := normalizeWhitespace(expected)
if normalizedResult != normalizedExpected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestVariousNamedCaptureFormats(t *testing.T) {
content := `
<data>
<entry id="1" value="100" />
<entry id="2" value="200" status="active" />
<entry id="3" value="300" status="inactive" />
</data>
`
expected := `
<data>
<entry id="ID-1" value="200" />
<entry id="ID-2" value="400" status="ACTIVE" />
<entry id="ID-3" value="300" status="inactive" />
</data>
`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`<entry id="(?<id_num>\d+)" value="(?<val>\d+)"(?: status="(?<status>[^"]*)")? />`,
`-- Prefix the ID with "ID-"
id_num = "ID-" .. id_num
print(id_num)
print(val)
print(status)
-- Double the value except for inactive status
if not status or status ~= "inactive" then
val = val * 2
end
-- Convert status to uppercase if present and active
if status and status == "active" then
status = string.upper(status)
end
-- Build the replacement based on whether status exists
if status then
replacement = string.format('<entry id="%s" value="%s" status="%s" />', id_num, val, status)
else
replacement = string.format('<entry id="%s" value="%s" />', id_num, val)
end`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 3 {
t.Errorf("Expected 3 matches, got %d", matches)
}
if mods != 3 {
t.Errorf("Expected 3 modifications, got %d", mods)
}
normalizedResult := normalizeWhitespace(result)
normalizedExpected := normalizeWhitespace(expected)
if normalizedResult != normalizedExpected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}
func TestSimpleNamedCapture(t *testing.T) {
content := `<product name="Widget" price="19.99"/>`
expected := `<product name="WIDGET" price="19.99"/>`
p := &RegexProcessor{}
result, mods, matches, err := p.ProcessContent(
content,
`name="(?<product_name>[^"]+)"`,
`product_name = string.upper(product_name)`)
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != expected {
t.Errorf("Expected content to be:\n%s\n\nGot:\n%s", expected, result)
}
}

View File

@@ -2,8 +2,8 @@ package processor
import (
"fmt"
"os"
"path/filepath"
"log"
"modify/processor/xpath"
"strings"
"github.com/antchfx/xmlquery"
@@ -13,44 +13,18 @@ import (
// XMLProcessor implements the Processor interface for XML documents
type XMLProcessor struct{}
// Process implements the Processor interface for XMLProcessor
func (p *XMLProcessor) Process(filename string, pattern string, luaExpr string) (int, int, error) {
// Read file content
fullPath := filepath.Join(".", filename)
content, err := os.ReadFile(fullPath)
if err != nil {
return 0, 0, fmt.Errorf("error reading file: %v", err)
}
fileContent := string(content)
// Process the content
modifiedContent, modCount, matchCount, err := p.ProcessContent(fileContent, pattern, luaExpr)
if err != nil {
return 0, 0, err
}
// If we made modifications, save the file
if modCount > 0 {
err = os.WriteFile(fullPath, []byte(modifiedContent), 0644)
if err != nil {
return 0, 0, fmt.Errorf("error writing file: %v", err)
}
}
return modCount, matchCount, nil
}
// ProcessContent implements the Processor interface for XMLProcessor
func (p *XMLProcessor) ProcessContent(content string, pattern string, luaExpr string) (string, int, int, error) {
func (p *XMLProcessor) ProcessContent(content string, path string, luaExpr string) (string, int, int, error) {
// Parse XML document
// We can't really use encoding/xml here because it requires a pre defined struct
// And we HAVE TO parse dynamic unknown XML
doc, err := xmlquery.Parse(strings.NewReader(content))
if err != nil {
return content, 0, 0, fmt.Errorf("error parsing XML: %v", err)
}
// Find nodes matching the XPath pattern
nodes, err := xmlquery.QueryAll(doc, pattern)
nodes, err := xpath.Get(doc, path)
if err != nil {
return content, 0, 0, fmt.Errorf("error executing XPath: %v", err)
}
@@ -60,104 +34,45 @@ func (p *XMLProcessor) ProcessContent(content string, pattern string, luaExpr st
return content, 0, 0, nil
}
// Initialize Lua
L := lua.NewState()
defer L.Close()
// Load math library
L.Push(L.GetGlobal("require"))
L.Push(lua.LString("math"))
if err := L.PCall(1, 1, nil); err != nil {
return content, 0, 0, fmt.Errorf("error loading Lua math library: %v", err)
}
// Load helper functions
if err := InitLuaHelpers(L); err != nil {
return content, 0, 0, err
}
// Apply modifications to each node
modCount := 0
for _, node := range nodes {
// Reset Lua state for each node
L.SetGlobal("v1", lua.LNil)
L.SetGlobal("s1", lua.LNil)
// Get the node value
var originalValue string
if node.Type == xmlquery.AttributeNode {
originalValue = node.InnerText()
} else if node.Type == xmlquery.TextNode {
originalValue = node.Data
} else {
originalValue = node.InnerText()
L, err := NewLuaState()
if err != nil {
return content, 0, 0, fmt.Errorf("error creating Lua state: %v", err)
}
defer L.Close()
// Convert to Lua variables
err = p.ToLua(L, originalValue)
err = p.ToLua(L, node)
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error converting to Lua: %v", err)
}
// Execute Lua script
if err := L.DoString(luaExpr); err != nil {
err = L.DoString(BuildLuaScript(luaExpr))
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error executing Lua: %v", err)
}
// Get modified value
result, err := p.FromLua(L)
if err != nil {
return content, modCount, matchCount, fmt.Errorf("error getting result from Lua: %v", err)
}
log.Printf("%#v", result)
newValue, ok := result.(string)
if !ok {
return content, modCount, matchCount, fmt.Errorf("expected string result from Lua, got %T", result)
}
// Skip if no change
if newValue == originalValue {
modified := false
modified = L.GetGlobal("modified").String() == "true"
if !modified {
log.Printf("No changes made to node at path: %s", node.Data)
continue
}
// Apply modification
if node.Type == xmlquery.AttributeNode {
// For attribute nodes, update the attribute value
node.Parent.Attr = append([]xmlquery.Attr{}, node.Parent.Attr...)
for i, attr := range node.Parent.Attr {
if attr.Name.Local == node.Data {
node.Parent.Attr[i].Value = newValue
break
}
}
} else if node.Type == xmlquery.TextNode {
// For text nodes, update the text content
node.Data = newValue
} else {
// For element nodes, replace inner text
// Simple approach: set the InnerText directly if there are no child elements
if node.FirstChild == nil || (node.FirstChild != nil && node.FirstChild.Type == xmlquery.TextNode && node.FirstChild.NextSibling == nil) {
if node.FirstChild != nil {
node.FirstChild.Data = newValue
} else {
// Create a new text node and add it as the first child
textNode := &xmlquery.Node{
Type: xmlquery.TextNode,
Data: newValue,
}
node.FirstChild = textNode
}
} else {
// Complex case: node has mixed content or child elements
// Replace just the text content while preserving child elements
// This is a simplified approach - more complex XML may need more robust handling
for child := node.FirstChild; child != nil; child = child.NextSibling {
if child.Type == xmlquery.TextNode {
child.Data = newValue
break // Update only the first text node
}
}
}
// Apply modification based on the result
if updatedValue, ok := result.(string); ok {
// If the result is a simple string, update the node value directly
xpath.Set(doc, path, updatedValue)
} else if nodeData, ok := result.(map[string]interface{}); ok {
// If the result is a map, apply more complex updates
updateNodeFromMap(node, nodeData)
}
modCount++
@@ -169,49 +84,329 @@ func (p *XMLProcessor) ProcessContent(content string, pattern string, luaExpr st
declaration := doc.FirstChild.OutputXML(true)
// Remove the firstChild (declaration) before serializing the rest of the document
doc.FirstChild = doc.FirstChild.NextSibling
return declaration + doc.OutputXML(true), modCount, matchCount, nil
return ConvertToNamedEntities(declaration + doc.OutputXML(true)), modCount, matchCount, nil
}
return doc.OutputXML(true), modCount, matchCount, nil
// Convert numeric entities to named entities for better readability
return ConvertToNamedEntities(doc.OutputXML(true)), modCount, matchCount, nil
}
func (p *XMLProcessor) ToLua(L *lua.LState, data interface{}) error {
table, err := p.ToLuaTable(L, data)
if err != nil {
return err
}
L.SetGlobal("v", table)
return nil
}
// ToLua converts XML node values to Lua variables
func (p *XMLProcessor) ToLua(L *lua.LState, data interface{}) error {
value, ok := data.(string)
func (p *XMLProcessor) ToLuaTable(L *lua.LState, data interface{}) (lua.LValue, error) {
// Check if data is an xmlquery.Node
node, ok := data.(*xmlquery.Node)
if !ok {
return fmt.Errorf("expected string value, got %T", data)
return nil, fmt.Errorf("expected xmlquery.Node, got %T", data)
}
// Set as string variable
L.SetGlobal("s1", lua.LString(value))
// Create a simple table with essential data
table := L.NewTable()
// Try to convert to number if possible
L.SetGlobal("v1", lua.LNumber(0)) // Default to 0
if err := L.DoString(fmt.Sprintf("v1 = tonumber(%q) or 0", value)); err != nil {
return fmt.Errorf("error converting value to number: %v", err)
// For element nodes, just provide basic info
L.SetField(table, "type", lua.LString(nodeTypeToString(node.Type)))
L.SetField(table, "name", lua.LString(node.Data))
L.SetField(table, "value", lua.LString(node.InnerText()))
// Add children if any
children := L.NewTable()
for child := node.FirstChild; child != nil; child = child.NextSibling {
childTable, err := p.ToLuaTable(L, child)
if err == nil {
children.Append(childTable)
}
}
L.SetField(table, "children", children)
return nil
attrs := L.NewTable()
if len(node.Attr) > 0 {
for _, attr := range node.Attr {
L.SetField(attrs, attr.Name.Local, lua.LString(attr.Value))
}
}
L.SetField(table, "attr", attrs)
return table, nil
}
// FromLua gets modified values from Lua
func (p *XMLProcessor) FromLua(L *lua.LState) (interface{}, error) {
// Check if string variable was modified
s1 := L.GetGlobal("s1")
if s1 != lua.LNil {
if s1Str, ok := s1.(lua.LString); ok {
return string(s1Str), nil
}
luaValue := L.GetGlobal("v")
// Handle string values directly
if luaValue.Type() == lua.LTString {
return luaValue.String(), nil
}
// Check if numeric variable was modified
v1 := L.GetGlobal("v1")
if v1 != lua.LNil {
if v1Num, ok := v1.(lua.LNumber); ok {
return fmt.Sprintf("%v", v1Num), nil
}
// Handle tables (for attributes and more complex updates)
if luaValue.Type() == lua.LTTable {
return luaTableToMap(L, luaValue.(*lua.LTable)), nil
}
// Default return empty string
return "", nil
return luaValue.String(), nil
}
// Simple helper to convert a Lua table to a Go map
func luaTableToMap(L *lua.LState, table *lua.LTable) map[string]interface{} {
result := make(map[string]interface{})
table.ForEach(func(k, v lua.LValue) {
if k.Type() == lua.LTString {
key := k.String()
if v.Type() == lua.LTTable {
result[key] = luaTableToMap(L, v.(*lua.LTable))
} else {
result[key] = v.String()
}
}
})
return result
}
// Simple helper to convert node type to string
func nodeTypeToString(nodeType xmlquery.NodeType) string {
switch nodeType {
case xmlquery.ElementNode:
return "element"
case xmlquery.TextNode:
return "text"
case xmlquery.AttributeNode:
return "attribute"
default:
return "other"
}
}
// Helper function to update an XML node from a map
func updateNodeFromMap(node *xmlquery.Node, data map[string]interface{}) {
// Update node value if present
if value, ok := data["value"]; ok {
if strValue, ok := value.(string); ok {
// For element nodes, replace text content
if node.Type == xmlquery.ElementNode {
// Find the first text child if it exists
var textNode *xmlquery.Node
for child := node.FirstChild; child != nil; child = child.NextSibling {
if child.Type == xmlquery.TextNode {
textNode = child
break
}
}
if textNode != nil {
// Update existing text node
textNode.Data = strValue
} else {
// Create new text node
newText := &xmlquery.Node{
Type: xmlquery.TextNode,
Data: strValue,
Parent: node,
}
// Insert at beginning of children
if node.FirstChild != nil {
newText.NextSibling = node.FirstChild
node.FirstChild.PrevSibling = newText
node.FirstChild = newText
} else {
node.FirstChild = newText
node.LastChild = newText
}
}
} else if node.Type == xmlquery.TextNode {
// Directly update text node
node.Data = strValue
} else if node.Type == xmlquery.AttributeNode {
// Update attribute value
if node.Parent != nil {
for i, attr := range node.Parent.Attr {
if attr.Name.Local == node.Data {
node.Parent.Attr[i].Value = strValue
break
}
}
}
}
}
}
// Update attributes if present
if attrs, ok := data["attr"].(map[string]interface{}); ok && node.Type == xmlquery.ElementNode {
for name, value := range attrs {
if strValue, ok := value.(string); ok {
// Look for existing attribute
found := false
for i, attr := range node.Attr {
if attr.Name.Local == name {
node.Attr[i].Value = strValue
found = true
break
}
}
// Add new attribute if not found
if !found {
node.Attr = append(node.Attr, xmlquery.Attr{
Name: struct {
Space, Local string
}{Local: name},
Value: strValue,
})
}
}
}
}
}
// Helper function to get a string representation of node type
func nodeTypeName(nodeType xmlquery.NodeType) string {
switch nodeType {
case xmlquery.ElementNode:
return "element"
case xmlquery.TextNode:
return "text"
case xmlquery.AttributeNode:
return "attribute"
case xmlquery.CommentNode:
return "comment"
case xmlquery.DeclarationNode:
return "declaration"
default:
return "unknown"
}
}
// ConvertToNamedEntities replaces numeric XML entities with their named counterparts
func ConvertToNamedEntities(xml string) string {
// Basic XML entities
replacements := map[string]string{
// Basic XML entities
"&#34;": "&quot;", // double quote
"&#39;": "&apos;", // single quote
"&#60;": "&lt;", // less than
"&#62;": "&gt;", // greater than
"&#38;": "&amp;", // ampersand
// Common symbols
"&#160;": "&nbsp;", // non-breaking space
"&#169;": "&copy;", // copyright
"&#174;": "&reg;", // registered trademark
"&#8364;": "&euro;", // euro
"&#163;": "&pound;", // pound
"&#165;": "&yen;", // yen
"&#162;": "&cent;", // cent
"&#167;": "&sect;", // section
"&#8482;": "&trade;", // trademark
"&#9824;": "&spades;", // spade
"&#9827;": "&clubs;", // club
"&#9829;": "&hearts;", // heart
"&#9830;": "&diams;", // diamond
// Special characters
"&#161;": "&iexcl;", // inverted exclamation
"&#191;": "&iquest;", // inverted question
"&#171;": "&laquo;", // left angle quotes
"&#187;": "&raquo;", // right angle quotes
"&#183;": "&middot;", // middle dot
"&#8226;": "&bull;", // bullet
"&#8230;": "&hellip;", // horizontal ellipsis
"&#8242;": "&prime;", // prime
"&#8243;": "&Prime;", // double prime
"&#8254;": "&oline;", // overline
"&#8260;": "&frasl;", // fraction slash
// Math symbols
"&#177;": "&plusmn;", // plus-minus
"&#215;": "&times;", // multiplication
"&#247;": "&divide;", // division
"&#8734;": "&infin;", // infinity
"&#8776;": "&asymp;", // almost equal
"&#8800;": "&ne;", // not equal
"&#8804;": "&le;", // less than or equal
"&#8805;": "&ge;", // greater than or equal
"&#8721;": "&sum;", // summation
"&#8730;": "&radic;", // square root
"&#8747;": "&int;", // integral
// Accented characters
"&#192;": "&Agrave;", // A grave
"&#193;": "&Aacute;", // A acute
"&#194;": "&Acirc;", // A circumflex
"&#195;": "&Atilde;", // A tilde
"&#196;": "&Auml;", // A umlaut
"&#197;": "&Aring;", // A ring
"&#198;": "&AElig;", // AE ligature
"&#199;": "&Ccedil;", // C cedilla
"&#200;": "&Egrave;", // E grave
"&#201;": "&Eacute;", // E acute
"&#202;": "&Ecirc;", // E circumflex
"&#203;": "&Euml;", // E umlaut
"&#204;": "&Igrave;", // I grave
"&#205;": "&Iacute;", // I acute
"&#206;": "&Icirc;", // I circumflex
"&#207;": "&Iuml;", // I umlaut
"&#208;": "&ETH;", // Eth
"&#209;": "&Ntilde;", // N tilde
"&#210;": "&Ograve;", // O grave
"&#211;": "&Oacute;", // O acute
"&#212;": "&Ocirc;", // O circumflex
"&#213;": "&Otilde;", // O tilde
"&#214;": "&Ouml;", // O umlaut
"&#216;": "&Oslash;", // O slash
"&#217;": "&Ugrave;", // U grave
"&#218;": "&Uacute;", // U acute
"&#219;": "&Ucirc;", // U circumflex
"&#220;": "&Uuml;", // U umlaut
"&#221;": "&Yacute;", // Y acute
"&#222;": "&THORN;", // Thorn
"&#223;": "&szlig;", // Sharp s
"&#224;": "&agrave;", // a grave
"&#225;": "&aacute;", // a acute
"&#226;": "&acirc;", // a circumflex
"&#227;": "&atilde;", // a tilde
"&#228;": "&auml;", // a umlaut
"&#229;": "&aring;", // a ring
"&#230;": "&aelig;", // ae ligature
"&#231;": "&ccedil;", // c cedilla
"&#232;": "&egrave;", // e grave
"&#233;": "&eacute;", // e acute
"&#234;": "&ecirc;", // e circumflex
"&#235;": "&euml;", // e umlaut
"&#236;": "&igrave;", // i grave
"&#237;": "&iacute;", // i acute
"&#238;": "&icirc;", // i circumflex
"&#239;": "&iuml;", // i umlaut
"&#240;": "&eth;", // eth
"&#241;": "&ntilde;", // n tilde
"&#242;": "&ograve;", // o grave
"&#243;": "&oacute;", // o acute
"&#244;": "&ocirc;", // o circumflex
"&#245;": "&otilde;", // o tilde
"&#246;": "&ouml;", // o umlaut
"&#248;": "&oslash;", // o slash
"&#249;": "&ugrave;", // u grave
"&#250;": "&uacute;", // u acute
"&#251;": "&ucirc;", // u circumflex
"&#252;": "&uuml;", // u umlaut
"&#253;": "&yacute;", // y acute
"&#254;": "&thorn;", // thorn
"&#255;": "&yuml;", // y umlaut
}
result := xml
for numeric, named := range replacements {
result = strings.ReplaceAll(result, numeric, named)
}
return result
}

View File

@@ -5,13 +5,21 @@ import (
"testing"
"regexp"
"github.com/antchfx/xmlquery"
lua "github.com/yuin/gopher-lua"
)
// Helper function to normalize whitespace for comparison
func normalizeXMLWhitespace(s string) string {
// Replace all whitespace sequences with a single space
re := regexp.MustCompile(`\s+`)
return re.ReplaceAllString(strings.TrimSpace(s), " ")
s = re.ReplaceAllString(strings.TrimSpace(s), " ")
// Normalize XML entities for comparison
s = ConvertToNamedEntities(s)
return s
}
func TestXMLProcessor_Process_NodeValues(t *testing.T) {
@@ -39,7 +47,7 @@ func TestXMLProcessor_Process_NodeValues(t *testing.T) {
<catalog>
<book id="bk101">
<author>Gambardella, Matthew</author>
<title>XML Developer's Guide</title>
<title>XML Developer&apos;s Guide</title>
<genre>Computer</genre>
<price>89.9</price>
<publish_date>2000-10-01</publish_date>
@@ -56,7 +64,7 @@ func TestXMLProcessor_Process_NodeValues(t *testing.T) {
</catalog>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -93,7 +101,7 @@ func TestXMLProcessor_Process_Attributes(t *testing.T) {
</items>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//item/@price", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//item/@price", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -130,7 +138,7 @@ func TestXMLProcessor_Process_ElementText(t *testing.T) {
</names>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//n/text()", "v = string.upper(v)")
result, modCount, matchCount, err := p.ProcessContent(content, "//n/text()", "v.value = string.upper(v.value)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -171,7 +179,7 @@ func TestXMLProcessor_Process_ElementAddition(t *testing.T) {
</config>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//settings/*", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//settings/*", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -236,7 +244,7 @@ func TestXMLProcessor_Process_ComplexXML(t *testing.T) {
</store>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v = v * 1.2")
result, modCount, matchCount, err := p.ProcessContent(content, "//price", "v.value = round(v.value * 1.2, 3)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -271,19 +279,21 @@ func TestXMLProcessor_ConditionalModification(t *testing.T) {
expected := `<?xml version="1.0" encoding="UTF-8"?>
<inventory>
<item id="1" stock="5" price="8.00"/>
<item id="2" stock="15" price="16.00"/>
<item id="3" stock="0" price="15.00"/>
<item id="1" stock="5" price="8.00"></item>
<item id="2" stock="15" price="16.00"></item>
<item id="3" stock="0" price="15.00"></item>
</inventory>`
p := &XMLProcessor{}
// Apply 20% discount but only for items with stock > 0
luaExpr := `
-- In the table-based approach, attributes are accessible directly
if v.stock and tonumber(v.stock) > 0 then
v.price = tonumber(v.price) * 0.8
if v.attr.stock and tonumber(v.attr.stock) > 0 then
v.attr.price = tonumber(v.attr.price) * 0.8
-- Format to 2 decimal places
v.price = string.format("%.2f", v.price)
v.attr.price = string.format("%.2f", v.attr.price)
else
return false
end
`
result, modCount, matchCount, err := p.ProcessContent(content, "//item", luaExpr)
@@ -327,7 +337,7 @@ func TestXMLProcessor_Process_SpecialCharacters(t *testing.T) {
</data>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//entry", "v = string.upper(v)")
result, modCount, matchCount, err := p.ProcessContent(content, "//entry", "v.value = string.upper(v.value)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -362,15 +372,14 @@ func TestXMLProcessor_Process_ChainedOperations(t *testing.T) {
// Apply multiple operations to the price: add tax, apply discount, round
luaExpr := `
-- When v is a numeric string, we can perform math operations directly
local price = v
local price = v.value
-- Add 15% tax
price = price * 1.15
-- Apply 10% discount
price = price * 0.9
-- Round to 2 decimal places
price = math.floor(price * 100 + 0.5) / 100
v = price
price = round(price, 2)
v.value = price
`
expected := `<?xml version="1.0" encoding="UTF-8"?>
@@ -422,7 +431,7 @@ func TestXMLProcessor_Process_MathFunctions(t *testing.T) {
</measurements>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//measurement", "v = round(v)")
result, modCount, matchCount, err := p.ProcessContent(content, "//measurement", "v.value = round(v.value)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -468,9 +477,9 @@ func TestXMLProcessor_Process_StringOperations(t *testing.T) {
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//email", `
-- With the table approach, v contains the text content directly
v = string.gsub(v, "@.+", "@anon.com")
local username = string.match(v, "(.+)@")
v = string.gsub(username, "%.", "") .. "@anon.com"
v.value = string.gsub(v.value, "@.+", "@anon.com")
local username = string.match(v.value, "(.+)@")
v.value = string.gsub(username, "%.", "") .. "@anon.com"
`)
if err != nil {
@@ -479,7 +488,7 @@ func TestXMLProcessor_Process_StringOperations(t *testing.T) {
// Test phone number masking
result, modCount2, matchCount2, err := p.ProcessContent(result, "//phone", `
v = string.gsub(v, "%d%d%d%-%d%d%d%-%d%d%d%d", function(match)
v.value = string.gsub(v.value, "%d%d%d%-%d%d%d%-%d%d%d%d", function(match)
return string.sub(match, 1, 3) .. "-XXX-XXXX"
end)
`)
@@ -536,14 +545,14 @@ func TestXMLProcessor_Process_DateManipulation(t *testing.T) {
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//date", `
local year, month, day = string.match(v, "(%d%d%d%d)-(%d%d)-(%d%d)")
local year, month, day = string.match(v.value, "(%d%d%d%d)-(%d%d)-(%d%d)")
-- Postpone events by 1 month
month = tonumber(month) + 1
if month > 12 then
month = 1
year = tonumber(year) + 1
end
v = string.format("%04d-%02d-%s", tonumber(year), month, day)
v.value = string.format("%04d-%02d-%s", tonumber(year), month, day)
`)
if err != nil {
@@ -609,36 +618,6 @@ func TestXMLProcessor_Process_Error_InvalidLua(t *testing.T) {
}
}
func TestXMLProcessor_Process_NoChanges(t *testing.T) {
content := `<?xml version="1.0" encoding="UTF-8"?>
<root>
<element>123</element>
</root>`
p := &XMLProcessor{}
result, modCount, matchCount, err := p.ProcessContent(content, "//element", "v1 = v1")
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matchCount != 1 {
t.Errorf("Expected 1 match, got %d", matchCount)
}
if modCount != 0 {
t.Errorf("Expected 0 modifications, got %d", modCount)
}
// Normalize whitespace for comparison
normalizedResult := normalizeXMLWhitespace(result)
normalizedContent := normalizeXMLWhitespace(content)
if normalizedResult != normalizedContent {
t.Errorf("Expected content to be unchanged")
}
}
func TestXMLProcessor_Process_ComplexXPathSelectors(t *testing.T) {
content := `<?xml version="1.0" encoding="UTF-8"?>
<library>
@@ -684,7 +663,7 @@ func TestXMLProcessor_Process_ComplexXPathSelectors(t *testing.T) {
p := &XMLProcessor{}
// Target only fiction books and apply 20% discount to price
result, modCount, matchCount, err := p.ProcessContent(content, "//book[@category='fiction']/price", "v = v * 0.8")
result, modCount, matchCount, err := p.ProcessContent(content, "//book[@category='fiction']/price", "v.value = round(v.value * 0.8, 2)")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -767,13 +746,13 @@ func TestXMLProcessor_Process_NestedStructureModification(t *testing.T) {
p := &XMLProcessor{}
// Boost hero stats by 20%
result, modCount, matchCount, err := p.ProcessContent(content, "//character[@id='hero']/stats/*", "v = math.floor(v * 1.2)")
result, modCount, matchCount, err := p.ProcessContent(content, "//character[@id='hero']/stats/*", "v.value = round(v.value * 1.2)")
if err != nil {
t.Fatalf("Error processing stats content: %v", err)
}
// Also upgrade hero equipment
result, modCount2, matchCount2, err := p.ProcessContent(result, "//character[@id='hero']/equipment/*/@damage|//character[@id='hero']/equipment/*/@defense", "v = v + 2")
result, modCount2, matchCount2, err := p.ProcessContent(result, "//character[@id='hero']/equipment/*/@damage|//character[@id='hero']/equipment/*/@defense", "v.value = v.value + 2")
if err != nil {
t.Fatalf("Error processing equipment content: %v", err)
}
@@ -835,8 +814,8 @@ func TestXMLProcessor_Process_ElementReplacement(t *testing.T) {
luaExpr := `
-- With a proper table approach, this becomes much simpler
local price = tonumber(v.price)
local quantity = tonumber(v.quantity)
local price = tonumber(v.attr.price)
local quantity = tonumber(v.attr.quantity)
-- Add a new total element
v.total = string.format("%.2f", price * quantity)
@@ -902,11 +881,11 @@ func TestXMLProcessor_Process_AttributeAddition(t *testing.T) {
-- We can access the "inStock" element directly
if v.inStock == "true" then
-- Add a new attribute directly
v._attr = v._attr or {}
v._attr.status = "available"
v.attr = v.attr or {}
v.attr.status = "available"
else
v._attr = v._attr or {}
v._attr.status = "out-of-stock"
v.attr = v.attr or {}
v.attr.status = "out-of-stock"
end
`
@@ -1031,9 +1010,9 @@ func TestXMLProcessor_Process_ElementReordering(t *testing.T) {
luaExpr := `
-- With table approach, we can reorder elements by redefining the table
-- Store the values
local artist = v.artist
local title = v.title
local year = v.year
local artist = v.attr.artist
local title = v.attr.title
local year = v.attr.year
-- Clear the table
for k in pairs(v) do
@@ -1041,9 +1020,9 @@ func TestXMLProcessor_Process_ElementReordering(t *testing.T) {
end
-- Add elements in the desired order
v.title = title
v.artist = artist
v.year = year
v.attr.title = title
v.attr.artist = artist
v.attr.year = year
`
result, modCount, matchCount, err := p.ProcessContent(content, "//song", luaExpr)
@@ -1194,13 +1173,13 @@ func TestXMLProcessor_Process_DynamicXPath(t *testing.T) {
expected := `<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<settings>
<setting name="timeout" value="60" />
<setting name="retries" value="3" />
<setting name="backoff" value="exponential" />
<setting name="timeout" value="60"></setting>
<setting name="retries" value="3"></setting>
<setting name="backoff" value="exponential"></setting>
</settings>
<advanced>
<setting name="logging" value="debug" />
<setting name="timeout" value="120" />
<setting name="logging" value="debug"></setting>
<setting name="timeout" value="120"></setting>
</advanced>
</configuration>`
@@ -1208,7 +1187,7 @@ func TestXMLProcessor_Process_DynamicXPath(t *testing.T) {
p := &XMLProcessor{}
// Double all timeout values in the configuration
result, modCount, matchCount, err := p.ProcessContent(content, "//setting[@name='timeout']/@value", "v = v * 2")
result, modCount, matchCount, err := p.ProcessContent(content, "//setting[@name='timeout']/@value", "v.value = v.value * 2")
if err != nil {
t.Fatalf("Error processing content: %v", err)
@@ -1263,34 +1242,34 @@ func TestXMLProcessor_Process_TableBasedStructureCreation(t *testing.T) {
local summary = ""
-- Process each child option
if v.settings and v.settings.option then
local options = v.settings.option
-- If there's just one option, wrap it in a table
if options._attr then
options = {options}
end
for i, opt in ipairs(options) do
count = count + 1
if opt._attr.name == "debug" then
summary = summary .. "Debug: " .. (opt._attr.value == "true" and "ON" or "OFF")
elseif opt._attr.name == "log_level" then
summary = summary .. "Logging: " .. opt._attr.value
end
if i < #options then
summary = summary .. ", "
end
end
end
local settings = v.children[1]
local options = settings.children
-- if settings and options then
-- if options.attr then
-- options = {options}
-- end
--
-- for i, opt in ipairs(options) do
-- count = count + 1
-- if opt.attr.name == "debug" then
-- summary = summary .. "Debug: " .. (opt.attr.value == "true" and "ON" or "OFF")
-- elseif opt.attr.name == "log_level" then
-- summary = summary .. "Logging: " .. opt.attr.value
-- end
--
-- if i < #options then
-- summary = summary .. ", "
-- end
-- end
-- end
-- Create a new calculated section
v.calculated = {
stats = {
count = tostring(count),
summary = summary
}
}
-- v.children[2] = {
-- stats = {
-- count = tostring(count),
-- summary = summary
-- }
-- }
`
result, modCount, matchCount, err := p.ProcessContent(content, "/data", luaExpr)
@@ -1530,3 +1509,267 @@ func TestXMLProcessor_Process_DeepPathNavigation(t *testing.T) {
// Add more test cases for specific XML manipulation scenarios below
// These tests would cover additional functionality as the implementation progresses
func TestXMLToLua(t *testing.T) {
// Sample XML to test with
xmlStr := `
<root id="1">
<person name="John" age="30">
<address type="home">
<street>123 Main St</street>
<city>Anytown</city>
<zip>12345</zip>
</address>
<contact type="email">john@example.com</contact>
</person>
<person name="Jane" age="28">
<address type="work">
<street>456 Business Ave</street>
<city>Worktown</city>
<zip>54321</zip>
</address>
<contact type="phone">555-1234</contact>
</person>
</root>
`
// Parse the XML
doc, err := xmlquery.Parse(strings.NewReader(xmlStr))
if err != nil {
t.Fatalf("Failed to parse XML: %v", err)
}
// Create a new Lua state
L := lua.NewState()
defer L.Close()
// Create an XML processor
processor := &XMLProcessor{}
// Test converting the root element to Lua
t.Run("RootElement", func(t *testing.T) {
// Find the root element
root := doc.SelectElement("root")
if root == nil {
t.Fatal("Failed to find root element")
}
// Convert to Lua
err := processor.ToLua(L, root)
if err != nil {
t.Fatalf("Failed to convert to Lua: %v", err)
}
// Verify the result
luaTable := L.GetGlobal("v")
if luaTable.Type() != lua.LTTable {
t.Fatalf("Expected table, got %s", luaTable.Type().String())
}
// Check element type
typeVal := L.GetField(luaTable, "type")
if typeVal.String() != "element" {
t.Errorf("Expected type 'element', got '%s'", typeVal.String())
}
// Check name
nameVal := L.GetField(luaTable, "name")
if nameVal.String() != "root" {
t.Errorf("Expected name 'root', got '%s'", nameVal.String())
}
// Check attributes
attrsTable := L.GetField(luaTable, "attributes")
if attrsTable.Type() != lua.LTTable {
t.Fatalf("Expected attributes table, got %s", attrsTable.Type().String())
}
idVal := L.GetField(attrsTable, "id")
if idVal.String() != "1" {
t.Errorf("Expected id '1', got '%s'", idVal.String())
}
// Check that we have children
childrenTable := L.GetField(luaTable, "children")
if childrenTable.Type() != lua.LTTable {
t.Fatalf("Expected children table, got %s", childrenTable.Type().String())
}
})
// Test converting a nested element to Lua
t.Run("NestedElement", func(t *testing.T) {
// Find a nested element
street := doc.SelectElement("//street")
if street == nil {
t.Fatal("Failed to find street element")
}
// Convert to Lua
err := processor.ToLua(L, street)
if err != nil {
t.Fatalf("Failed to convert to Lua: %v", err)
}
// Verify the result
luaTable := L.GetGlobal("v")
if luaTable.Type() != lua.LTTable {
t.Fatalf("Expected table, got %s", luaTable.Type().String())
}
// Check element type
typeVal := L.GetField(luaTable, "type")
if typeVal.String() != "element" {
t.Errorf("Expected type 'element', got '%s'", typeVal.String())
}
// Check name
nameVal := L.GetField(luaTable, "name")
if nameVal.String() != "street" {
t.Errorf("Expected name 'street', got '%s'", nameVal.String())
}
// Check value
valueVal := L.GetField(luaTable, "value")
if valueVal.String() != "123 Main St" {
t.Errorf("Expected value '123 Main St', got '%s'", valueVal.String())
}
})
// Test FromLua with a simple string update
t.Run("FromLuaString", func(t *testing.T) {
// Set up a Lua state with a string value
L := lua.NewState()
defer L.Close()
L.SetGlobal("v", lua.LString("New Value"))
// Convert from Lua
result, err := processor.FromLua(L)
if err != nil {
t.Fatalf("Failed to convert from Lua: %v", err)
}
// Verify the result
strResult, ok := result.(string)
if !ok {
t.Fatalf("Expected string result, got %T", result)
}
if strResult != "New Value" {
t.Errorf("Expected 'New Value', got '%s'", strResult)
}
})
// Test FromLua with a complex table update
t.Run("FromLuaTable", func(t *testing.T) {
// Set up a Lua state with a table value
L := lua.NewState()
defer L.Close()
table := L.NewTable()
L.SetField(table, "value", lua.LString("Updated Text"))
attrTable := L.NewTable()
L.SetField(attrTable, "id", lua.LString("new-id"))
L.SetField(attrTable, "class", lua.LString("highlight"))
L.SetField(table, "attributes", attrTable)
L.SetGlobal("v", table)
// Convert from Lua
result, err := processor.FromLua(L)
if err != nil {
t.Fatalf("Failed to convert from Lua: %v", err)
}
// Verify the result
mapResult, ok := result.(map[string]interface{})
if !ok {
t.Fatalf("Expected map result, got %T", result)
}
// Check value
if value, ok := mapResult["value"]; !ok || value != "Updated Text" {
t.Errorf("Expected value 'Updated Text', got '%v'", value)
}
// Check attributes
attrs, ok := mapResult["attributes"].(map[string]interface{})
if !ok {
t.Fatalf("Expected attributes map, got %T", mapResult["attributes"])
}
if id, ok := attrs["id"]; !ok || id != "new-id" {
t.Errorf("Expected id 'new-id', got '%v'", id)
}
if class, ok := attrs["class"]; !ok || class != "highlight" {
t.Errorf("Expected class 'highlight', got '%v'", class)
}
})
// Test updateNodeFromMap with a simple value update
t.Run("UpdateNodeValue", func(t *testing.T) {
// Create a simple element to update
xmlStr := `<test>Original Text</test>`
doc, _ := xmlquery.Parse(strings.NewReader(xmlStr))
node := doc.SelectElement("test")
// Create update data
updateData := map[string]interface{}{
"value": "Updated Text",
}
// Update the node
updateNodeFromMap(node, updateData)
// Verify the update
if node.InnerText() != "Updated Text" {
t.Errorf("Expected value 'Updated Text', got '%s'", node.InnerText())
}
})
// Test updateNodeFromMap with attribute updates
t.Run("UpdateNodeAttributes", func(t *testing.T) {
// Create an element with attributes
xmlStr := `<test id="old">Text</test>`
doc, _ := xmlquery.Parse(strings.NewReader(xmlStr))
node := doc.SelectElement("test")
// Create update data
updateData := map[string]interface{}{
"attributes": map[string]interface{}{
"id": "new",
"class": "added",
},
}
// Update the node
updateNodeFromMap(node, updateData)
// Verify the id attribute was updated
idFound := false
classFound := false
for _, attr := range node.Attr {
if attr.Name.Local == "id" {
idFound = true
if attr.Value != "new" {
t.Errorf("Expected id 'new', got '%s'", attr.Value)
}
}
if attr.Name.Local == "class" {
classFound = true
if attr.Value != "added" {
t.Errorf("Expected class 'added', got '%s'", attr.Value)
}
}
}
if !idFound {
t.Error("Expected to find 'id' attribute but didn't")
}
if !classFound {
t.Error("Expected to find 'class' attribute but didn't")
}
})
}

View File

@@ -0,0 +1,4 @@
// The package is now using github.com/antchfx/xmlquery for XPath parsing.
// The parsing functionality tests have been removed since we're now
// delegating XPath parsing to the xmlquery library.
package xpath

View File

@@ -0,0 +1,4 @@
// The package is now using github.com/antchfx/xmlquery for XPath parsing.
// The parsing functionality tests have been removed since we're now
// delegating XPath parsing to the xmlquery library.
package xpath

133
processor/xpath/xpath.go Normal file
View File

@@ -0,0 +1,133 @@
package xpath
import (
"errors"
"fmt"
"github.com/antchfx/xmlquery"
)
// Get retrieves nodes from XML data using an XPath expression
func Get(node *xmlquery.Node, path string) ([]*xmlquery.Node, error) {
if node == nil {
return nil, errors.New("nil node provided")
}
// Execute xpath query directly
nodes, err := xmlquery.QueryAll(node, path)
if err != nil {
return nil, fmt.Errorf("failed to execute XPath query: %v", err)
}
return nodes, nil
}
// Set updates a single node in the XML data using an XPath expression
func Set(node *xmlquery.Node, path string, value interface{}) error {
if node == nil {
return errors.New("nil node provided")
}
// Find the node to update
nodes, err := xmlquery.QueryAll(node, path)
if err != nil {
return fmt.Errorf("failed to execute XPath query: %v", err)
}
if len(nodes) == 0 {
return fmt.Errorf("no nodes found for path: %s", path)
}
// Update the first matching node
updateNodeValue(nodes[0], value)
return nil
}
// SetAll updates all nodes that match the XPath expression
func SetAll(node *xmlquery.Node, path string, value interface{}) error {
if node == nil {
return errors.New("nil node provided")
}
// Find all nodes to update
nodes, err := xmlquery.QueryAll(node, path)
if err != nil {
return fmt.Errorf("failed to execute XPath query: %v", err)
}
if len(nodes) == 0 {
return fmt.Errorf("no nodes found for path: %s", path)
}
// Update all matching nodes
for _, matchNode := range nodes {
updateNodeValue(matchNode, value)
}
return nil
}
// Helper function to update a node's value
func updateNodeValue(node *xmlquery.Node, value interface{}) {
strValue := fmt.Sprintf("%v", value)
// Handle different node types
switch node.Type {
case xmlquery.AttributeNode:
// For attribute nodes, update the attribute value
parent := node.Parent
if parent != nil {
for i, attr := range parent.Attr {
if attr.Name.Local == node.Data {
parent.Attr[i].Value = strValue
break
}
}
}
case xmlquery.TextNode:
// For text nodes, update the text content
node.Data = strValue
case xmlquery.ElementNode:
// For element nodes, clear existing text children and add a new text node
// First, remove all existing text children
var nonTextChildren []*xmlquery.Node
for child := node.FirstChild; child != nil; child = child.NextSibling {
if child.Type != xmlquery.TextNode {
nonTextChildren = append(nonTextChildren, child)
}
}
// Clear all children
node.FirstChild = nil
node.LastChild = nil
// Add a new text node
textNode := &xmlquery.Node{
Type: xmlquery.TextNode,
Data: strValue,
Parent: node,
}
// Set the text node as the first child
node.FirstChild = textNode
node.LastChild = textNode
// Add back non-text children
for _, child := range nonTextChildren {
child.Parent = node
// If this is the first child being added back
if node.FirstChild == textNode && node.LastChild == textNode {
node.FirstChild.NextSibling = child
child.PrevSibling = node.FirstChild
node.LastChild = child
} else {
// Add to the end of the chain
node.LastChild.NextSibling = child
child.PrevSibling = node.LastChild
node.LastChild = child
}
}
}
}

View File

@@ -0,0 +1,474 @@
package xpath
import (
"strings"
"testing"
"github.com/antchfx/xmlquery"
)
// Parse test XML data once at the beginning for use in multiple tests
func parseTestXML(t *testing.T, xmlData string) *xmlquery.Node {
doc, err := xmlquery.Parse(strings.NewReader(xmlData))
if err != nil {
t.Fatalf("Failed to parse test XML: %v", err)
}
return doc
}
// XML test data as a string for our tests
var testXML = `
<store>
<book category="fiction">
<title lang="en">The Fellowship of the Ring</title>
<author>J.R.R. Tolkien</author>
<year>1954</year>
<price>22.99</price>
</book>
<book category="fiction">
<title lang="en">The Two Towers</title>
<author>J.R.R. Tolkien</author>
<year>1954</year>
<price>23.45</price>
</book>
<book category="technical">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
<bicycle>
<color>red</color>
<price>199.95</price>
</bicycle>
</store>
`
func TestEvaluator(t *testing.T) {
// Parse the test XML data once for all test cases
doc := parseTestXML(t, testXML)
tests := []struct {
name string
path string
error bool
}{
{
name: "simple_element_access",
path: "/store/bicycle/color",
},
{
name: "recursive_element_access",
path: "//price",
},
{
name: "wildcard_element_access",
path: "/store/book/*",
},
{
name: "attribute_exists_predicate",
path: "//title[@lang]",
},
{
name: "attribute_equals_predicate",
path: "//title[@lang='en']",
},
{
name: "value_comparison_predicate",
path: "/store/book[price>35.00]/title",
error: true,
},
{
name: "last_predicate",
path: "/store/book[last()]/title",
error: true,
},
{
name: "last_minus_predicate",
path: "/store/book[last()-1]/title",
error: true,
},
{
name: "position_predicate",
path: "/store/book[position()<3]/title",
error: true,
},
{
name: "invalid_index",
path: "/store/book[10]/title",
error: true,
},
{
name: "nonexistent_element",
path: "/store/nonexistent",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, err := Get(doc, tt.path)
// Handle expected errors
if tt.error {
if err == nil && len(result) == 0 {
// If we expected an error but got empty results instead, that's okay
return
}
if err != nil {
// If we got an error as expected, that's okay
return
}
} else if err != nil {
// If we didn't expect an error but got one, that's a test failure
t.Errorf("Get(%q) returned unexpected error: %v", tt.path, err)
return
}
// Special cases where we don't care about exact matches
switch tt.name {
case "wildcard_element_access":
// Just check that we got some elements
if len(result) == 0 {
t.Errorf("Expected multiple elements for wildcard, got none")
}
return
case "attribute_exists_predicate", "attribute_equals_predicate":
// Just check that we got some titles
if len(result) == 0 {
t.Errorf("Expected titles with lang attribute, got none")
}
// Ensure all are title elements
for _, node := range result {
if node.Data != "title" {
t.Errorf("Expected title elements, got: %s", node.Data)
}
}
return
case "nonexistent_element":
// Just check that we got empty results
if len(result) != 0 {
t.Errorf("Expected empty results for nonexistent element, got %d items", len(result))
}
return
}
// For other cases, just verify we got results
if len(result) == 0 {
t.Errorf("Expected results for path %s, got none", tt.path)
}
})
}
}
func TestEdgeCases(t *testing.T) {
t.Run("nil_node", func(t *testing.T) {
result, err := Get(nil, "/store/book")
if err == nil {
t.Errorf("Expected error for nil node")
return
}
if len(result) > 0 {
t.Errorf("Expected empty result, got %v", result)
}
})
t.Run("invalid_xml", func(t *testing.T) {
invalidXML, err := xmlquery.Parse(strings.NewReader("<invalid>xml"))
if err != nil {
// If parsing fails, that's expected
return
}
_, err = Get(invalidXML, "/store")
if err == nil {
t.Error("Expected error for invalid XML structure")
}
})
// For these tests with the simple XML, we expect just one result
simpleXML := `<root><book><title lang="en">Test</title></book></root>`
doc := parseTestXML(t, simpleXML)
t.Run("current_node", func(t *testing.T) {
result, err := Get(doc, "/root/book/.")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) > 1 {
t.Errorf("Expected at most 1 result, got %d", len(result))
}
if len(result) > 0 {
// Verify it's the book node
if result[0].Data != "book" {
t.Errorf("Expected book node, got %v", result[0].Data)
}
}
})
t.Run("attributes", func(t *testing.T) {
result, err := Get(doc, "/root/book/title/@lang")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 1 || result[0].InnerText() != "en" {
t.Errorf("Expected 'en', got %v", result[0].InnerText())
}
})
}
func TestGetWithPaths(t *testing.T) {
// Use a simplified, well-formed XML document
simpleXML := `<store>
<book category="fiction">
<title lang="en">The Book Title</title>
<author>Author Name</author>
<price>19.99</price>
</book>
<bicycle>
<color>red</color>
<price>199.95</price>
</bicycle>
</store>`
// Parse the XML for testing
doc := parseTestXML(t, simpleXML)
// Debug: Print the test XML
t.Logf("Test XML:\n%s", simpleXML)
tests := []struct {
name string
path string
expectedValue string
}{
{
name: "simple_element_access",
path: "/store/bicycle/color",
expectedValue: "red",
},
{
name: "attribute_access",
path: "/store/book/title/@lang",
expectedValue: "en",
},
{
name: "recursive_with_attribute",
path: "//title[@lang='en']",
expectedValue: "The Book Title",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Debug: Print the path we're looking for
t.Logf("Looking for path: %s", tt.path)
result, err := Get(doc, tt.path)
if err != nil {
t.Errorf("Get(%q) returned error: %v", tt.path, err)
return
}
// Debug: Print the results
t.Logf("Got %d results", len(result))
for i, r := range result {
t.Logf("Result %d: Node=%s, Value=%v", i, r.Data, r.InnerText())
}
// Check that we got results
if len(result) == 0 {
t.Errorf("Get(%q) returned no results", tt.path)
return
}
// For attribute access test, do more specific checks
if tt.name == "attribute_access" {
// Check the first result's value matches expected
if result[0].InnerText() != tt.expectedValue {
t.Errorf("Attribute value: got %v, expected %s", result[0].InnerText(), tt.expectedValue)
}
}
// For simple element access, check the text content
if tt.name == "simple_element_access" {
if text := result[0].InnerText(); text != tt.expectedValue {
t.Errorf("Element text: got %s, expected %s", text, tt.expectedValue)
}
}
// For recursive with attribute test, check title elements with lang="en"
if tt.name == "recursive_with_attribute" {
for _, node := range result {
// Check the node is a title
if node.Data != "title" {
t.Errorf("Expected title element, got %s", node.Data)
}
// Check text content
if text := node.InnerText(); text != tt.expectedValue {
t.Errorf("Text content: got %s, expected %s", text, tt.expectedValue)
}
// Check attributes - find the lang attribute
hasLang := false
for _, attr := range node.Attr {
if attr.Name.Local == "lang" && attr.Value == "en" {
hasLang = true
break
}
}
if !hasLang {
t.Errorf("Expected lang=\"en\" attribute, but it was not found")
}
}
}
})
}
}
func TestSet(t *testing.T) {
t.Run("simple element", func(t *testing.T) {
xmlData := `<root><name>John</name></root>`
doc := parseTestXML(t, xmlData)
err := Set(doc, "/root/name", "Jane")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}
// Verify the change
result, err := Get(doc, "/root/name")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 1 {
t.Errorf("Expected 1 result, got %d", len(result))
return
}
// Check text content
if text := result[0].InnerText(); text != "Jane" {
t.Errorf("Expected text 'Jane', got '%s'", text)
}
})
t.Run("attribute", func(t *testing.T) {
xmlData := `<root><element id="123"></element></root>`
doc := parseTestXML(t, xmlData)
err := Set(doc, "/root/element/@id", "456")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}
// Verify the change
result, err := Get(doc, "/root/element/@id")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 1 {
t.Errorf("Expected 1 result, got %d", len(result))
return
}
// For attributes, check the inner text
if text := result[0].InnerText(); text != "456" {
t.Errorf("Expected attribute value '456', got '%s'", text)
}
})
t.Run("indexed element", func(t *testing.T) {
xmlData := `<root><items><item>first</item><item>second</item></items></root>`
doc := parseTestXML(t, xmlData)
err := Set(doc, "/root/items/item[1]", "changed")
if err != nil {
t.Errorf("Set() returned error: %v", err)
return
}
// Verify the change using XPath that specifically targets the first item
result, err := Get(doc, "/root/items/item[1]")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
// Check if we have results
if len(result) == 0 {
t.Errorf("Expected at least one result for /root/items/item[1]")
return
}
// Check text content
if text := result[0].InnerText(); text != "changed" {
t.Errorf("Expected text 'changed', got '%s'", text)
}
})
}
func TestSetAll(t *testing.T) {
t.Run("multiple elements", func(t *testing.T) {
xmlData := `<root><items><item>first</item><item>second</item></items></root>`
doc := parseTestXML(t, xmlData)
err := SetAll(doc, "//item", "changed")
if err != nil {
t.Errorf("SetAll() returned error: %v", err)
return
}
// Verify all items are changed
result, err := Get(doc, "//item")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 2 {
t.Errorf("Expected 2 results, got %d", len(result))
return
}
// Check each node
for i, node := range result {
if text := node.InnerText(); text != "changed" {
t.Errorf("Item %d: expected text 'changed', got '%s'", i, text)
}
}
})
t.Run("attributes", func(t *testing.T) {
xmlData := `<root><item id="1"/><item id="2"/></root>`
doc := parseTestXML(t, xmlData)
err := SetAll(doc, "//item/@id", "new")
if err != nil {
t.Errorf("SetAll() returned error: %v", err)
return
}
// Verify all attributes are changed
result, err := Get(doc, "//item/@id")
if err != nil {
t.Errorf("Get() returned error: %v", err)
return
}
if len(result) != 2 {
t.Errorf("Expected 2 results, got %d", len(result))
return
}
// For attributes, check inner text
for i, node := range result {
if text := node.InnerText(); text != "new" {
t.Errorf("Attribute %d: expected value 'new', got '%s'", i, text)
}
}
})
}

View File

@@ -0,0 +1,83 @@
package regression
import (
"modify/processor"
"testing"
)
func TestTalentsMechanicOutOfRange(t *testing.T) {
given := `<Talent identifier="quickfixer">
<Icon texture="Content/UI/TalentsIcons2.png" sheetindex="5,2" sheetelementsize="128,128"/>
<Description tag="talentdescription.quickfixer">
<Replace tag="[amount]" value="20" color="gui.green"/>
<Replace tag="[duration]" value="10" color="gui.green"/>
</Description>
<Description tag="talentdescription.repairmechanicaldevicestwiceasfast"/>
<AbilityGroupEffect abilityeffecttype="None">
<Abilities>
<CharacterAbilityGiveStat stattype="MechanicalRepairSpeed" value="1"/>
</Abilities>
</AbilityGroupEffect>
<AbilityGroupEffect abilityeffecttype="OnRepairComplete">
<Conditions>
<AbilityConditionItem tags="fabricator,door,engine,oxygengenerator,pump,turretammosource,deconstructor,medicalfabricator,ductblock"/>
</Conditions>
<Abilities>
<CharacterAbilityApplyStatusEffects>
<StatusEffects>
<StatusEffect type="OnAbility" target="Character" disabledeltatime="true">
<Affliction identifier="quickfixer" amount="10.0"/>
</StatusEffect>
</StatusEffects>
</CharacterAbilityApplyStatusEffects>
</Abilities>
</AbilityGroupEffect>
</Talent>`
actual := `<Talent identifier="quickfixer">
<Icon texture="Content/UI/TalentsIcons2.png" sheetindex="5,2" sheetelementsize="128,128"/>
<Description tag="talentdescription.quickfixer">
<Replace tag="[amount]" value="30" color="gui.green"/>
<Replace tag="[duration]" value="20" color="gui.green"/>
</Description>
<Description tag="talentdescription.repairmechanicaldevicestwiceasfast"/>
<AbilityGroupEffect abilityeffecttype="None">
<Abilities>
<CharacterAbilityGiveStat stattype="MechanicalRepairSpeed" value="2"/>
</Abilities>
</AbilityGroupEffect>
<AbilityGroupEffect abilityeffecttype="OnRepairComplete">
<Conditions>
<AbilityConditionItem tags="fabricator,door,engine,oxygengenerator,pump,turretammosource,deconstructor,medicalfabricator,ductblock"/>
</Conditions>
<Abilities>
<CharacterAbilityApplyStatusEffects>
<StatusEffects>
<StatusEffect type="OnAbility" target="Character" disabledeltatime="true">
<Affliction identifier="quickfixer" amount="20"/>
</StatusEffect>
</StatusEffects>
</CharacterAbilityApplyStatusEffects>
</Abilities>
</AbilityGroupEffect>
</Talent>`
p := &processor.RegexProcessor{}
result, mods, matches, err := p.ProcessContent(given, `<Talent identifier="quickfixer">!anyvalue="(?<movementspeed>!num)"!anyvalue="(?<duration>!num)"!anyvalue="(?<repairspeed>!num)"!anyamount="(?<durationv>!num)"`, "movementspeed=round(movementspeed*1.5, 2) duration=round(duration*2, 2) repairspeed=round(repairspeed*2, 2) durationv=duration")
if err != nil {
t.Fatalf("Error processing content: %v", err)
}
if matches != 1 {
t.Errorf("Expected 1 match, got %d", matches)
}
if mods != 1 {
t.Errorf("Expected 1 modification, got %d", mods)
}
if result != actual {
t.Errorf("expected %s, got %s", actual, result)
}
}

49
release.sh Normal file
View File

@@ -0,0 +1,49 @@
#!/bin/bash
echo "Figuring out the tag..."
TAG=$(git describe --tags --exact-match 2>/dev/null || echo "")
if [ -z "$TAG" ]; then
# Get the latest tag
LATEST_TAG=$(git describe --tags $(git rev-list --tags --max-count=1))
# Increment the patch version
IFS='.' read -r -a VERSION_PARTS <<< "$LATEST_TAG"
VERSION_PARTS[2]=$((VERSION_PARTS[2]+1))
TAG="${VERSION_PARTS[0]}.${VERSION_PARTS[1]}.${VERSION_PARTS[2]}"
# Create a new tag
git tag $TAG
git push origin $TAG
fi
echo "Tag: $TAG"
echo "Building the thing..."
go build -o chef.exe .
go install .
echo "Creating a release..."
TOKEN="$GITEA_API_KEY"
GITEA="https://git.site.quack-lab.dev"
REPO="dave/BigChef"
# Create a release
RELEASE_RESPONSE=$(curl -s -X POST \
-H "Authorization: token $TOKEN" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d '{
"tag_name": "'"$TAG"'",
"name": "'"$TAG"'",
"draft": false,
"prerelease": false
}' \
$GITEA/api/v1/repos/$REPO/releases)
# Extract the release ID
echo $RELEASE_RESPONSE
RELEASE_ID=$(echo $RELEASE_RESPONSE | awk -F'"id":' '{print $2+0; exit}')
echo "Release ID: $RELEASE_ID"
echo "Uploading the things..."
curl -X POST \
-H "Authorization: token $TOKEN" \
-F "attachment=@chef.exe" \
"$GITEA/api/v1/repos/$REPO/releases/${RELEASE_ID}/assets?name=chef.exe"
rm chef.exe