Commit Graph

213 Commits

Author SHA1 Message Date
e51efb4289 Run local database 2026-01-26 15:44:17 +01:00
3c6f296927 Add user to ddl 2026-01-26 15:33:56 +01:00
b5aa63928a Increase the backoff
Usually it'll be resource contention or rate limits
So we want to wait a while
2026-01-26 14:53:20 +01:00
c9abd3936c Refactor NConsumers to nsq.go 2026-01-26 14:39:19 +01:00
1fbe478378 Handle 404 and 400 with names 2026-01-26 14:36:36 +01:00
adde43563e Add db polling to readers
So we may let it run over night
2026-01-26 14:31:49 +01:00
61d2d14fb8 Implement name and solar system resolvers as well 2026-01-26 14:23:49 +01:00
02baec430c Start touching messages 2026-01-26 14:15:57 +01:00
e2ff432d57 Update NSQ configuration and improve error handling in resolvers 2026-01-26 13:52:12 +01:00
d1c0a6ace0 Run multiple consumers for resolvers 2026-01-26 13:44:54 +01:00
d9cfceaceb Resolve all the other type information 2026-01-26 13:44:05 +01:00
fb57616d0d Refactor all the constants to lnsq 2026-01-26 13:37:32 +01:00
0f926a139c Fix reading int32s for some readers 2026-01-26 13:29:15 +01:00
dc0796a547 Rename all the readers to MSB 2026-01-26 13:28:34 +01:00
d765c0335a Refactor reader and resolver to a generic interface 2026-01-26 13:24:00 +01:00
f42fb81594 Remove polling, again 2026-01-26 12:59:19 +01:00
5968f0a8bf Great use of json you clanker... 2026-01-26 12:58:04 +01:00
3cdf4036b2 lol... "completed"... 2026-01-26 12:55:51 +01:00
7b2a63b55c Don't poll clickhouse
Duplicates :(
2026-01-26 12:53:25 +01:00
f8ea0232b5 Remove the fucking signals that, again, nobody ever asked for 2026-01-26 12:51:22 +01:00
6a87a8bcda Ditch the memo for now 2026-01-26 12:47:39 +01:00
8ab2a69654 FUCK OFF with the json and remove batching from resolver 2026-01-26 12:45:07 +01:00
3db461a80a Update 2026-01-26 12:32:21 +01:00
201f24a429 Print stats on nsq consumer 2026-01-26 12:30:45 +01:00
43a49c4dbd Add character name readers 2026-01-26 12:22:28 +01:00
61a0c835b0 Fix the db pathing 2026-01-26 12:20:05 +01:00
5b015620c1 Implement item type resolver step 2026-01-26 12:11:43 +01:00
004c273958 Rip clickhouse out of db.go 2026-01-26 12:11:32 +01:00
424d2b9213 cLiEnT WAHBG JUSAGHEJWBFDRIOUASnaefhsbgktw jvskfgaehji bsšućaf 2026-01-26 12:05:03 +01:00
794f0b0d04 Let go of fucking nsqlookupd 2026-01-26 11:57:39 +01:00
b6b7b9ec56 Fix the god damn fucking nsq containers 2026-01-26 11:53:31 +01:00
ecdb41217e Add item type reader 2026-01-26 11:53:26 +01:00
de5bffe9b1 Refactor nsq and clickhouse clients to their individual packages 2026-01-26 11:51:25 +01:00
2448f76c1e Refactor connecting to nsq to separate package 2026-01-26 11:41:15 +01:00
caa1d97617 Refactor connecting to nsq 2026-01-26 11:40:08 +01:00
233c67ff00 Implement stage2 - write into clickhouse 2026-01-26 11:05:15 +01:00
0dfae49f94 Add a simple clickhouse client 2026-01-26 10:27:00 +01:00
2b79c51837 Have stage1 flatten after reading 2026-01-26 09:42:57 +01:00
1bc05dc81d Make pipelines programs on their own 2026-01-26 09:41:30 +01:00
e2e1ebe48f Hallucinate some wires 2026-01-26 09:35:12 +01:00
dfdd74f52e Hallucinate stage 1 disk reader 2026-01-26 09:32:13 +01:00
725d6b8ee3 Move shit around to separate packages 2026-01-26 09:26:53 +01:00
600fc0d7ff Rework the types to separate package 2026-01-26 09:24:39 +01:00
045930025c Don't log to file 2026-01-26 08:11:34 +01:00
5b5e5ddf6b Duplicate inserter 8 times
Because tmux doesn't let use S-C :)))))))))))))))))))))))))))
2026-01-25 19:56:45 +01:00
c1c0291c2d Bigger nsq messages 2026-01-25 18:26:18 +01:00
783284e3f5 Remove entirely gRaCeFuL eXiT 2026-01-25 18:20:58 +01:00
342b0be467 Parallelize and memoize esi name lookup 2026-01-25 15:38:12 +01:00
99f5899fbe Pause consumer on rate limit 2026-01-25 14:46:35 +01:00
1dfed10c77 Remove the clickhouse port that we weren't previously using and was added just to break everything 2026-01-25 14:15:02 +01:00