Commit Graph

182 Commits

Author SHA1 Message Date
ecdb41217e Add item type reader 2026-01-26 11:53:26 +01:00
de5bffe9b1 Refactor nsq and clickhouse clients to their individual packages 2026-01-26 11:51:25 +01:00
2448f76c1e Refactor connecting to nsq to separate package 2026-01-26 11:41:15 +01:00
caa1d97617 Refactor connecting to nsq 2026-01-26 11:40:08 +01:00
233c67ff00 Implement stage2 - write into clickhouse 2026-01-26 11:05:15 +01:00
0dfae49f94 Add a simple clickhouse client 2026-01-26 10:27:00 +01:00
2b79c51837 Have stage1 flatten after reading 2026-01-26 09:42:57 +01:00
1bc05dc81d Make pipelines programs on their own 2026-01-26 09:41:30 +01:00
e2e1ebe48f Hallucinate some wires 2026-01-26 09:35:12 +01:00
dfdd74f52e Hallucinate stage 1 disk reader 2026-01-26 09:32:13 +01:00
725d6b8ee3 Move shit around to separate packages 2026-01-26 09:26:53 +01:00
600fc0d7ff Rework the types to separate package 2026-01-26 09:24:39 +01:00
045930025c Don't log to file 2026-01-26 08:11:34 +01:00
5b5e5ddf6b Duplicate inserter 8 times
Because tmux doesn't let use S-C :)))))))))))))))))))))))))))
2026-01-25 19:56:45 +01:00
c1c0291c2d Bigger nsq messages 2026-01-25 18:26:18 +01:00
783284e3f5 Remove entirely gRaCeFuL eXiT 2026-01-25 18:20:58 +01:00
342b0be467 Parallelize and memoize esi name lookup 2026-01-25 15:38:12 +01:00
99f5899fbe Pause consumer on rate limit 2026-01-25 14:46:35 +01:00
1dfed10c77 Remove the clickhouse port that we weren't previously using and was added just to break everything 2026-01-25 14:15:02 +01:00
ef42e1f52d Insert to prod clickhouse 2026-01-25 14:09:07 +01:00
34fe95a911 Fix up the logs to not be as spammy 2026-01-25 14:09:02 +01:00
6ce1758672 Memoize database functions to reduce disk reads 2026-01-25 14:02:45 +01:00
2a6b903b58 Fix more god damn nsq garbage 2026-01-25 13:47:18 +01:00
6bba290cd4 Get rid of broken nsq batching 2026-01-25 13:42:33 +01:00
ac67045ded Use 127.0.0.1 2026-01-25 13:31:28 +01:00
25e8c3335a Batch insert to nsq 2026-01-25 12:58:01 +01:00
c0dd32e90e Run nsq locally
http is a nightmare...
2026-01-25 12:54:51 +01:00
386cf59523 Refactor all run args to env variables 2026-01-25 12:50:35 +01:00
4d969f1641 Rework main to be an nsq pipeline instead of direct ingest 2026-01-25 12:31:58 +01:00
e7f14278be Remove more legacy shit 2026-01-25 12:19:27 +01:00
322aa8ab81 Add a docker compose 2026-01-25 12:17:26 +01:00
cd5716b7f0 Add a build script 2026-01-25 12:08:56 +01:00
98fff24d94 Update 2026-01-25 03:33:42 +01:00
d16c3471cf Update
Update
2026-01-25 03:32:02 +01:00
5e79f1db1b Fix finding modules in cargo 2026-01-25 03:06:17 +01:00
734a90ba7e Fix date strings 2026-01-25 02:57:01 +01:00
e3af9cd15b Implement an endpoint for fetching complete entire killmails 2026-01-25 02:53:25 +01:00
d5048bae89 Add a simple killmail fetch api 2026-01-25 02:41:01 +01:00
b2b06c93bb Update 2026-01-25 02:26:35 +01:00
5a9ab78070 Use ids instead of names in the wheres 2026-01-25 02:23:14 +01:00
36dbe79cc5 Proxy images 2026-01-25 01:46:09 +01:00
5e7cdde5b8 Update 2026-01-25 01:25:17 +01:00
22a76ff8ec Return ids with analytics 2026-01-25 01:20:41 +01:00
0a7d911da9 Remove legacy code 2026-01-24 23:51:51 +01:00
7afd388c54 Hallucinate a system name handler 2026-01-24 23:42:26 +01:00
b8826d37e8 Aggregate the statistics in api 2026-01-24 23:35:21 +01:00
058db2a6e2 Fix some fucking stupid type issues 2026-01-24 23:25:45 +01:00
6eb0cad899 Remove legacy code 2026-01-24 23:18:26 +01:00
ae02ded7e6 Remove limits that nobody ever asked for 2026-01-24 23:17:12 +01:00
e87f476b43 Wire up the API with the new methods 2026-01-24 23:09:57 +01:00