Drew DeVault
|
82d73c6e31
|
schema: use rum index
https://github.com/postgrespro/rum
|
2022-07-13 10:13:54 +02:00 |
|
Umar Getagazov
|
fde8b75efd
|
Drop crawl schedule-related fields
They were unused.
|
2022-07-11 17:50:44 +02:00 |
|
Drew DeVault
|
c6777e21a7
|
schema.sql: set default exclusion list to {}
|
2022-07-11 17:48:36 +02:00 |
|
Umar Getagazov
|
5471687556
|
Add per-domain page exclusion mechanism
|
2022-07-11 13:20:31 +02:00 |
|
Drew DeVault
|
e44770b9b7
|
schema: add "source" column to page
|
2022-07-10 10:13:11 +02:00 |
|
Drew DeVault
|
01b2b1349b
|
crawler: compute checksum and make unique
Fixes: https://todo.sr.ht/~sircmpwn/searchhut/30
|
2022-07-10 09:36:07 +02:00 |
|
Drew DeVault
|
9790813a55
|
Track pages with JavaScript and total crawl time
|
2022-07-10 09:12:07 +02:00 |
|
Drew DeVault
|
050694c4f2
|
Initial commit
|
2022-07-08 19:46:11 +02:00 |
|