Commit Graph

22 Commits

Author SHA1 Message Date
Damian Bednarczyk
130fdb6b5b add bing bot + create generic struct for bots 2023-11-30 19:55:34 -06:00
Damian Bednarczyk
ba252f5750 add other headers 2023-11-30 19:24:50 -06:00
Damian Bednarczyk
8862b7de8b rough draft of more modular design 2023-11-30 16:05:14 -06:00
Damian Bednarczyk
0122b2f1cf add option to use random ip from googlebot pool 2023-11-29 23:24:32 -06:00
Damian Bednarczyk
dc558b38fa fix import orders 2023-11-29 15:02:05 -06:00
Kevin Pham
e5bd5df2c0 structured article json first draft 2023-11-29 00:33:46 -06:00
Kevin Pham
31200cf9e2 replace cycletls with bogdanfinn/tls-client for better tls fingerprint spoofing reliability 2023-11-28 12:33:53 -06:00
Kevin Pham
214790f905 fix fwd response headers 2023-11-28 00:06:41 -06:00
Kevin Pham
9a8d4ac261 fix cycletls imports 2023-11-27 22:40:34 -06:00
Kevin Pham
0fe02c397d fix srcset handler in html rewriter 2023-11-27 21:36:50 -06:00
Kevin Pham
78e15d8342 fix archivist response modifier 2023-11-27 20:29:03 -06:00
Kevin Pham
81cda1101c improve wayback/archist display / refactor cycletls transport with PR in ext lib 2023-11-27 20:09:03 -06:00
Kevin Pham
7b3aecca7d add TLS fingerprint (ja3) spoofer request modifier 2023-11-27 16:30:13 -06:00
Kevin Pham
2dccc7ca35 add req/resp header forwarding modifiers 2023-11-27 12:45:23 -06:00
Kevin Pham
59cf0317ec improve ua modifier with client side spoofing 2023-11-27 12:00:12 -06:00
Kevin Pham
fb63dba8a0 add response / request modifier API to only fire once 2023-11-26 23:15:00 -06:00
Kevin Pham
854dafbcfa improve js rewriting functionality by not relying on js to get proxy and proxified URLs; direct injection from golang 2023-11-21 20:54:09 -06:00
Kevin Pham
a4e016b36c add common referrer options 2023-11-21 20:33:52 -06:00
Kevin Pham
dab77d786f url rewriter tweaks 2023-11-21 14:10:37 -06:00
Kevin Pham
79a229f28c handle srcset resource URL rewrites; monkey patch JS for URL rewrites 2023-11-20 23:42:50 -06:00
Kevin Pham
5035f65d6b wip 2023-11-19 20:59:55 -06:00
Kevin Pham
ee9066dedb refactor wip 2023-11-19 15:03:11 -06:00