Merge pull request #21 from joncrangle/csp-override

Allow the user to specify the Content Security Policy for a domain

README.md

@@ -3,6 +3,8 @@
 </p>
 
 <h1 align="center">Ladder</h1>
+<div><img alt="License" src="https://img.shields.io/github/license/kubero-dev/ladder"> <img alt="go.mod Go version " src="https://img.shields.io/github/go-mod/go-version/kubero-dev/ladder"> <img alt="GitHub tag (with filter)" src="https://img.shields.io/github/v/tag/kubero-dev/ladder"> <img alt="GitHub (Pre-)Release Date" src="https://img.shields.io/github/release-date-pre/kubero-dev/ladder"> <img alt="GitHub Downloads all releases" src="https://img.shields.io/github/downloads/kubero-dev/ladder/total"> <img alt="GitHub Build Status (with event)" src="https://img.shields.io/github/actions/workflow/status/kubero-dev/ladder/release-binaries.yaml"></div>
+
 
 *Ladder is a web proxy to help bypass paywalls.* This is a selfhosted version of [1ft.io](https://1ft.io) and [12ft.io](https://12ft.io). It is inspired by [13ft](https://github.com/wasi-master/13ft).
 
@@ -10,8 +12,6 @@
 
 Freedom of information is an essential pillar of democracy and informed decision-making. While media organizations have legitimate financial interests, it is crucial to strike a balance between profitability and the public's right to access information. The proliferation of paywalls raises concerns about the erosion of this fundamental freedom, and it is imperative for society to find innovative ways to preserve access to vital information without compromising the sustainability of journalism. In a world where knowledge should be shared and not commodified, paywalls should be critically examined to ensure that they do not undermine the principles of an open and informed society.
 
-Certain sites may display missing images or encounter formatting issues. This can be attributed to the site's reliance on JavaScript or CSS for image and resource loading, which presents a limitation when accessed through this proxy. If you prefer a full experience, please concider buying a subscription for the site.
-
 > **Disclaimer:** This project is intended for educational purposes only. The author does not endorse or encourage any unethical or illegal activity. Use this tool at your own risk.
 
 ### Features
@@ -23,15 +23,24 @@ Certain sites may display missing images or encounter formatting issues. This ca
 - [x] Fetch RAW HTML
 - [x] Custom User Agent
 - [x] Custom X-Forwarded-For IP
-- [x] [Docker container](https://github.com/kubero-dev/ladder/pkgs/container/ladder)
+- [x] [Docker container](https://github.com/kubero-dev/ladder/pkgs/container/ladder) (amd64, arm64)
 - [x] Linux binary
 - [x] Mac OS binary
 - [x] Windows binary (untested)
-- [x] Removes most of the ads (unexpected side effect)
+- [x] Removes most of the ads (unexpected side effect ¯\\\_(ツ)_/¯ )
 - [x] Basic Auth
 - [x] Disable logs
 - [x] No Tracking
+- [x] Limit the proxy to a list of domains
+- [x] Expose Ruleset to other ladders
 - [ ] Optional TOR proxy
+- [ ] A key to share only one URL
+- [ ] Fetch from Google Cache if not available
+
+### Limitations
+Certain sites may display missing images or encounter formatting issues. This can be attributed to the site's reliance on JavaScript or CSS for image and resource loading, which presents a limitation when accessed through this proxy. If you prefer a full experience, please consider buying a subscription for the site.
+
+Some sites do not expose their content to search engines, which means that the proxy cannot access the content. A future version will try to fetch the content from Google Cache.
 
 ## Installation
 
@@ -53,6 +62,9 @@ curl https://raw.githubusercontent.com/kubero-dev/ladder/main/docker-compose.yam
 docker-compose up -d
 ```
 
+### Helm
+See [README.md](/helm-chart/README.md) in helm-chart sub-directory for more information.
+
 ## Usage
 
 ### Browser
@@ -63,6 +75,11 @@ docker-compose up -d
 Or direct by appending the URL to the end of the proxy URL:
 http://localhost:8080/https://www.example.com
 
+Or create a bookmark with the following URL:
+```javascript
+javascript:window.location.href="http://localhost:8080/"+location.href
+```
+
 ### API
 ```bash
 curl -X GET "http://localhost:8080/api/https://www.example.com"
@@ -71,11 +88,15 @@ curl -X GET "http://localhost:8080/api/https://www.example.com"
 ### RAW
 http://localhost:8080/raw/https://www.example.com
 
+
+### Running Ruleset
+http://localhost:8080/ruleset
+
 ## Configuration
 
 ### Environment Variables
 
-| Variable | Description | Default |
+| Variable | Description | Value |
 | --- | --- | --- |
 | `PORT` | Port to listen on | `8080` |
 | `PREFORK` | Spawn multiple server instances | `false` |
@@ -85,10 +106,53 @@ http://localhost:8080/raw/https://www.example.com
 | `LOG_URLS` | Log fetched URL's | `true` |
 | `DISABLE_FORM` | Disables URL Form Frontpage | `false` |
 | `FORM_PATH` | Path to custom Form HTML | `` |
-| `RULES_URL` | URL to a ruleset file | `https://raw.githubusercontent.com/kubero-dev/ladder/main/ruleset.yaml` |
+| `RULESET` | URL to a ruleset file | `https://raw.githubusercontent.com/kubero-dev/ladder/main/ruleset.yaml` or `/path/to/my/rules.yaml` |
+| `EXPOSE_RULESET` | Make your Ruleset available to other ladders | `true` |
+| `ALLOWED_DOMAINS` | Comma separated list of allowed domains. Empty = no limitations | `` |
+| `ALLOWED_DOMAINS_RULESET` | Allow Domains from Ruleset. false = no limitations | `false` |
+
+`ALLOWED_DOMAINS` and `ALLOWED_DOMAINS_RULESET` are joined together. If both are empty, no limitations are applied.
 
 ### Ruleset
 
 It is possible to apply custom rules to modify the response. This can be used to remove unwanted or modify elements from the page. The ruleset is a YAML file that contains a list of rules for each domain and is loaded on startup
 
 See in [ruleset.yaml](ruleset.yaml) for an example.
+
+```yaml
+- domain: example.com          # Inbcludes all subdomains
+  domains:                     # Additional domains to apply the rule
+    - www.example.de
+    - www.beispiel.de
+  headers:
+    x-forwarded-for: none      # override X-Forwarded-For header or delete with none
+    referer: none              # override Referer header or delete with none
+    user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36
+    content-security-policy: script-src 'self'; # override response header
+    cookie: privacy=1
+  regexRules:
+    - match: <script\s+([^>]*\s+)?src="(/)([^"]*)"
+      replace: <script $1 script="/https://www.example.com/$3"
+  injections:
+    - position: head # Position where to inject the code
+      append: |      # possible keys: append, prepend, replace
+        <script>
+          window.localStorage.clear();
+          console.log("test");
+          alert("Hello!");
+        </script>
+- domain: www.anotherdomain.com # Domain where the rule applies
+  paths:                        # Paths where the rule applies
+    - /article
+  googleCache: false            # Use Google Cache to fetch the content
+  regexRules:                   # Regex rules to apply
+    - match: <script\s+([^>]*\s+)?src="(/)([^"]*)"
+      replace: <script $1 script="/https://www.example.com/$3"
+  injections:
+    - position: .left-content article .post-title # Position where to inject the code into DOM
+      replace: | 
+        <h1>My Custom Title</h1>
+    - position: .left-content article # Position where to inject the code into DOM
+      prepend: | 
+        <h2>Suptitle</h2>
+```

cmd/main.go

@@ -7,7 +7,6 @@ import (
 	"ladder/handlers"
 	"log"
 	"os"
-	"strconv"
 	"strings"
 
 	"github.com/akamensky/argparse"
@@ -23,31 +22,32 @@ func main() {
 
 	parser := argparse.NewParser("ladder", "Every Wall needs a Ladder")
 
-	p := os.Getenv("PORT")
+	portEnv := os.Getenv("PORT")
 	if os.Getenv("PORT") == "" {
-		p = "8080"
+		portEnv = "8080"
 	}
 	port := parser.String("p", "port", &argparse.Options{
 		Required: false,
-		Default:  p,
+		Default:  portEnv,
 		Help:     "Port the webserver will listen on"})
 
-	pf, _ := strconv.ParseBool(os.Getenv("PREFORK"))
-
 	prefork := parser.Flag("P", "prefork", &argparse.Options{
 		Required: false,
-		Default:  pf,
 		Help:     "This will spawn multiple processes listening"})
 
-	// Parse input
 	err := parser.Parse(os.Args)
 	if err != nil {
 		fmt.Print(parser.Usage(err))
 	}
 
+	if os.Getenv("PREFORK") == "true" {
+		*prefork = true
+	}
+
 	app := fiber.New(
 		fiber.Config{
 			Prefork: *prefork,
+			GETOnly: true,
 		},
 	)
 
@@ -74,9 +74,11 @@ func main() {
 	}
 
 	app.Get("/", handlers.Form)
+	app.Get("ruleset", handlers.Ruleset)
 
 	app.Get("raw/*", handlers.Raw)
 	app.Get("api/*", handlers.Api)
+	app.Get("ruleset", handlers.Raw)
 	app.Get("/*", handlers.ProxySite)
 
 	log.Fatal(app.Listen(":" + *port))

docker-compose.yaml

@@ -3,12 +3,17 @@ services:
   ladder:
     image: ghcr.io/kubero-dev/ladder:latest
     container_name: ladder
-    build: .
+    #build: .
     #restart: always
     #command: sh -c ./ladder
     environment:
       - PORT=8080
-      #- PREFORK=true
+      - RULESET=/app/ruleset.yaml
+      #- ALLOWED_DOMAINS_RULESET=false
+      #- EXPOSE_RULESET=true
+      #- PREFORK=false
+      #- DISABLE_FORM=fase
+      #- FORM_PATH=/app/form.html
       #- X_FORWARDED_FOR=66.249.66.1
       #- USER_AGENT=Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
       #- USERPASS=foo:bar
@@ -16,11 +21,6 @@ services:
       #- GODEBUG=netdns=go
     ports:
       - "8080:8080"
-    deploy:
+    volumes:
-      resources:
+      - ./ruleset.yaml:/app/ruleset.yaml
-        limits:
+      - ./handlers/form.html:/app/form.html
-          cpus: "0.50"
-          memory: 512M
-        reservations:
-          cpus: "0.25"
-          memory: 128M

handlers/api.go

@@ -27,7 +27,8 @@ func Api(c *fiber.Ctx) error {
 		Version: version,
 		Body:    body,
 	}
-	response.Request.Headers = make([]interface{}, 0)
+
+	response.Request.Headers = make([]any, 0, len(req.Header))
 	for k, v := range req.Header {
 		response.Request.Headers = append(response.Request.Headers, map[string]string{
 			"key":   k,
@@ -35,7 +36,7 @@ func Api(c *fiber.Ctx) error {
 		})
 	}
 
-	response.Response.Headers = make([]interface{}, 0)
+	response.Response.Headers = make([]any, 0, len(resp.Header))
 	for k, v := range resp.Header {
 		response.Response.Headers = append(response.Response.Headers, map[string]string{
 			"key":   k,

handlers/proxy.go

@@ -18,6 +18,7 @@ import (
 var UserAgent = getenv("USER_AGENT", "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)")
 var ForwardedFor = getenv("X_FORWARDED_FOR", "66.249.66.1")
 var rulesSet = loadRules()
+var allowedDomains = strings.Split(os.Getenv("ALLOWED_DOMAINS"), ",")
 
 func ProxySite(c *fiber.Ctx) error {
 	// Get the url from the URL
@@ -32,6 +33,8 @@ func ProxySite(c *fiber.Ctx) error {
 	}
 
 	c.Set("Content-Type", resp.Header.Get("Content-Type"))
+	c.Set("Content-Security-Policy", resp.Header.Get("Content-Security-Policy"))
+
 	return c.SendString(body)
 }
 
@@ -51,17 +54,53 @@ func fetchSite(urlpath string, queries map[string]string) (string, *http.Request
 		return "", nil, nil, err
 	}
 
-	if os.Getenv("DEBUG	") == "true" {
+	if len(allowedDomains) > 0 && !StringInSlice(u.Host, allowedDomains) {
+		return "", nil, nil, fmt.Errorf("domain not allowed. %s not in %s", u.Host, allowedDomains)
+	}
+
+	if os.Getenv("LOG_URLS") == "true" {
 		log.Println(u.String() + urlQuery)
 	}
 
+	rule := fetchRule(u.Host, u.Path)
+
+	if rule.GoogleCache {
+		u, err = url.Parse("https://webcache.googleusercontent.com/search?q=cache:" + u.String())
+		if err != nil {
+			return "", nil, nil, err
+		}
+	}
+
 	// Fetch the site
 	client := &http.Client{}
 	req, _ := http.NewRequest("GET", u.String()+urlQuery, nil)
+
+	if rule.Headers.UserAgent != "" {
+		req.Header.Set("User-Agent", rule.Headers.UserAgent)
+	} else {
 		req.Header.Set("User-Agent", UserAgent)
+	}
+
+	if rule.Headers.XForwardedFor != "" {
+		if rule.Headers.XForwardedFor != "none" {
+			req.Header.Set("X-Forwarded-For", rule.Headers.XForwardedFor)
+		}
+	} else {
 		req.Header.Set("X-Forwarded-For", ForwardedFor)
+	}
+
+	if rule.Headers.Referer != "" {
+		if rule.Headers.Referer != "none" {
+			req.Header.Set("Referer", rule.Headers.Referer)
+		}
+	} else {
 		req.Header.Set("Referer", u.String())
-	req.Header.Set("Host", u.Host)
+	}
+
+	if rule.Headers.Cookie != "" {
+		req.Header.Set("Cookie", rule.Headers.Cookie)
+	}
+
 	resp, err := client.Do(req)
 
 	if err != nil {
@@ -74,11 +113,16 @@ func fetchSite(urlpath string, queries map[string]string) (string, *http.Request
 		return "", nil, nil, err
 	}
 
-	body := rewriteHtml(bodyB, u)
+	if rule.Headers.CSP != "" {
+		resp.Header.Set("Content-Security-Policy", rule.Headers.CSP)
+	}
+
+	log.Print("rule", rule)
+	body := rewriteHtml(bodyB, u, rule)
 	return body, req, resp, nil
 }
 
-func rewriteHtml(bodyB []byte, u *url.URL) string {
+func rewriteHtml(bodyB []byte, u *url.URL, rule Rule) string {
 	// Rewrite the HTML
 	body := string(bodyB)
 
@@ -98,8 +142,8 @@ func rewriteHtml(bodyB []byte, u *url.URL) string {
 	body = strings.ReplaceAll(body, "url(/", "url(/https://"+u.Host+"/")
 	body = strings.ReplaceAll(body, "href=\"https://"+u.Host, "href=\"/https://"+u.Host+"/")
 
-	if os.Getenv("RULES_URL") != "" {
+	if os.Getenv("RULESET") != "" {
-		body = applyRules(u.Host, u.Path, body)
+		body = applyRules(body, rule)
 	}
 	return body
 }
@@ -113,13 +157,16 @@ func getenv(key, fallback string) string {
 }
 
 func loadRules() RuleSet {
-	rulesUrl := os.Getenv("RULES_URL")
+	rulesUrl := os.Getenv("RULESET")
 	if rulesUrl == "" {
 		RulesList := RuleSet{}
 		return RulesList
 	}
 	log.Println("Loading rules")
 
+	var ruleSet RuleSet
+	if strings.HasPrefix(rulesUrl, "http") {
+
 		resp, err := http.Get(rulesUrl)
 		if err != nil {
 			log.Println("ERROR:", err)
@@ -134,29 +181,61 @@ func loadRules() RuleSet {
 		if err != nil {
 			log.Println("ERROR:", err)
 		}
-
-	var ruleSet RuleSet
 		yaml.Unmarshal(body, &ruleSet)
+
 		if err != nil {
 			log.Println("ERROR:", err)
 		}
+	} else {
+		yamlFile, err := os.ReadFile(rulesUrl)
+		if err != nil {
+			log.Println("ERROR:", err)
+		}
+		yaml.Unmarshal(yamlFile, &ruleSet)
+	}
 
-	log.Println("Loaded rules for", len(ruleSet), "Domains")
+	domains := []string{}
+	for _, rule := range ruleSet {
+
+		domains = append(domains, rule.Domain)
+		domains = append(domains, rule.Domains...)
+		if os.Getenv("ALLOWED_DOMAINS_RULESET") == "true" {
+			allowedDomains = append(allowedDomains, domains...)
+		}
+	}
+
+	log.Println("Loaded ", len(ruleSet), " rules for", len(domains), "Domains")
 	return ruleSet
 }
 
-func applyRules(domain string, path string, body string) string {
+func fetchRule(domain string, path string) Rule {
+	if len(rulesSet) == 0 {
+		return Rule{}
+	}
+	rule := Rule{}
+	for _, rule := range rulesSet {
+		domains := rule.Domains
+		if rule.Domain != "" {
+			domains = append(domains, rule.Domain)
+		}
+		for _, ruleDomain := range domains {
+			if ruleDomain == domain || strings.HasSuffix(domain, ruleDomain) {
+				if len(rule.Paths) > 0 && !StringInSlice(path, rule.Paths) {
+					continue
+				}
+				// return first match
+				return rule
+			}
+		}
+	}
+	return rule
+}
+
+func applyRules(body string, rule Rule) string {
 	if len(rulesSet) == 0 {
 		return body
 	}
 
-	for _, rule := range rulesSet {
-		if rule.Domain != domain {
-			continue
-		}
-		if rule.Path != "" && rule.Path != path {
-			continue
-		}
 	for _, regexRule := range rule.RegexRules {
 		re := regexp.MustCompile(regexRule.Match)
 		body = re.ReplaceAllString(body, regexRule.Replace)
@@ -180,25 +259,15 @@ func applyRules(domain string, path string, body string) string {
 			log.Fatal(err)
 		}
 	}
-	}
 
 	return body
 }
 
-type Rule struct {
+func StringInSlice(s string, list []string) bool {
-	Match   string `yaml:"match"`
+	for _, x := range list {
-	Replace string `yaml:"replace"`
+		if strings.HasPrefix(s, x) {
-}
+			return true
-
+		}
-type RuleSet []struct {
+	}
-	Domain      string `yaml:"domain"`
+	return false
-	Path        string `yaml:"path,omitempty"`
-	GoogleCache bool   `yaml:"googleCache,omitempty"`
-	RegexRules  []Rule `yaml:"regexRules"`
-	Injections  []struct {
-		Position string `yaml:"position"`
-		Append   string `yaml:"append"`
-		Prepend  string `yaml:"prepend"`
-		Replace  string `yaml:"replace"`
-	} `yaml:"injections"`
 }

handlers/proxy.test.go

@@ -51,7 +51,7 @@ func TestRewriteHtml(t *testing.T) {
 		</html>
 	`
 
-	actual := rewriteHtml(bodyB, u)
+	actual := rewriteHtml(bodyB, u, Rule{})
 	assert.Equal(t, expected, actual)
 }
 

handlers/ruleset.go

@@ -0,0 +1,24 @@
+package handlers
+
+import (
+	"os"
+
+	"github.com/gofiber/fiber/v2"
+	"gopkg.in/yaml.v3"
+)
+
+func Ruleset(c *fiber.Ctx) error {
+
+	if os.Getenv("EXPOSE_RULESET") == "false" {
+		c.SendStatus(fiber.StatusForbidden)
+		return c.SendString("Rules Disabled")
+	}
+
+	body, err := yaml.Marshal(rulesSet)
+	if err != nil {
+		c.SendStatus(fiber.StatusInternalServerError)
+		return c.SendString(err.Error())
+	}
+
+	return c.SendString(string(body))
+}

handlers/types.go

@@ -0,0 +1,29 @@
+package handlers
+
+type Regex struct {
+	Match   string `yaml:"match"`
+	Replace string `yaml:"replace"`
+}
+
+type RuleSet []Rule
+
+type Rule struct {
+	Domain  string   `yaml:"domain,omitempty"`
+	Domains []string `yaml:"domains,omitempty"`
+	Paths   []string `yaml:"paths,omitempty"`
+	Headers struct {
+		UserAgent     string `yaml:"user-agent,omitempty"`
+		XForwardedFor string `yaml:"x-forwarded-for,omitempty"`
+		Referer       string `yaml:"referer,omitempty"`
+		Cookie        string `yaml:"cookie,omitempty"`
+		CSP           string `yaml:"content-security-policy,omitempty"`
+	} `yaml:"headers,omitempty"`
+	GoogleCache bool    `yaml:"googleCache,omitempty"`
+	RegexRules  []Regex `yaml:"regexRules"`
+	Injections  []struct {
+		Position string `yaml:"position"`
+		Append   string `yaml:"append"`
+		Prepend  string `yaml:"prepend"`
+		Replace  string `yaml:"replace"`
+	} `yaml:"injections"`
+}

helm-chart/Chart.yaml

@@ -0,0 +1,6 @@
+apiVersion: v2
+name: ladder
+description: A helm chart to deploy kubero-dev/ladder 
+type: application
+version: "1.0"
+appVersion: "v0.0.11"

helm-chart/README.md

@@ -0,0 +1,27 @@
+# Helm Chart for deployment of Ladder
+This folder contains a basic helm chart deployment for the ladder app.  
+
+# Deployment pre-reqs
+## Values
+Edit the values to your own preferences, with the only minimum requirement being `ingress.HOST` (line 19) being updated to your intended domain name.  
+
+Other variables in `values.yaml` can be updated as to your preferences, with details on each variable being listed in the main [README.md](/README.md) in the root of this repo.  
+
+## Defaults in K8s
+No ingress default has been specified. 
+You can set this manually by adding an annotation to the ingress.yaml - if needed.  
+For example, to use Traefik - 
+```yaml
+metadata:
+  name: ladder-ingress
+  annotations:
+    kubernetes.io/ingress.class: traefik
+```
+
+## Helm Install
+`helm install <name> <location> -n <namespace-name> --create-namespace`  
+`helm install ladder .\ladder\ -n ladder --create-namespace`  
+
+## Helm Upgrade
+`helm upgrade <name> <location> -n <namespace-name>`  
+`helm upgrade ladder .\ladder\ -n ladder`  

helm-chart/templates/deployment.yaml

@@ -0,0 +1,55 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: ladder
+  name: ladder
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: ladder
+  template:
+    metadata:
+      labels:
+        app: ladder
+    spec:
+      containers:
+      - image: "{{ .Values.image.RELEASE }}"
+        imagePullPolicy: Always
+        name: ladder
+        resources:
+          limits:
+            cpu: 250m
+            memory: 128Mi
+          requests:
+            cpu: 250m
+            memory: 128Mi
+        env:
+          - name: PORT
+            value: "{{ .Values.env.PORT }}"
+          - name: PREFORK
+            value: "{{ .Values.env.PREFORK }}"
+          - name: USER_AGENT
+            value: "{{ .Values.env.USER_AGENT }}"
+          - name: X_FORWARDED_FOR
+            value: "{{ .Values.env.X_FORWARDED_FOR }}"
+          - name: USERPASS
+            value: "{{ .Values.env.USERPASS }}"
+          - name: LOG_URLS
+            value: "{{ .Values.env.LOG_URLS }}"
+          - name: DISABLE_FORM
+            value: "{{ .Values.env.DISABLE_FORM }}"
+          - name: FORM_PATH
+            value: "{{ .Values.env.FORM_PATH }}"
+          - name: RULESET
+            value: "{{ .Values.env.RULESET }}"
+          - name: EXPOSE_RULESET
+            value: "{{ .Values.env.EXPOSE_RULESET }}"
+          - name: ALLOWED_DOMAINS
+            value: "{{ .Values.env.ALLOWED_DOMAINS }}"
+          - name: ALLOWED_DOMAINS_RULESET
+            value: "{{ .Values.env.ALLOWED_DOMAINS_RULESET }}"
+      restartPolicy: Always
+      terminationGracePeriodSeconds: 30

helm-chart/templates/ingress.yaml

@@ -0,0 +1,17 @@
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: ladder-ingress
+spec:
+  rules:
+  - host: "{{ .Values.ingress.HOST }}"
+    http:
+      paths:
+      - path: /
+        pathType: Prefix
+        backend:
+          service:
+            name: ladder-service
+            port:
+              number: {{ .Values.ingress.PORT }}

helm-chart/templates/service.yaml

@@ -0,0 +1,14 @@
+---
+kind: Service
+apiVersion: v1
+metadata:
+  name: ladder-service
+spec:
+  type: ClusterIP
+  selector:
+    app: ladder
+  ports:
+  - name: http
+    port: {{ .Values.ingress.PORT }}
+    protocol: TCP
+    targetPort: {{ .Values.env.PORT }}

helm-chart/values.yaml

@@ -0,0 +1,20 @@
+image:
+  RELEASE: ghcr.io/kubero-dev/ladder:v0.0.11
+  
+env:
+  PORT: 8080
+  PREFORK: "false"
+  USER_AGENT: "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
+  X_FORWARDED_FOR:
+  USERPASS: ""
+  LOG_URLS: "true"
+  DISABLE_FORM: "false"
+  FORM_PATH: ""
+  RULESET: "https://raw.githubusercontent.com/kubero-dev/ladder/main/ruleset.yaml"
+  EXPOSE_RULESET: "true"
+  ALLOWED_DOMAINS: ""
+  ALLOWED_DOMAINS_RULESET: "false"
+
+ingress:
+  HOST: "ladder.domain.com"
+  PORT: 80

ruleset.yaml

@@ -1,4 +1,12 @@
-- domain: www.example.com
+- domain: example.com
+  domains: 
+  - www.beispiel.de
+  googleCache: true
+  headers:
+    x-forwarded-for: none
+    referer: none
+    user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36
+    cookie: privacy=1
   regexRules:
     - match: <script\s+([^>]*\s+)?src="(/)([^"]*)"
       replace: <script $1 script="/https://www.example.com/$3"
@@ -10,16 +18,148 @@
           console.log("test");
           alert("Hello!");
         </script>
-- domain: www.anotherdomain.com # Domain where the rule applies
+    - position: h1
-  path: /article                # Path where the rule applies
-  googleCache: false            # Search also in Google Cache
-  regexRules:                   # Regex rules to apply
-    - match: <script\s+([^>]*\s+)?src="(/)([^"]*)"
-      replace: <script $1 script="/https://www.example.com/$3"
-  injections:
-    - position: .left-content article .post-title # Position where to inject the code into DOM
       replace: |
-        <h1>My Custom Title</h1>
+        <h1>An example with a ladder ;-)</h1>
-    - position: .left-content article # Position where to inject the code into DOM
+- domain: www.americanbanker.com
-      prepend: | 
+  paths:
-        <h2>Suptitle</h2>
+    - /news
+  injections:
+    - position: head
+      append: |
+        <script>
+          document.addEventListener("DOMContentLoaded", () => {
+            const inlineGate = document.querySelector('.inline-gate');
+            if (inlineGate) {
+              inlineGate.classList.remove('inline-gate');
+              const inlineGated = document.querySelectorAll('.inline-gated');
+              for (const elem of inlineGated) { elem.classList.remove('inline-gated'); }
+            }
+          });
+        </script>
+- domain: www.nzz.ch
+  paths:
+    - /international
+    - /sport
+    - /wirtschaft
+    - /technologie
+    - /feuilleton
+    - /zuerich
+    - /wissenschaft
+    - /gesellschaft
+    - /panorama
+    - /mobilitaet
+    - /reisen
+    - /meinung
+    - /finanze
+  injections:
+    - position: head
+      append: |
+        <script>
+          document.addEventListener("DOMContentLoaded", () => {
+            const paywall = document.querySelector('.dynamic-regwall');
+            removeDOMElement(paywall)
+          });
+        </script>
+- domains: 
+  - www.architecturaldigest.com
+  - www.bonappetit.com
+  - www.cntraveler.com
+  - www.epicurious.com
+  - www.gq.com
+  - www.newyorker.com
+  - www.vanityfair.com
+  - www.vogue.com
+  - www.wired.com
+  injections:
+    - position: head
+      append: |
+        <script>
+          document.addEventListener("DOMContentLoaded", () => {
+            const banners = document.querySelectorAll('.paywall-bar, div[class^="MessageBannerWrapper-"');
+            banners.forEach(el => { el.remove(); });
+          });
+        </script>
+- domains: 
+  - www.nytimes.com
+  - www.time.com
+  headers:
+    ueser-agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
+    cookie: nyt-a=; nyt-gdpr=0; nyt-geo=DE; nyt-privacy=1
+    referer: https://www.google.com/ 
+  injections:
+    - position: head
+      append: |
+        <script>
+          window.localStorage.clear();
+          document.addEventListener("DOMContentLoaded", () => {
+            const banners = document.querySelectorAll('div[data-testid="inline-message"], div[id^="ad-"], div[id^="leaderboard-"], div.expanded-dock, div.pz-ad-box, div[id="top-wrapper"], div[id="bottom-wrapper"]');
+            banners.forEach(el => { el.remove(); });
+          });
+        </script>
+- domains: 
+  - www.thestar.com
+  - www.niagarafallsreview.ca
+  - www.stcatharinesstandard.ca
+  - www.thepeterboroughexaminer.com
+  - www.therecord.com
+  - www.thespec.com
+  - www.wellandtribune.ca
+  injections:
+    - position: head
+      append: |
+        <script>
+          window.localStorage.clear();
+          document.addEventListener("DOMContentLoaded", () => {
+            const paywall = document.querySelectorAll('div.subscriber-offers');
+            paywall.forEach(el => { el.remove(); });
+            const subscriber_only = document.querySelectorAll('div.subscriber-only');
+            for (const elem of subscriber_only) {
+              if (elem.classList.contains('encrypted-content') && dompurify_loaded) {
+                const parser = new DOMParser();
+                const doc = parser.parseFromString('<div>' + DOMPurify.sanitize(unscramble(elem.innerText)) + '</div>', 'text/html');
+                const content_new = doc.querySelector('div');
+                elem.parentNode.replaceChild(content_new, elem);
+              }
+              elem.removeAttribute('style');
+              elem.removeAttribute('class');
+            }
+            const banners = document.querySelectorAll('div.subscription-required, div.redacted-overlay, div.subscriber-hide, div.tnt-ads-container');
+            banners.forEach(el => { el.remove(); });
+            const ads = document.querySelectorAll('div.tnt-ads-container, div[class*="adLabelWrapper"]');
+            ads.forEach(el => { el.remove(); });
+            const recommendations = document.querySelectorAll('div[id^="tncms-region-article"]');
+            recommendations.forEach(el => { el.remove(); });
+          });
+        </script>
+- domain: www.usatoday.com
+  injections:
+    - position: head
+      append: |
+        <script>
+          document.addEventListener("DOMContentLoaded", () => {
+            const banners = document.querySelectorAll('div.roadblock-container, .gnt_nb, [aria-label="advertisement"], div[id="main-frame-error"]');
+            banners.forEach(el => { el.remove(); });
+          });
+        </script>
+- domain: www.washingtonpost.com
+  injections:
+    - position: head
+      append: |
+        <script>
+          document.addEventListener("DOMContentLoaded", () => {
+            let paywall = document.querySelectorAll('div[data-qa$="-ad"], div[id="leaderboard-wrapper"], div[data-qa="subscribe-promo"]');
+            paywall.forEach(el => { el.remove(); });
+            const images = document.querySelectorAll('img');
+            images.forEach(image => { image.parentElement.style.filter = ''; });
+            const headimage = document.querySelectorAll('div .aspect-custom');
+            headimage.forEach(image => { image.style.filter = ''; });
+          });
+        </script>
+- domain: medium.com
+  headers:
+    referer: https://t.co/x?amp=1
+    x-forwarded-for: none
+    user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36
+    content-security-policy: script-src 'self';
+    cookie: