🔖 release 1.2.0

docs/_sidebar.md

@@ -2,6 +2,7 @@
 - [quickstart](quickstart.md)
 - [responses](responses.md)
 - [routes](routes.md)
+- [static files](static_files.md)
 - middleware
   - [overview](middleware/overview.md)
   - [session](middleware/sessions.md)
@@ -9,4 +10,4 @@
   - [cors](middleware/cors.md)
   - [pydantic](middleware/pydantic.md)
   - [writing your own](middleware/custom_middleware.md)
-- [databases](db.md)
+- [databases](db.md)

docs/routes.md

@@ -103,6 +103,7 @@ You can pass integers, strings, and positive floats with the following types:
 - str
 - int
 - float
+- path (see below)
 
 A URL can also have multiple capture groups:
 
@@ -113,6 +114,15 @@ def example(request, id, name):
 ```
 In this case, a valid URL might be `/example/3/james`, and both sections will be split out and passed to the view.
 
+The `path` option is special; this is used when you want to capture everything after the slash. For example:
+
+```python
+@app.route("/example/<path:rest>")
+def example(request, rest):
+    return HttpResponse(body=f"Example with {rest}")
+```
+It will come in as a string, but it will include all the slashes and other characters that are in the URL.
+
 ## Adding Error Views
 
 For some apps, you may want to have your own error views that are themed to your particular application. For this, there's a slightly different process, but the gist is the same. There are also three ways to handle error views, all very similar to adding regular views.
@@ -184,4 +194,4 @@ path = app.reverse("example", {'obj_id': 3}, query={'name': 'james'})
 print(path)  # -> "/example/3?name=james"
 ```
 
-The arguments you pass in must match what the path expects, or you'll get a `SpiderwebException`. If there's no route with that name, you'll get a `ReverseNotFound` exception instead.
+The arguments you pass in must match what the path expects, or you'll get a `SpiderwebException`. If there's no route with that name, you'll get a `ReverseNotFound` exception instead.

docs/static_files.md

@@ -0,0 +1,65 @@
+# serving static files for local development
+
+When you're developing locally, it's often useful to be able to serve static files directly from your application, especially when you're working on the frontend. Spiderweb does have a mechanism for serving static files, but it's _not recommended_ (read: this is a Very Bad Idea) for production use. Instead, you should use a reverse proxy like nginx or Apache to serve them.
+
+To serve static files locally, you'll need to tell Spiderweb where they are. Once you fill this out, Spiderweb will automatically handle the routing to find them.
+
+Before we get started:
+
+> [!DANGER]
+> Having Spiderweb handle your static files in production is a **critical safety issue**. It does its best to identify if a request is malicious, but it is much safer to have this be handled by a reverse proxy.
+
+```python
+from spiderweb import SpiderwebRouter
+
+app = SpiderwebRouter(
+    staticfiles_dirs=[
+        "my_static_files", 
+        "maybe_other_static_files_here"
+    ],
+    debug=True,
+    static_url="assets",
+)
+```
+
+> [!NOTE]
+> Note the `debug` attribute in the example above; even if `staticfiles_dirs` is set, Spiderweb will only serve the files if `debug` is set to `True`. This is a safety check for you and an easy toggle for deployment.
+
+## `staticfiles_dirs`
+
+> default: `[]`
+
+This is a list of directories that Spiderweb will look in for static files. When a request comes in for a static file, Spiderweb will look in each of these directories in order to find the file. If it doesn't find the file in any of the directories, it will return a 404.
+
+## `static_url`
+
+> default: `static`
+
+This is the URL that Spiderweb will use to serve static files. In the example above, the URL would be `http://localhost:8000/assets/`. If you don't set this, Spiderweb will default to `/static/`.
+
+## `debug`
+
+> default: `False`
+
+This is a boolean that tells Spiderweb whether it is running in debug mode or not. Among other things, it's used in serving static files. If this value is not included, it defaults to False, and Spiderweb will not serve static files. For local development, you will want to set it to True.
+
+## Linking to static files
+
+There is a tag in the templates that you can use to link to static files. This tag will automatically generate the correct URL for the file based on the `static_url` attribute you set in the router.
+
+```html
+<img 
+    src="{% static 'hello_world.gif' %}" 
+    alt="A rotating globe with the caption, 'hello world'."
+>
+```
+
+In this example, the `static` tag will generate a URL that looks like `/assets/hello_world.gif`. This is the URL that the browser will use to request the file. If you have a file that is in a folder, you can specify that in the tag:
+
+```html
+<img 
+    src="{% static 'gifs/landing/hello_world.gif' %}" 
+    alt="A rotating globe with the caption, 'hello world'."
+>
+```
+This will pull the gif from `{your static folder}/gifs/landing/hello_world.gif`.

example.py

@@ -26,6 +26,8 @@ app = SpiderwebRouter(
     staticfiles_dirs=["static_files"],
     append_slash=False,  # default
     cors_allow_all_origins=True,
+    static_url="static_stuff",
+    debug=True,
 )
 
 

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "spiderweb-framework"
-version = "1.1.0"
+version = "1.2.0"
 description = "A small web framework, just big enough for a spider."
 authors = ["Joe Kaufeld <opensource@joekaufeld.com>"]
 readme = "README.md"

spiderweb/constants.py

@@ -2,7 +2,7 @@ from peewee import DatabaseProxy
 
 DEFAULT_ALLOWED_METHODS = ["POST", "GET", "PUT", "PATCH", "DELETE"]
 DEFAULT_ENCODING = "UTF-8"
-__version__ = "1.1.0"
+__version__ = "1.2.0"
 
 # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie
 REGEX_COOKIE_NAME = r"^[a-zA-Z0-9\s\(\)<>@,;:\/\\\[\]\?=\{\}\"\t]*$"

spiderweb/converters.py

@@ -20,3 +20,11 @@ class FloatConverter:
 
     def to_python(self, value):
         return float(value)
+
+
+class PathConverter:
+    regex = r".+"
+    name = "path"
+
+    def to_python(self, value):
+        return str(value)

spiderweb/jinja_core.py

@@ -0,0 +1,16 @@
+from typing import TYPE_CHECKING
+
+from jinja2 import Environment
+
+
+if TYPE_CHECKING:
+    from spiderweb import SpiderwebRouter
+
+
+class SpiderwebEnvironment(Environment):
+    # Contains all the normal abilities of the Jinja environment, but with a link
+    # back to the server for easy access to settings and other server-related
+    # information.
+    def __init__(self, server=None, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.server: "SpiderwebRouter" = server

spiderweb/jinja_extensions.py

@@ -0,0 +1,19 @@
+import posixpath
+
+from jinja2 import nodes
+from jinja2.ext import Extension
+
+
+class StaticFilesExtension(Extension):
+    # Take things that look like `{% static "file" %}` and replace them with `/static/file`
+    tags = {"static"}
+
+    def parse(self, parser):
+        token = next(parser.stream)
+        args = [parser.parse_expression()]
+        return nodes.Output([self.call_method("_static", args)]).set_lineno(
+            token.lineno
+        )
+
+    def _static(self, file):
+        return posixpath.join(f"/{self.environment.server.static_url}", file)

spiderweb/main.py

@@ -6,10 +6,10 @@ import traceback
 import urllib.parse as urlparse
 from logging import Logger
 from threading import Thread
-from typing import Optional, Callable, Sequence, LiteralString, Literal
+from typing import Optional, Callable, Sequence, Literal
 from wsgiref.simple_server import WSGIServer
 
-from jinja2 import BaseLoader, Environment, FileSystemLoader
+from jinja2 import BaseLoader, FileSystemLoader
 from peewee import Database, SqliteDatabase
 
 from spiderweb.middleware import MiddlewareMixin
@@ -31,6 +31,7 @@ from spiderweb.exceptions import (
     NoResponseError,
     SpiderwebNetworkException,
 )
+from spiderweb.jinja_core import SpiderwebEnvironment
 from spiderweb.local_server import LocalServerMixin
 from spiderweb.request import Request
 from spiderweb.response import HttpResponse, TemplateResponse, JsonResponse
@@ -61,10 +62,12 @@ class SpiderwebRouter(LocalServerMixin, MiddlewareMixin, RoutesMixin, FernetMixi
         cors_allow_private_network: bool = False,
         csrf_trusted_origins: Sequence[str] = None,
         db: Optional[Database] = None,
+        debug: bool = False,
         templates_dirs: Sequence[str] = None,
         middleware: Sequence[str] = None,
         append_slash: bool = False,
         staticfiles_dirs: Sequence[str] = None,
+        static_url: str = "static",
         routes: Sequence[tuple[str, Callable] | tuple[str, Callable, dict]] = None,
         error_routes: dict[int, Callable] = None,
         secret_key: str = None,
@@ -87,6 +90,7 @@ class SpiderwebRouter(LocalServerMixin, MiddlewareMixin, RoutesMixin, FernetMixi
         self.append_slash = append_slash
         self.templates_dirs = templates_dirs
         self.staticfiles_dirs = staticfiles_dirs
+        self.static_url = static_url
         self._middleware: list[str] = middleware or []
         self.middleware: list[Callable] = []
         self.secret_key = secret_key if secret_key else self.generate_key()
@@ -109,6 +113,8 @@ class SpiderwebRouter(LocalServerMixin, MiddlewareMixin, RoutesMixin, FernetMixi
             convert_url_to_regex(i) for i in self._csrf_trusted_origins
         ]
 
+        self.debug = debug
+
         self.extra_data = kwargs
 
         # session middleware
@@ -144,13 +150,23 @@ class SpiderwebRouter(LocalServerMixin, MiddlewareMixin, RoutesMixin, FernetMixi
         if self.error_routes:
             self.add_error_routes()
 
+        template_env_args = {
+            "server": self,
+            "extensions": [
+                "spiderweb.jinja_extensions.StaticFilesExtension",
+            ],
+        }
+
         if self.templates_dirs:
-            self.template_loader = Environment(
-                loader=FileSystemLoader(self.templates_dirs)
+            self.template_loader = SpiderwebEnvironment(
+                loader=FileSystemLoader(self.templates_dirs),
+                **template_env_args,
             )
         else:
             self.template_loader = None
-        self.string_loader = Environment(loader=BaseLoader())
+        self.string_loader = SpiderwebEnvironment(
+            loader=BaseLoader(), **template_env_args
+        )
 
         if self.staticfiles_dirs:
             for static_dir in self.staticfiles_dirs:
@@ -160,7 +176,14 @@ class SpiderwebRouter(LocalServerMixin, MiddlewareMixin, RoutesMixin, FernetMixi
                         f"Static files directory '{str(static_dir)}' does not exist."
                     )
                     raise ConfigError
-            self.add_route(r"/static/<str:filename>", send_file)  # noqa: F405
+            if self.debug:
+                # We don't need a log message here because this is the expected behavior
+                self.add_route(rf"/{self.static_url}/<path:filename>", send_file)  # noqa: F405
+            else:
+                self.log.warning(
+                    "`staticfiles_dirs` is set, but `debug` is set to FALSE. Static"
+                    " files will not be served."
+                )
 
         # finally, run the startup checks to verify everything is correct and happy.
         self.log.info("Run startup checks...")
@@ -187,7 +210,14 @@ class SpiderwebRouter(LocalServerMixin, MiddlewareMixin, RoutesMixin, FernetMixi
 
             start_response(status, headers)
 
-            rendered_output = resp.render()
+            try:
+                rendered_output = resp.render()
+            except Exception as e:
+                self.log.error("Fatal error!")
+                self.log.error(e)
+                self.log.error(traceback.format_exc())
+                return [f"Internal Server Error: {e}".encode(DEFAULT_ENCODING)]
+
             if not isinstance(rendered_output, list):
                 rendered_output = [rendered_output]
             encoded_resp = [

spiderweb/middleware/__init__.py

@@ -2,9 +2,6 @@ from typing import Callable, ClassVar
 import sys
 
 from .base import SpiderwebMiddleware as SpiderwebMiddleware
-from .cors import CorsMiddleware as CorsMiddleware
-from .csrf import CSRFMiddleware as CSRFMiddleware
-from .sessions import SessionMiddleware as SessionMiddleware
 from ..exceptions import ConfigError, UnusedMiddleware, StartupErrors
 from ..request import Request
 from ..response import HttpResponse

spiderweb/tests/test_middleware.py

@@ -4,7 +4,7 @@ from datetime import timedelta
 import pytest
 from peewee import SqliteDatabase
 
-from spiderweb import SpiderwebRouter, HttpResponse, StartupErrors
+from spiderweb import SpiderwebRouter, HttpResponse, StartupErrors, ConfigError
 from spiderweb.constants import DEFAULT_ENCODING
 from spiderweb.middleware.cors import (
     ACCESS_CONTROL_ALLOW_ORIGIN,
@@ -94,6 +94,13 @@ def test_exploding_middleware():
     assert len(app.middleware) == 0
 
 
+def test_invalid_middleware():
+    with pytest.raises(ConfigError) as e:
+        SpiderwebRouter(middleware=["nonexistent.middleware"])
+
+    assert e.value.args[0] == "Middleware 'nonexistent.middleware' not found."
+
+
 def test_csrf_middleware_without_session_middleware():
     with pytest.raises(StartupErrors) as e:
         SpiderwebRouter(

spiderweb/utils.py

@@ -1,3 +1,4 @@
+import importlib
 import json
 import re
 import secrets
@@ -13,12 +14,10 @@ VALID_CHARS = string.ascii_letters + string.digits
 
 
 def import_by_string(name):
-    # https://stackoverflow.com/a/547867
-    components = name.split(".")
-    mod = __import__(components[0])
-    for comp in components[1:]:
-        mod = getattr(mod, comp)
-    return mod
+    mod_name, klass_name = name.rsplit(".", 1)
+    module = importlib.import_module(mod_name)
+    klass = getattr(module, klass_name)
+    return klass
 
 
 def is_safe_path(path: str) -> bool:

templates/test.html

@@ -13,7 +13,7 @@
         middleware is working.
     </p>
     <p>
-        <img src="/static/aaaaaa.gif" alt="AAAAAAAAAA">
+        <img src="{% static 'aaaaaa.gif' %}" alt="AAAAAAAAAA">
     </p>
     <p>
         {{ request.META }}