Skip to content

Support for non-serializable component parameters #120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 57 commits into from
Feb 2, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
48ef41d
fix use_location bug
Archmonger Jan 9, 2023
89a59d6
add missing migration
Archmonger Jan 10, 2023
2247a18
enable django mypy plugin
Archmonger Jan 10, 2023
f2b5a3a
Functional DB-backed component params
Archmonger Jan 10, 2023
0b11cf7
remove broken django stubs
Archmonger Jan 10, 2023
587cfa3
fix type hints
Archmonger Jan 10, 2023
e93253c
skip migrations
Archmonger Jan 10, 2023
8a84f1d
bump idom version
Archmonger Jan 10, 2023
3760dde
Use idom 0.43.0 hooks
Archmonger Jan 10, 2023
e686894
don't render component if args/kwargs are wrong
Archmonger Jan 10, 2023
4b7676c
docs and changelog
Archmonger Jan 10, 2023
0f57702
Add object in templatetag tests
Archmonger Jan 10, 2023
9432349
fix styling errors
Archmonger Jan 10, 2023
a87a7b5
type hints for user
Archmonger Jan 10, 2023
4f41975
validate function signature
Archmonger Jan 10, 2023
ec46469
use lowercase tuple
Archmonger Jan 11, 2023
5e75488
functioning eviction strategy
Archmonger Jan 11, 2023
7344a97
don't clean DB on startup if running django tests
Archmonger Jan 11, 2023
d2d4573
switch sys check to contextlib suppress
Archmonger Jan 11, 2023
ace4a34
view_to_component uses del_html_head_body_transform
Archmonger Jan 11, 2023
8cf2884
update docs
Archmonger Jan 11, 2023
d985e14
add serializable to dictionary
Archmonger Jan 11, 2023
3011e75
fix #35
Archmonger Jan 11, 2023
e9a1364
fix #29
Archmonger Jan 11, 2023
e0fcbd9
Make some utils public if they're harmless
Archmonger Jan 12, 2023
2adb676
Make DATE_FORMAT a global
Archmonger Jan 12, 2023
319dda2
validate params_query expiration within fetch queries
Archmonger Jan 12, 2023
00e3c85
refactor db_cleanup
Archmonger Jan 12, 2023
43f4a32
more deprivatization
Archmonger Jan 12, 2023
f79f936
document django_query_postprocessor
Archmonger Jan 12, 2023
83730cd
comment for _register_component
Archmonger Jan 12, 2023
5dd8d16
add postprocessor to dictionary
Archmonger Jan 12, 2023
9f6b757
docs revision
Archmonger Jan 14, 2023
5d56d23
misc docs updates
Archmonger Jan 14, 2023
827adf3
move hooks back into `django_idom.hooks`
Archmonger Jan 14, 2023
6a08531
Merge remote-tracking branch 'upstream/main' into database-backed-kwargs
Archmonger Jan 14, 2023
03f315b
v3.0.0
Archmonger Jan 14, 2023
8f83c1b
docs formatting
Archmonger Jan 14, 2023
be07851
better link for keys
Archmonger Jan 15, 2023
6209c07
subclass core's Connection
Archmonger Jan 15, 2023
04a2efc
Complete types list
Archmonger Jan 15, 2023
196c756
remove unused ignore
Archmonger Jan 15, 2023
ac415dd
tests for ComponentParams expiration
Archmonger Jan 16, 2023
6447885
use tuple for ComponentParamData
Archmonger Jan 16, 2023
1a9b06e
better type hints for ComponentParamData
Archmonger Jan 16, 2023
cb0bd4f
Better type hints for postprocessor_kwargs
Archmonger Jan 16, 2023
648d3a2
type hint fixes
Archmonger Jan 16, 2023
54b9840
type hint for ComponentPreloader
Archmonger Jan 16, 2023
976f10f
timezone might not always be available, so don't rely on it.
Archmonger Jan 20, 2023
3f6069c
more robust ComponentParams tests
Archmonger Jan 22, 2023
abc3439
always lazy load config
Archmonger Jan 22, 2023
d7809f9
remove defaults.py
Archmonger Jan 22, 2023
a2a2c8c
remove version bump
Archmonger Jan 28, 2023
468d52f
fix type hint issues
Archmonger Jan 28, 2023
0d9d4a9
change mypy to incremental
Archmonger Feb 1, 2023
a0194c0
format using new black version
Archmonger Feb 1, 2023
f5cc229
log when idom database is not ready at startup
Archmonger Feb 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ Using the following categories, list your changes in this order:

## [Unreleased]

- Nothing (yet)
### Security

- Fixed a potential method of component argument spoofing

## [2.2.1] - 2022-01-09

Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ line_length = 88
lines_after_imports = 2

[tool.mypy]
exclude = [
'migrations/.*',
]
ignore_missing_imports = true
warn_unused_configs = true
warn_redundant_casts = true
Expand Down
3 changes: 2 additions & 1 deletion requirements/pkg-deps.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
channels >=4.0.0
idom >=0.40.2, <0.41.0
idom >=0.43.0, <0.44.0
aiofile >=3.0
dill >=0.3.5
typing_extensions
6 changes: 3 additions & 3 deletions src/django_idom/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@
"IDOM_WEBSOCKET_URL",
"idom/",
)
IDOM_WS_MAX_RECONNECT_TIMEOUT = getattr(
IDOM_MAX_RECONNECT_TIMEOUT = getattr(
settings,
"IDOM_WS_MAX_RECONNECT_TIMEOUT",
604800,
"IDOM_MAX_RECONNECT_TIMEOUT",
259200, # Default to 3 days
)
IDOM_CACHE: BaseCache = (
caches["idom"]
Expand Down
4 changes: 3 additions & 1 deletion src/django_idom/hooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,9 @@ def use_location() -> Location:
# TODO: Use the browser's current page, rather than the WS route
scope = use_scope()
search = scope["query_string"].decode()
return Location(scope["path"], f"?{search}" if search else "")
return Location(
scope["path"], f"?{search}" if (search and (search != "undefined")) else ""
)


def use_origin() -> str | None:
Expand Down
26 changes: 26 additions & 0 deletions src/django_idom/migrations/0001_initial.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Generated by Django 4.1.3 on 2023-01-10 00:54

from django.db import migrations, models


class Migration(migrations.Migration):

initial = True

dependencies = []

operations = [
migrations.CreateModel(
name="ComponentParams",
fields=[
(
"uuid",
models.UUIDField(
editable=False, primary_key=True, serialize=False, unique=True
),
),
("data", models.BinaryField()),
("created_at", models.DateTimeField(auto_now_add=True)),
],
),
]
Empty file.
7 changes: 7 additions & 0 deletions src/django_idom/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from django.db import models


class ComponentParams(models.Model):
uuid = models.UUIDField(primary_key=True, editable=False, unique=True) # type: ignore
data = models.BinaryField(editable=False) # type: ignore
created_at = models.DateTimeField(auto_now_add=True, editable=False) # type: ignore
7 changes: 3 additions & 4 deletions src/django_idom/templates/idom/component.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,12 @@
<div id="{{ idom_mount_uuid }}" class="{{ class }}"></div>
<script type="module" crossorigin="anonymous">
import { mountViewToElement } from "{% static 'django_idom/client.js' %}";
const mountPoint = document.getElementById("{{ idom_mount_uuid }}");
const mountElement = document.getElementById("{{ idom_mount_uuid }}");
mountViewToElement(
mountPoint,
mountElement,
"{{ idom_websocket_url }}",
"{{ idom_web_modules_url }}",
"{{ idom_ws_max_reconnect_timeout }}",
"{{ idom_component_id }}",
"{{ idom_component_params }}"
"{{ idom_component_path }}",
);
</script>
35 changes: 21 additions & 14 deletions src/django_idom/templatetags/idom.py
Original file line number Diff line number Diff line change
@@ -1,28 +1,29 @@
import json
from typing import Any
from urllib.parse import urlencode
from uuid import uuid4

import dill as pickle
from django import template
from django.urls import reverse

from django_idom.config import IDOM_WEBSOCKET_URL, IDOM_WS_MAX_RECONNECT_TIMEOUT
from django_idom.utils import _register_component
from django_idom import models
from django_idom.config import IDOM_MAX_RECONNECT_TIMEOUT, IDOM_WEBSOCKET_URL
from django_idom.types import ComponentParamData
from django_idom.utils import _register_component, func_has_params


IDOM_WEB_MODULES_URL = reverse("idom:web_modules", args=["x"])[:-1][1:]
register = template.Library()


@register.inclusion_tag("idom/component.html")
def component(dotted_path: str, **kwargs: Any):
def component(dotted_path: str, *args, **kwargs):
"""This tag is used to embed an existing IDOM component into your HTML template.

Args:
dotted_path: The dotted path to the component to render.
*args: The positional arguments to provide to the component.

Keyword Args:
**kwargs: The keyword arguments to pass to the component.
**kwargs: The keyword arguments to provide to the component.

Example ::

Expand All @@ -34,17 +35,23 @@ def component(dotted_path: str, **kwargs: Any):
</body>
</html>
"""
_register_component(dotted_path)

component = _register_component(dotted_path)
uuid = uuid4().hex
class_ = kwargs.pop("class", "")
json_kwargs = json.dumps(kwargs, separators=(",", ":"))

# Store the component's args/kwargs in the database if needed
# This will be fetched by the websocket consumer later
if func_has_params(component):
params = ComponentParamData(args, kwargs)
model = models.ComponentParams(uuid=uuid, data=pickle.dumps(params))
model.full_clean()
model.save()

return {
"class": class_,
"idom_websocket_url": IDOM_WEBSOCKET_URL,
"idom_web_modules_url": IDOM_WEB_MODULES_URL,
"idom_ws_max_reconnect_timeout": IDOM_WS_MAX_RECONNECT_TIMEOUT,
"idom_mount_uuid": uuid4().hex,
"idom_component_id": dotted_path,
"idom_component_params": urlencode({"kwargs": json_kwargs}),
"idom_ws_max_reconnect_timeout": IDOM_MAX_RECONNECT_TIMEOUT,
"idom_mount_uuid": uuid,
"idom_component_path": f"{dotted_path}/{uuid}",
}
11 changes: 10 additions & 1 deletion src/django_idom/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ class IdomWebsocket:
scope: dict
close: Callable[[Optional[int]], Awaitable[None]]
disconnect: Callable[[int], Awaitable[None]]
view_id: str
dotted_path: str


@dataclass
Expand Down Expand Up @@ -89,3 +89,12 @@ class QueryOptions:

postprocessor_kwargs: dict[str, Any] = field(default_factory=lambda: {})
"""Keyworded arguments directly passed into the `postprocessor` for configuration."""


@dataclass
class ComponentParamData:
"""Container used for serializing component parameters.
This dataclass is pickled & stored in the database, then unpickled when needed."""

args: tuple
kwargs: dict
11 changes: 9 additions & 2 deletions src/django_idom/utils.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from __future__ import annotations

import contextlib
import inspect
import logging
import os
import re
Expand Down Expand Up @@ -74,14 +75,15 @@ async def render_view(
return response


def _register_component(dotted_path: str) -> None:
def _register_component(dotted_path: str) -> Callable:
from django_idom.config import IDOM_REGISTERED_COMPONENTS

if dotted_path in IDOM_REGISTERED_COMPONENTS:
return
return IDOM_REGISTERED_COMPONENTS[dotted_path]

IDOM_REGISTERED_COMPONENTS[dotted_path] = _import_dotted_path(dotted_path)
_logger.debug("IDOM has registered component %s", dotted_path)
return IDOM_REGISTERED_COMPONENTS[dotted_path]


def _import_dotted_path(dotted_path: str) -> Callable:
Expand Down Expand Up @@ -257,3 +259,8 @@ def django_query_postprocessor(
)

return data


def func_has_params(func: Callable) -> bool:
"""Checks if a function has any args or kwarg parameters."""
return str(inspect.signature(func)) != "()"
36 changes: 25 additions & 11 deletions src/django_idom/websocket/consumer.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
"""Anything used to construct a websocket endpoint"""
import asyncio
import json
import logging
from typing import Any
from urllib.parse import parse_qsl

import dill as pickle
from channels.auth import login
from channels.db import database_sync_to_async as convert_to_async
from channels.generic.websocket import AsyncJsonWebsocketConsumer
Expand All @@ -13,7 +12,8 @@

from django_idom.config import IDOM_REGISTERED_COMPONENTS
from django_idom.hooks import WebsocketContext
from django_idom.types import IdomWebsocket
from django_idom.types import ComponentParamData, IdomWebsocket
from django_idom.utils import func_has_params


_logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -48,22 +48,36 @@ async def receive_json(self, content: Any, **kwargs: Any) -> None:
await self._idom_recv_queue.put(LayoutEvent(**content))

async def _run_dispatch_loop(self):
view_id = self.scope["url_route"]["kwargs"]["view_id"]
from django_idom import models

dotted_path = self.scope["url_route"]["kwargs"]["dotted_path"]
uuid = self.scope["url_route"]["kwargs"]["uuid"]

try:
component_constructor = IDOM_REGISTERED_COMPONENTS[view_id]
component_constructor = IDOM_REGISTERED_COMPONENTS[dotted_path]
except KeyError:
_logger.warning(f"Unknown IDOM view ID {view_id!r}")
_logger.warning(
f"Attempt to access invalid IDOM component: {dotted_path!r}"
)
return

query_dict = dict(parse_qsl(self.scope["query_string"].decode()))
component_kwargs = json.loads(query_dict.get("kwargs", "{}"))

# Provide developer access to parts of this websocket
socket = IdomWebsocket(self.scope, self.close, self.disconnect, view_id)
socket = IdomWebsocket(self.scope, self.close, self.disconnect, dotted_path)

try:
component_instance = component_constructor(**component_kwargs)
# Fetch the component's args/kwargs from the database, if needed
component_args: tuple[Any, ...] = tuple()
component_kwargs: dict = {}
if func_has_params(component_constructor):
params_query = await models.ComponentParams.objects.aget(uuid=uuid)
component_params: ComponentParamData = pickle.loads(params_query.data)
component_args = component_params.args
component_kwargs = component_params.kwargs

# Generate the initial component instance
component_instance = component_constructor(
*component_args, **component_kwargs
)
except Exception:
_logger.exception(
f"Failed to construct component {component_constructor} "
Expand Down
2 changes: 1 addition & 1 deletion src/django_idom/websocket/paths.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@


IDOM_WEBSOCKET_PATH = path(
f"{IDOM_WEBSOCKET_URL}<view_id>/", IdomAsyncWebsocketConsumer.as_asgi()
f"{IDOM_WEBSOCKET_URL}<dotted_path>/<uuid>/", IdomAsyncWebsocketConsumer.as_asgi()
)

"""A URL path for :class:`IdomAsyncWebsocketConsumer`.
Expand Down
Loading