Archive snapshots of websites locally (#672)

* Add basic HTML snapshots

* Implement asset list

* Add snapshot creation tests

* Add deletion tests

* Show file size

* Remove snapshots

* Create new snapshots

* Switch to single-file

* CSS tweak

* Remove auto refresh

* Show delete link when there is no file yet

* Add current date to display name

* Add flag for snapshot support

* Add option for disabling automatic snapshots

* Make snapshots sharable

* Document image variants

* Update README.md

* Add migrations

* Fix tests

README.md

@@ -33,16 +33,13 @@ The name comes from:
 **Feature Overview:**
 - Clean UI optimized for readability
 - Organize bookmarks with tags
-- Add notes using Markdown
+- Bulk editing, Markdown notes, read it later functionality
-- Read it later functionality
+- Share bookmarks with other users or guests
-- Share bookmarks with other users
-- Bulk editing
 - Automatically provides titles, descriptions and icons of bookmarked websites
-- Automatically creates snapshots of bookmarked websites on [the Internet Archive Wayback Machine](https://archive.org/web/)
+- Automatically creates snapshots of websites, either as local HTML file or on Internet Archive
 - Import and export bookmarks in Netscape HTML format
 - Installable as a Progressive Web App (PWA)
 - Extensions for [Firefox](https://addons.mozilla.org/firefox/addon/linkding-extension/) and [Chrome](https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe), as well as a bookmarklet
-- Light and dark themes
 - SSO support via OIDC or authentication proxies
 - REST API for developing 3rd party apps
 - Admin panel for user self-service and raw data access
@@ -62,27 +59,45 @@ The Docker image is compatible with ARM platforms, so it can be run on a Raspber
 linkding uses an SQLite database by default.
 Alternatively linkding supports PostgreSQL, see the [database options](docs/Options.md#LD_DB_ENGINE) for more information.
 
-<details>
-
-<summary>🧪 Alpine-based image</summary>
-
-The default Docker image (`latest` tag) is based on a slim variant of Debian Linux.
-Alternatively, there is an image based on Alpine Linux (`latest-alpine` tag) which has a smaller size, resulting in a smaller download and less disk space required.
-The Alpine image is currently about 45 MB in compressed size, compared to about 130 MB for the Debian image.
-
-To use it, replace the `latest` tag with `latest-alpine`, either in the CLI command below when using Docker, or in the `docker-compose.yml` file when using docker-compose.
-
-> [!WARNING]
-> The image is currently considered experimental in order to gather feedback and iron out any issues.
-> Only use it if you are comfortable running experimental software or want to help out with testing.
-> While there should be no issues with creating new installations, there might be issues when migrating existing installations.
-> If you plan to migrate your existing installation, make sure to create proper [backups](https://github.com/sissbruecker/linkding/blob/master/docs/backup.md) first.
-
-</details>
-
 ###  Using Docker
 
-To install linkding using Docker you can just run the [latest image](https://hub.docker.com/repository/docker/sissbruecker/linkding) from Docker Hub:
+The Docker image comes in several variants. To use a different image than the default, replace `latest` with the desired tag in the commands below, or in the docker-compose file.
+
+<table>
+  <thead>
+    <tr>
+      <th>Tag</th>
+      <th>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><code>latest</code></td>
+      <td>Provides the basic functionality of linkding</td>
+    </tr>
+    <tr>
+      <td><code>latest-plus</code></td>
+      <td>
+        Includes feature for saving HTML snapshots of websites
+        <ul>
+          <li>Significantly larger image size as it includes a Chromium installation</li>
+          <li>Requires more runtime memory to run Chromium</li>
+          <li>Requires more disk space for storing HTML snapshots</li>
+        </ul>            
+      </td>
+    </tr>
+    <tr>
+      <td><code>latest-alpine</code></td>
+      <td><code>latest</code>, but based on Alpine Linux. 🧪 Experimental</td>
+    </tr>    
+    <tr>
+      <td><code>latest-plus-alpine</code></td>
+      <td><code>latest-plus</code>, but based on Alpine Linux. 🧪 Experimental</td>
+    </tr>    
+  </tbody>
+</table>
+
+To install linkding using Docker you can just run the image from [Docker Hub](https://hub.docker.com/repository/docker/sissbruecker/linkding):
 ```shell
 docker run --name linkding -p 9090:9090 -v {host-data-folder}:/etc/linkding/data -d sissbruecker/linkding:latest
 ```

bookmarks/frontend/behaviors/bookmark-details.js

@@ -1,38 +0,0 @@
-import { registerBehavior } from "./index";
-
-class BookmarkDetails {
-  constructor(element) {
-    this.form = element.querySelector(".status form");
-    if (!this.form) {
-      // Form may not exist if user does not own the bookmark
-      return;
-    }
-    this.form.addEventListener("submit", (event) => {
-      event.preventDefault();
-      this.submitForm();
-    });
-
-    const inputs = this.form.querySelectorAll("input");
-    inputs.forEach((input) => {
-      input.addEventListener("change", () => {
-        this.submitForm();
-      });
-    });
-  }
-
-  async submitForm() {
-    const url = this.form.action;
-    const formData = new FormData(this.form);
-
-    await fetch(url, {
-      method: "POST",
-      body: formData,
-      redirect: "manual", // ignore redirect
-    });
-
-    // Refresh bookmark page if it exists
-    document.dispatchEvent(new CustomEvent("bookmark-page-refresh"));
-  }
-}
-
-registerBehavior("ld-bookmark-details", BookmarkDetails);

bookmarks/frontend/behaviors/bookmark-page.js

@@ -1,4 +1,4 @@
-import { registerBehavior, swap } from "./index";
+import { registerBehavior, swapContent } from "./index";
 
 class BookmarkPage {
   constructor(element) {
@@ -37,8 +37,8 @@ class BookmarkPage {
       fetch(`${bookmarksUrl}${query}`).then((response) => response.text()),
       fetch(`${tagsUrl}${query}`).then((response) => response.text()),
     ]).then(([bookmarkListHtml, tagCloudHtml]) => {
-      swap(this.bookmarkList, bookmarkListHtml);
+      swapContent(this.bookmarkList, bookmarkListHtml);
-      swap(this.tagCloud, tagCloudHtml);
+      swapContent(this.tagCloud, tagCloudHtml);
 
       // Dispatch list updated event
       const listElement = this.bookmarkList.querySelector(

bookmarks/frontend/behaviors/form.js

@@ -0,0 +1,54 @@
+import { registerBehavior, swap } from "./index";
+
+class FormBehavior {
+  constructor(element) {
+    this.element = element;
+    element.addEventListener("submit", this.onFormSubmit.bind(this));
+  }
+
+  async onFormSubmit(event) {
+    event.preventDefault();
+
+    const url = this.element.action;
+    const formData = new FormData(this.element);
+    if (event.submitter) {
+      formData.append(event.submitter.name, event.submitter.value);
+    }
+
+    await fetch(url, {
+      method: "POST",
+      body: formData,
+      redirect: "manual", // ignore redirect
+    });
+
+    // Dispatch refresh events
+    const refreshEvents = this.element.getAttribute("refresh-events");
+    if (refreshEvents) {
+      refreshEvents.split(",").forEach((eventName) => {
+        document.dispatchEvent(new CustomEvent(eventName));
+      });
+    }
+
+    // Refresh form
+    await this.refresh();
+  }
+
+  async refresh() {
+    const refreshUrl = this.element.getAttribute("refresh-url");
+    const html = await fetch(refreshUrl).then((response) => response.text());
+    swap(this.element, html);
+  }
+}
+
+class FormAutoSubmitBehavior {
+  constructor(element) {
+    this.element = element;
+    this.element.addEventListener("change", () => {
+      const form = this.element.closest("form");
+      form.dispatchEvent(new Event("submit", { cancelable: true }));
+    });
+  }
+}
+
+registerBehavior("ld-form", FormBehavior);
+registerBehavior("ld-form-auto-submit", FormAutoSubmitBehavior);

bookmarks/frontend/behaviors/index.js

@@ -12,7 +12,14 @@ export function applyBehaviors(container, behaviorNames = null) {
 
   behaviorNames.forEach((behaviorName) => {
     const behavior = behaviorRegistry[behaviorName];
-    const elements = container.querySelectorAll(`[${behaviorName}]`);
+    const elements = Array.from(
+      container.querySelectorAll(`[${behaviorName}]`),
+    );
+
+    // Include the container element if it has the behavior
+    if (container.hasAttribute && container.hasAttribute(behaviorName)) {
+      elements.push(container);
+    }
 
     elements.forEach((element) => {
       element.__behaviors = element.__behaviors || [];
@@ -31,6 +38,13 @@ export function applyBehaviors(container, behaviorNames = null) {
 }
 
 export function swap(element, html) {
+  const dom = new DOMParser().parseFromString(html, "text/html");
+  const newElement = dom.body.firstChild;
+  element.replaceWith(newElement);
+  applyBehaviors(newElement);
+}
+
+export function swapContent(element, html) {
   element.innerHTML = html;
   applyBehaviors(element);
 }

bookmarks/frontend/index.js

@@ -1,8 +1,8 @@
-import "./behaviors/bookmark-details";
 import "./behaviors/bookmark-page";
 import "./behaviors/bulk-edit";
 import "./behaviors/confirm-button";
 import "./behaviors/dropdown";
+import "./behaviors/form";
 import "./behaviors/modal";
 import "./behaviors/global-shortcuts";
 import "./behaviors/tag-autocomplete";

bookmarks/migrations/0030_bookmarkasset.py

@@ -0,0 +1,43 @@
+# Generated by Django 5.0.2 on 2024-03-31 08:21
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0029_bookmark_list_actions_toast"),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name="BookmarkAsset",
+            fields=[
+                (
+                    "id",
+                    models.AutoField(
+                        auto_created=True,
+                        primary_key=True,
+                        serialize=False,
+                        verbose_name="ID",
+                    ),
+                ),
+                ("date_created", models.DateTimeField(auto_now_add=True)),
+                ("file", models.CharField(blank=True, max_length=2048)),
+                ("file_size", models.IntegerField(null=True)),
+                ("asset_type", models.CharField(max_length=64)),
+                ("content_type", models.CharField(max_length=128)),
+                ("display_name", models.CharField(blank=True, max_length=2048)),
+                ("status", models.CharField(max_length=64)),
+                ("gzip", models.BooleanField(default=False)),
+                (
+                    "bookmark",
+                    models.ForeignKey(
+                        on_delete=django.db.models.deletion.CASCADE,
+                        to="bookmarks.bookmark",
+                    ),
+                ),
+            ],
+        ),
+    ]

bookmarks/migrations/0031_userprofile_enable_automatic_html_snapshots.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.0.2 on 2024-04-01 10:29
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0030_bookmarkasset"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="enable_automatic_html_snapshots",
+            field=models.BooleanField(default=True),
+        ),
+    ]

bookmarks/migrations/0032_html_snapshots_hint_toast.py

@@ -0,0 +1,34 @@
+# Generated by Django 5.0.2 on 2024-04-01 12:17
+
+from django.db import migrations
+from django.contrib.auth import get_user_model
+
+from bookmarks.models import Toast
+
+User = get_user_model()
+
+
+def forwards(apps, schema_editor):
+
+    for user in User.objects.all():
+        toast = Toast(
+            key="html_snapshots_hint",
+            message="This version adds a new feature for archiving snapshots of websites locally. To use it, you need to switch to a different Docker image. See the installation instructions on GitHub for details.",
+            owner=user,
+        )
+        toast.save()
+
+
+def reverse(apps, schema_editor):
+    Toast.objects.filter(key="bookmark_list_actions_hint").delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0031_userprofile_enable_automatic_html_snapshots"),
+    ]
+
+    operations = [
+        migrations.RunPython(forwards, reverse),
+    ]

bookmarks/models.py

@@ -1,18 +1,22 @@
-import binascii
+import logging
 import os
 from typing import List
 
+import binascii
 from django import forms
+from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.models import User
 from django.db import models
-from django.db.models.signals import post_save
+from django.db.models.signals import post_save, post_delete
 from django.dispatch import receiver
 from django.http import QueryDict
 
 from bookmarks.utils import unique
 from bookmarks.validators import BookmarkURLValidator
 
+logger = logging.getLogger(__name__)
+
 
 class Tag(models.Model):
     name = models.CharField(max_length=64)
@@ -85,6 +89,47 @@ class Bookmark(models.Model):
         return self.resolved_title + " (" + self.url[:30] + "...)"
 
 
+class BookmarkAsset(models.Model):
+    TYPE_SNAPSHOT = "snapshot"
+
+    CONTENT_TYPE_HTML = "text/html"
+
+    STATUS_PENDING = "pending"
+    STATUS_COMPLETE = "complete"
+    STATUS_FAILURE = "failure"
+
+    bookmark = models.ForeignKey(Bookmark, on_delete=models.CASCADE)
+    date_created = models.DateTimeField(auto_now_add=True, null=False)
+    file = models.CharField(max_length=2048, blank=True, null=False)
+    file_size = models.IntegerField(null=True)
+    asset_type = models.CharField(max_length=64, blank=False, null=False)
+    content_type = models.CharField(max_length=128, blank=False, null=False)
+    display_name = models.CharField(max_length=2048, blank=True, null=False)
+    status = models.CharField(max_length=64, blank=False, null=False)
+    gzip = models.BooleanField(default=False, null=False)
+
+    def save(self, *args, **kwargs):
+        if self.file:
+            try:
+                file_path = os.path.join(settings.LD_ASSET_FOLDER, self.file)
+                if os.path.isfile(file_path):
+                    self.file_size = os.path.getsize(file_path)
+            except Exception:
+                pass
+        super().save(*args, **kwargs)
+
+
+@receiver(post_delete, sender=BookmarkAsset)
+def bookmark_asset_deleted(sender, instance, **kwargs):
+    if instance.file:
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, instance.file)
+        if os.path.isfile(filepath):
+            try:
+                os.remove(filepath)
+            except Exception as error:
+                logger.error(f"Failed to delete asset file: {filepath}", exc_info=error)
+
+
 class BookmarkForm(forms.ModelForm):
     # Use URLField for URL
     url = forms.CharField(validators=[BookmarkURLValidator()])
@@ -353,6 +398,7 @@ class UserProfile(models.Model):
     permanent_notes = models.BooleanField(default=False, null=False)
     custom_css = models.TextField(blank=True, null=False)
     search_preferences = models.JSONField(default=dict, null=False)
+    enable_automatic_html_snapshots = models.BooleanField(default=True, null=False)
 
 
 class UserProfileForm(forms.ModelForm):
@@ -369,6 +415,7 @@ class UserProfileForm(forms.ModelForm):
             "enable_sharing",
             "enable_public_sharing",
             "enable_favicons",
+            "enable_automatic_html_snapshots",
             "display_url",
             "display_view_bookmark_action",
             "display_edit_bookmark_action",

bookmarks/services/bookmarks.py

@@ -34,6 +34,9 @@ def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
     tasks.create_web_archive_snapshot(current_user, bookmark, False)
     # Load favicon
     tasks.load_favicon(current_user, bookmark)
+    # Create HTML snapshot
+    if current_user.profile.enable_automatic_html_snapshots:
+        tasks.create_html_snapshot(bookmark)
 
     return bookmark
 

bookmarks/services/monolith.py

@@ -0,0 +1,32 @@
+import gzip
+import shutil
+import subprocess
+import os
+
+from django.conf import settings
+
+
+class MonolithError(Exception):
+    pass
+
+
+# Monolith isn't used at the moment, as the local snapshot implementation
+# switched to single-file after the prototype. Keeping this around in case
+# it turns out to be useful in the future.
+def create_snapshot(url: str, filepath: str):
+    monolith_path = settings.LD_MONOLITH_PATH
+    monolith_options = settings.LD_MONOLITH_OPTIONS
+    temp_filepath = filepath + ".tmp"
+
+    try:
+        command = f"{monolith_path} '{url}' {monolith_options} -o {temp_filepath}"
+        subprocess.run(command, check=True, shell=True)
+
+        with open(temp_filepath, "rb") as raw_file, gzip.open(
+            filepath, "wb"
+        ) as gz_file:
+            shutil.copyfileobj(raw_file, gz_file)
+
+        os.remove(temp_filepath)
+    except subprocess.CalledProcessError as error:
+        raise MonolithError(f"Failed to create snapshot: {error.stderr}")

bookmarks/services/singlefile.py

@@ -0,0 +1,33 @@
+import gzip
+import os
+import shutil
+import subprocess
+
+from django.conf import settings
+
+
+class SingeFileError(Exception):
+    pass
+
+
+def create_snapshot(url: str, filepath: str):
+    singlefile_path = settings.LD_SINGLEFILE_PATH
+    singlefile_options = settings.LD_SINGLEFILE_OPTIONS
+    temp_filepath = filepath + ".tmp"
+
+    try:
+        command = f"{singlefile_path} '{url}' {singlefile_options} {temp_filepath}"
+        subprocess.run(command, check=True, shell=True)
+
+        # single-file doesn't return exit codes apparently, so check if the file was created
+        if not os.path.exists(temp_filepath):
+            raise SingeFileError("Failed to create snapshot")
+
+        with open(temp_filepath, "rb") as raw_file, gzip.open(
+            filepath, "wb"
+        ) as gz_file:
+            shutil.copyfileobj(raw_file, gz_file)
+
+        os.remove(temp_filepath)
+    except subprocess.CalledProcessError as error:
+        raise SingeFileError(f"Failed to create snapshot: {error.stderr}")

bookmarks/services/tasks.py

@@ -1,4 +1,5 @@
 import logging
+import os
 
 import waybackpy
 from background_task import background
@@ -7,10 +8,11 @@ from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.models import User
 from waybackpy.exceptions import WaybackError, TooManyRequestsError, NoCDXRecordFound
+from django.utils import timezone, formats
 
 import bookmarks.services.wayback
-from bookmarks.models import Bookmark, UserProfile
+from bookmarks.models import Bookmark, BookmarkAsset, UserProfile
-from bookmarks.services import favicon_loader
+from bookmarks.services import favicon_loader, singlefile
 from bookmarks.services.website_loader import DEFAULT_USER_AGENT
 
 logger = logging.getLogger(__name__)
@@ -193,3 +195,64 @@ def _schedule_refresh_favicons_task(user_id: int):
         tasks.append(task)
 
     Task.objects.bulk_create(tasks)
+
+
+def is_html_snapshot_feature_active() -> bool:
+    return settings.LD_ENABLE_SNAPSHOTS and not settings.LD_DISABLE_BACKGROUND_TASKS
+
+
+def create_html_snapshot(bookmark: Bookmark):
+    if not is_html_snapshot_feature_active():
+        return
+
+    timestamp = formats.date_format(timezone.now(), "SHORT_DATE_FORMAT")
+    asset = BookmarkAsset(
+        bookmark=bookmark,
+        asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+        content_type="text/html",
+        display_name=f"HTML snapshot from {timestamp}",
+        status=BookmarkAsset.STATUS_PENDING,
+    )
+    asset.save()
+    _create_html_snapshot_task(asset.id)
+
+
+def _generate_snapshot_filename(asset: BookmarkAsset) -> str:
+    def sanitize_char(char):
+        if char.isalnum() or char in ("-", "_", "."):
+            return char
+        else:
+            return "_"
+
+    formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
+    sanitized_url = "".join(sanitize_char(char) for char in asset.bookmark.url)
+
+    return f"{asset.asset_type}_{formatted_datetime}_{sanitized_url}.html.gz"
+
+
+@background()
+def _create_html_snapshot_task(asset_id: int):
+    try:
+        asset = BookmarkAsset.objects.get(id=asset_id)
+    except BookmarkAsset.DoesNotExist:
+        return
+
+    logger.info(f"Create HTML snapshot for bookmark. url={asset.bookmark.url}")
+
+    try:
+        filename = _generate_snapshot_filename(asset)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        singlefile.create_snapshot(asset.bookmark.url, filepath)
+        asset.status = BookmarkAsset.STATUS_COMPLETE
+        asset.file = filename
+        asset.gzip = True
+        logger.info(
+            f"Successfully created HTML snapshot for bookmark. url={asset.bookmark.url}"
+        )
+    except singlefile.SingeFileError as error:
+        asset.status = BookmarkAsset.STATUS_FAILURE
+        logger.error(
+            f"Failed to create HTML snapshot for bookmark. url={asset.bookmark.url}",
+            exc_info=error,
+        )
+    asset.save()

bookmarks/styles/bookmark-details.scss

@@ -37,6 +37,53 @@
     margin-bottom: 0;
   }
 
+  .assets {
+    margin-top: $unit-2;
+  }
+
+  .assets .asset {
+    display: flex;
+    align-items: center;
+    gap: $unit-3;
+    padding: $unit-2 0;
+    border-top: $unit-o solid $border-color-light;
+  }
+
+  .assets .asset:last-child {
+    border-bottom: $unit-o solid $border-color-light;
+  }
+
+  .assets .asset-icon {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+  }
+
+  .assets .asset-text {
+    flex: 1 1 0;
+  }
+
+  .assets .asset-text .filesize {
+    color: $gray-color;
+    margin-left: $unit-2;
+  }
+
+  .assets .asset-actions, .assets-actions {
+    display: flex;
+    gap: $unit-3;
+    align-items: center;
+  }
+
+  .assets .asset-actions .btn, .assets-actions .btn {
+    height: unset;
+    padding: 0;
+    border: none;
+  }
+
+  .assets-actions {
+    margin-top: $unit-2;
+  }
+
   .tags a {
     color: $alternative-color;
   }
@@ -46,7 +93,7 @@
     gap: $unit-2;
   }
 
-  .status form .form-group, .status form .form-switch {
+  .status .form-group, .status .form-switch {
     margin: 0;
   }
 

bookmarks/templates/bookmarks/details.html

@@ -1,13 +1,13 @@
 {% extends 'bookmarks/layout.html' %}
 
 {% block content %}
-  <div ld-bookmark-details class="bookmark-details page">
+  <div class="bookmark-details page">
-    {% if request.user == bookmark.owner %}
+    {% if details.is_editable %}
       {% include 'bookmarks/details/actions.html' %}
     {% endif %}
     {% include 'bookmarks/details/title.html' %}
     <div>
-      {% include 'bookmarks/details/content.html' %}
+      {% include 'bookmarks/details/form.html' %}
     </div>
   </div>
 {% endblock %}

bookmarks/templates/bookmarks/details/actions.html

@@ -1,11 +1,14 @@
 <div class="actions">
   <div class="left-actions">
-    <a class="btn" href="{% url 'bookmarks:edit' bookmark.id %}?return_url={{ edit_return_url|urlencode }}">Edit</a>
+    <a class="btn"
+       href="{% url 'bookmarks:edit' details.bookmark.id %}?return_url={{ details.edit_return_url|urlencode }}">Edit</a>
   </div>
   <div class="right-actions">
-    <form action="{% url 'bookmarks:index.action' %}?return_url={{ delete_return_url|urlencode }}" method="post">
+    <form action="{% url 'bookmarks:index.action' %}?return_url={{ details.delete_return_url|urlencode }}"
+          method="post">
       {% csrf_token %}
-      <button ld-confirm-button type="submit" name="remove" value="{{ bookmark.id }}" class="btn btn-link text-error">
+      <button ld-confirm-button type="submit" name="remove" value="{{ details.bookmark.id }}"
+              class="btn btn-link text-error">
         Delete...
       </button>
     </form>

bookmarks/templates/bookmarks/details/asset_icon.html

@@ -0,0 +1,42 @@
+{% if asset.content_type == 'text/html' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/>
+    <path d="M2 21v-6"/>
+    <path d="M5 15v6"/>
+    <path d="M2 18h3"/>
+    <path d="M20 15v6h2"/>
+    <path d="M13 21v-6l2 3l2 -3v6"/>
+    <path d="M7.5 15h3"/>
+    <path d="M9 15v6"/>
+  </svg>
+{% elif asset.content_type == 'application/pdf' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/>
+    <path d="M5 18h1.5a1.5 1.5 0 0 0 0 -3h-1.5v6"/>
+    <path d="M17 18h2"/>
+    <path d="M20 15h-3v6"/>
+    <path d="M11 15v6h1a2 2 0 0 0 2 -2v-2a2 2 0 0 0 -2 -2h-1z"/>
+  </svg>
+{% elif asset.content_type == 'image/png' or asset.content_type == 'image/jpeg' or asset.content_type == 'image.gif' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M15 8h.01"/>
+    <path d="M3 6a3 3 0 0 1 3 -3h12a3 3 0 0 1 3 3v12a3 3 0 0 1 -3 3h-12a3 3 0 0 1 -3 -3v-12z"/>
+    <path d="M3 16l5 -5c.928 -.893 2.072 -.893 3 0l5 5"/>
+    <path d="M14 14l1 -1c.928 -.893 2.072 -.893 3 0l3 3"/>
+  </svg>
+{% else %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor"
+       stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M17 21h-10a2 2 0 0 1 -2 -2v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2z"/>
+  </svg>
+{% endif %}

bookmarks/templates/bookmarks/details/assets.html

@@ -0,0 +1,37 @@
+{% if details.assets %}
+  <div class="assets">
+    {% for asset in details.assets %}
+      <div class="asset" data-asset-id="{{ asset.id }}">
+        <div class="asset-icon {{ asset.icon_classes }}">
+          {% include 'bookmarks/details/asset_icon.html' %}
+        </div>
+        <div class="asset-text truncate {{ asset.text_classes }}">
+        <span>
+          {{ asset.display_name }}
+          {% if asset.status == 'pending' %}(queued){% endif %}
+          {% if asset.status == 'failure' %}(failed){% endif %}
+        </span>
+          {% if asset.file_size %}
+            <span class="filesize">{{ asset.file_size|filesizeformat }}</span>
+          {% endif %}
+        </div>
+        <div class="asset-actions">
+          {% if asset.file %}
+            <a class="btn btn-link" href="{% url 'bookmarks:assets.view' asset.id %}" target="_blank">View</a>
+          {% endif %}
+          {% if details.is_editable %}
+            <button ld-confirm-button type="submit" name="remove_asset" value="{{ asset.id }}" class="btn btn-link">
+              Remove
+            </button>
+          {% endif %}
+        </div>
+      </div>
+    {% endfor %}
+  </div>
+{% endif %}
+
+{% if details.is_editable %}
+  <div class="assets-actions">
+    <button type="submit" name="create_snapshot" class="btn btn-link">Create HTML snapshot</button>
+  </div>
+{% endif %}

bookmarks/templates/bookmarks/details/content.html

@@ -1,85 +0,0 @@
-{% load static %}
-{% load shared %}
-
-<div class="weblinks">
-  <a class="weblink" href="{{ bookmark.url }}" rel="noopener"
-     target="{{ request.user_profile.bookmark_link_target }}">
-    {% if bookmark.favicon_file and request.user_profile.enable_favicons %}
-      <img class="favicon" src="{% static bookmark.favicon_file %}" alt="">
-    {% endif %}
-    <span>{{ bookmark.url }}</span>
-  </a>
-  {% if bookmark.web_archive_snapshot_url %}
-    <a class="weblink" href="{{ bookmark.web_archive_snapshot_url }}"
-       target="{{ request.user_profile.bookmark_link_target }}">
-      {% if bookmark.favicon_file and request.user_profile.enable_favicons %}
-        <svg class="favicon" viewBox="0 0 76 86" xmlns="http://www.w3.org/2000/svg">
-          <path
-              d="m76 82v4h-76l.00080851-4zm-3-6v5h-70v-5zm-62.6696277-54 .8344146.4217275.4176066 6.7436084.4176065 10.9576581v10.5383496l-.4176065 13.1364492-.0694681 8.8498268-1.1825531.3523804h-4.17367003l-1.25202116-.3523804-.48627608-8.8498268-.41840503-13.0662957v-10.5375432l.41840503-11.028618.38167482-6.7798947.87034634-.3854412zm60.0004653 0 .8353798.4217275.4168913 6.7436084.4168913 10.9576581v10.5383496l-.4168913 13.1364492-.0686832 8.8498268-1.1835879.3523804h-4.1737047l-1.2522712-.3523804-.4879704-8.8498268-.4168913-13.0662957v-10.5375432l.4168913-11.028618.3833483-6.7798947.8697215-.3854412zm-42.000632 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2529447-.3523804-.4863246-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8688361-.3854412zm23 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2521462-.3523804-.4871231-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8696347-.3854412zm21.6697944-9v7h-70v-7zm-35.7200748-13 36.7200748 8.4088317-1.4720205 2.5911683h-70.32799254l-2.19998696-2.10140371z"
-              fill="currentColor" fill-rule="evenodd"/>
-        </svg>
-      {% endif %}
-      <span>View on Internet Archive</span>
-    </a>
-  {% endif %}
-</div>
-<dl class="grid columns-2 columns-sm-1 gap-0">
-  {% if request.user == bookmark.owner %}
-    <div class="status col-2">
-      <dt>Status</dt>
-      <dd class="d-flex" style="gap: .8rem">
-        <form action="{% url 'bookmarks:details' bookmark.id %}" method="post">
-          {% csrf_token %}
-          <div class="form-group">
-            <label class="form-switch">
-              <input type="checkbox" name="is_archived" {% if bookmark.is_archived %}checked{% endif %}>
-              <i class="form-icon"></i> Archived
-            </label>
-          </div>
-          <div class="form-group">
-            <label class="form-switch">
-              <input type="checkbox" name="unread" {% if bookmark.unread %}checked{% endif %}>
-              <i class="form-icon"></i> Unread
-            </label>
-          </div>
-          {% if request.user_profile.enable_sharing %}
-            <div class="form-group">
-              <label class="form-switch">
-                <input type="checkbox" name="shared" {% if bookmark.shared %}checked{% endif %}>
-                <i class="form-icon"></i> Shared
-              </label>
-            </div>
-          {% endif %}
-        </form>
-      </dd>
-    </div>
-  {% endif %}
-  {% if bookmark.tag_names %}
-    <div class="tags col-1">
-      <dt>Tags</dt>
-      <dd>
-        {% for tag_name in bookmark.tag_names %}
-          <a href="{% url 'bookmarks:index' %}?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
-        {% endfor %}
-      </dd>
-    </div>
-  {% endif %}
-  <div class="date-added col-1">
-    <dt>Date added</dt>
-    <dd>
-      <span>{{ bookmark.date_added }}</span>
-    </dd>
-  </div>
-  {% if bookmark.resolved_description %}
-    <div class="description col-2">
-      <dt>Description</dt>
-      <dd>{{ bookmark.resolved_description }}</dd>
-    </div>
-  {% endif %}
-  {% if bookmark.notes %}
-    <div class="notes col-2">
-      <dt>Notes</dt>
-      <dd class="markdown">{% markdown bookmark.notes %}</dd>
-    </div>
-  {% endif %}
-</dl>

bookmarks/templates/bookmarks/details/form.html

@@ -0,0 +1,99 @@
+{% load static %}
+{% load shared %}
+
+<form ld-form action="{% url 'bookmarks:details' details.bookmark.id %}"
+      refresh-url="{% url 'bookmarks:partials.details_form' details.bookmark.id %}"
+      refresh-events="bookmark-page-refresh"
+      method="post">
+  <div class="weblinks">
+    <a class="weblink" href="{{ details.bookmark.url }}" rel="noopener"
+       target="{{ details.profile.bookmark_link_target }}">
+      {% if details.show_link_icons %}
+        <img class="favicon" src="{% static details.bookmark.favicon_file %}" alt="">
+      {% endif %}
+      <span>{{ details.bookmark.url }}</span>
+    </a>
+    {% if details.bookmark.web_archive_snapshot_url %}
+      <a class="weblink" href="{{ details.bookmark.web_archive_snapshot_url }}"
+         target="{{ details.profile.bookmark_link_target }}">
+        {% if details.show_link_icons %}
+          <svg class="favicon" viewBox="0 0 76 86" xmlns="http://www.w3.org/2000/svg">
+            <path
+                d="m76 82v4h-76l.00080851-4zm-3-6v5h-70v-5zm-62.6696277-54 .8344146.4217275.4176066 6.7436084.4176065 10.9576581v10.5383496l-.4176065 13.1364492-.0694681 8.8498268-1.1825531.3523804h-4.17367003l-1.25202116-.3523804-.48627608-8.8498268-.41840503-13.0662957v-10.5375432l.41840503-11.028618.38167482-6.7798947.87034634-.3854412zm60.0004653 0 .8353798.4217275.4168913 6.7436084.4168913 10.9576581v10.5383496l-.4168913 13.1364492-.0686832 8.8498268-1.1835879.3523804h-4.1737047l-1.2522712-.3523804-.4879704-8.8498268-.4168913-13.0662957v-10.5375432l.4168913-11.028618.3833483-6.7798947.8697215-.3854412zm-42.000632 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2529447-.3523804-.4863246-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8688361-.3854412zm23 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2521462-.3523804-.4871231-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8696347-.3854412zm21.6697944-9v7h-70v-7zm-35.7200748-13 36.7200748 8.4088317-1.4720205 2.5911683h-70.32799254l-2.19998696-2.10140371z"
+                fill="currentColor" fill-rule="evenodd"/>
+          </svg>
+        {% endif %}
+        <span>View on Internet Archive</span>
+      </a>
+    {% endif %}
+  </div>
+  <dl class="grid columns-2 columns-sm-1 gap-0">
+    {% if details.is_editable %}
+      <div class="status col-2">
+        <dt>Status</dt>
+        <dd class="d-flex" style="gap: .8rem">
+          {% csrf_token %}
+          <div class="form-group">
+            <label class="form-switch">
+              <input ld-form-auto-submit type="checkbox" name="is_archived"
+                     {% if details.bookmark.is_archived %}checked{% endif %}>
+              <i class="form-icon"></i> Archived
+            </label>
+          </div>
+          <div class="form-group">
+            <label class="form-switch">
+              <input ld-form-auto-submit type="checkbox" name="unread"
+                     {% if details.bookmark.unread %}checked{% endif %}>
+              <i class="form-icon"></i> Unread
+            </label>
+          </div>
+          {% if details.profile.enable_sharing %}
+            <div class="form-group">
+              <label class="form-switch">
+                <input ld-form-auto-submit type="checkbox" name="shared"
+                       {% if details.bookmark.shared %}checked{% endif %}>
+                <i class="form-icon"></i> Shared
+              </label>
+            </div>
+          {% endif %}
+        </dd>
+      </div>
+    {% endif %}
+    {% if details.show_files %}
+      <div class="files col-2">
+        <dt>Files</dt>
+        <dd>
+          {% include 'bookmarks/details/assets.html' %}
+        </dd>
+      </div>
+    {% endif %}
+    {% if details.bookmark.tag_names %}
+      <div class="tags col-1">
+        <dt>Tags</dt>
+        <dd>
+          {% for tag_name in details.bookmark.tag_names %}
+            <a href="{% url 'bookmarks:index' %}?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
+          {% endfor %}
+        </dd>
+      </div>
+    {% endif %}
+    <div class="date-added col-1">
+      <dt>Date added</dt>
+      <dd>
+        <span>{{ details.bookmark.date_added }}</span>
+      </dd>
+    </div>
+    {% if details.bookmark.resolved_description %}
+      <div class="description col-2">
+        <dt>Description</dt>
+        <dd>{{ details.bookmark.resolved_description }}</dd>
+      </div>
+    {% endif %}
+    {% if details.bookmark.notes %}
+      <div class="notes col-2">
+        <dt>Notes</dt>
+        <dd class="markdown">{% markdown details.bookmark.notes %}</dd>
+      </div>
+    {% endif %}
+  </dl>
+</form>

bookmarks/templates/bookmarks/details/title.html

@@ -1,3 +1,3 @@
 <h2>
-  {{ bookmark.resolved_title }}
+  {{ details.bookmark.resolved_title }}
 </h2>

bookmarks/templates/bookmarks/details_modal.html

@@ -1,4 +1,4 @@
-<div ld-bookmark-details class="modal active bookmark-details">
+<div class="modal active bookmark-details">
   <div class="modal-overlay" aria-label="Close"></div>
   <div class="modal-container">
     <div class="modal-header">
@@ -14,11 +14,11 @@
     </div>
     <div class="modal-body">
       <div class="content">
-        {% include 'bookmarks/details/content.html' %}
+        {% include 'bookmarks/details/form.html' %}
       </div>
     </div>
 
-    {% if request.user == bookmark.owner %}
+    {% if details.is_editable %}
       <div class="modal-footer">
         {% include 'bookmarks/details/actions.html' %}
       </div>

bookmarks/templates/settings/general.html

@@ -163,6 +163,18 @@
               href="{% url 'bookmarks:shared' %}">shared bookmarks page</a>.
           </div>
         </div>
+        {% if has_snapshot_support %}
+          <div class="form-group">
+            <label for="{{ form.enable_automatic_html_snapshots.id_for_label }}" class="form-checkbox">
+              {{ form.enable_automatic_html_snapshots }}
+              <i class="form-icon"></i> Automatically create HTML snapshots
+            </label>
+            <div class="form-input-hint">
+              Automatically creates HTML snapshots when adding bookmarks. Alternatively, when disabled, snapshots can be
+              created manually in the details view of a bookmark.
+            </div>
+          </div>
+        {% endif %}
         <div class="form-group">
           <details {% if form.custom_css.value %}open{% endif %}>
             <summary>Custom CSS</summary>

bookmarks/tests/helpers.py

@@ -1,6 +1,6 @@
 import random
 import logging
-import datetime
+from datetime import datetime
 from typing import List
 
 from bs4 import BeautifulSoup
@@ -10,7 +10,7 @@ from django.utils.crypto import get_random_string
 from rest_framework import status
 from rest_framework.test import APITestCase
 
-from bookmarks.models import Bookmark, Tag
+from bookmarks.models import Bookmark, BookmarkAsset, Tag
 
 
 class BookmarkFactoryMixin:
@@ -133,6 +133,38 @@ class BookmarkFactoryMixin:
     def get_numbered_bookmark(self, title: str):
         return Bookmark.objects.get(title=title)
 
+    def setup_asset(
+        self,
+        bookmark: Bookmark,
+        date_created: datetime = None,
+        file: str = None,
+        file_size: int = None,
+        asset_type: str = BookmarkAsset.TYPE_SNAPSHOT,
+        content_type: str = "image/html",
+        display_name: str = None,
+        status: str = BookmarkAsset.STATUS_COMPLETE,
+        gzip: bool = False,
+    ):
+        if date_created is None:
+            date_created = timezone.now()
+        if not file:
+            file = get_random_string(length=32)
+        if not display_name:
+            display_name = file
+        asset = BookmarkAsset(
+            bookmark=bookmark,
+            date_created=date_created,
+            file=file,
+            file_size=file_size,
+            asset_type=asset_type,
+            content_type=content_type,
+            display_name=display_name,
+            status=status,
+            gzip=gzip,
+        )
+        asset.save()
+        return asset
+
     def setup_tag(self, user: User = None, name: str = ""):
         if user is None:
             user = self.get_or_create_test_user()

bookmarks/tests/test_bookmark_asset_view.py

@@ -0,0 +1,125 @@
+import os
+
+from django.conf import settings
+from django.test import TestCase
+from django.urls import reverse
+
+from bookmarks.tests.helpers import (
+    BookmarkFactoryMixin,
+)
+
+
+class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
+    def setUp(self) -> None:
+        user = self.get_or_create_test_user()
+        self.client.force_login(user)
+
+    def tearDown(self):
+        temp_files = [
+            f for f in os.listdir(settings.LD_ASSET_FOLDER) if f.startswith("temp")
+        ]
+        for temp_file in temp_files:
+            os.remove(os.path.join(settings.LD_ASSET_FOLDER, temp_file))
+
+    def setup_asset_file(self, filename):
+        if not os.path.exists(settings.LD_ASSET_FOLDER):
+            os.makedirs(settings.LD_ASSET_FOLDER)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "w") as f:
+            f.write("test")
+
+    def setup_asset_with_file(self, bookmark):
+        filename = f"temp_{bookmark.id}.html.gzip"
+        self.setup_asset_file(filename)
+        asset = self.setup_asset(bookmark=bookmark, file=filename)
+        return asset
+
+    def test_view_access(self):
+        # own bookmark
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 200)
+
+        # other user's bookmark
+        other_user = self.setup_user()
+        bookmark = self.setup_bookmark(user=other_user)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing disabled
+        bookmark = self.setup_bookmark(user=other_user, shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, sharing enabled
+        profile = other_user.profile
+        profile.enable_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(user=other_user, shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing enabled
+        bookmark = self.setup_bookmark(user=other_user, shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 200)
+
+    def test_view_access_guest_user(self):
+        self.client.logout()
+
+        # unshared, sharing disabled
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing disabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, sharing enabled
+        profile = self.get_or_create_test_user().profile
+        profile.enable_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing enabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, public sharing enabled
+        profile.enable_public_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, public sharing enabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 200)

bookmarks/tests/test_bookmark_assets.py

@@ -0,0 +1,89 @@
+import os
+
+from django.conf import settings
+from django.test import TestCase
+
+from bookmarks.tests.helpers import (
+    BookmarkFactoryMixin,
+)
+from bookmarks.services import bookmarks
+
+
+class BookmarkAssetsTestCase(TestCase, BookmarkFactoryMixin):
+    def tearDown(self):
+        temp_files = [
+            f for f in os.listdir(settings.LD_ASSET_FOLDER) if f.startswith("temp")
+        ]
+        for temp_file in temp_files:
+            os.remove(os.path.join(settings.LD_ASSET_FOLDER, temp_file))
+
+    def setup_asset_file(self, filename):
+        if not os.path.exists(settings.LD_ASSET_FOLDER):
+            os.makedirs(settings.LD_ASSET_FOLDER)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "w") as f:
+            f.write("test")
+
+    def setup_asset_with_file(self, bookmark):
+        filename = f"temp_{bookmark.id}.html.gzip"
+        self.setup_asset_file(filename)
+        asset = self.setup_asset(bookmark=bookmark, file=filename)
+        return asset
+
+    def test_delete_bookmark_deletes_asset_file(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset.file))
+        )
+
+        bookmark.delete()
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset.file))
+        )
+
+    def test_bulk_delete_bookmarks_deletes_asset_files(self):
+        bookmark1 = self.setup_bookmark()
+        asset1 = self.setup_asset_with_file(bookmark1)
+        bookmark2 = self.setup_bookmark()
+        asset2 = self.setup_asset_with_file(bookmark2)
+        bookmark3 = self.setup_bookmark()
+        asset3 = self.setup_asset_with_file(bookmark3)
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset1.file))
+        )
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset2.file))
+        )
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset3.file))
+        )
+
+        bookmarks.delete_bookmarks(
+            [bookmark1.id, bookmark2.id, bookmark3.id], self.get_or_create_test_user()
+        )
+
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset1.file))
+        )
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset2.file))
+        )
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset3.file))
+        )
+
+    def test_save_updates_file_size(self):
+        # File does not exist initially
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark=bookmark, file="temp.html.gz")
+        self.assertIsNone(asset.file_size)
+
+        # Add file, save again
+        self.setup_asset_file(asset.file)
+        asset.save()
+        self.assertEqual(asset.file_size, 4)
+
+        # Create asset with initial file
+        asset = self.setup_asset(bookmark=bookmark, file="temp.html.gz")
+        self.assertEqual(asset.file_size, 4)

bookmarks/tests/test_bookmark_details_modal.py

@@ -1,8 +1,11 @@
-from django.test import TestCase
+from unittest.mock import patch
+
+from django.test import TestCase, override_settings
 from django.urls import reverse
 from django.utils import formats
 
-from bookmarks.models import UserProfile
+from bookmarks.models import BookmarkAsset, UserProfile
+from bookmarks.services import tasks
 from bookmarks.tests.helpers import BookmarkFactoryMixin, HtmlTestMixin
 
 
@@ -11,8 +14,15 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         user = self.get_or_create_test_user()
         self.client.force_login(user)
 
+    def get_view_name(self):
+        return "bookmarks:details_modal"
+
     def get_base_url(self, bookmark):
-        return reverse("bookmarks:details_modal", args=[bookmark.id])
+        return reverse(self.get_view_name(), args=[bookmark.id])
+
+    def get_details_form(self, soup, bookmark):
+        expected_url = reverse("bookmarks:details", args=[bookmark.id])
+        return soup.find("form", {"action": expected_url})
 
     def get_details(self, bookmark, return_url=""):
         url = self.get_base_url(bookmark)
@@ -35,43 +45,38 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
     def find_weblink(self, soup, url):
         return soup.find("a", {"class": "weblink", "href": url})
 
-    def test_access(self):
+    def find_asset(self, soup, asset):
+        return soup.find("div", {"data-asset-id": asset.id})
+
+    def details_route_access_test(self, view_name: str, shareable: bool):
         # own bookmark
         bookmark = self.setup_bookmark()
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
         self.assertEqual(response.status_code, 200)
 
         # other user's bookmark
         other_user = self.setup_user()
         bookmark = self.setup_bookmark(user=other_user)
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
         self.assertEqual(response.status_code, 404)
 
         # non-existent bookmark
-        response = self.client.get(reverse("bookmarks:details_modal", args=[9999]))
+        response = self.client.get(reverse(view_name, args=[9999]))
         self.assertEqual(response.status_code, 404)
 
         # guest user
         self.client.logout()
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 404 if shareable else 302)
-        )
-        self.assertEqual(response.status_code, 404)
 
-    def test_access_with_sharing(self):
+    def details_route_sharing_access_test(self, view_name: str, shareable: bool):
         # shared bookmark, sharing disabled
         other_user = self.setup_user()
         bookmark = self.setup_bookmark(shared=True, user=other_user)
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
         self.assertEqual(response.status_code, 404)
 
         # shared bookmark, sharing enabled
@@ -79,26 +84,38 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         profile.enable_sharing = True
         profile.save()
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 200 if shareable else 404)
-        )
-        self.assertEqual(response.status_code, 200)
 
         # shared bookmark, guest user, no public sharing
         self.client.logout()
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 404 if shareable else 302)
-        )
-        self.assertEqual(response.status_code, 404)
 
         # shared bookmark, guest user, public sharing
         profile.enable_public_sharing = True
         profile.save()
 
-        response = self.client.get(
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
-            reverse("bookmarks:details_modal", args=[bookmark.id])
+        self.assertEqual(response.status_code, 200 if shareable else 302)
-        )
+
-        self.assertEqual(response.status_code, 200)
+    def test_access(self):
+        self.details_route_access_test(self.get_view_name(), True)
+
+    def test_access_with_sharing(self):
+        self.details_route_sharing_access_test(self.get_view_name(), True)
+
+    def test_form_partial_access(self):
+        # form partial is only used when submitting forms, which should be only
+        # accessible to the owner of the bookmark. As such assume it requires
+        # login.
+        self.details_route_access_test("bookmarks:partials.details_form", False)
+
+    def test_form_partial_access_with_sharing(self):
+        # form partial is only used when submitting forms, which should be only
+        # accessible to the owner of the bookmark. As such assume it requires
+        # login.
+        self.details_route_sharing_access_test("bookmarks:partials.details_form", False)
 
     def test_displays_title(self):
         # with title
@@ -246,9 +263,8 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         # renders form
         bookmark = self.setup_bookmark()
         soup = self.get_details(bookmark)
-        section = self.get_section(soup, "Status")
 
-        form = section.find("form")
+        form = self.get_details_form(soup, bookmark)
         self.assertIsNotNone(form)
         self.assertEqual(
             form["action"], reverse("bookmarks:details", args=[bookmark.id])
@@ -312,30 +328,21 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         bookmark = self.setup_bookmark()
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNotNone(section)
-        self.assertIsNotNone(form)
 
         # other user's bookmark
         other_user = self.setup_user(enable_sharing=True)
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNone(section)
-        self.assertIsNone(form)
 
         # guest user
         self.client.logout()
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNone(section)
-        self.assertIsNone(form)
 
     def test_status_update(self):
         bookmark = self.setup_bookmark()
@@ -560,3 +567,215 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         delete_button = soup.find("button", {"type": "submit", "name": "remove"})
         self.assertIsNone(edit_link)
         self.assertIsNone(delete_button)
+
+    def test_assets_visibility_no_snapshot_support(self):
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.find_section(soup, "Files")
+        self.assertIsNone(section)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_assets_visibility_with_snapshot_support(self):
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.find_section(soup, "Files")
+        self.assertIsNotNone(section)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_list_visibility(self):
+        # no assets
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+        self.assertIsNone(asset_list)
+
+        # with assets
+        bookmark = self.setup_bookmark()
+        self.setup_asset(bookmark)
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+        self.assertIsNotNone(asset_list)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_list(self):
+        bookmark = self.setup_bookmark()
+        assets = [
+            self.setup_asset(bookmark),
+            self.setup_asset(bookmark),
+            self.setup_asset(bookmark),
+        ]
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+
+        for asset in assets:
+            asset_item = self.find_asset(asset_list, asset)
+            self.assertIsNotNone(asset_item)
+
+            asset_icon = asset_item.select_one(".asset-icon svg")
+            self.assertIsNotNone(asset_icon)
+
+            asset_text = asset_item.select_one(".asset-text span")
+            self.assertIsNotNone(asset_text)
+            self.assertIn(asset.display_name, asset_text.text)
+
+            view_url = reverse("bookmarks:assets.view", args=[asset.id])
+            view_link = asset_item.find("a", {"href": view_url})
+            self.assertIsNotNone(view_link)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_without_file(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark)
+        asset.file = ""
+        asset.save()
+
+        soup = self.get_details(bookmark)
+        asset_item = self.find_asset(soup, asset)
+        view_url = reverse("bookmarks:assets.view", args=[asset.id])
+        view_link = asset_item.find("a", {"href": view_url})
+        self.assertIsNone(view_link)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_status(self):
+        bookmark = self.setup_bookmark()
+        pending_asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_PENDING)
+        failed_asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_FAILURE)
+
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, pending_asset)
+        asset_text = asset_item.select_one(".asset-text span")
+        self.assertIn("(queued)", asset_text.text)
+
+        asset_item = self.find_asset(soup, failed_asset)
+        asset_text = asset_item.select_one(".asset-text span")
+        self.assertIn("(failed)", asset_text.text)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_file_size(self):
+        bookmark = self.setup_bookmark()
+        asset1 = self.setup_asset(bookmark, file_size=None)
+        asset2 = self.setup_asset(bookmark, file_size=54639)
+        asset3 = self.setup_asset(bookmark, file_size=11492020)
+
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset1)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertEqual(asset_text.text.strip(), asset1.display_name)
+
+        asset_item = self.find_asset(soup, asset2)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertIn("53.4\xa0KB", asset_text.text)
+
+        asset_item = self.find_asset(soup, asset3)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertIn("11.0\xa0MB", asset_text.text)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_actions_visibility(self):
+        bookmark = self.setup_bookmark()
+
+        # with file
+        asset = self.setup_asset(bookmark)
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNotNone(delete_button)
+
+        # without file
+        asset.file = ""
+        asset.save()
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNone(view_link)
+        self.assertIsNotNone(delete_button)
+
+        # shared bookmark
+        other_user = self.setup_user(enable_sharing=True, enable_public_sharing=True)
+        bookmark = self.setup_bookmark(shared=True, user=other_user)
+        asset = self.setup_asset(bookmark)
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNone(delete_button)
+
+        # shared bookmark, guest user
+        self.client.logout()
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNone(delete_button)
+
+    def test_remove_asset(self):
+        # remove asset
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark)
+
+        response = self.client.post(
+            self.get_base_url(bookmark), {"remove_asset": asset.id}
+        )
+        self.assertEqual(response.status_code, 302)
+        self.assertFalse(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+        # non-existent asset
+        response = self.client.post(self.get_base_url(bookmark), {"remove_asset": 9999})
+        self.assertEqual(response.status_code, 404)
+
+        # post without asset ID does not remove
+        asset = self.setup_asset(bookmark)
+        response = self.client.post(self.get_base_url(bookmark))
+        self.assertEqual(response.status_code, 302)
+        self.assertTrue(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+        # guest user
+        asset = self.setup_asset(bookmark)
+        self.client.logout()
+        response = self.client.post(
+            self.get_base_url(bookmark), {"remove_asset": asset.id}
+        )
+        self.assertEqual(response.status_code, 404)
+        self.assertTrue(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_snapshot(self):
+        with patch.object(
+            tasks, "_create_html_snapshot_task"
+        ) as mock_create_html_snapshot_task:
+            bookmark = self.setup_bookmark()
+            response = self.client.post(
+                self.get_base_url(bookmark), {"create_snapshot": ""}
+            )
+            self.assertEqual(response.status_code, 302)
+
+            mock_create_html_snapshot_task.assert_called_with(bookmark.id)
+
+            self.assertEqual(bookmark.bookmarkasset_set.count(), 1)

bookmarks/tests/test_bookmark_details_view.py

@@ -1,8 +1,6 @@
-from django.urls import reverse
-
 from bookmarks.tests.test_bookmark_details_modal import BookmarkDetailsModalTestCase
 
 
 class BookmarkDetailsViewTestCase(BookmarkDetailsModalTestCase):
-    def get_base_url(self, bookmark):
+    def get_view_name(self):
-        return reverse("bookmarks:details", args=[bookmark.id])
+        return "bookmarks:details"

bookmarks/tests/test_bookmarks_service.py

@@ -105,6 +105,24 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
 
             mock_load_favicon.assert_called_once_with(self.user, bookmark)
 
+    def test_create_should_load_html_snapshot(self):
+        with patch.object(tasks, "create_html_snapshot") as mock_create_html_snapshot:
+            bookmark_data = Bookmark(url="https://example.com")
+            bookmark = create_bookmark(bookmark_data, "tag1,tag2", self.user)
+
+            mock_create_html_snapshot.assert_called_once_with(bookmark)
+
+    def test_create_should_not_load_html_snapshot_when_setting_is_disabled(self):
+        profile = self.get_or_create_test_user().profile
+        profile.enable_automatic_html_snapshots = False
+        profile.save()
+
+        with patch.object(tasks, "create_html_snapshot") as mock_create_html_snapshot:
+            bookmark_data = Bookmark(url="https://example.com")
+            create_bookmark(bookmark_data, "tag1,tag2", self.user)
+
+            mock_create_html_snapshot.assert_not_called()
+
     def test_update_should_create_web_archive_snapshot_if_url_did_change(self):
         with patch.object(
             tasks, "create_web_archive_snapshot"
@@ -167,6 +185,14 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
 
             mock_load_favicon.assert_called_once_with(self.user, bookmark)
 
+    def test_update_should_not_create_html_snapshot(self):
+        with patch.object(tasks, "create_html_snapshot") as mock_create_html_snapshot:
+            bookmark = self.setup_bookmark()
+            bookmark.title = "updated title"
+            update_bookmark(bookmark, "tag1,tag2", self.user)
+
+            mock_create_html_snapshot.assert_not_called()
+
     def test_archive_bookmark(self):
         bookmark = Bookmark(
             url="https://example.com",

bookmarks/tests/test_bookmarks_tasks.py

@@ -1,18 +1,20 @@
 import datetime
+import os.path
 from dataclasses import dataclass
 from typing import Any
 from unittest import mock
 
 import waybackpy
 from background_task.models import Task
+from django.conf import settings
 from django.contrib.auth.models import User
 from django.test import TestCase, override_settings
 from waybackpy.exceptions import WaybackError
 
 import bookmarks.services.favicon_loader
 import bookmarks.services.wayback
-from bookmarks.models import UserProfile
+from bookmarks.models import BookmarkAsset, UserProfile
-from bookmarks.services import tasks
+from bookmarks.services import tasks, singlefile
 from bookmarks.tests.helpers import BookmarkFactoryMixin, disable_logging
 
 
@@ -626,3 +628,86 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         tasks.schedule_refresh_favicons(self.get_or_create_test_user())
 
         self.assertEqual(Task.objects.count(), 0)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_create_pending_asset(self):
+        bookmark = self.setup_bookmark()
+
+        with mock.patch("bookmarks.services.monolith.create_snapshot"):
+            tasks.create_html_snapshot(bookmark)
+            self.assertEqual(BookmarkAsset.objects.count(), 1)
+
+            tasks.create_html_snapshot(bookmark)
+            self.assertEqual(BookmarkAsset.objects.count(), 2)
+
+            assets = BookmarkAsset.objects.filter(bookmark=bookmark)
+            for asset in assets:
+                self.assertEqual(asset.bookmark, bookmark)
+                self.assertEqual(asset.asset_type, BookmarkAsset.TYPE_SNAPSHOT)
+                self.assertEqual(asset.content_type, BookmarkAsset.CONTENT_TYPE_HTML)
+                self.assertIn("HTML snapshot", asset.display_name)
+                self.assertEqual(asset.status, BookmarkAsset.STATUS_PENDING)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_update_file_info(self):
+        bookmark = self.setup_bookmark(url="https://example.com")
+
+        with mock.patch("bookmarks.services.singlefile.create_snapshot") as mock_create:
+            tasks.create_html_snapshot(bookmark)
+            asset = BookmarkAsset.objects.get(bookmark=bookmark)
+            asset.date_created = datetime.datetime(2021, 1, 2, 3, 44, 55)
+            asset.save()
+            expected_filename = "snapshot_2021-01-02_034455_https___example.com.html.gz"
+
+            self.run_pending_task(tasks._create_html_snapshot_task)
+
+            mock_create.assert_called_once_with(
+                "https://example.com",
+                os.path.join(settings.LD_ASSET_FOLDER, expected_filename),
+            )
+
+            asset = BookmarkAsset.objects.get(bookmark=bookmark)
+            self.assertEqual(asset.status, BookmarkAsset.STATUS_COMPLETE)
+            self.assertEqual(asset.file, expected_filename)
+            self.assertTrue(asset.gzip)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_handle_error(self):
+        bookmark = self.setup_bookmark(url="https://example.com")
+
+        with mock.patch("bookmarks.services.singlefile.create_snapshot") as mock_create:
+            mock_create.side_effect = singlefile.SingeFileError("Error")
+
+            tasks.create_html_snapshot(bookmark)
+            self.run_pending_task(tasks._create_html_snapshot_task)
+
+            asset = BookmarkAsset.objects.get(bookmark=bookmark)
+            self.assertEqual(asset.status, BookmarkAsset.STATUS_FAILURE)
+            self.assertEqual(asset.file, "")
+            self.assertFalse(asset.gzip)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_handle_missing_bookmark(self):
+        with mock.patch("bookmarks.services.singlefile.create_snapshot") as mock_create:
+            tasks._create_html_snapshot_task(123)
+            self.run_pending_task(tasks._create_html_snapshot_task)
+
+            mock_create.assert_not_called()
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=False)
+    def test_create_html_snapshot_should_not_run_when_single_file_is_disabled(
+        self,
+    ):
+        bookmark = self.setup_bookmark()
+        tasks.create_html_snapshot(bookmark)
+
+        self.assertEqual(Task.objects.count(), 0)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True, LD_DISABLE_BACKGROUND_TASKS=True)
+    def test_create_html_snapshot_should_not_run_when_background_tasks_are_disabled(
+        self,
+    ):
+        bookmark = self.setup_bookmark()
+        tasks.create_html_snapshot(bookmark)
+
+        self.assertEqual(Task.objects.count(), 0)

bookmarks/tests/test_monolith_service.py

@@ -0,0 +1,44 @@
+import gzip
+import os
+from unittest import mock
+import subprocess
+
+from django.test import TestCase
+
+from bookmarks.services import monolith
+
+
+class MonolithServiceTestCase(TestCase):
+    html_content = "<html><body><h1>Hello, World!</h1></body></html>"
+    html_filepath = "temp.html.gz"
+    temp_html_filepath = "temp.html.gz.tmp"
+
+    def tearDown(self):
+        if os.path.exists(self.html_filepath):
+            os.remove(self.html_filepath)
+        if os.path.exists(self.temp_html_filepath):
+            os.remove(self.temp_html_filepath)
+
+    def create_test_file(self, *args, **kwargs):
+        with open(self.temp_html_filepath, "w") as file:
+            file.write(self.html_content)
+
+    def test_create_snapshot(self):
+        with mock.patch("subprocess.run") as mock_run:
+            mock_run.side_effect = self.create_test_file
+
+            monolith.create_snapshot("http://example.com", self.html_filepath)
+
+            self.assertTrue(os.path.exists(self.html_filepath))
+            self.assertFalse(os.path.exists(self.temp_html_filepath))
+
+            with gzip.open(self.html_filepath, "rt") as file:
+                content = file.read()
+                self.assertEqual(content, self.html_content)
+
+    def test_create_snapshot_failure(self):
+        with mock.patch("subprocess.run") as mock_run:
+            mock_run.side_effect = subprocess.CalledProcessError(1, "command")
+
+            with self.assertRaises(monolith.MonolithError):
+                monolith.create_snapshot("http://example.com", self.html_filepath)

bookmarks/tests/test_settings_general_view.py

@@ -31,6 +31,7 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             "enable_sharing": False,
             "enable_public_sharing": False,
             "enable_favicons": False,
+            "enable_automatic_html_snapshots": True,
             "tag_search": UserProfile.TAG_SEARCH_STRICT,
             "display_url": False,
             "display_view_bookmark_action": True,
@@ -69,6 +70,7 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             "enable_sharing": True,
             "enable_public_sharing": True,
             "enable_favicons": True,
+            "enable_automatic_html_snapshots": False,
             "tag_search": UserProfile.TAG_SEARCH_LAX,
             "display_url": True,
             "display_view_bookmark_action": False,
@@ -110,6 +112,10 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
         self.assertEqual(
             self.user.profile.enable_favicons, form_data["enable_favicons"]
         )
+        self.assertEqual(
+            self.user.profile.enable_automatic_html_snapshots,
+            form_data["enable_automatic_html_snapshots"],
+        )
         self.assertEqual(self.user.profile.tag_search, form_data["tag_search"])
         self.assertEqual(self.user.profile.display_url, form_data["display_url"])
         self.assertEqual(
@@ -285,6 +291,35 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             count=0,
         )
 
+    def test_automatic_html_snapshots_should_be_hidden_when_snapshots_not_supported(
+        self,
+    ):
+        response = self.client.get(reverse("bookmarks:settings.general"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            """
+            <input type="checkbox" name="enable_automatic_html_snapshots" id="id_enable_automatic_html_snapshots" checked="">
+            """,
+            html,
+            count=0,
+        )
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_automatic_html_snapshots_should_be_visible_when_snapshots_supported(
+        self,
+    ):
+        response = self.client.get(reverse("bookmarks:settings.general"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            """
+            <input type="checkbox" name="enable_automatic_html_snapshots" id="id_enable_automatic_html_snapshots" checked="">
+            """,
+            html,
+            count=1,
+        )
+
     def test_about_shows_version_info(self):
         response = self.client.get(reverse("bookmarks:settings.general"))
         html = response.content.decode()

bookmarks/tests/test_singlefile_service.py

@@ -0,0 +1,50 @@
+import gzip
+import os
+from unittest import mock
+import subprocess
+
+from django.test import TestCase
+
+from bookmarks.services import singlefile
+
+
+class SingleFileServiceTestCase(TestCase):
+    html_content = "<html><body><h1>Hello, World!</h1></body></html>"
+    html_filepath = "temp.html.gz"
+    temp_html_filepath = "temp.html.gz.tmp"
+
+    def tearDown(self):
+        if os.path.exists(self.html_filepath):
+            os.remove(self.html_filepath)
+        if os.path.exists(self.temp_html_filepath):
+            os.remove(self.temp_html_filepath)
+
+    def create_test_file(self, *args, **kwargs):
+        with open(self.temp_html_filepath, "w") as file:
+            file.write(self.html_content)
+
+    def test_create_snapshot(self):
+        with mock.patch("subprocess.run") as mock_run:
+            mock_run.side_effect = self.create_test_file
+
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            self.assertTrue(os.path.exists(self.html_filepath))
+            self.assertFalse(os.path.exists(self.temp_html_filepath))
+
+            with gzip.open(self.html_filepath, "rt") as file:
+                content = file.read()
+                self.assertEqual(content, self.html_content)
+
+    def test_create_snapshot_failure(self):
+        # subprocess fails - which it probably doesn't as single-file doesn't return exit codes
+        with mock.patch("subprocess.run") as mock_run:
+            mock_run.side_effect = subprocess.CalledProcessError(1, "command")
+
+            with self.assertRaises(singlefile.SingeFileError):
+                singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+        # so also check that it raises error if output file isn't created
+        with mock.patch("subprocess.run") as mock_run:
+            with self.assertRaises(singlefile.SingeFileError):
+                singlefile.create_snapshot("http://example.com", self.html_filepath)

bookmarks/urls.py

@@ -44,6 +44,12 @@ urlpatterns = [
         views.bookmarks.details_modal,
         name="details_modal",
     ),
+    # Assets
+    path(
+        "assets/<int:asset_id>",
+        views.assets.view,
+        name="assets.view",
+    ),
     # Partials
     path(
         "bookmarks/partials/bookmark-list/active",
@@ -75,6 +81,11 @@ urlpatterns = [
         partials.shared_tag_cloud,
         name="partials.tag_cloud.shared",
     ),
+    path(
+        "bookmarks/partials/details-form/<int:bookmark_id>",
+        partials.details_form,
+        name="partials.details_form",
+    ),
     # Settings
     path("settings", views.settings.general, name="settings.index"),
     path("settings/general", views.settings.general, name="settings.general"),

bookmarks/views/__init__.py

@@ -1,3 +1,4 @@
+from .assets import *
 from .bookmarks import *
 from .settings import *
 from .toasts import *

bookmarks/views/assets.py

@@ -0,0 +1,43 @@
+import gzip
+import os
+
+from django.conf import settings
+from django.http import (
+    HttpResponse,
+    Http404,
+)
+
+from bookmarks.models import BookmarkAsset
+
+
+def view(request, asset_id: int):
+    try:
+        asset = BookmarkAsset.objects.get(pk=asset_id)
+    except BookmarkAsset.DoesNotExist:
+        raise Http404("Asset does not exist")
+
+    bookmark = asset.bookmark
+    is_owner = bookmark.owner == request.user
+    is_shared = (
+        request.user.is_authenticated
+        and bookmark.shared
+        and bookmark.owner.profile.enable_sharing
+    )
+    is_public_shared = bookmark.shared and bookmark.owner.profile.enable_public_sharing
+
+    if not is_owner and not is_shared and not is_public_shared:
+        raise Http404("Bookmark does not exist")
+
+    filepath = os.path.join(settings.LD_ASSET_FOLDER, asset.file)
+
+    if not os.path.exists(filepath):
+        raise Http404("Asset file does not exist")
+
+    if asset.gzip:
+        with gzip.open(filepath, "rb") as f:
+            content = f.read()
+    else:
+        with open(filepath, "rb") as f:
+            content = f.read()
+
+    return HttpResponse(content, content_type=asset.content_type)

bookmarks/views/bookmarks.py

@@ -12,7 +12,13 @@ from django.shortcuts import render
 from django.urls import reverse
 
 from bookmarks import queries
-from bookmarks.models import Bookmark, BookmarkForm, BookmarkSearch, build_tag_string
+from bookmarks.models import (
+    Bookmark,
+    BookmarkAsset,
+    BookmarkForm,
+    BookmarkSearch,
+    build_tag_string,
+)
 from bookmarks.services.bookmarks import (
     create_bookmark,
     update_bookmark,
@@ -28,6 +34,7 @@ from bookmarks.services.bookmarks import (
     share_bookmarks,
     unshare_bookmarks,
 )
+from bookmarks.services import tasks
 from bookmarks.utils import get_safe_return_url
 from bookmarks.views.partials import contexts
 
@@ -120,31 +127,39 @@ def _details(request, bookmark_id: int, template: str):
     if not is_owner and not is_shared and not is_public_shared:
         raise Http404("Bookmark does not exist")
 
-    edit_return_url = get_safe_return_url(
-        request.GET.get("return_url"), reverse("bookmarks:details", args=[bookmark_id])
-    )
-    delete_return_url = get_safe_return_url(
-        request.GET.get("return_url"), reverse("bookmarks:index")
-    )
-
-    # handles status actions form
     if request.method == "POST":
         if not is_owner:
             raise Http404("Bookmark does not exist")
-        bookmark.is_archived = request.POST.get("is_archived") == "on"
-        bookmark.unread = request.POST.get("unread") == "on"
-        bookmark.shared = request.POST.get("shared") == "on"
-        bookmark.save()
 
-        return HttpResponseRedirect(edit_return_url)
+        return_url = get_safe_return_url(
+            request.GET.get("return_url"),
+            reverse("bookmarks:details", args=[bookmark.id]),
+        )
+
+        if "remove_asset" in request.POST:
+            asset_id = request.POST["remove_asset"]
+            try:
+                asset = bookmark.bookmarkasset_set.get(pk=asset_id)
+            except BookmarkAsset.DoesNotExist:
+                raise Http404("Asset does not exist")
+            asset.delete()
+        if "create_snapshot" in request.POST:
+            tasks.create_html_snapshot(bookmark)
+        else:
+            bookmark.is_archived = request.POST.get("is_archived") == "on"
+            bookmark.unread = request.POST.get("unread") == "on"
+            bookmark.shared = request.POST.get("shared") == "on"
+            bookmark.save()
+
+        return HttpResponseRedirect(return_url)
+
+    details_context = contexts.BookmarkDetailsContext(request, bookmark)
 
     return render(
         request,
         template,
         {
-            "bookmark": bookmark,
+            "details": details_context,
-            "edit_return_url": edit_return_url,
-            "delete_return_url": delete_return_url,
         },
     )
 

bookmarks/views/partials/__init__.py

@@ -1,6 +1,8 @@
 from django.contrib.auth.decorators import login_required
+from django.http import Http404
 from django.shortcuts import render
 
+from bookmarks.models import Bookmark
 from bookmarks.views.partials import contexts
 
 
@@ -56,3 +58,15 @@ def shared_tag_cloud(request):
     tag_cloud_context = contexts.SharedTagCloudContext(request)
 
     return render(request, "bookmarks/tag_cloud.html", {"tag_cloud": tag_cloud_context})
+
+
+@login_required
+def details_form(request, bookmark_id: int):
+    try:
+        bookmark = Bookmark.objects.get(pk=bookmark_id, owner=request.user)
+    except Bookmark.DoesNotExist:
+        raise Http404("Bookmark does not exist")
+
+    details_context = contexts.BookmarkDetailsContext(request, bookmark)
+
+    return render(request, "bookmarks/details/form.html", {"details": details_context})

bookmarks/views/partials/contexts.py

@@ -6,11 +6,13 @@ from django.core.handlers.wsgi import WSGIRequest
 from django.core.paginator import Paginator
 from django.db import models
 from django.urls import reverse
+from django.conf import settings
 
 from bookmarks import queries
 from bookmarks import utils
 from bookmarks.models import (
     Bookmark,
+    BookmarkAsset,
     BookmarkSearch,
     User,
     UserProfile,
@@ -274,3 +276,55 @@ class SharedTagCloudContext(TagCloudContext):
         return queries.query_shared_bookmark_tags(
             user, self.request.user_profile, self.search, public_only
         )
+
+
+class BookmarkAssetItem:
+    def __init__(self, asset: BookmarkAsset):
+        self.asset = asset
+
+        self.id = asset.id
+        self.display_name = asset.display_name
+        self.content_type = asset.content_type
+        self.file = asset.file
+        self.file_size = asset.file_size
+        self.status = asset.status
+
+        icon_classes = []
+        text_classes = []
+        if asset.status == BookmarkAsset.STATUS_PENDING:
+            icon_classes.append("text-gray")
+            text_classes.append("text-gray")
+        elif asset.status == BookmarkAsset.STATUS_FAILURE:
+            icon_classes.append("text-error")
+            text_classes.append("text-error")
+        else:
+            icon_classes.append("text-primary")
+
+        self.icon_classes = " ".join(icon_classes)
+        self.text_classes = " ".join(text_classes)
+
+
+class BookmarkDetailsContext:
+    def __init__(self, request: WSGIRequest, bookmark: Bookmark):
+        user = request.user
+        user_profile = request.user_profile
+
+        self.edit_return_url = utils.get_safe_return_url(
+            request.GET.get("return_url"),
+            reverse("bookmarks:details", args=[bookmark.id]),
+        )
+        self.delete_return_url = utils.get_safe_return_url(
+            request.GET.get("return_url"), reverse("bookmarks:index")
+        )
+
+        self.bookmark = bookmark
+        self.profile = request.user_profile
+        self.is_editable = bookmark.owner == user
+        self.sharing_enabled = user_profile.enable_sharing
+        self.show_link_icons = user_profile.enable_favicons and bookmark.favicon_file
+        # For now hide files section if snapshots are not supported
+        self.show_files = settings.LD_ENABLE_SNAPSHOTS
+
+        self.assets = [
+            BookmarkAssetItem(asset) for asset in bookmark.bookmarkasset_set.all()
+        ]

bookmarks/views/settings.py

@@ -12,7 +12,7 @@ from django.shortcuts import render
 from django.urls import reverse
 from rest_framework.authtoken.models import Token
 
-from bookmarks.models import Bookmark, BookmarkSearch, UserProfileForm, FeedToken
+from bookmarks.models import Bookmark, UserProfileForm, FeedToken
 from bookmarks.services import exporter, tasks
 from bookmarks.services import importer
 from bookmarks.utils import app_version
@@ -24,6 +24,7 @@ logger = logging.getLogger(__name__)
 def general(request):
     profile_form = None
     enable_refresh_favicons = django_settings.LD_ENABLE_REFRESH_FAVICONS
+    has_snapshot_support = django_settings.LD_ENABLE_SNAPSHOTS
     update_profile_success_message = None
     refresh_favicons_success_message = None
     import_success_message = _find_message_with_tag(
@@ -53,6 +54,7 @@ def general(request):
         {
             "form": profile_form,
             "enable_refresh_favicons": enable_refresh_favicons,
+            "has_snapshot_support": has_snapshot_support,
             "update_profile_success_message": update_profile_success_message,
             "refresh_favicons_success_message": refresh_favicons_success_message,
             "import_success_message": import_success_message,

bootstrap.sh

@@ -7,6 +7,8 @@ LD_SERVER_PORT="${LD_SERVER_PORT:-9090}"
 mkdir -p data
 # Create favicon folder if it does not exist
 mkdir -p data/favicons
+# Create assets folder if it does not exist
+mkdir -p data/assets
 
 # Generate secret key file if it does not exist
 python manage.py generate_secret_key

docker/alpine.Dockerfile

@@ -67,7 +67,7 @@ RUN wget https://www.sqlite.org/${SQLITE_RELEASE_YEAR}/sqlite-amalgamation-${SQL
     gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-uc icu-io` -o libicu.so
 
 
-FROM python:3.11.8-alpine3.19 AS final
+FROM python:3.11.8-alpine3.19 AS linkding
 # install runtime dependencies
 RUN apk update && apk add bash curl icu libpq mailcap libssl3
 # create www-data user and group
@@ -96,3 +96,10 @@ HEALTHCHECK --interval=30s --retries=3 --timeout=1s \
 CMD curl -f http://localhost:${LD_SERVER_PORT:-9090}/${LD_CONTEXT_PATH}health || exit 1
 
 CMD ["./bootstrap.sh"]
+
+
+FROM linkding AS linkding-plus
+# install node, chromium and single-file
+RUN apk update && apk add nodejs npm chromium && npm install -g single-file-cli
+# enable snapshot support
+ENV LD_ENABLE_SNAPSHOTS=True

docker/default.Dockerfile

@@ -69,7 +69,7 @@ RUN wget https://www.sqlite.org/${SQLITE_RELEASE_YEAR}/sqlite-amalgamation-${SQL
     gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-uc icu-io` -o libicu.so
 
 
-FROM python:3.11.8-slim-bookworm as final
+FROM python:3.11.8-slim-bookworm as linkding
 RUN apt-get update && apt-get -y install mime-support libpq-dev libicu-dev libssl3 curl
 WORKDIR /etc/linkding
 # copy prod dependencies
@@ -94,3 +94,9 @@ HEALTHCHECK --interval=30s --retries=3 --timeout=1s \
 CMD curl -f http://localhost:${LD_SERVER_PORT:-9090}/${LD_CONTEXT_PATH}health || exit 1
 
 CMD ["./bootstrap.sh"]
+
+FROM linkding AS linkding-plus
+# install node, chromium and single-file
+RUN apt-get update && apt-get -y install nodejs npm chromium && npm install -g single-file-cli
+# enable snapshot support
+ENV LD_ENABLE_SNAPSHOTS=True

scripts/build-docker.sh

@@ -2,14 +2,29 @@
 
 version=$(<version.txt)
 
-docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 \
+# Base image
+docker buildx build --target linkding --platform linux/amd64,linux/arm64,linux/arm/v7 \
   -f docker/default.Dockerfile \
   -t sissbruecker/linkding:latest \
   -t sissbruecker/linkding:$version \
   --push .
 
-docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 \
+docker buildx build --target linkding --platform linux/amd64,linux/arm64,linux/arm/v7 \
   -f docker/alpine.Dockerfile \
   -t sissbruecker/linkding:latest-alpine \
   -t sissbruecker/linkding:$version-alpine \
   --push .
+
+# Plus image with support for single-file snapshots
+# Needs checking if this works with ARMv7, excluded for now
+docker buildx build --target linkding-plus --platform linux/amd64,linux/arm64 \
+  -f docker/default.Dockerfile \
+  -t sissbruecker/linkding:latest-plus \
+  -t sissbruecker/linkding:$version-plus \
+  --push .
+
+docker buildx build --target linkding-plus --platform linux/amd64,linux/arm64 \
+  -f docker/alpine.Dockerfile \
+  -t sissbruecker/linkding:latest-alpine-plus \
+  -t sissbruecker/linkding:$version-alpine-plus \
+  --push .

siteroot/settings/base.py

@@ -269,3 +269,20 @@ LD_ENABLE_REFRESH_FAVICONS = os.getenv("LD_ENABLE_REFRESH_FAVICONS", True) in (
     "True",
     "1",
 )
+
+# Asset / snapshot settings
+LD_ASSET_FOLDER = os.path.join(BASE_DIR, "data", "assets")
+
+LD_ENABLE_SNAPSHOTS = os.getenv("LD_ENABLE_SNAPSHOTS", False) in (
+    True,
+    "True",
+    "1",
+)
+LD_SINGLEFILE_PATH = os.getenv("LD_SINGLEFILE_PATH", "single-file")
+LD_SINGLEFILE_OPTIONS = os.getenv("LD_SINGLEFILE_OPTIONS", "")
+
+# Monolith isn't used at the moment, as the local snapshot implementation
+# switched to single-file after the prototype. Keeping this around in case
+# it turns out to be useful in the future.
+LD_MONOLITH_PATH = os.getenv("LD_MONOLITH_PATH", "monolith")
+LD_MONOLITH_OPTIONS = os.getenv("LD_MONOLITH_OPTIONS", "-a -v -s")