Merge tag 'v0.4.4' into nix

.github/workflows/django-tests.yml

@@ -55,5 +55,6 @@ jobs:
         EMAIL_HOST_PASSWORD: ""
         EMAIL_USE_TLS: true
         ENABLE_PREVIEW_IMAGES: false
+        ENABLE_THUMBNAIL_GENERATION: true
       run: |
         pytest -n 3

.github/workflows/pylint.yml

@@ -21,8 +21,7 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         pip install -r requirements.txt
-        pip install pylint
     - name: Analysing the code with pylint
       run: |
-        pylint bookwyrm/ --ignore=migrations --disable=E1101,E1135,E1136,R0903,R0901,R0902,W0707,W0511,W0406,R0401,R0801
+        pylint bookwyrm/
 

.pylintrc

@@ -0,0 +1,9 @@
+[MAIN]
+ignore=migrations
+load-plugins=pylint.extensions.no_self_use
+
+[MESSAGES CONTROL]
+disable=E1101,E1135,E1136,R0903,R0901,R0902,W0707,W0511,W0406,R0401,R0801,C3001,import-error
+
+[FORMAT]
+max-line-length=88

Dockerfile

@@ -6,8 +6,8 @@ RUN mkdir /app /app/static /app/images
 
 WORKDIR /app
 
+RUN apt-get update && apt-get install -y gettext libgettextpo-dev tidy && apt-get clean
+
 
 COPY pyproject.toml poetry.lock ./
 RUN poetry install --no-dev
-
-COPY ./bookwyrm ./celerywyrm ${APP_DIR}/

README.md

@@ -1,60 +1,49 @@
-# BookWyrm Nix Edition
+# BookWyrm
 
-Social reading and reviewing, decentralized with ActivityPub. This branch is a fork of upstream Bookwyrm adapted to run on NixOS (outside of Docker). For more details on this, see [nix/README.md](./nix/README.md) The rest of this document is the upstream README.
+**Note:** This branch is a fork of upstream Bookwyrm adapted to run on NixOS (outside of Docker). For more details on this, see [nix/README.md](./nix/README.md) The rest of this document is the upstream README.
 
-## Contents
-- [Joining BookWyrm](#joining-bookwyrm)
-- [Contributing](#contributing)
-- [About BookWyrm](#about-bookwyrm)
-    - [What it is and isn't](#what-it-is-and-isnt)
-    - [The role of federation](#the-role-of-federation)
-    - [Features](#features)
-- [Set up BookWyrm](#set-up-bookwyrm)
+---- 
 
-## Joining BookWyrm
-If you'd like to join an instance, you can check out the [instances](https://joinbookwyrm.com/instances/) list.
+[![](https://img.shields.io/github/release/bookwyrm-social/bookwyrm.svg?colorB=58839b)](https://github.com/bookwyrm-social/bookwyrm/releases)
+[![Run Python Tests](https://github.com/bookwyrm-social/bookwyrm/actions/workflows/django-tests.yml/badge.svg)](https://github.com/bookwyrm-social/bookwyrm/actions/workflows/django-tests.yml)
+[![Pylint](https://github.com/bookwyrm-social/bookwyrm/actions/workflows/pylint.yml/badge.svg)](https://github.com/bookwyrm-social/bookwyrm/actions/workflows/pylint.yml)
+
+BookWyrm is a social network for tracking your reading, talking about books, writing reviews, and discovering what to read next. Federation allows BookWyrm users to join small, trusted communities that can connect with one another, and with other ActivityPub services like [Mastodon](https://joinmastodon.org/) and [Pleroma](http://pleroma.social/).
 
 
-## Contributing
-See [contributing](https://docs.joinbookwyrm.com/contributing.html) for code, translation or monetary contributions.
+## Links
+
+[![Mastodon Follow](https://img.shields.io/mastodon/follow/000146121?domain=https%3A%2F%2Ftech.lgbt&style=social)](https://tech.lgbt/@bookwyrm)
+[![Twitter Follow](https://img.shields.io/twitter/follow/BookWyrmSocial?style=social)](https://twitter.com/BookWyrmSocial)
+
+ - [Project homepage](https://joinbookwyrm.com/)
+ - [Support](https://patreon.com/bookwyrm)
+ - [Documentation](https://docs.joinbookwyrm.com/)
+
 
 ## About BookWyrm
-### What it is and isn't
 BookWyrm is a platform for social reading. You can use it to track what you're reading, review books, and follow your friends. It isn't primarily meant for cataloguing or as a data-source for books, but it does do both of those things to some degree.
 
-### The role of federation
+## Federation
 BookWyrm is built on [ActivityPub](http://activitypub.rocks/). With ActivityPub, it inter-operates with different instances of BookWyrm, and other ActivityPub compliant services, like Mastodon. This means you can run an instance for your book club, and still follow your friend who posts on a server devoted to 20th century Russian speculative fiction. It also means that your friend on mastodon can read and comment on a book review that you post on your BookWyrm instance.
 
 Federation makes it possible to have small, self-determining communities, in contrast to the monolithic service you find on GoodReads or Twitter. An instance can be focused on a particular interest, be just for a group of friends, or anything else that brings people together. Each community can choose which other instances they want to federate with, and moderate and run their community autonomously. Check out https://runyourown.social/ to get a sense of the philosophy and logistics behind small, high-trust social networks.
 
-### Features
-Since the project is still in its early stages, the features are growing every day, and there is plenty of room for suggestions and ideas. Open an [issue](https://github.com/bookwyrm-social/bookwyrm/issues) to get the conversation going!
-- Posting about books
-    - Compose reviews, with or without ratings, which are aggregated in the book page
-    - Compose other kinds of statuses about books, such as:
-        - Comments on a book
-        - Quotes or excerpts
-    - Reply to statuses
-    - View aggregate reviews of a book across connected BookWyrm instances
-    - Differentiate local and federated reviews and rating in your activity feed
-- Track reading activity
-    - Shelve books on default "to-read," "currently reading," and "read" shelves
-    - Create custom shelves
-    - Store started reading/finished reading dates, as well as progress updates along the way
-    - Update followers about reading activity (optionally, and with granular privacy controls)
-    - Create lists of books which can be open to submissions from anyone, curated, or only edited by the creator
-- Federation with ActivityPub
-    - Broadcast and receive user statuses and activity
-    - Share book data between instances to create a networked database of metadata
-    - Identify shared books across instances and aggregate related content
-    - Follow and interact with users across BookWyrm instances
-    - Inter-operate with non-BookWyrm ActivityPub services (currently, Mastodon is supported)
-- Granular privacy controls
-    - Private, followers-only, and public privacy levels for posting, shelves, and lists
-    - Option for users to manually approve followers
-    - Allow blocking and flagging for moderation
+## Features
 
-### The Tech Stack
+### Post about books
+Compose reviews, comment on what you're reading, and post quotes from books. You can converse with other BookWyrm users across the network about what they're reading.
+
+### Track reading activity
+Keep track of what books you've read, and what books you'd like to read in the future.
+
+### Federation with ActivityPub
+Federation allows you to interact with users on other instances and services, and also shares metadata about books and authors, which collaboratively builds a decentralized database of books.
+
+### Privacy and moderation
+Users and administrators can control who can see thier posts and what other instances to federate with.
+
+## Tech Stack
 Web backend
 - [Django](https://www.djangoproject.com/) web server
 - [PostgreSQL](https://www.postgresql.org/) database

SECURITY.md

@@ -0,0 +1,5 @@
+# Security Policy
+
+## Reporting a Vulnerability
+
+Please report security issues to `mousereeve@riseup.net`

bookwyrm/book_search.py

@@ -148,8 +148,8 @@ class SearchResult:
 
     def __repr__(self):
         # pylint: disable=consider-using-f-string
-        return "<SearchResult key={!r} title={!r} author={!r}>".format(
-            self.key, self.title, self.author
+        return "<SearchResult key={!r} title={!r} author={!r} confidence={!r}>".format(
+            self.key, self.title, self.author, self.confidence
         )
 
     def json(self):

bookwyrm/connectors/abstract_connector.py

@@ -1,9 +1,8 @@
 """ functionality outline for a book data connector """
 from abc import ABC, abstractmethod
 import imghdr
-import ipaddress
 import logging
-from urllib.parse import urlparse
+import re
 
 from django.core.files.base import ContentFile
 from django.db import transaction
@@ -11,7 +10,7 @@ import requests
 from requests.exceptions import RequestException
 
 from bookwyrm import activitypub, models, settings
-from .connector_manager import load_more_data, ConnectorException
+from .connector_manager import load_more_data, ConnectorException, raise_not_valid_url
 from .format_mappings import format_mappings
 
 
@@ -39,62 +38,34 @@ class AbstractMinimalConnector(ABC):
         for field in self_fields:
             setattr(self, field, getattr(info, field))
 
-    def search(self, query, min_confidence=None, timeout=settings.QUERY_TIMEOUT):
-        """free text search"""
-        params = {}
-        if min_confidence:
-            params["min_confidence"] = min_confidence
+    def get_search_url(self, query):
+        """format the query url"""
+        # Check if the query resembles an ISBN
+        if maybe_isbn(query) and self.isbn_search_url and self.isbn_search_url != "":
+            return f"{self.isbn_search_url}{query}"
 
-        data = self.get_search_data(
-            f"{self.search_url}{query}",
-            params=params,
-            timeout=timeout,
-        )
-        results = []
+        # NOTE: previously, we tried searching isbn and if that produces no results,
+        # searched as free text. This, instead, only searches isbn if it's isbn-y
+        return f"{self.search_url}{query}"
 
-        for doc in self.parse_search_data(data)[:10]:
-            results.append(self.format_search_result(doc))
-        return results
-
-    def isbn_search(self, query, timeout=settings.QUERY_TIMEOUT):
-        """isbn search"""
-        params = {}
-        data = self.get_search_data(
-            f"{self.isbn_search_url}{query}",
-            params=params,
-            timeout=timeout,
-        )
-        results = []
-
-        # this shouldn't be returning mutliple results, but just in case
-        for doc in self.parse_isbn_search_data(data)[:10]:
-            results.append(self.format_isbn_search_result(doc))
-        return results
-
-    def get_search_data(self, remote_id, **kwargs):  # pylint: disable=no-self-use
-        """this allows connectors to override the default behavior"""
-        return get_data(remote_id, **kwargs)
+    def process_search_response(self, query, data, min_confidence):
+        """Format the search results based on the formt of the query"""
+        if maybe_isbn(query):
+            return list(self.parse_isbn_search_data(data))[:10]
+        return list(self.parse_search_data(data, min_confidence))[:10]
 
     @abstractmethod
     def get_or_create_book(self, remote_id):
         """pull up a book record by whatever means possible"""
 
     @abstractmethod
-    def parse_search_data(self, data):
+    def parse_search_data(self, data, min_confidence):
         """turn the result json from a search into a list"""
 
-    @abstractmethod
-    def format_search_result(self, search_result):
-        """create a SearchResult obj from json"""
-
     @abstractmethod
     def parse_isbn_search_data(self, data):
         """turn the result json from a search into a list"""
 
-    @abstractmethod
-    def format_isbn_search_result(self, search_result):
-        """create a SearchResult obj from json"""
-
 
 class AbstractConnector(AbstractMinimalConnector):
     """generic book data connector"""
@@ -254,9 +225,6 @@ def get_data(url, params=None, timeout=10):
     # check if the url is blocked
     raise_not_valid_url(url)
 
-    if models.FederatedServer.is_blocked(url):
-        raise ConnectorException(f"Attempting to load data from blocked url: {url}")
-
     try:
         resp = requests.get(
             url,
@@ -311,20 +279,6 @@ def get_image(url, timeout=10):
     return image_content, extension
 
 
-def raise_not_valid_url(url):
-    """do some basic reality checks on the url"""
-    parsed = urlparse(url)
-    if not parsed.scheme in ["http", "https"]:
-        raise ConnectorException("Invalid scheme: ", url)
-
-    try:
-        ipaddress.ip_address(parsed.netloc)
-        raise ConnectorException("Provided url is an IP address: ", url)
-    except ValueError:
-        # it's not an IP address, which is good
-        pass
-
-
 class Mapping:
     """associate a local database field with a field in an external dataset"""
 
@@ -366,3 +320,9 @@ def unique_physical_format(format_text):
         # try a direct match, so saving this would be redundant
         return None
     return format_text
+
+
+def maybe_isbn(query):
+    """check if a query looks like an isbn"""
+    isbn = re.sub(r"[\W_]", "", query)  # removes filler characters
+    return len(isbn) in [10, 13]  # ISBN10 or ISBN13

bookwyrm/connectors/bookwyrm_connector.py

@@ -10,15 +10,12 @@ class Connector(AbstractMinimalConnector):
     def get_or_create_book(self, remote_id):
         return activitypub.resolve_remote_id(remote_id, model=models.Edition)
 
-    def parse_search_data(self, data):
-        return data
-
-    def format_search_result(self, search_result):
-        search_result["connector"] = self
-        return SearchResult(**search_result)
+    def parse_search_data(self, data, min_confidence):
+        for search_result in data:
+            search_result["connector"] = self
+            yield SearchResult(**search_result)
 
     def parse_isbn_search_data(self, data):
-        return data
-
-    def format_isbn_search_result(self, search_result):
-        return self.format_search_result(search_result)
+        for search_result in data:
+            search_result["connector"] = self
+            yield SearchResult(**search_result)

bookwyrm/connectors/connector_manager.py

@@ -1,17 +1,18 @@
 """ interface with whatever connectors the app has """
-from datetime import datetime
+import asyncio
 import importlib
+import ipaddress
 import logging
-import re
 from urllib.parse import urlparse
 
+import aiohttp
 from django.dispatch import receiver
 from django.db.models import signals
 
 from requests import HTTPError
 
 from bookwyrm import book_search, models
-from bookwyrm.settings import SEARCH_TIMEOUT
+from bookwyrm.settings import SEARCH_TIMEOUT, USER_AGENT
 from bookwyrm.tasks import app
 
 logger = logging.getLogger(__name__)
@@ -21,53 +22,85 @@ class ConnectorException(HTTPError):
     """when the connector can't do what was asked"""
 
 
+async def get_results(session, url, min_confidence, query, connector):
+    """try this specific connector"""
+    # pylint: disable=line-too-long
+    headers = {
+        "Accept": (
+            'application/json, application/activity+json, application/ld+json; profile="https://www.w3.org/ns/activitystreams"; charset=utf-8'
+        ),
+        "User-Agent": USER_AGENT,
+    }
+    params = {"min_confidence": min_confidence}
+    try:
+        async with session.get(url, headers=headers, params=params) as response:
+            if not response.ok:
+                logger.info("Unable to connect to %s: %s", url, response.reason)
+                return
+
+            try:
+                raw_data = await response.json()
+            except aiohttp.client_exceptions.ContentTypeError as err:
+                logger.exception(err)
+                return
+
+            return {
+                "connector": connector,
+                "results": connector.process_search_response(
+                    query, raw_data, min_confidence
+                ),
+            }
+    except asyncio.TimeoutError:
+        logger.info("Connection timed out for url: %s", url)
+    except aiohttp.ClientError as err:
+        logger.info(err)
+
+
+async def async_connector_search(query, items, min_confidence):
+    """Try a number of requests simultaneously"""
+    timeout = aiohttp.ClientTimeout(total=SEARCH_TIMEOUT)
+    async with aiohttp.ClientSession(timeout=timeout) as session:
+        tasks = []
+        for url, connector in items:
+            tasks.append(
+                asyncio.ensure_future(
+                    get_results(session, url, min_confidence, query, connector)
+                )
+            )
+
+        results = await asyncio.gather(*tasks)
+        return results
+
+
 def search(query, min_confidence=0.1, return_first=False):
     """find books based on arbitary keywords"""
     if not query:
         return []
     results = []
 
-    # Have we got a ISBN ?
-    isbn = re.sub(r"[\W_]", "", query)
-    maybe_isbn = len(isbn) in [10, 13]  # ISBN10 or ISBN13
-
-    start_time = datetime.now()
+    items = []
     for connector in get_connectors():
-        result_set = None
-        if maybe_isbn and connector.isbn_search_url and connector.isbn_search_url != "":
-            # Search on ISBN
-            try:
-                result_set = connector.isbn_search(isbn)
-            except Exception as err:  # pylint: disable=broad-except
-                logger.info(err)
-                # if this fails, we can still try regular search
+        # get the search url from the connector before sending
+        url = connector.get_search_url(query)
+        try:
+            raise_not_valid_url(url)
+        except ConnectorException:
+            # if this URL is invalid we should skip it and move on
+            logger.info("Request denied to blocked domain: %s", url)
+            continue
+        items.append((url, connector))
 
-        # if no isbn search results, we fallback to generic search
-        if not result_set:
-            try:
-                result_set = connector.search(query, min_confidence=min_confidence)
-            except Exception as err:  # pylint: disable=broad-except
-                # we don't want *any* error to crash the whole search page
-                logger.info(err)
-                continue
-
-        if return_first and result_set:
-            # if we found anything, return it
-            return result_set[0]
-
-        if result_set:
-            results.append(
-                {
-                    "connector": connector,
-                    "results": result_set,
-                }
-            )
-        if (datetime.now() - start_time).seconds >= SEARCH_TIMEOUT:
-            break
+    # load as many results as we can
+    results = asyncio.run(async_connector_search(query, items, min_confidence))
+    results = [r for r in results if r]
 
     if return_first:
-        return None
+        # find the best result from all the responses and return that
+        all_results = [r for con in results for r in con["results"]]
+        all_results = sorted(all_results, key=lambda r: r.confidence, reverse=True)
+        return all_results[0] if all_results else None
 
+    # failed requests will return None, so filter those out
     return results
 
 
@@ -119,6 +152,15 @@ def load_more_data(connector_id, book_id):
     connector.expand_book_data(book)
 
 
+@app.task(queue="low_priority")
+def create_edition_task(connector_id, work_id, data):
+    """separate task for each of the 10,000 editions of LoTR"""
+    connector_info = models.Connector.objects.get(id=connector_id)
+    connector = load_connector(connector_info)
+    work = models.Work.objects.select_subclasses().get(id=work_id)
+    connector.create_edition_from_data(work, data)
+
+
 def load_connector(connector_info):
     """instantiate the connector class"""
     connector = importlib.import_module(
@@ -133,3 +175,20 @@ def create_connector(sender, instance, created, *args, **kwargs):
     """create a connector to an external bookwyrm server"""
     if instance.application_type == "bookwyrm":
         get_or_create_connector(f"https://{instance.server_name}")
+
+
+def raise_not_valid_url(url):
+    """do some basic reality checks on the url"""
+    parsed = urlparse(url)
+    if not parsed.scheme in ["http", "https"]:
+        raise ConnectorException("Invalid scheme: ", url)
+
+    try:
+        ipaddress.ip_address(parsed.netloc)
+        raise ConnectorException("Provided url is an IP address: ", url)
+    except ValueError:
+        # it's not an IP address, which is good
+        pass
+
+    if models.FederatedServer.is_blocked(url):
+        raise ConnectorException(f"Attempting to load data from blocked url: {url}")

bookwyrm/connectors/inventaire.py

@@ -5,7 +5,7 @@ from bookwyrm import models
 from bookwyrm.book_search import SearchResult
 from .abstract_connector import AbstractConnector, Mapping
 from .abstract_connector import get_data
-from .connector_manager import ConnectorException
+from .connector_manager import ConnectorException, create_edition_task
 
 
 class Connector(AbstractConnector):
@@ -77,53 +77,42 @@ class Connector(AbstractConnector):
             **{k: data.get(k) for k in ["uri", "image", "labels", "sitelinks", "type"]},
         }
 
-    def search(self, query, min_confidence=None):  # pylint: disable=arguments-differ
-        """overrides default search function with confidence ranking"""
-        results = super().search(query)
-        if min_confidence:
-            # filter the search results after the fact
-            return [r for r in results if r.confidence >= min_confidence]
-        return results
-
-    def parse_search_data(self, data):
-        return data.get("results")
-
-    def format_search_result(self, search_result):
-        images = search_result.get("image")
-        cover = f"{self.covers_url}/img/entities/{images[0]}" if images else None
-        # a deeply messy translation of inventaire's scores
-        confidence = float(search_result.get("_score", 0.1))
-        confidence = 0.1 if confidence < 150 else 0.999
-        return SearchResult(
-            title=search_result.get("label"),
-            key=self.get_remote_id(search_result.get("uri")),
-            author=search_result.get("description"),
-            view_link=f"{self.base_url}/entity/{search_result.get('uri')}",
-            cover=cover,
-            confidence=confidence,
-            connector=self,
-        )
+    def parse_search_data(self, data, min_confidence):
+        for search_result in data.get("results", []):
+            images = search_result.get("image")
+            cover = f"{self.covers_url}/img/entities/{images[0]}" if images else None
+            # a deeply messy translation of inventaire's scores
+            confidence = float(search_result.get("_score", 0.1))
+            confidence = 0.1 if confidence < 150 else 0.999
+            if confidence < min_confidence:
+                continue
+            yield SearchResult(
+                title=search_result.get("label"),
+                key=self.get_remote_id(search_result.get("uri")),
+                author=search_result.get("description"),
+                view_link=f"{self.base_url}/entity/{search_result.get('uri')}",
+                cover=cover,
+                confidence=confidence,
+                connector=self,
+            )
 
     def parse_isbn_search_data(self, data):
         """got some daaaata"""
         results = data.get("entities")
         if not results:
-            return []
-        return list(results.values())
-
-    def format_isbn_search_result(self, search_result):
-        """totally different format than a regular search result"""
-        title = search_result.get("claims", {}).get("wdt:P1476", [])
-        if not title:
-            return None
-        return SearchResult(
-            title=title[0],
-            key=self.get_remote_id(search_result.get("uri")),
-            author=search_result.get("description"),
-            view_link=f"{self.base_url}/entity/{search_result.get('uri')}",
-            cover=self.get_cover_url(search_result.get("image")),
-            connector=self,
-        )
+            return
+        for search_result in list(results.values()):
+            title = search_result.get("claims", {}).get("wdt:P1476", [])
+            if not title:
+                continue
+            yield SearchResult(
+                title=title[0],
+                key=self.get_remote_id(search_result.get("uri")),
+                author=search_result.get("description"),
+                view_link=f"{self.base_url}/entity/{search_result.get('uri')}",
+                cover=self.get_cover_url(search_result.get("image")),
+                connector=self,
+            )
 
     def is_work_data(self, data):
         return data.get("type") == "work"
@@ -167,12 +156,17 @@ class Connector(AbstractConnector):
 
         for edition_uri in edition_options.get("uris"):
             remote_id = self.get_remote_id(edition_uri)
+            create_edition_task.delay(self.connector.id, work.id, remote_id)
+
+    def create_edition_from_data(self, work, edition_data, instance=None):
+        """pass in the url as data and then call the version in abstract connector"""
+        if isinstance(edition_data, str):
             try:
-                data = self.get_book_data(remote_id)
+                edition_data = self.get_book_data(edition_data)
             except ConnectorException:
                 # who, indeed, knows
-                continue
-            self.create_edition_from_data(work, data)
+                return
+        super().create_edition_from_data(work, edition_data, instance=instance)
 
     def get_cover_url(self, cover_blob, *_):
         """format the relative cover url into an absolute one:

bookwyrm/connectors/openlibrary.py

@@ -5,7 +5,7 @@ from bookwyrm import models
 from bookwyrm.book_search import SearchResult
 from .abstract_connector import AbstractConnector, Mapping
 from .abstract_connector import get_data, infer_physical_format, unique_physical_format
-from .connector_manager import ConnectorException
+from .connector_manager import ConnectorException, create_edition_task
 from .openlibrary_languages import languages
 
 
@@ -152,39 +152,41 @@ class Connector(AbstractConnector):
         image_name = f"{cover_id}-{size}.jpg"
         return f"{self.covers_url}/b/id/{image_name}"
 
-    def parse_search_data(self, data):
-        return data.get("docs")
+    def parse_search_data(self, data, min_confidence):
+        for idx, search_result in enumerate(data.get("docs")):
+            # build the remote id from the openlibrary key
+            key = self.books_url + search_result["key"]
+            author = search_result.get("author_name") or ["Unknown"]
+            cover_blob = search_result.get("cover_i")
+            cover = self.get_cover_url([cover_blob], size="M") if cover_blob else None
 
-    def format_search_result(self, search_result):
-        # build the remote id from the openlibrary key
-        key = self.books_url + search_result["key"]
-        author = search_result.get("author_name") or ["Unknown"]
-        cover_blob = search_result.get("cover_i")
-        cover = self.get_cover_url([cover_blob], size="M") if cover_blob else None
-        return SearchResult(
-            title=search_result.get("title"),
-            key=key,
-            author=", ".join(author),
-            connector=self,
-            year=search_result.get("first_publish_year"),
-            cover=cover,
-        )
+            # OL doesn't provide confidence, but it does sort by an internal ranking, so
+            # this confidence value is relative to the list position
+            confidence = 1 / (idx + 1)
+
+            yield SearchResult(
+                title=search_result.get("title"),
+                key=key,
+                author=", ".join(author),
+                connector=self,
+                year=search_result.get("first_publish_year"),
+                cover=cover,
+                confidence=confidence,
+            )
 
     def parse_isbn_search_data(self, data):
-        return list(data.values())
-
-    def format_isbn_search_result(self, search_result):
-        # build the remote id from the openlibrary key
-        key = self.books_url + search_result["key"]
-        authors = search_result.get("authors") or [{"name": "Unknown"}]
-        author_names = [author.get("name") for author in authors]
-        return SearchResult(
-            title=search_result.get("title"),
-            key=key,
-            author=", ".join(author_names),
-            connector=self,
-            year=search_result.get("publish_date"),
-        )
+        for search_result in list(data.values()):
+            # build the remote id from the openlibrary key
+            key = self.books_url + search_result["key"]
+            authors = search_result.get("authors") or [{"name": "Unknown"}]
+            author_names = [author.get("name") for author in authors]
+            yield SearchResult(
+                title=search_result.get("title"),
+                key=key,
+                author=", ".join(author_names),
+                connector=self,
+                year=search_result.get("publish_date"),
+            )
 
     def load_edition_data(self, olkey):
         """query openlibrary for editions of a work"""
@@ -208,7 +210,7 @@ class Connector(AbstractConnector):
             # does this edition have ANY interesting data?
             if ignore_edition(edition_data):
                 continue
-            self.create_edition_from_data(work, edition_data)
+            create_edition_task.delay(self.connector.id, work.id, edition_data)
 
 
 def ignore_edition(edition_data):

bookwyrm/emailing.py

@@ -45,7 +45,8 @@ def moderation_report_email(report):
     """a report was created"""
     data = email_data()
     data["reporter"] = report.reporter.localname or report.reporter.username
-    data["reportee"] = report.user.localname or report.user.username
+    if report.user:
+        data["reportee"] = report.user.localname or report.user.username
     data["report_link"] = report.remote_id
 
     for admin in models.User.objects.filter(

bookwyrm/forms/__init__.py

@@ -10,3 +10,4 @@ from .landing import *
 from .links import *
 from .lists import *
 from .status import *
+from .user_admin import *

bookwyrm/forms/edit_user.py

@@ -1,5 +1,8 @@
 """ using django model forms """
 from django import forms
+from django.contrib.auth.password_validation import validate_password
+from django.core.exceptions import ValidationError
+from django.utils.translation import gettext_lazy as _
 
 from bookwyrm import models
 from bookwyrm.models.fields import ClearableFileInputWithWarning
@@ -66,3 +69,33 @@ class DeleteUserForm(CustomForm):
     class Meta:
         model = models.User
         fields = ["password"]
+
+
+class ChangePasswordForm(CustomForm):
+    current_password = forms.CharField(widget=forms.PasswordInput)
+    confirm_password = forms.CharField(widget=forms.PasswordInput)
+
+    class Meta:
+        model = models.User
+        fields = ["password"]
+        widgets = {
+            "password": forms.PasswordInput(),
+        }
+
+    def clean(self):
+        """Make sure passwords match and are valid"""
+        current_password = self.data.get("current_password")
+        if not self.instance.check_password(current_password):
+            self.add_error("current_password", _("Incorrect password"))
+
+        cleaned_data = super().clean()
+        new_password = cleaned_data.get("password")
+        confirm_password = self.data.get("confirm_password")
+
+        if new_password != confirm_password:
+            self.add_error("confirm_password", _("Password does not match"))
+
+        try:
+            validate_password(new_password)
+        except ValidationError as err:
+            self.add_error("password", err)

bookwyrm/forms/groups.py

@@ -4,12 +4,6 @@ from .custom_form import CustomForm
 
 
 # pylint: disable=missing-class-docstring
-class UserGroupForm(CustomForm):
-    class Meta:
-        model = models.User
-        fields = ["groups"]
-
-
 class GroupForm(CustomForm):
     class Meta:
         model = models.Group

bookwyrm/forms/landing.py

@@ -1,5 +1,7 @@
 """ Forms for the landing pages """
-from django.forms import PasswordInput
+from django import forms
+from django.contrib.auth.password_validation import validate_password
+from django.core.exceptions import ValidationError
 from django.utils.translation import gettext_lazy as _
 
 from bookwyrm import models
@@ -13,7 +15,7 @@ class LoginForm(CustomForm):
         fields = ["localname", "password"]
         help_texts = {f: None for f in fields}
         widgets = {
-            "password": PasswordInput(),
+            "password": forms.PasswordInput(),
         }
 
 
@@ -22,12 +24,16 @@ class RegisterForm(CustomForm):
         model = models.User
         fields = ["localname", "email", "password"]
         help_texts = {f: None for f in fields}
-        widgets = {"password": PasswordInput()}
+        widgets = {"password": forms.PasswordInput()}
 
     def clean(self):
         """Check if the username is taken"""
         cleaned_data = super().clean()
         localname = cleaned_data.get("localname").strip()
+        try:
+            validate_password(cleaned_data.get("password"))
+        except ValidationError as err:
+            self.add_error("password", err)
         if models.User.objects.filter(localname=localname).first():
             self.add_error("localname", _("User with this username already exists"))
 
@@ -43,3 +49,28 @@ class InviteRequestForm(CustomForm):
     class Meta:
         model = models.InviteRequest
         fields = ["email", "answer"]
+
+
+class PasswordResetForm(CustomForm):
+    confirm_password = forms.CharField(widget=forms.PasswordInput)
+
+    class Meta:
+        model = models.User
+        fields = ["password"]
+        widgets = {
+            "password": forms.PasswordInput(),
+        }
+
+    def clean(self):
+        """Make sure the passwords match and are valid"""
+        cleaned_data = super().clean()
+        new_password = cleaned_data.get("password")
+        confirm_password = self.data.get("confirm_password")
+
+        if new_password != confirm_password:
+            self.add_error("confirm_password", _("Password does not match"))
+
+        try:
+            validate_password(new_password)
+        except ValidationError as err:
+            self.add_error("password", err)

bookwyrm/forms/user_admin.py

@@ -0,0 +1,10 @@
+""" using django model forms """
+from bookwyrm import models
+from .custom_form import CustomForm
+
+
+# pylint: disable=missing-class-docstring
+class UserGroupForm(CustomForm):
+    class Meta:
+        model = models.User
+        fields = ["groups"]

bookwyrm/importers/calibre_import.py

@@ -24,5 +24,5 @@ class CalibreImporter(Importer):
         super().__init__(*args, **kwargs)
 
     def get_shelf(self, normalized_row):
-        # Calibre export does not indicate which shelf to use. Go with a default one for now
+        # Calibre export does not indicate which shelf to use. Use a default one for now
         return Shelf.TO_READ

bookwyrm/lists_stream.py

@@ -114,12 +114,20 @@ class ListsStream(RedisStore):
 
 @receiver(signals.post_save, sender=models.List)
 # pylint: disable=unused-argument
-def add_list_on_create(sender, instance, created, *args, **kwargs):
-    """add newly created lists streamsstreams"""
-    if not created:
+def add_list_on_create(sender, instance, created, *args, update_fields=None, **kwargs):
+    """add newly created lists streams"""
+    if created:
+        # when creating new things, gotta wait on the transaction
+        transaction.on_commit(lambda: add_list_on_create_command(instance.id))
         return
-    # when creating new things, gotta wait on the transaction
-    transaction.on_commit(lambda: add_list_on_create_command(instance.id))
+
+    # if update_fields was specified, we can check if privacy was updated, but if
+    # it wasn't specified (ie, by an activitypub update), there's no way to know
+    if update_fields and "privacy" not in update_fields:
+        return
+
+    # the privacy may have changed, so we need to re-do the whole thing
+    remove_list_task.delay(instance.id, re_add=True)
 
 
 @receiver(signals.post_delete, sender=models.List)
@@ -217,7 +225,7 @@ def populate_lists_task(user_id):
 
 
 @app.task(queue=MEDIUM)
-def remove_list_task(list_id):
+def remove_list_task(list_id, re_add=False):
     """remove a list from any stream it might be in"""
     stores = models.User.objects.filter(local=True, is_active=True).values_list(
         "id", flat=True
@@ -227,6 +235,9 @@ def remove_list_task(list_id):
     stores = [ListsStream().stream_id(idx) for idx in stores]
     ListsStream().remove_object_from_related_stores(list_id, stores=stores)
 
+    if re_add:
+        add_list_task.delay(list_id)
+
 
 @app.task(queue=HIGH)
 def add_list_task(list_id):

bookwyrm/management/commands/generate_preview_images.py

@@ -56,12 +56,17 @@ class Command(BaseCommand):
             self.stdout.write(" OK 🖼")
 
             # Books
-            books = models.Book.objects.select_subclasses().filter()
-            self.stdout.write(
-                "   → Book preview images ({}): ".format(len(books)), ending=""
+            book_ids = (
+                models.Book.objects.select_subclasses()
+                .filter()
+                .values_list("id", flat=True)
             )
-            for book in books:
-                preview_images.generate_edition_preview_image_task.delay(book.id)
+
+            self.stdout.write(
+                "   → Book preview images ({}): ".format(len(book_ids)), ending=""
+            )
+            for book_id in book_ids:
+                preview_images.generate_edition_preview_image_task.delay(book_id)
                 self.stdout.write(".", ending="")
             self.stdout.write(" OK 🖼")
 

bookwyrm/management/commands/initdb.py

@@ -89,7 +89,7 @@ def init_connectors():
         covers_url="https://inventaire.io",
         search_url="https://inventaire.io/api/search?types=works&types=works&search=",
         isbn_search_url="https://inventaire.io/api/entities?action=by-uris&uris=isbn%3A",
-        priority=3,
+        priority=1,
     )
 
     models.Connector.objects.create(
@@ -101,7 +101,7 @@ def init_connectors():
         covers_url="https://covers.openlibrary.org",
         search_url="https://openlibrary.org/search?q=",
         isbn_search_url="https://openlibrary.org/api/books?jscmd=data&format=json&bibkeys=ISBN:",
-        priority=3,
+        priority=1,
     )
 
 

bookwyrm/migrations/0151_alter_report_user.py

@@ -0,0 +1,24 @@
+# Generated by Django 3.2.13 on 2022-07-05 23:54
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookwyrm", "0150_readthrough_stopped_date"),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name="report",
+            name="user",
+            field=models.ForeignKey(
+                null=True,
+                on_delete=django.db.models.deletion.PROTECT,
+                to=settings.AUTH_USER_MODEL,
+            ),
+        ),
+    ]

bookwyrm/migrations/0151_auto_20220705_0049.py

@@ -0,0 +1,90 @@
+# Generated by Django 3.2.13 on 2022-07-05 00:49
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookwyrm", "0150_readthrough_stopped_date"),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name="notification",
+            name="related_book",
+        ),
+        migrations.AddField(
+            model_name="notification",
+            name="related_list_items",
+            field=models.ManyToManyField(
+                related_name="notifications", to="bookwyrm.ListItem"
+            ),
+        ),
+        migrations.AddField(
+            model_name="notification",
+            name="related_reports",
+            field=models.ManyToManyField(to="bookwyrm.Report"),
+        ),
+        migrations.AddField(
+            model_name="notification",
+            name="related_users",
+            field=models.ManyToManyField(
+                related_name="notifications", to=settings.AUTH_USER_MODEL
+            ),
+        ),
+        migrations.AlterField(
+            model_name="notification",
+            name="related_list_item",
+            field=models.ForeignKey(
+                null=True,
+                on_delete=django.db.models.deletion.CASCADE,
+                related_name="notifications_tmp",
+                to="bookwyrm.listitem",
+            ),
+        ),
+        migrations.AlterField(
+            model_name="notification",
+            name="related_report",
+            field=models.ForeignKey(
+                null=True,
+                on_delete=django.db.models.deletion.CASCADE,
+                related_name="notifications_tmp",
+                to="bookwyrm.report",
+            ),
+        ),
+        migrations.RunSQL(
+            sql="""
+            INSERT INTO bookwyrm_notification_related_users (notification_id, user_id)
+            SELECT id, related_user_id
+            FROM bookwyrm_notification
+                WHERE bookwyrm_notification.related_user_id IS NOT NULL;
+
+            INSERT INTO bookwyrm_notification_related_list_items (notification_id, listitem_id)
+            SELECT id, related_list_item_id
+            FROM bookwyrm_notification
+                WHERE bookwyrm_notification.related_list_item_id IS NOT NULL;
+
+            INSERT INTO bookwyrm_notification_related_reports (notification_id, report_id)
+            SELECT id, related_report_id
+            FROM bookwyrm_notification
+                WHERE bookwyrm_notification.related_report_id IS NOT NULL;
+
+            """,
+            reverse_sql=migrations.RunSQL.noop,
+        ),
+        migrations.RemoveField(
+            model_name="notification",
+            name="related_list_item",
+        ),
+        migrations.RemoveField(
+            model_name="notification",
+            name="related_report",
+        ),
+        migrations.RemoveField(
+            model_name="notification",
+            name="related_user",
+        ),
+    ]

bookwyrm/migrations/0152_alter_report_user.py

@@ -0,0 +1,25 @@
+# Generated by Django 3.2.13 on 2022-07-06 19:16
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookwyrm", "0151_alter_report_user"),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name="report",
+            name="user",
+            field=models.ForeignKey(
+                blank=True,
+                null=True,
+                on_delete=django.db.models.deletion.PROTECT,
+                to=settings.AUTH_USER_MODEL,
+            ),
+        ),
+    ]

bookwyrm/migrations/0152_remove_notification_notification_type_valid.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.2.13 on 2022-07-05 03:16
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookwyrm", "0151_auto_20220705_0049"),
+    ]
+
+    operations = [
+        migrations.RemoveConstraint(
+            model_name="notification",
+            name="notification_type_valid",
+        ),
+    ]

bookwyrm/migrations/0153_merge_20220706_2141.py

@@ -0,0 +1,13 @@
+# Generated by Django 3.2.13 on 2022-07-06 21:41
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookwyrm", "0152_alter_report_user"),
+        ("bookwyrm", "0152_remove_notification_notification_type_valid"),
+    ]
+
+    operations = []

bookwyrm/models/antispam.py

@@ -3,7 +3,7 @@ from functools import reduce
 import operator
 
 from django.apps import apps
-from django.db import models
+from django.db import models, transaction
 from django.db.models import Q
 from django.utils.translation import gettext_lazy as _
 
@@ -58,25 +58,20 @@ def automod_task():
         return
     reporter = AutoMod.objects.first().created_by
     reports = automod_users(reporter) + automod_statuses(reporter)
-    if reports:
-        admins = User.objects.filter(
-            models.Q(user_permissions__name__in=["moderate_user", "moderate_post"])
-            | models.Q(is_superuser=True)
-        ).all()
-        notification_model = apps.get_model(
-            "bookwyrm", "Notification", require_ready=True
-        )
+    if not reports:
+        return
+
+    admins = User.objects.filter(
+        models.Q(user_permissions__name__in=["moderate_user", "moderate_post"])
+        | models.Q(is_superuser=True)
+    ).all()
+    notification_model = apps.get_model("bookwyrm", "Notification", require_ready=True)
+    with transaction.atomic():
         for admin in admins:
-            notification_model.objects.bulk_create(
-                [
-                    notification_model(
-                        user=admin,
-                        related_report=r,
-                        notification_type="REPORT",
-                    )
-                    for r in reports
-                ]
+            notification, _ = notification_model.objects.get_or_create(
+                user=admin, notification_type=notification_model.REPORT, read=False
             )
+            notification.related_repors.add(reports)
 
 
 def automod_users(reporter):

bookwyrm/models/base_model.py

@@ -132,7 +132,7 @@ class BookWyrmModel(models.Model):
             return
 
         # but generally moderators can delete other people's stuff
-        if self.user == viewer or viewer.has_perm("moderate_post"):
+        if self.user == viewer or viewer.has_perm("bookwyrm.moderate_post"):
             return
 
         raise PermissionDenied()

bookwyrm/models/fields.py

@@ -16,7 +16,7 @@ from django.utils.encoding import filepath_to_uri
 
 from bookwyrm import activitypub
 from bookwyrm.connectors import get_image
-from bookwyrm.sanitize_html import InputHtmlParser
+from bookwyrm.utils.sanitizer import clean
 from bookwyrm.settings import MEDIA_FULL_URL
 
 
@@ -497,9 +497,7 @@ class HtmlField(ActivitypubFieldMixin, models.TextField):
     def field_from_activity(self, value):
         if not value or value == MISSING:
             return None
-        sanitizer = InputHtmlParser()
-        sanitizer.feed(value)
-        return sanitizer.get_output()
+        return clean(value)
 
 
 class ArrayField(ActivitypubFieldMixin, DjangoArrayField):

bookwyrm/models/group.py

@@ -140,16 +140,6 @@ class GroupMemberInvitation(models.Model):
         # make an invitation
         super().save(*args, **kwargs)
 
-        # now send the invite
-        model = apps.get_model("bookwyrm.Notification", require_ready=True)
-        notification_type = "INVITE"
-        model.objects.create(
-            user=self.user,
-            related_user=self.group.user,
-            related_group=self.group,
-            notification_type=notification_type,
-        )
-
     @transaction.atomic
     def accept(self):
         """turn this request into the real deal"""
@@ -157,25 +147,24 @@ class GroupMemberInvitation(models.Model):
 
         model = apps.get_model("bookwyrm.Notification", require_ready=True)
         # tell the group owner
-        model.objects.create(
-            user=self.group.user,
-            related_user=self.user,
+        model.notify(
+            self.group.user,
+            self.user,
             related_group=self.group,
-            notification_type="ACCEPT",
+            notification_type=model.ACCEPT,
         )
 
         # let the other members know about it
         for membership in self.group.memberships.all():
             member = membership.user
             if member not in (self.user, self.group.user):
-                model.objects.create(
-                    user=member,
-                    related_user=self.user,
+                model.notify(
+                    member,
+                    self.user,
                     related_group=self.group,
-                    notification_type="JOIN",
+                    notification_type=model.JOIN,
                 )
 
     def reject(self):
         """generate a Reject for this membership request"""
-
         self.delete()

bookwyrm/models/link.py

@@ -84,7 +84,7 @@ class LinkDomain(BookWyrmModel):
     )
 
     def raise_not_editable(self, viewer):
-        if viewer.has_perm("moderate_post"):
+        if viewer.has_perm("bookwyrm.moderate_post"):
             return
         raise PermissionDenied()
 

bookwyrm/models/list.py

@@ -1,7 +1,6 @@
 """ make a list of books!! """
 import uuid
 
-from django.apps import apps
 from django.core.exceptions import PermissionDenied
 from django.db import models
 from django.db.models import Q
@@ -129,7 +128,7 @@ class List(OrderedCollectionMixin, BookWyrmModel):
         """on save, update embed_key and avoid clash with existing code"""
         if not self.embed_key:
             self.embed_key = uuid.uuid4()
-        return super().save(*args, **kwargs)
+        super().save(*args, **kwargs)
 
 
 class ListItem(CollectionItemMixin, BookWyrmModel):
@@ -151,33 +150,11 @@ class ListItem(CollectionItemMixin, BookWyrmModel):
     collection_field = "book_list"
 
     def save(self, *args, **kwargs):
-        """create a notification too"""
-        created = not bool(self.id)
+        """Update the list's date"""
         super().save(*args, **kwargs)
         # tick the updated date on the parent list
         self.book_list.updated_date = timezone.now()
-        self.book_list.save(broadcast=False)
-
-        list_owner = self.book_list.user
-        model = apps.get_model("bookwyrm.Notification", require_ready=True)
-        # create a notification if somoene ELSE added to a local user's list
-        if created and list_owner.local and list_owner != self.user:
-            model.objects.create(
-                user=list_owner,
-                related_user=self.user,
-                related_list_item=self,
-                notification_type="ADD",
-            )
-
-        if self.book_list.group:
-            for membership in self.book_list.group.memberships.all():
-                if membership.user != self.user:
-                    model.objects.create(
-                        user=membership.user,
-                        related_user=self.user,
-                        related_list_item=self,
-                        notification_type="ADD",
-                    )
+        self.book_list.save(broadcast=False, update_fields=["updated_date"])
 
     def raise_not_deletable(self, viewer):
         """the associated user OR the list owner can delete"""

bookwyrm/models/notification.py

@@ -1,77 +1,125 @@
 """ alert a user to activity """
-from django.db import models
+from django.db import models, transaction
 from django.dispatch import receiver
 from .base_model import BookWyrmModel
-from . import Boost, Favorite, ImportJob, Report, Status, User
-
-# pylint: disable=line-too-long
-NotificationType = models.TextChoices(
-    "NotificationType",
-    "FAVORITE REPLY MENTION TAG FOLLOW FOLLOW_REQUEST BOOST IMPORT ADD REPORT INVITE ACCEPT JOIN LEAVE REMOVE GROUP_PRIVACY GROUP_NAME GROUP_DESCRIPTION",
-)
+from . import Boost, Favorite, GroupMemberInvitation, ImportJob, ListItem, Report
+from . import Status, User, UserFollowRequest
 
 
 class Notification(BookWyrmModel):
     """you've been tagged, liked, followed, etc"""
 
+    # Status interactions
+    FAVORITE = "FAVORITE"
+    BOOST = "BOOST"
+    REPLY = "REPLY"
+    MENTION = "MENTION"
+    TAG = "TAG"
+
+    # Relationships
+    FOLLOW = "FOLLOW"
+    FOLLOW_REQUEST = "FOLLOW_REQUEST"
+
+    # Imports
+    IMPORT = "IMPORT"
+
+    # List activity
+    ADD = "ADD"
+
+    # Admin
+    REPORT = "REPORT"
+
+    # Groups
+    INVITE = "INVITE"
+    ACCEPT = "ACCEPT"
+    JOIN = "JOIN"
+    LEAVE = "LEAVE"
+    REMOVE = "REMOVE"
+    GROUP_PRIVACY = "GROUP_PRIVACY"
+    GROUP_NAME = "GROUP_NAME"
+    GROUP_DESCRIPTION = "GROUP_DESCRIPTION"
+
+    # pylint: disable=line-too-long
+    NotificationType = models.TextChoices(
+        # there has got be a better way to do this
+        "NotificationType",
+        f"{FAVORITE} {REPLY} {MENTION} {TAG} {FOLLOW} {FOLLOW_REQUEST} {BOOST} {IMPORT} {ADD} {REPORT} {INVITE} {ACCEPT} {JOIN} {LEAVE} {REMOVE} {GROUP_PRIVACY} {GROUP_NAME} {GROUP_DESCRIPTION}",
+    )
+
     user = models.ForeignKey("User", on_delete=models.CASCADE)
-    related_book = models.ForeignKey("Edition", on_delete=models.CASCADE, null=True)
-    related_user = models.ForeignKey(
-        "User", on_delete=models.CASCADE, null=True, related_name="related_user"
+    read = models.BooleanField(default=False)
+    notification_type = models.CharField(
+        max_length=255, choices=NotificationType.choices
+    )
+
+    related_users = models.ManyToManyField(
+        "User", symmetrical=False, related_name="notifications"
     )
     related_group = models.ForeignKey(
         "Group", on_delete=models.CASCADE, null=True, related_name="notifications"
     )
     related_status = models.ForeignKey("Status", on_delete=models.CASCADE, null=True)
     related_import = models.ForeignKey("ImportJob", on_delete=models.CASCADE, null=True)
-    related_list_item = models.ForeignKey(
-        "ListItem", on_delete=models.CASCADE, null=True
-    )
-    related_report = models.ForeignKey("Report", on_delete=models.CASCADE, null=True)
-    read = models.BooleanField(default=False)
-    notification_type = models.CharField(
-        max_length=255, choices=NotificationType.choices
+    related_list_items = models.ManyToManyField(
+        "ListItem", symmetrical=False, related_name="notifications"
     )
+    related_reports = models.ManyToManyField("Report", symmetrical=False)
 
-    def save(self, *args, **kwargs):
-        """save, but don't make dupes"""
-        # there's probably a better way to do this
-        if self.__class__.objects.filter(
-            user=self.user,
-            related_book=self.related_book,
-            related_user=self.related_user,
-            related_group=self.related_group,
-            related_status=self.related_status,
-            related_import=self.related_import,
-            related_list_item=self.related_list_item,
-            related_report=self.related_report,
-            notification_type=self.notification_type,
-        ).exists():
+    @classmethod
+    @transaction.atomic
+    def notify(cls, user, related_user, **kwargs):
+        """Create a notification"""
+        if related_user and (not user.local or user == related_user):
             return
-        super().save(*args, **kwargs)
+        notification = cls.objects.filter(user=user, **kwargs).first()
+        if not notification:
+            notification = cls.objects.create(user=user, **kwargs)
+        if related_user:
+            notification.related_users.add(related_user)
+        notification.read = False
+        notification.save()
 
-    class Meta:
-        """checks if notifcation is in enum list for valid types"""
-
-        constraints = [
-            models.CheckConstraint(
-                check=models.Q(notification_type__in=NotificationType.values),
-                name="notification_type_valid",
+    @classmethod
+    @transaction.atomic
+    def notify_list_item(cls, user, list_item):
+        """Group the notifications around the list items, not the user"""
+        related_user = list_item.user
+        notification = cls.objects.filter(
+            user=user,
+            related_users=related_user,
+            related_list_items__book_list=list_item.book_list,
+            notification_type=Notification.ADD,
+        ).first()
+        if not notification:
+            notification = cls.objects.create(
+                user=user, notification_type=Notification.ADD
             )
-        ]
+            notification.related_users.add(related_user)
+        notification.related_list_items.add(list_item)
+        notification.read = False
+        notification.save()
+
+    @classmethod
+    def unnotify(cls, user, related_user, **kwargs):
+        """Remove a user from a notification and delete it if that was the only user"""
+        try:
+            notification = cls.objects.filter(user=user, **kwargs).get()
+        except Notification.DoesNotExist:
+            return
+        notification.related_users.remove(related_user)
+        if not notification.related_users.count():
+            notification.delete()
 
 
 @receiver(models.signals.post_save, sender=Favorite)
 # pylint: disable=unused-argument
 def notify_on_fav(sender, instance, *args, **kwargs):
     """someone liked your content, you ARE loved"""
-    if not instance.status.user.local or instance.status.user == instance.user:
-        return
-    Notification.objects.create(
-        user=instance.status.user,
-        notification_type="FAVORITE",
-        related_user=instance.user,
+    Notification.notify(
+        instance.status.user,
+        instance.user,
         related_status=instance.status,
+        notification_type=Notification.FAVORITE,
     )
 
 
@@ -81,15 +129,16 @@ def notify_on_unfav(sender, instance, *args, **kwargs):
     """oops, didn't like that after all"""
     if not instance.status.user.local:
         return
-    Notification.objects.filter(
-        user=instance.status.user,
-        related_user=instance.user,
+    Notification.unnotify(
+        instance.status.user,
+        instance.user,
         related_status=instance.status,
-        notification_type="FAVORITE",
-    ).delete()
+        notification_type=Notification.FAVORITE,
+    )
 
 
 @receiver(models.signals.post_save)
+@transaction.atomic
 # pylint: disable=unused-argument
 def notify_user_on_mention(sender, instance, *args, **kwargs):
     """creating and deleting statuses with @ mentions and replies"""
@@ -105,22 +154,23 @@ def notify_user_on_mention(sender, instance, *args, **kwargs):
         and instance.reply_parent.user != instance.user
         and instance.reply_parent.user.local
     ):
-        Notification.objects.create(
-            user=instance.reply_parent.user,
-            notification_type="REPLY",
-            related_user=instance.user,
+        Notification.notify(
+            instance.reply_parent.user,
+            instance.user,
             related_status=instance,
+            notification_type=Notification.REPLY,
         )
+
     for mention_user in instance.mention_users.all():
         # avoid double-notifying about this status
         if not mention_user.local or (
             instance.reply_parent and mention_user == instance.reply_parent.user
         ):
             continue
-        Notification.objects.create(
-            user=mention_user,
-            notification_type="MENTION",
-            related_user=instance.user,
+        Notification.notify(
+            mention_user,
+            instance.user,
+            notification_type=Notification.MENTION,
             related_status=instance,
         )
 
@@ -135,11 +185,11 @@ def notify_user_on_boost(sender, instance, *args, **kwargs):
     ):
         return
 
-    Notification.objects.create(
-        user=instance.boosted_status.user,
+    Notification.notify(
+        instance.boosted_status.user,
+        instance.user,
         related_status=instance.boosted_status,
-        related_user=instance.user,
-        notification_type="BOOST",
+        notification_type=Notification.BOOST,
     )
 
 
@@ -147,12 +197,12 @@ def notify_user_on_boost(sender, instance, *args, **kwargs):
 # pylint: disable=unused-argument
 def notify_user_on_unboost(sender, instance, *args, **kwargs):
     """unboosting a status"""
-    Notification.objects.filter(
-        user=instance.boosted_status.user,
+    Notification.unnotify(
+        instance.boosted_status.user,
+        instance.user,
         related_status=instance.boosted_status,
-        related_user=instance.user,
-        notification_type="BOOST",
-    ).delete()
+        notification_type=Notification.BOOST,
+    )
 
 
 @receiver(models.signals.post_save, sender=ImportJob)
@@ -166,23 +216,94 @@ def notify_user_on_import_complete(
         return
     Notification.objects.create(
         user=instance.user,
-        notification_type="IMPORT",
+        notification_type=Notification.IMPORT,
         related_import=instance,
     )
 
 
 @receiver(models.signals.post_save, sender=Report)
+@transaction.atomic
 # pylint: disable=unused-argument
-def notify_admins_on_report(sender, instance, *args, **kwargs):
+def notify_admins_on_report(sender, instance, created, *args, **kwargs):
     """something is up, make sure the admins know"""
+    if not created:
+        # otherwise you'll get a notification when you resolve a report
+        return
+
     # moderators and superusers should be notified
     admins = User.objects.filter(
         models.Q(user_permissions__name__in=["moderate_user", "moderate_post"])
         | models.Q(is_superuser=True)
     ).all()
     for admin in admins:
-        Notification.objects.create(
+        notification, _ = Notification.objects.get_or_create(
             user=admin,
-            related_report=instance,
-            notification_type="REPORT",
+            notification_type=Notification.REPORT,
+            read=False,
+        )
+        notification.related_reports.add(instance)
+
+
+@receiver(models.signals.post_save, sender=GroupMemberInvitation)
+# pylint: disable=unused-argument
+def notify_user_on_group_invite(sender, instance, *args, **kwargs):
+    """Cool kids club here we come"""
+    Notification.notify(
+        instance.user,
+        instance.group.user,
+        related_group=instance.group,
+        notification_type=Notification.INVITE,
+    )
+
+
+@receiver(models.signals.post_save, sender=ListItem)
+@transaction.atomic
+# pylint: disable=unused-argument
+def notify_user_on_list_item_add(sender, instance, created, *args, **kwargs):
+    """Someone added to your list"""
+    if not created:
+        return
+
+    list_owner = instance.book_list.user
+    # create a notification if somoene ELSE added to a local user's list
+    if list_owner.local and list_owner != instance.user:
+        # keep the related_user singular, group the items
+        Notification.notify_list_item(list_owner, instance)
+
+    if instance.book_list.group:
+        for membership in instance.book_list.group.memberships.all():
+            if membership.user != instance.user:
+                Notification.notify_list_item(membership.user, instance)
+
+
+@receiver(models.signals.post_save, sender=UserFollowRequest)
+@transaction.atomic
+# pylint: disable=unused-argument
+def notify_user_on_follow(sender, instance, created, *args, **kwargs):
+    """Someone added to your list"""
+    if not created or not instance.user_object.local:
+        return
+
+    manually_approves = instance.user_object.manually_approves_followers
+    if manually_approves:
+        # don't group notifications
+        notification = Notification.objects.filter(
+            user=instance.user_object,
+            related_users=instance.user_subject,
+            notification_type=Notification.FOLLOW_REQUEST,
+        ).first()
+        if not notification:
+            notification = Notification.objects.create(
+                user=instance.user_object, notification_type=Notification.FOLLOW_REQUEST
+            )
+        notification.related_users.set([instance.user_subject])
+        notification.read = False
+        notification.save()
+    else:
+        # Only group unread follows
+        Notification.notify(
+            instance.user_object,
+            instance.user_subject,
+            notification_type=Notification.FOLLOW,
+            read=False,
         )

bookwyrm/models/relationship.py

@@ -1,5 +1,4 @@
 """ defines relationships between users """
-from django.apps import apps
 from django.core.cache import cache
 from django.db import models, transaction, IntegrityError
 from django.db.models import Q
@@ -148,14 +147,6 @@ class UserFollowRequest(ActivitypubMixin, UserRelationship):
             if not manually_approves:
                 self.accept()
 
-            model = apps.get_model("bookwyrm.Notification", require_ready=True)
-            notification_type = "FOLLOW_REQUEST" if manually_approves else "FOLLOW"
-            model.objects.create(
-                user=self.user_object,
-                related_user=self.user_subject,
-                notification_type=notification_type,
-            )
-
     def get_accept_reject_id(self, status):
         """get id for sending an accept or reject of a local user"""
 
@@ -218,7 +209,7 @@ def clear_cache(user_subject, user_object):
     """clear relationship cache"""
     cache.delete_many(
         [
-            f"relationship-{user_subject.id}-{user_object.id}",
-            f"relationship-{user_object.id}-{user_subject.id}",
+            f"cached-relationship-{user_subject.id}-{user_object.id}",
+            f"cached-relationship-{user_object.id}-{user_subject.id}",
         ]
     )

bookwyrm/models/report.py

@@ -11,7 +11,7 @@ class Report(BookWyrmModel):
         "User", related_name="reporter", on_delete=models.PROTECT
     )
     note = models.TextField(null=True, blank=True)
-    user = models.ForeignKey("User", on_delete=models.PROTECT)
+    user = models.ForeignKey("User", on_delete=models.PROTECT, null=True, blank=True)
     status = models.ForeignKey(
         "Status",
         null=True,

bookwyrm/models/shelf.py

@@ -103,12 +103,25 @@ class ShelfBook(CollectionItemMixin, BookWyrmModel):
         if not self.user:
             self.user = self.shelf.user
         if self.id and self.user.local:
-            cache.delete(f"book-on-shelf-{self.book.id}-{self.shelf.id}")
+            # remove all caches related to all editions of this book
+            cache.delete_many(
+                [
+                    f"book-on-shelf-{book.id}-{self.shelf.id}"
+                    for book in self.book.parent_work.editions.all()
+                ]
+            )
         super().save(*args, **kwargs)
 
     def delete(self, *args, **kwargs):
         if self.id and self.user.local:
-            cache.delete(f"book-on-shelf-{self.book.id}-{self.shelf.id}")
+            cache.delete_many(
+                [
+                    f"book-on-shelf-{book}-{self.shelf.id}"
+                    for book in self.book.parent_work.editions.values_list(
+                        "id", flat=True
+                    )
+                ]
+            )
         super().delete(*args, **kwargs)
 
     class Meta:

bookwyrm/models/status.py

@@ -303,10 +303,17 @@ class Comment(BookStatus):
     @property
     def pure_content(self):
         """indicate the book in question for mastodon (or w/e) users"""
-        return (
-            f'{self.content}<p>(comment on <a href="{self.book.remote_id}">'
-            f'"{self.book.title}"</a>)</p>'
-        )
+        if self.progress_mode == "PG" and self.progress and (self.progress > 0):
+            return_value = (
+                f'{self.content}<p>(comment on <a href="{self.book.remote_id}">'
+                f'"{self.book.title}"</a>, page {self.progress})</p>'
+            )
+        else:
+            return_value = (
+                f'{self.content}<p>(comment on <a href="{self.book.remote_id}">'
+                f'"{self.book.title}"</a>)</p>'
+            )
+        return return_value
 
     activity_serializer = activitypub.Comment
 
@@ -332,10 +339,17 @@ class Quotation(BookStatus):
         """indicate the book in question for mastodon (or w/e) users"""
         quote = re.sub(r"^<p>", '<p>"', self.quote)
         quote = re.sub(r"</p>$", '"</p>', quote)
-        return (
-            f'{quote} <p>-- <a href="{self.book.remote_id}">'
-            f'"{self.book.title}"</a></p>{self.content}'
-        )
+        if self.position_mode == "PG" and self.position and (self.position > 0):
+            return_value = (
+                f'{quote} <p>-- <a href="{self.book.remote_id}">'
+                f'"{self.book.title}"</a>, page {self.position}</p>{self.content}'
+            )
+        else:
+            return_value = (
+                f'{quote} <p>-- <a href="{self.book.remote_id}">'
+                f'"{self.book.title}"</a></p>{self.content}'
+            )
+        return return_value
 
     activity_serializer = activitypub.Quotation
 
@@ -374,7 +388,7 @@ class Review(BookStatus):
     def save(self, *args, **kwargs):
         """clear rating caches"""
         if self.book.parent_work:
-            cache.delete(f"book-rating-{self.book.parent_work.id}-*")
+            cache.delete(f"book-rating-{self.book.parent_work.id}")
         super().save(*args, **kwargs)
 
 

bookwyrm/sanitize_html.py

@@ -1,71 +0,0 @@
-""" html parser to clean up incoming text from unknown sources """
-from html.parser import HTMLParser
-
-
-class InputHtmlParser(HTMLParser):  # pylint: disable=abstract-method
-    """Removes any html that isn't allowed_tagsed from a block"""
-
-    def __init__(self):
-        HTMLParser.__init__(self)
-        self.allowed_tags = [
-            "p",
-            "blockquote",
-            "br",
-            "b",
-            "i",
-            "strong",
-            "em",
-            "pre",
-            "a",
-            "span",
-            "ul",
-            "ol",
-            "li",
-        ]
-        self.allowed_attrs = ["href", "rel", "src", "alt"]
-        self.tag_stack = []
-        self.output = []
-        # if the html appears invalid, we just won't allow any at all
-        self.allow_html = True
-
-    def handle_starttag(self, tag, attrs):
-        """check if the tag is valid"""
-        if self.allow_html and tag in self.allowed_tags:
-            allowed_attrs = " ".join(
-                f'{a}="{v}"' for a, v in attrs if a in self.allowed_attrs
-            )
-            reconstructed = f"<{tag}"
-            if allowed_attrs:
-                reconstructed += " " + allowed_attrs
-            reconstructed += ">"
-            self.output.append(("tag", reconstructed))
-            self.tag_stack.append(tag)
-        else:
-            self.output.append(("data", ""))
-
-    def handle_endtag(self, tag):
-        """keep the close tag"""
-        if not self.allow_html or tag not in self.allowed_tags:
-            self.output.append(("data", ""))
-            return
-
-        if not self.tag_stack or self.tag_stack[-1] != tag:
-            # the end tag doesn't match the most recent start tag
-            self.allow_html = False
-            self.output.append(("data", ""))
-            return
-
-        self.tag_stack = self.tag_stack[:-1]
-        self.output.append(("tag", f"</{tag}>"))
-
-    def handle_data(self, data):
-        """extract the answer, if we're in an answer tag"""
-        self.output.append(("data", data))
-
-    def get_output(self):
-        """convert the output from a list of tuples to a string"""
-        if self.tag_stack:
-            self.allow_html = False
-        if not self.allow_html:
-            return "".join(v for (k, v) in self.output if k == "data")
-        return "".join(v for (k, v) in self.output)

bookwyrm/settings.py

@@ -11,7 +11,7 @@ from django.utils.translation import gettext_lazy as _
 env = Env()
 env.read_env()
 DOMAIN = env("DOMAIN")
-VERSION = "0.4.0"
+VERSION = "0.4.3"
 
 RELEASE_API = env(
     "RELEASE_API",
@@ -217,7 +217,7 @@ STREAMS = [
 
 # Search configuration
 # total time in seconds that the instance will spend searching connectors
-SEARCH_TIMEOUT = int(env("SEARCH_TIMEOUT", 15))
+SEARCH_TIMEOUT = int(env("SEARCH_TIMEOUT", 8))
 # timeout for a query to an individual connector
 QUERY_TIMEOUT = int(env("QUERY_TIMEOUT", 5))
 

bookwyrm/static/css/bookwyrm/components/_book_list.scss

@@ -6,11 +6,11 @@ ol.ordered-list {
     counter-reset: list-counter;
 }
 
-ol.ordered-list li {
+ol.ordered-list > li {
     counter-increment: list-counter;
 }
 
-ol.ordered-list li::before {
+ol.ordered-list > li::before {
     content: counter(list-counter);
     position: absolute;
     left: -20px;

bookwyrm/status.py

@@ -2,15 +2,13 @@
 from django.db import transaction
 
 from bookwyrm import models
-from bookwyrm.sanitize_html import InputHtmlParser
+from bookwyrm.utils import sanitizer
 
 
 def create_generated_note(user, content, mention_books=None, privacy="public"):
     """a note created by the app about user activity"""
     # sanitize input html
-    parser = InputHtmlParser()
-    parser.feed(content)
-    content = parser.get_output()
+    content = sanitizer.clean(content)
 
     with transaction.atomic():
         # create but don't save

bookwyrm/templates/about/layout.html

@@ -50,7 +50,7 @@
         </ul>
     </nav>
 
-    <div class="column">
+    <div class="column is-clipped">
         {% block about_content %}{% endblock %}
     </div>
 </div>

bookwyrm/templates/book/file_links/edit_links.html

@@ -42,7 +42,11 @@
                     <a href="{{ link.url }}" target="_blank" rel="noopener noreferrer">{{ link.url }}</a>
                 </td>
                 <td>
+                    {% if link.added_by %}
                     <a href="{% url 'user-feed' link.added_by.id %}">{{ link.added_by.display_name }}</a>
+                    {% else %}
+                    <em>{% trans "Unknown user" %}</em>
+                    {% endif %}
                 </td>
                 <td>
                     {{ link.filelink.filetype }}
@@ -50,7 +54,7 @@
                 <td>
                     {{ link.domain.name }}
                     <p>
-                        <a href="{% url 'report-link' link.added_by.id link.id %}">{% trans "Report spam" %}</a>
+                        <a href="{% url 'report-link' link.id %}">{% trans "Report spam" %}</a>
                     </p>
                 </td>
                 <td>

bookwyrm/templates/book/file_links/verification_modal.html

@@ -19,7 +19,7 @@ Is that where you'd like to go?
 {% block modal-footer %}
 {% if request.user.is_authenticated %}
 <div class="is-flex-grow-1">
-    <a href="{% url 'report-link' link.added_by.id link.id %}">{% trans "Report spam" %}</a>
+    <a href="{% url 'report-link' link.id %}">{% trans "Report spam" %}</a>
 </div>
 
 <button type="button" class="button" data-modal-close>{% trans "Cancel" %}</button>

bookwyrm/templates/confirm_email/resend_modal.html

@@ -19,16 +19,8 @@
             name="email"
             class="input"
             id="email"
-            aria-described-by="id_email_errors"
             required
         >
-        {% if error %}
-        <div id="id_email_errors">
-            <p class="help is-danger">
-                {% trans "No user matching this email address found." %}
-            </p>
-        </div>
-        {% endif %}
     </div>
 </div>
 {% endblock %}

bookwyrm/templates/email/moderation_report/html_content.html

@@ -3,7 +3,19 @@
 
 {% block content %}
 <p>
-{% blocktrans %}@{{ reporter }} has flagged behavior by @{{ reportee }} for moderation. {% endblocktrans %}
+{% if report_link %}
+
+    {% blocktrans trimmed %}
+    @{{ reporter }} has flagged a link domain for moderation.
+    {% endblocktrans %}
+
+{% else %}
+
+    {% blocktrans trimmed %}
+    @{{ reporter }} has flagged behavior by @{{ reportee }} for moderation.
+    {% endblocktrans %}
+
+{% endif %}
 </p>
 
 {% trans "View report" as text %}

bookwyrm/templates/email/moderation_report/text_content.html

@@ -2,7 +2,15 @@
 {% load i18n %}
 {% block content %}
 
-{% blocktrans %}@{{ reporter}} has flagged behavior by @{{ reportee }} for moderation. {% endblocktrans %}
+{% if report_link %}
+{% blocktrans trimmed %}
+@{{ reporter }} has flagged a link domain for moderation.
+{% endblocktrans %}
+{% else %}
+{% blocktrans trimmed %}
+@{{ reporter }} has flagged behavior by @{{ reportee }} for moderation.
+{% endblocktrans %}
+{% endif %}
 
 {% trans "View report" %}
 {{ report_link }}

bookwyrm/templates/feed/direct_messages.html

@@ -14,7 +14,7 @@
 </header>
 
 <div class="box">
-    {% include 'snippets/create_status/status.html' with type="direct" uuid=1 mention=partner no_script=True %}
+    {% include 'snippets/create_status/status.html' with type="direct" uuid=1 mention=partner %}
 </div>
 
 <section class="block">

bookwyrm/templates/landing/password_reset.html

@@ -26,7 +26,16 @@
                         {% trans "Password:" %}
                     </label>
                     <div class="control">
-                        <input type="password" name="password" maxlength="128" class="input" required="" id="id_new_password" aria-describedby="form_errors">
+                        <input
+                            type="password"
+                            name="password"
+                            maxlength="128"
+                            class="input"
+                            required=""
+                            id="id_new_password"
+                            aria-describedby="desc_password"
+                        >
+                        {% include 'snippets/form_errors.html' with errors_list=form.password.errors id="desc_password" %}
                     </div>
                 </div>
                 <div class="field">
@@ -34,7 +43,8 @@
                         {% trans "Confirm password:" %}
                     </label>
                     <div class="control">
-                        <input type="password" name="confirm-password" maxlength="128" class="input" required="" id="id_confirm_password" aria-describedby="form_errors">
+                        {{ form.confirm_password }}
+                        {% include 'snippets/form_errors.html' with errors_list=form.confirm_password.errors id="desc_confirm_password" %}
                     </div>
                 </div>
                 <div class="field is-grouped">

bookwyrm/templates/landing/password_reset_request.html

@@ -9,7 +9,13 @@
         <div class="block">
             <h1 class="title">{% trans "Reset Password" %}</h1>
 
-            {% if message %}<p class="notification is-primary">{{ message }}</p>{% endif %}
+            {% if sent_message %}
+            <p class="notification is-primary">
+            {% blocktrans trimmed %}
+            A password reset link will be sent to <strong>{{ email }}</strong> if there is an account using that email address.
+            {% endblocktrans %}
+            </p>
+            {% endif %}
 
             <p>{% trans "A link to reset your password will be sent to your email address" %}</p>
             <form name="password-reset" method="post" action="/password-reset">

bookwyrm/templates/notifications/items/accept.html

@@ -13,8 +13,34 @@
 
 {% block description %}
 
+{% if other_user_count == 0 %}
+
     {% blocktrans trimmed with group_name=notification.related_group.name group_path=notification.related_group.local_path %}
-    accepted your invitation to join group "<a href="{{ group_path }}">{{ group_name }}</a>"
+    <a href="{{ related_user_link }}">{{ related_user }}</a>
+    accepted your invitation to join group
+    "<a href="{{ group_path }}">{{ group_name }}</a>"
     {% endblocktrans %}
 
+{% elif other_user_count == 1 %}
+
+    {% blocktrans trimmed with group_name=notification.related_group.name group_path=notification.related_group.local_path %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a>
+    and
+    <a href="{{ second_user_link }}">{{ second_user }}</a>
+    accepted your invitation to join group
+    "<a href="{{ group_path }}">{{ group_name }}</a>"
+    {% endblocktrans %}
+
+{% else %}
+
+    {% blocktrans trimmed with group_name=notification.related_group.name group_path=notification.related_group.local_path %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a>
+    and
+    {{ other_user_display_count }} others
+    accepted your invitation to join group
+    "<a href="{{ group_path }}">{{ group_name }}</a>"
+    {% endblocktrans %}
+
+{% endif %}
+
 {% endblock %}

bookwyrm/templates/notifications/items/add.html

@@ -1,14 +1,16 @@
 {% extends 'notifications/items/layout.html' %}
-
 {% load i18n %}
 {% load utilities %}
+{% load humanize %}
 
 {% block primary_link %}{% spaceless %}
-{% if notification.related_list_item.approved %}
-    {{ notification.related_list_item.book_list.local_path }}
+{% with related_list=notification.related_list_items.first.book_list %}
+{% if related_list.curation != "curated" %}
+    {{ related_list.local_path }}
 {% else %}
-    {% url 'list-curate' notification.related_list_item.book_list.id %}
+    {% url 'list-curate' related_list.id %}
 {% endif %}
+{% endwith %}
 {% endspaceless %}{% endblock %}
 
 {% block icon %}
@@ -16,25 +18,89 @@
 {% endblock %}
 
 {% block description %}
-{% with book_path=notification.related_list_item.book.local_path %}
-{% with book_title=notification.related_list_item.book|book_title %}
-{% with list_name=notification.related_list_item.book_list.name %}
+{% with related_list=notification.related_list_items.first.book_list %}
+{% with book_path=notification.related_list_items.first.book.local_path %}
+{% with book_title=notification.related_list_items.first.book|book_title %}
+{% with second_book_path=notification.related_list_items.all.1.book.local_path %}
+{% with second_book_title=notification.related_list_items.all.1.book|book_title %}
+{% with list_name=related_list.name %}
 
-    {% if notification.related_list_item.approved %}
-    {% blocktrans trimmed with list_path=notification.related_list_item.book_list.local_path %}
+{% url 'list' related_list.id as list_path %}
+{% url 'list-curate' related_list.id as list_curate_path %}
 
-    added <em><a href="{{ book_path }}">{{ book_title }}</a></em> to your list "<a href="{{ list_path }}">{{ list_name }}</a>"
-
-    {% endblocktrans %}
+{% if notification.related_list_items.count == 1 %}
+    {% if related_list.curation != "curated" %}
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        added <em><a href="{{ book_path }}">{{ book_title }}</a></em>
+        to your list "<a href="{{ list_path }}">{{ list_name }}</a>"
+        {% endblocktrans %}
     {% else %}
-    {% url 'list-curate' notification.related_list_item.book_list.id as list_path %}
-    {% blocktrans trimmed with list_path=list_path %}
-
-    suggested adding <em><a href="{{ book_path }}">{{ book_title }}</a></em> to your list "<a href="{{ list_path }}">{{ list_name }}</a>"
-
-    {% endblocktrans %}
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        suggested adding <em><a href="{{ book_path }}">{{ book_title }}</a></em>
+        to your list "<a href="{{ list_curate_path }}">{{ list_name }}</a>"
+        {% endblocktrans %}
     {% endif %}
+{% elif notification.related_list_items.count == 2 %}
+    {% if related_list.curation != "curated" %}
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        added <em><a href="{{ book_path }}">{{ book_title }}</a></em>
+        and <em><a href="{{ second_book_path }}">{{ second_book_title }}</a></em>
+        to your list "<a href="{{ list_path }}">{{ list_name }}</a>"
+        {% endblocktrans %}
+    {% else %}
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        suggested adding <em><a href="{{ book_path }}">{{ book_title }}</a></em>
+        and <em><a href="{{ second_book_path }}">{{ second_book_title }}</a></em>
+        to your list "<a href="{{ list_curate_path }}">{{ list_name }}</a>"
+        {% endblocktrans %}
+    {% endif %}
+{% else %}
+    {% with count=notification.related_list_items.count|add:"-2" %}
+    {% with display_count=count|intcomma %}
+    {% if related_list.curation != "curated" %}
 
+        {% blocktrans trimmed count counter=count %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        added <em><a href="{{ book_path }}">{{ book_title }}</a></em>,
+        <em><a href="{{ second_book_path }}">{{ second_book_title }}</a></em>,
+        and {{ display_count }} other book
+        to your list "<a href="{{ list_path }}">{{ list_name }}</a>"
+        {% plural %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        added <em><a href="{{ book_path }}">{{ book_title }}</a></em>,
+        <em><a href="{{ second_book_path }}">{{ second_book_title }}</a></em>,
+        and {{ display_count }} other books
+        to your list "<a href="{{ list_path }}">{{ list_name }}</a>"
+        {% endblocktrans %}
+
+    {% else %}
+
+        {% blocktrans trimmed count counter=count %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        suggested adding <em><a href="{{ book_path }}">{{ book_title }}</a></em>,
+        <em><a href="{{ second_book_path }}">{{ second_book_title }}</a></em>,
+        and {{ display_count }} other book
+        to your list "<a href="{{ list_curate_path }}">{{ list_name }}</a>"
+        {% plural %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        suggested adding <em><a href="{{ book_path }}">{{ book_title }}</a></em>,
+        <em><a href="{{ second_book_path }}">{{ second_book_title }}</a></em>,
+        and {{ display_count }} other books
+        to your list "<a href="{{ list_curate_path }}">{{ list_name }}</a>"
+        {% endblocktrans %}
+
+    {% endif %}
+    {% endwith %}
+    {% endwith %}
+{% endif %}
+
+{% endwith %}
+{% endwith %}
+{% endwith %}
 {% endwith %}
 {% endwith %}
 {% endwith %}

bookwyrm/templates/notifications/items/boost.html

@@ -16,29 +16,97 @@
     {% with related_status.local_path as related_path %}
 
     {% if related_status.status_type == 'Review' %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    boosted your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> boosted your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
 
-    {% endblocktrans %}
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        boosted your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others boosted your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% endif %}
     {% elif related_status.status_type == 'Comment' %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    boosted your <a href="{{ related_path }}">comment on<em>{{ book_title }}</em></a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> boosted your <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
 
-    {% endblocktrans %}
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        boosted your <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others boosted your <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% endif %}
     {% elif related_status.status_type == 'Quotation' %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    boosted your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> boosted your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
 
-    {% endblocktrans %}
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        boosted your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others boosted your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% endif %}
     {% else %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    boosted your <a href="{{ related_path }}">status</a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> boosted your <a href="{{ related_path }}">status</a>
+        {% endblocktrans %}
 
-    {% endblocktrans %}
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        boosted your <a href="{{ related_path }}">status</a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others boosted your <a href="{{ related_path }}">status</a>
+        {% endblocktrans %}
+
+        {% endif %}
     {% endif %}
 
     {% endwith %}
@@ -50,7 +118,7 @@
 <div class="notification py-2 {% if notification.id in unread %}is-primary is-light{% else %}has-background-body has-text-muted{% endif %}">
     <div class="columns">
         <div class="column is-clipped">
-            {% include 'snippets/status_preview.html' with status=related_status %}
+            {% include 'notifications/items/status_preview.html' with status=related_status %}
         </div>
         <div class="column is-narrow has-text-muted">
             {{ related_status.published_date|timesince }}

bookwyrm/templates/notifications/items/fav.html

@@ -16,29 +16,98 @@
     {% with related_status.local_path as related_path %}
 
     {% if related_status.status_type == 'Review' %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    liked your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> liked your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
 
-    {% endblocktrans %}
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        liked your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others liked your <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% endif %}
     {% elif related_status.status_type == 'Comment' %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    liked your <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> liked your <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
 
-    {% endblocktrans %}
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        liked your <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others liked your <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% endif %}
     {% elif related_status.status_type == 'Quotation' %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    liked your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> liked your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
 
-    {% endblocktrans %}
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        liked your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others liked your <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        {% endblocktrans %}
+
+        {% endif %}
     {% else %}
-    {% blocktrans trimmed %}
+        {% if other_user_count == 0 %}
 
-    liked your <a href="{{ related_path }}">status</a>
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> liked your <a href="{{ related_path }}">status</a>
+        {% endblocktrans %}
+
+        {% elif other_user_count == 1 %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a>
+        and
+        <a href="{{ second_user_link }}">{{ second_user }}</a>
+        liked your <a href="{{ related_path }}">status</a>
+        {% endblocktrans %}
+
+        {% else %}
+
+        {% blocktrans trimmed %}
+        <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others liked your <a href="{{ related_path }}">status</a>
+        {% endblocktrans %}
+
+        {% endif %}
 
-    {% endblocktrans %}
     {% endif %}
 
     {% endwith %}
@@ -50,7 +119,7 @@
 <div class="notification py-2 {% if notification.id in unread %}is-primary is-light{% else %}has-background-body has-text-muted{% endif %}">
     <div class="columns">
         <div class="column is-clipped">
-            {% include 'snippets/status_preview.html' with status=related_status %}
+            {% include 'notifications/items/status_preview.html' with status=related_status %}
         </div>
         <div class="column is-narrow has-text-muted">
             {{ related_status.published_date|timesince }}

bookwyrm/templates/notifications/items/follow.html

@@ -4,7 +4,7 @@
 {% load utilities %}
 
 {% block primary_link %}{% spaceless %}
-    {{ notification.related_user.local_path }}
+{% url 'user-followers' request.user.localname %}
 {% endspaceless %}{% endblock %}
 
 {% block icon %}
@@ -12,6 +12,19 @@
 {% endblock %}
 
 {% block description %}
-    {% trans "followed you" %}
-    {% include 'snippets/follow_button.html' with user=notification.related_user %}
+{% if related_user_count == 1 %}
+    {% blocktrans trimmed %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a> followed you
+    {% endblocktrans %}
+{% elif related_user_count == 2 %}
+    {% blocktrans trimmed %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a> and
+    <a href="{{ second_user_link }}">{{ second_user }}</a> followed you
+    {% endblocktrans %}
+{% else %}
+    {% blocktrans trimmed %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a> and {{ other_user_display_count }} others followed you
+    {% endblocktrans %}
+{% endif %}
+
 {% endblock %}

bookwyrm/templates/notifications/items/follow_request.html

@@ -3,13 +3,19 @@
 {% load i18n %}
 {% load utilities %}
 
+{% block primary_link %}{% spaceless %}
+{% url 'user-followers' request.user.localname %}
+{% endspaceless %}{% endblock %}
+
 {% block icon %}
 <span class="icon icon-local"></span>
 {% endblock %}
 
 {% block description %}
-    {% trans "sent you a follow request" %}
+    {% blocktrans trimmed %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a> sent you a follow request
+    {% endblocktrans %}
     <div class="row shrink">
-        {% include 'snippets/follow_request_buttons.html' with user=notification.related_user %}
+        {% include 'snippets/follow_request_buttons.html' with user=notification.related_users.first %}
     </div>
 {% endblock %}

bookwyrm/templates/notifications/items/invite.html

@@ -12,11 +12,15 @@
 {% endblock %}
 
 {% block description %}
+
 {% blocktrans trimmed with group_name=notification.related_group.name group_path=notification.related_group.local_path %}
-invited you to join the group "<a href="{{ group_path }}">{{ group_name }}</a>"
+    <a href="{{ related_user_link }}">{{ related_user }}</a>
+    invited you to join the group
+    "<a href="{{ group_path }}">{{ group_name }}</a>"
 {% endblocktrans %}
+
 <div class="row shrink">
     {% include 'snippets/join_invitation_buttons.html' with group=notification.related_group %}
 </div>
-{% endblock %}
 
+{% endblock %}

bookwyrm/templates/notifications/items/layout.html

@@ -1,5 +1,9 @@
 {% load notification_page_tags %}
+{% load humanize %}
 {% related_status notification as related_status %}
+
+{% get_related_users notification as related_users %}
+{% with related_user_count=notification.related_users.count %}
 <div class="notification {% if notification.id in unread %}has-background-primary{% endif %}">
     <div class="columns is-mobile {% if notification.id in unread %}has-text-white{% else %}has-text-more-muted{% endif %}">
         <div class="column is-narrow is-size-3">
@@ -9,14 +13,43 @@
         </div>
 
         <div class="column is-clipped">
+            {% if related_user_count > 1 %}
+            <div class="block">
+                <ul class="is-flex">
+                    {% for user in related_users %}
+                    <li class="mr-2">
+                        <a href="{{ user.local_path }}">
+                            {% include 'snippets/avatar.html' with user=user %}
+                        </a>
+                    </li>
+                    {% endfor %}
+                </ul>
+            </div>
+            {% endif %}
             <div class="block content">
-                <p>
-                    {% if notification.related_user %}
-                        <a href="{{ notification.related_user.local_path }}">{% include 'snippets/avatar.html' with user=notification.related_user %}
-                            {{ notification.related_user.display_name }}</a>
-                    {% endif %}
+                {% if related_user_count == 1 %}
+                    {% with user=related_users.0 %}
+                    {% spaceless %}
+                        <a href="{{ user.local_path }}" class="mr-2">
+                            {% include 'snippets/avatar.html' with user=user %}
+                        </a>
+                        {% endspaceless %}
+                    {% endwith %}
+                {% endif %}
+
+                {% with related_user=related_users.0.display_name %}
+                {% with related_user_link=related_users.0.local_path %}
+                {% with second_user=related_users.1.display_name %}
+                {% with second_user_link=related_users.1.local_path %}
+                {% with other_user_count=related_user_count|add:"-1" %}
+                {% with other_user_display_count=other_user_count|intcomma %}
                     {% block description %}{% endblock %}
-                </p>
+                {% endwith %}
+                {% endwith %}
+                {% endwith %}
+                {% endwith %}
+                {% endwith %}
+                {% endwith %}
             </div>
 
             {% if related_status %}
@@ -27,4 +60,4 @@
         </div>
     </div>
 </div>
-
+{% endwith %}

bookwyrm/templates/notifications/items/leave.html

@@ -13,8 +13,34 @@
 
 {% block description %}
 
+{% if other_user_count == 0 %}
+
     {% blocktrans trimmed with group_name=notification.related_group.name group_path=notification.related_group.local_path %}
-    has left your group "<a href="{{ group_path }}">{{ group_name }}</a>"
+    <a href="{{ related_user_link }}">{{ related_user }}</a>
+    has left your group
+    "<a href="{{ group_path }}">{{ group_name }}</a>"
     {% endblocktrans %}
 
+{% elif other_user_count == 1 %}
+
+    {% blocktrans trimmed with group_name=notification.related_group.name group_path=notification.related_group.local_path %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a>
+    and
+    <a href="{{ second_user_link }}">{{ second_user }}</a>
+    have left your group
+    "<a href="{{ group_path }}">{{ group_name }}</a>"
+    {% endblocktrans %}
+
+{% else %}
+
+    {% blocktrans trimmed with group_name=notification.related_group.name group_path=notification.related_group.local_path %}
+    <a href="{{ related_user_link }}">{{ related_user }}</a>
+    and
+    {{ other_user_display_count }} others
+    have left your group
+    "<a href="{{ group_path }}">{{ group_name }}</a>"
+    {% endblocktrans %}
+
+{% endif %}
+
 {% endblock %}

bookwyrm/templates/notifications/items/mention.html

@@ -19,25 +19,25 @@
     {% if related_status.status_type == 'Review' %}
     {% blocktrans trimmed %}
 
-        mentioned you in a <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
+        <a href="{{ related_user_link }}">{{ related_user }}</a> mentioned you in a <a href="{{ related_path }}">review of <em>{{ book_title }}</em></a>
 
     {% endblocktrans %}
     {% elif related_status.status_type == 'Comment' %}
     {% blocktrans trimmed %}
 
-        mentioned you in a <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
+        <a href="{{ related_user_link }}">{{ related_user }}</a> mentioned you in a <a href="{{ related_path }}">comment on <em>{{ book_title }}</em></a>
 
     {% endblocktrans %}
     {% elif related_status.status_type == 'Quotation' %}
     {% blocktrans trimmed %}
 
-        mentioned you in a <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
+        <a href="{{ related_user_link }}">{{ related_user }}</a> mentioned you in a <a href="{{ related_path }}">quote from <em>{{ book_title }}</em></a>
 
     {% endblocktrans %}
     {% else %}
     {% blocktrans trimmed %}
 
-        mentioned you in a <a href="{{ related_path }}">status</a>
+        <a href="{{ related_user_link }}">{{ related_user }}</a> mentioned you in a <a href="{{ related_path }}">status</a>
 
     {% endblocktrans %}
     {% endif %}
@@ -51,7 +51,7 @@
 <div class="notification py-2 {% if notification.id in unread %}is-primary is-light{% else %}has-background-body has-text-default{% endif %}">
     <div class="columns">
         <div class="column is-clipped">
-            {% include 'snippets/status_preview.html' with status=related_status %}
+            {% include 'notifications/items/status_preview.html' with status=related_status %}
         </div>
         <div class="column is-narrow has-text-default">
             {{ related_status.published_date|timesince }}

bookwyrm/templates/notifications/items/reply.html

@@ -20,25 +20,25 @@
     {% if related_status.reply_parent.status_type == 'Review' %}
     {% blocktrans trimmed %}
 
-    <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">review of <em>{{ book_title }}</em></a>
+    <a href="{{ related_user_link }}">{{ related_user }}</a> <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">review of <em>{{ book_title }}</em></a>
 
     {% endblocktrans %}
     {% elif related_status.reply_parent.status_type == 'Comment' %}
     {% blocktrans trimmed %}
 
-    <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">comment on <em>{{ book_title }}</em></a>
+    <a href="{{ related_user_link }}">{{ related_user }}</a> <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">comment on <em>{{ book_title }}</em></a>
 
     {% endblocktrans %}
     {% elif related_status.reply_parent.status_type == 'Quotation' %}
     {% blocktrans trimmed %}
 
-    <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">quote from <em>{{ book_title }}</em></a>
+    <a href="{{ related_user_link }}">{{ related_user }}</a> <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">quote from <em>{{ book_title }}</em></a>
 
     {% endblocktrans %}
     {% else %}
     {% blocktrans trimmed %}
 
-    <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">status</a>
+    <a href="{{ related_user_link }}">{{ related_user }}</a> <a href="{{ related_path }}">replied</a> to your <a href="{{ parent_path }}">status</a>
 
     {% endblocktrans %}
     {% endif %}
@@ -54,7 +54,7 @@
 <div class="notification py-2 {% if notification.id in unread %}is-primary is-light{% else %}has-background-body has-text-default{% endif %}">
     <div class="columns">
         <div class="column is-clipped">
-            {% include 'snippets/status_preview.html' with status=related_status %}
+            {% include 'notifications/items/status_preview.html' with status=related_status %}
         </div>
         <div class="column is-narrow has-text-default">
             {{ related_status.published_date|timesince }}

bookwyrm/templates/notifications/items/report.html

@@ -1,9 +1,9 @@
 {% extends 'notifications/items/layout.html' %}
-
+{% load humanize %}
 {% load i18n %}
 
 {% block primary_link %}{% spaceless %}
-    {% url 'settings-report' notification.related_report.id %}
+{% url 'settings-reports' %}
 {% endspaceless %}{% endblock %}
 
 {% block icon %}
@@ -11,6 +11,10 @@
 {% endblock %}
 
 {% block description %}
-    {% url 'settings-report' notification.related_report.id as path %}
-    {% blocktrans %}A new <a href="{{ path }}">report</a> needs moderation.{% endblocktrans %}
+    {% url 'settings-reports' as path %}
+    {% blocktrans trimmed count counter=notification.related_reports.count with display_count=notification.related_reports.count|intcomma %}
+    A new <a href="{{ path }}">report</a> needs moderation
+    {% plural %}
+    {{ display_count }} new <a href="{{ path }}">reports</a> need moderation
+    {% endblocktrans %}
 {% endblock %}

bookwyrm/templates/notifications/items/status_preview.html

@@ -1,4 +1,17 @@
-{% if status.content %}
+{% load i18n %}
+{% if status.content_warning %}
+
+{% trans "Content warning" as text %}
+<span>
+    <span class="icon icon-warning is-size-5" title="{{ text }}">
+        <span class="is-sr-only">{{ text }}</span>
+    </span>
+
+    <a href="{{ status.local_path }}">
+        {{ status.content_warning }}
+    </a>
+</span>
+{% elif status.content %}
 <a href="{{ status.local_path }}">
     {{ status.content | safe | truncatewords_html:10 }}{% if status.mention_books %} <em>{{ status.mention_books.first.title }}</em>{% endif %}
 </a>

bookwyrm/templates/preferences/change_password.html

@@ -8,15 +8,31 @@
 {% endblock %}
 
 {% block panel %}
+{% if success %}
+<div class="notification is-success is-light">
+    <span class="icon icon-check" aria-hidden="true"></span>
+    <span>
+        {% trans "Successfully changed password" %}
+    </span>
+</div>
+{% endif %}
 <form name="edit-profile" action="{% url 'prefs-password' %}" method="post" enctype="multipart/form-data">
     {% csrf_token %}
+    <div class="field">
+        <label class="label" for="id_password">{% trans "Current password:" %}</label>
+        {{ form.current_password }}
+        {% include 'snippets/form_errors.html' with errors_list=form.current_password.errors id="desc_current_password" %}
+    </div>
+    <hr aria-hidden="true" />
     <div class="field">
         <label class="label" for="id_password">{% trans "New password:" %}</label>
-        <input type="password" name="password" maxlength="128" class="input" required="" id="id_password">
+        {{ form.password }}
+        {% include 'snippets/form_errors.html' with errors_list=form.password.errors id="desc_current_password" %}
     </div>
     <div class="field">
         <label class="label" for="id_confirm_password">{% trans "Confirm password:" %}</label>
-        <input type="password" name="confirm-password" maxlength="128" class="input" required="" id="id_confirm_password">
+        {{ form.confirm_password }}
+        {% include 'snippets/form_errors.html' with errors_list=form.confirm_password.errors id="desc_confirm_password" %}
     </div>
     <button class="button is-primary" type="submit">{% trans "Change Password" %}</button>
 </form>

bookwyrm/templates/preferences/export.html

@@ -13,10 +13,13 @@
         {% trans "Your export will include all the books on your shelves, books you have reviewed, and books with reading activity." %}
     </p>
     <p>
-        <a href="{% url 'prefs-export-file' %}" class="button">
-            <span class="icon icon-download" aria-hidden="true"></span>
-            <span>Download file</span>
-        </a>
+        <form name="export" method="POST" href="{% url 'prefs-export' %}">
+            {% csrf_token %}
+            <button type="submit" class="button">
+                <span class="icon icon-download" aria-hidden="true"></span>
+                <span>{% trans "Download file" %}</span>
+            </button>
+        </form>
     </p>
 </div>
 {% endblock %}

bookwyrm/templates/search/barcode_modal.html

@@ -16,11 +16,11 @@
         <select>
         </select>
     </div>
-    
+
     <div id="barcode-status" class="block">
         <div class="grant-access is-hidden">
             <span class="icon icon-lock"></span>
-            <span class="is-size-5">{% trans "Requesting camera..." %}</span></br>
+            <span class="is-size-5">{% trans "Requesting camera..." %}</span><br/>
             <span>{% trans "Grant access to the camera to scan a book's barcode." %}</span>
         </div>
         <div class="access-denied is-hidden">

bookwyrm/templates/search/book.html

@@ -36,7 +36,7 @@
     {% if result_set.results %}
     <section class="mb-5">
         {% if not result_set.connector.local %}
-        <details class="details-panel box" {% if forloop.first %}open{% endif %}>
+        <details class="details-panel box" open>
         {% endif %}
             {% if not result_set.connector.local %}
             <summary class="is-flex is-align-items-center is-flex-wrap-wrap is-gap-2">

bookwyrm/templates/settings/dashboard/dashboard.html

@@ -37,50 +37,40 @@
 </div>
 
 <div class="columns block is-multiline">
-    {% if reports %}
-    <div class="column is-flex">
-        <a href="{% url 'settings-reports' %}" class="notification is-warning is-block is-flex-grow-1">
-            {% blocktrans trimmed count counter=reports with display_count=reports|intcomma %}
-            {{ display_count }} open report
-            {% plural %}
-            {{ display_count }} open reports
-            {% endblocktrans %}
-        </a>
-    </div>
+    {% if email_config_error %}
+    {% include 'settings/dashboard/warnings/email_config.html' with warning_level="danger" fullwidth=True %}
     {% endif %}
 
-    {% if pending_domains %}
-    <div class="column is-flex">
-        <a href="{% url 'settings-link-domain' %}" class="notification is-primary is-block is-flex-grow-1">
-            {% blocktrans trimmed count counter=pending_domains with display_count=pending_domains|intcomma %}
-            {{ display_count }} domain needs review
-            {% plural %}
-            {{ display_count }} domains need review
-            {% endblocktrans %}
-        </a>
-    </div>
+    {% if current_version %}
+    {% include 'settings/dashboard/warnings/update_version.html' with warning_level="warning" fullwidth=True %}
     {% endif %}
 
-    {% if not site.allow_registration and site.allow_invite_requests and invite_requests %}
-    <div class="column is-flex">
-        <a href="{% url 'settings-invite-requests' %}" class="notification is-block is-success is-flex-grow-1">
-            {% blocktrans trimmed count counter=invite_requests with display_count=invite_requests|intcomma %}
-            {{ display_count }} invite request
-            {% plural %}
-            {{ display_count }} invite requests
-            {% endblocktrans %}
-        </a>
+    {% if missing_privacy or missing_conduct %}
+    <div class="column is-12 columns m-0 p-0">
+        {% if missing_privacy %}
+        {% include 'settings/dashboard/warnings/missing_privacy.html' with warning_level="danger" %}
+        {% endif %}
+
+        {% if missing_conduct %}
+        {% include 'settings/dashboard/warnings/missing_conduct.html' with warning_level="warning" %}
+        {% endif %}
     </div>
     {% endif %}
 
     {% if current_version %}
-    <div class="column is-flex">
-        <a href="https://docs.joinbookwyrm.com/updating-your-instance.html" class="notification is-block is-warning is-flex-grow-1" target="_blank">
-            {% blocktrans trimmed with current=current_version available=available_version %}
-            An update is available! You're running v{{ current }} and the latest release is {{ available }}.
-            {% endblocktrans %}
-        </a>
-    </div>
+    {% include 'settings/dashboard/warnings/update_version.html' with warning_level="warning" fullwidth=True %}
+    {% endif %}
+
+    {% if reports %}
+    {% include 'settings/dashboard/warnings/reports.html' with warning_level="warning" %}
+    {% endif %}
+
+    {% if pending_domains %}
+    {% include 'settings/dashboard/warnings/domain_review.html' with warning_level="primary" %}
+    {% endif %}
+
+    {% if not site.allow_registration and site.allow_invite_requests and invite_requests %}
+    {% include 'settings/dashboard/warnings/invites.html' with warning_level="success" %}
     {% endif %}
 </div>
 

bookwyrm/templates/settings/dashboard/warnings/domain_review.html

@@ -0,0 +1,15 @@
+{% extends 'settings/dashboard/warnings/layout.html' %}
+{% load i18n %}
+{% load humanize %}
+
+{% block warning_link %}{% url 'settings-link-domain' %}{% endblock %}
+
+{% block warning_text %}
+
+{% blocktrans trimmed count counter=pending_domains with display_count=pending_domains|intcomma %}
+{{ display_count }} domain needs review
+{% plural %}
+{{ display_count }} domains need review
+{% endblocktrans %}
+
+{% endblock %}

bookwyrm/templates/settings/dashboard/warnings/email_config.html

@@ -0,0 +1,13 @@
+{% extends 'settings/dashboard/warnings/layout.html' %}
+{% load i18n %}
+
+{% block warning_link %}https://docs.joinbookwyrm.com/install-prod.html{% endblock %}
+
+{% block warning_text %}
+
+{% blocktrans trimmed %}
+Your outgoing email address, <code>{{ email_sender }}</code>, may be misconfigured.
+{% endblocktrans %}
+{% trans "Check the <code>EMAIL_SENDER_NAME</code> and <code>EMAIL_SENDER_DOMAIN</code> in your <code>.env</code> file." %}
+
+{% endblock %}

bookwyrm/templates/settings/dashboard/warnings/invites.html

@@ -0,0 +1,15 @@
+{% extends 'settings/dashboard/warnings/layout.html' %}
+{% load i18n %}
+{% load humanize %}
+
+{% block warning_link %}{% url 'settings-invite-requests' %}{% endblock %}
+
+{% block warning_text %}
+
+{% blocktrans trimmed count counter=invite_requests with display_count=invite_requests|intcomma %}
+{{ display_count }} invite request
+{% plural %}
+{{ display_count }} invite requests
+{% endblocktrans %}
+
+{% endblock %}

bookwyrm/templates/settings/dashboard/warnings/layout.html

@@ -0,0 +1,5 @@
+<div class="column is-flex{% if fullwidth %} is-12{% endif %}">
+    <a href="{% block warning_link %}{% endblock %}" class="notification is-{{ warning_level }} is-block is-flex-grow-1">
+        {% block warning_text %}{% endblock %}
+    </a>
+</div>

bookwyrm/templates/settings/dashboard/warnings/missing_conduct.html

@@ -0,0 +1,10 @@
+{% extends 'settings/dashboard/warnings/layout.html' %}
+{% load i18n %}
+
+{% block warning_link %}{% url 'settings-site' %}#instance-info{% endblock %}
+
+{% block warning_text %}
+
+{% trans "Your instance is missing a code of conduct." %}
+
+{% endblock %}

bookwyrm/templates/settings/dashboard/warnings/missing_privacy.html

@@ -0,0 +1,10 @@
+{% extends 'settings/dashboard/warnings/layout.html' %}
+{% load i18n %}
+
+{% block warning_link %}{% url 'settings-site' %}#instance-info{% endblock %}
+
+{% block warning_text %}
+
+{% trans "Your instance is missing a privacy policy." %}
+
+{% endblock %}

bookwyrm/templates/settings/dashboard/warnings/reports.html

@@ -0,0 +1,15 @@
+{% extends 'settings/dashboard/warnings/layout.html' %}
+{% load i18n %}
+{% load humanize %}
+
+{% block warning_link %}{% url 'settings-reports' %}{% endblock %}
+
+{% block warning_text %}
+
+{% blocktrans trimmed count counter=reports with display_count=reports|intcomma %}
+{{ display_count }} open report
+{% plural %}
+{{ display_count }} open reports
+{% endblocktrans %}
+
+{% endblock %}

bookwyrm/templates/settings/dashboard/warnings/update_version.html

@@ -0,0 +1,12 @@
+{% extends 'settings/dashboard/warnings/layout.html' %}
+{% load i18n %}
+
+{% block warning_link %}https://docs.joinbookwyrm.com/updating.html{% endblock %}
+
+{% block warning_text %}
+
+{% blocktrans trimmed with current=current_version available=available_version %}
+An update is available! You're running v{{ current }} and the latest release is {{ available }}.
+{% endblocktrans %}
+
+{% endblock %}

bookwyrm/templates/settings/federation/instance.html

@@ -56,7 +56,7 @@
                 <dt class="is-pulled-left mr-5">{% trans "Users:" %}</dt>
                 <dd>
                     {{ users.count }}
-                    {% if server.user_set.count %}(<a href="{% url 'settings-users' %}?server={{ server.server_name }}">{% trans "View all" %}</a>){% endif %}
+                    {% if server.user_set.count %}(<a href="{% url 'settings-users' status="federated" %}?server={{ server.server_name }}">{% trans "View all" %}</a>){% endif %}
                 </dd>
 
                 <dt class="is-pulled-left mr-5">{% trans "Reports:" %}</dt>

bookwyrm/templates/settings/federation/instance_filters.html

@@ -2,6 +2,7 @@
 
 {% block filter_fields %}
 {% include 'settings/federation/software_filter.html' %}
+{% include 'settings/users/server_filter.html' %}
 
 {% endblock %}
 

bookwyrm/templates/settings/link_domains/link_table.html

@@ -15,7 +15,11 @@
             <a href="{{ link.url }}" target="_blank" rel="noopener noreferrer">{{ link.url }}</a>
         </td>
         <td>
+            {% if link.added_by %}
             <a href="{% url 'settings-user' link.added_by.id %}">@{{ link.added_by|username }}</a>
+            {% else %}
+            <em>{% trans "Unknown user" %}</em>
+            {% endif %}
         </td>
         <td>
             {% if link.filelink.filetype %}

bookwyrm/templates/settings/reports/report.html

@@ -55,9 +55,11 @@
 </div>
 {% endif %}
 
+{% if report.user %}
 {% include 'settings/users/user_info.html' with user=report.user %}
 
 {% include 'settings/users/user_moderation_actions.html' with user=report.user %}
+{% endif %}
 
 <div class="block">
     <h3 class="title is-4">{% trans "Moderator Comments" %}</h3>

bookwyrm/templates/settings/reports/report_header.html

@@ -9,9 +9,15 @@ Report #{{ report_id }}: Status posted by @{{ username }}
 
 {% elif report.links.exists %}
 
-{% blocktrans trimmed with report_id=report.id username=report.user|username %}
-Report #{{ report_id }}: Link added by @{{ username }}
-{% endblocktrans %}
+    {% if report.user %}
+    {% blocktrans trimmed with report_id=report.id username=report.user|username %}
+    Report #{{ report_id }}: Link added by @{{ username }}
+    {% endblocktrans %}
+    {% else %}
+    {% blocktrans trimmed with report_id=report.id %}
+    Report #{{ report_id }}: Link domain
+    {% endblocktrans %}
+    {% endif %}
 
 {% else %}
 

bookwyrm/templates/settings/users/user_admin.html

@@ -24,6 +24,10 @@
             <li {% if request.path in url %}class="is-active" aria-current="page"{% endif %}>
                 <a href="{{ url }}">{% trans "Local users" %}</a>
             </li>
+            {% url 'settings-users' status="deleted" as url %}
+            <li {% if url in request.path %}class="is-active" aria-current="page"{% endif %}>
+                <a href="{{ url }}">{% trans "Deleted users" %}</a>
+            </li>
             {% url 'settings-users' status="federated" as url %}
             <li {% if url in request.path %}class="is-active" aria-current="page"{% endif %}>
                 <a href="{{ url }}">{% trans "Federated community" %}</a>
@@ -36,7 +40,7 @@
     <table class="table is-striped is-fullwidth">
         <tr>
             {% url 'settings-users' as url %}
-            <th>
+            <th colspan="2">
                 {% trans "Username" as text %}
                 {% include 'snippets/table-sort-header.html' with field="username" sort=sort text=text %}
             </th>
@@ -52,7 +56,7 @@
                 {% trans "Status" as text %}
                 {% include 'snippets/table-sort-header.html' with field="is_active" sort=sort text=text %}
             </th>
-            {% if status != "local" %}
+            {% if status == "federated" %}
             <th>
                 {% trans "Remote instance" as text %}
                 {% include 'snippets/table-sort-header.html' with field="federated_server__server_name" sort=sort text=text %}
@@ -61,7 +65,10 @@
         </tr>
         {% for user in users %}
         <tr>
-            <td class="overflow-wrap-anywhere">
+            <td class="pr-0">
+                {% include 'snippets/avatar.html' with user=user %}
+            </td>
+            <td class="overflow-wrap-anywhere pl-1">
                 <a href="{% url 'settings-user' user.id %}">{{ user|username }}</a>
             </td>
             <td>{{ user.created_date }}</td>
@@ -72,6 +79,12 @@
                         <span class="icon icon-check"></span>
                     </span>
                     {% trans "Active" %}
+                {% elif user.deactivation_reason == "moderator_deletion" or user.deactivation_reason == "self_deletion" %}
+                    <span class="tag is-danger" aria-hidden="true">
+                        <span class="icon icon-x"></span>
+                    </span>
+                    {% trans "Deleted" %}
+                    <span class="help">({{ user.get_deactivation_reason_display }})</span>
                 {% else %}
                     <span class="tag is-warning" aria-hidden="true">
                         <span class="icon icon-x"></span>
@@ -80,7 +93,7 @@
                     <span class="help">({{ user.get_deactivation_reason_display }})</span>
                 {% endif %}
             </td>
-            {% if status != "local" %}
+            {% if status == "federated" %}
             <td>
                 {% if user.federated_server %}
                 <a href="{% url 'settings-federated-server' user.federated_server.id %}">{{ user.federated_server.server_name }}</a>

bookwyrm/templates/setup/config.html

@@ -144,7 +144,7 @@
             {% blocktrans trimmed %}
                 You can change your instance settings in the <code>.env</code> file on your server.
             {% endblocktrans %}
-            <a href="https://docs.joinbookwyrm.com/installing-in-production.html" target="_blank">
+            <a href="https://docs.joinbookwyrm.com/install-prod.html" target="_blank">
                 {% trans "View installation instructions" %}
             </a>
         </p>

bookwyrm/templates/snippets/follow_button.html

@@ -26,7 +26,7 @@
         <form action="{% url 'unfollow' %}" method="POST" class="interaction follow_{{ user.id }} {% if not relationship.is_following and not relationship.is_follow_pending %}is-hidden{%endif %}" data-id="follow_{{ user.id }}">
             {% csrf_token %}
             <input type="hidden" name="user" value="{{ user.username }}">
-            {% if user.manually_approves_followers and not relationship.is_following %}
+            {% if relationship.is_follow_pending %}
             <button class="button is-small is-danger is-light" type="submit">
                 {% trans "Undo follow request" %}
             </button>

bookwyrm/templates/user_menu.html

@@ -68,9 +68,15 @@
             <li class="navbar-divider" role="presentation" aria-hidden="true">&nbsp;</li>
 
             <li role="menuitem">
-                <a href="{% url 'logout' %}" class="navbar-item">
-                    {% trans 'Log out' %}
-                </a>
+                <form
+                    name="logout"
+                    method="POST"
+                    action="{% url 'logout' %}"
+                    class="navbar-item"
+                >
+                    {% csrf_token %}
+                    <button type="submit">{% trans 'Log out' %}</button>
+                </form>
             </li>
         </ul>
     </div>

bookwyrm/templatetags/interaction.py

@@ -46,7 +46,7 @@ def get_relationship(context, user_object):
         get_relationship_name,
         user,
         user_object,
-        timeout=259200,
+        timeout=60 * 60,
     )
 
 
@@ -61,6 +61,6 @@ def get_relationship_name(user, user_object):
         types["is_blocked"] = True
     elif user_object in user.following.all():
         types["is_following"] = True
-    elif user_object in user.follower_requests.all():
+    elif user in user_object.follower_requests.all():
         types["is_follow_pending"] = True
     return types

bookwyrm/templatetags/notification_page_tags.py

@@ -12,3 +12,9 @@ def related_status(notification):
     if not notification.related_status:
         return None
     return load_subclass(notification.related_status)
+
+
+@register.simple_tag(takes_context=False)
+def get_related_users(notification):
+    """Who actually was it who liked your post"""
+    return list(reversed(list(notification.related_users.distinct())))[:10]

bookwyrm/templatetags/rating_tags.py

@@ -13,10 +13,10 @@ register = template.Library()
 def get_rating(book, user):
     """get the overall rating of a book"""
     return cache.get_or_set(
-        f"book-rating-{book.parent_work.id}-{user.id}",
-        lambda u, b: models.Review.privacy_filter(u)
-        .filter(book__parent_work__editions=b, rating__gt=0)
-        .aggregate(Avg("rating"))["rating__avg"]
+        f"book-rating-{book.parent_work.id}",
+        lambda u, b: models.Review.objects.filter(
+            book__parent_work__editions=b, rating__gt=0
+        ).aggregate(Avg("rating"))["rating__avg"]
         or 0,
         user,
         book,

bookwyrm/templatetags/shelf_tags.py

@@ -17,7 +17,7 @@ def get_is_book_on_shelf(book, shelf):
         lambda b, s: s.books.filter(id=b.id).exists(),
         book,
         shelf,
-        timeout=15552000,
+        timeout=60 * 60,  # just cache this for an hour
     )
 
 
@@ -68,7 +68,7 @@ def active_shelf(context, book):
         ),
         user,
         book,
-        timeout=15552000,
+        timeout=60 * 60,
     ) or {"book": book}
 
 
@@ -85,5 +85,5 @@ def latest_read_through(book, user):
         ),
         user,
         book,
-        timeout=15552000,
+        timeout=60 * 60,
     )

bookwyrm/templatetags/utilities.py

@@ -53,7 +53,7 @@ def comparison_bool(str1, str2, reverse=False):
 
 @register.filter(is_safe=True)
 def truncatepath(value, arg):
-    """Truncate a path by removing all directories except the first and truncating ."""
+    """Truncate a path by removing all directories except the first and truncating"""
     path = os.path.normpath(value.name)
     path_list = path.split(os.sep)
     try:

bookwyrm/tests/connectors/test_abstract_connector.py

@@ -42,15 +42,9 @@ class AbstractConnector(TestCase):
 
             generated_remote_link_field = "openlibrary_link"
 
-            def format_search_result(self, search_result):
-                return search_result
-
-            def parse_search_data(self, data):
+            def parse_search_data(self, data, min_confidence):
                 return data
 
-            def format_isbn_search_result(self, search_result):
-                return search_result
-
             def parse_isbn_search_data(self, data):
                 return data
 

bookwyrm/tests/connectors/test_abstract_minimal_connector.py

@@ -1,6 +1,5 @@
 """ testing book data connectors """
 from django.test import TestCase
-import responses
 
 from bookwyrm import models
 from bookwyrm.connectors import abstract_connector
@@ -25,18 +24,12 @@ class AbstractConnector(TestCase):
         class TestConnector(abstract_connector.AbstractMinimalConnector):
             """nothing added here"""
 
-            def format_search_result(self, search_result):
-                return search_result
-
             def get_or_create_book(self, remote_id):
                 pass
 
-            def parse_search_data(self, data):
+            def parse_search_data(self, data, min_confidence):
                 return data
 
-            def format_isbn_search_result(self, search_result):
-                return search_result
-
             def parse_isbn_search_data(self, data):
                 return data
 
@@ -54,45 +47,6 @@ class AbstractConnector(TestCase):
         self.assertIsNone(connector.name)
         self.assertEqual(connector.identifier, "example.com")
 
-    @responses.activate
-    def test_search(self):
-        """makes an http request to the outside service"""
-        responses.add(
-            responses.GET,
-            "https://example.com/search?q=a%20book%20title",
-            json=["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l"],
-            status=200,
-        )
-        results = self.test_connector.search("a book title")
-        self.assertEqual(len(results), 10)
-        self.assertEqual(results[0], "a")
-        self.assertEqual(results[1], "b")
-        self.assertEqual(results[2], "c")
-
-    @responses.activate
-    def test_search_min_confidence(self):
-        """makes an http request to the outside service"""
-        responses.add(
-            responses.GET,
-            "https://example.com/search?q=a%20book%20title&min_confidence=1",
-            json=["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l"],
-            status=200,
-        )
-        results = self.test_connector.search("a book title", min_confidence=1)
-        self.assertEqual(len(results), 10)
-
-    @responses.activate
-    def test_isbn_search(self):
-        """makes an http request to the outside service"""
-        responses.add(
-            responses.GET,
-            "https://example.com/isbn?q=123456",
-            json=["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l"],
-            status=200,
-        )
-        results = self.test_connector.isbn_search("123456")
-        self.assertEqual(len(results), 10)
-
     def test_create_mapping(self):
         """maps remote fields for book data to bookwyrm activitypub fields"""
         mapping = Mapping("isbn")

bookwyrm/tests/connectors/test_bookwyrm_connector.py

@@ -30,14 +30,11 @@ class BookWyrmConnector(TestCase):
         result = self.connector.get_or_create_book(book.remote_id)
         self.assertEqual(book, result)
 
-    def test_format_search_result(self):
+    def test_parse_search_data(self):
         """create a SearchResult object from search response json"""
         datafile = pathlib.Path(__file__).parent.joinpath("../data/bw_search.json")
         search_data = json.loads(datafile.read_bytes())
-        results = self.connector.parse_search_data(search_data)
-        self.assertIsInstance(results, list)
-
-        result = self.connector.format_search_result(results[0])
+        result = list(self.connector.parse_search_data(search_data, 0))[0]
         self.assertIsInstance(result, SearchResult)
         self.assertEqual(result.title, "Jonathan Strange and Mr Norrell")
         self.assertEqual(result.key, "https://example.com/book/122")
@@ -45,10 +42,9 @@ class BookWyrmConnector(TestCase):
         self.assertEqual(result.year, 2017)
         self.assertEqual(result.connector, self.connector)
 
-    def test_format_isbn_search_result(self):
+    def test_parse_isbn_search_data(self):
         """just gotta attach the connector"""
         datafile = pathlib.Path(__file__).parent.joinpath("../data/bw_search.json")
         search_data = json.loads(datafile.read_bytes())
-        results = self.connector.parse_isbn_search_data(search_data)
-        result = self.connector.format_isbn_search_result(results[0])
+        result = list(self.connector.parse_isbn_search_data(search_data))[0]
         self.assertEqual(result.connector, self.connector)

bookwyrm/tests/connectors/test_connector_manager.py

@@ -49,39 +49,11 @@ class ConnectorManager(TestCase):
         self.assertEqual(len(connectors), 1)
         self.assertIsInstance(connectors[0], BookWyrmConnector)
 
-    @responses.activate
-    def test_search_plaintext(self):
-        """search all connectors"""
-        responses.add(
-            responses.GET,
-            "http://fake.ciom/search/Example?min_confidence=0.1",
-            json=[{"title": "Hello", "key": "https://www.example.com/search/1"}],
-        )
-        results = connector_manager.search("Example")
-        self.assertEqual(len(results), 1)
-        self.assertEqual(len(results[0]["results"]), 1)
-        self.assertEqual(results[0]["connector"].identifier, "test_connector_remote")
-        self.assertEqual(results[0]["results"][0].title, "Hello")
-
     def test_search_empty_query(self):
         """don't panic on empty queries"""
         results = connector_manager.search("")
         self.assertEqual(results, [])
 
-    @responses.activate
-    def test_search_isbn(self):
-        """special handling if a query resembles an isbn"""
-        responses.add(
-            responses.GET,
-            "http://fake.ciom/isbn/0000000000",
-            json=[{"title": "Hello", "key": "https://www.example.com/search/1"}],
-        )
-        results = connector_manager.search("0000000000")
-        self.assertEqual(len(results), 1)
-        self.assertEqual(len(results[0]["results"]), 1)
-        self.assertEqual(results[0]["connector"].identifier, "test_connector_remote")
-        self.assertEqual(results[0]["results"][0].title, "Hello")
-
     def test_first_search_result(self):
         """only get one search result"""
         result = connector_manager.first_search_result("Example")

bookwyrm/tests/connectors/test_inventaire_connector.py

@@ -66,38 +66,14 @@ class Inventaire(TestCase):
         with self.assertRaises(ConnectorException):
             self.connector.get_book_data("https://test.url/ok")
 
-    @responses.activate
-    def test_search(self):
-        """min confidence filtering"""
-        responses.add(
-            responses.GET,
-            "https://inventaire.io/search?q=hi",
-            json={
-                "results": [
-                    {
-                        "_score": 200,
-                        "label": "hello",
-                    },
-                    {
-                        "_score": 100,
-                        "label": "hi",
-                    },
-                ],
-            },
-        )
-        results = self.connector.search("hi", min_confidence=0.5)
-        self.assertEqual(len(results), 1)
-        self.assertEqual(results[0].title, "hello")
-
-    def test_format_search_result(self):
+    def test_parse_search_data(self):
         """json to search result objs"""
         search_file = pathlib.Path(__file__).parent.joinpath(
             "../data/inventaire_search.json"
         )
         search_results = json.loads(search_file.read_bytes())
 
-        results = self.connector.parse_search_data(search_results)
-        formatted = self.connector.format_search_result(results[0])
+        formatted = list(self.connector.parse_search_data(search_results, 0))[0]
 
         self.assertEqual(formatted.title, "The Stories of Vladimir Nabokov")
         self.assertEqual(
@@ -178,15 +154,14 @@ class Inventaire(TestCase):
         result = self.connector.resolve_keys(keys)
         self.assertEqual(result, ["epistolary novel", "crime novel"])
 
-    def test_isbn_search(self):
+    def test_pase_isbn_search_data(self):
         """another search type"""
         search_file = pathlib.Path(__file__).parent.joinpath(
             "../data/inventaire_isbn_search.json"
         )
         search_results = json.loads(search_file.read_bytes())
 
-        results = self.connector.parse_isbn_search_data(search_results)
-        formatted = self.connector.format_isbn_search_result(results[0])
+        formatted = list(self.connector.parse_isbn_search_data(search_results))[0]
 
         self.assertEqual(formatted.title, "L'homme aux cercles bleus")
         self.assertEqual(
@@ -198,25 +173,12 @@ class Inventaire(TestCase):
             "https://covers.inventaire.io/img/entities/12345",
         )
 
-    def test_isbn_search_empty(self):
+    def test_parse_isbn_search_data_empty(self):
         """another search type"""
         search_results = {}
-        results = self.connector.parse_isbn_search_data(search_results)
+        results = list(self.connector.parse_isbn_search_data(search_results))
         self.assertEqual(results, [])
 
-    def test_isbn_search_no_title(self):
-        """another search type"""
-        search_file = pathlib.Path(__file__).parent.joinpath(
-            "../data/inventaire_isbn_search.json"
-        )
-        search_results = json.loads(search_file.read_bytes())
-        search_results["entities"]["isbn:9782290349229"]["claims"]["wdt:P1476"] = None
-
-        result = self.connector.format_isbn_search_result(
-            search_results.get("entities")
-        )
-        self.assertIsNone(result)
-
     def test_is_work_data(self):
         """is it a work"""
         work_file = pathlib.Path(__file__).parent.joinpath(

bookwyrm/tests/connectors/test_openlibrary_connector.py

@@ -122,21 +122,11 @@ class Openlibrary(TestCase):
         self.assertEqual(result, "https://covers.openlibrary.org/b/id/image-L.jpg")
 
     def test_parse_search_result(self):
-        """extract the results from the search json response"""
-        datafile = pathlib.Path(__file__).parent.joinpath("../data/ol_search.json")
-        search_data = json.loads(datafile.read_bytes())
-        result = self.connector.parse_search_data(search_data)
-        self.assertIsInstance(result, list)
-        self.assertEqual(len(result), 2)
-
-    def test_format_search_result(self):
         """translate json from openlibrary into SearchResult"""
         datafile = pathlib.Path(__file__).parent.joinpath("../data/ol_search.json")
         search_data = json.loads(datafile.read_bytes())
-        results = self.connector.parse_search_data(search_data)
-        self.assertIsInstance(results, list)
+        result = list(self.connector.parse_search_data(search_data, 0))[0]
 
-        result = self.connector.format_search_result(results[0])
         self.assertIsInstance(result, SearchResult)
         self.assertEqual(result.title, "This Is How You Lose the Time War")
         self.assertEqual(result.key, "https://openlibrary.org/works/OL20639540W")
@@ -148,18 +138,10 @@ class Openlibrary(TestCase):
         """extract the results from the search json response"""
         datafile = pathlib.Path(__file__).parent.joinpath("../data/ol_isbn_search.json")
         search_data = json.loads(datafile.read_bytes())
-        result = self.connector.parse_isbn_search_data(search_data)
-        self.assertIsInstance(result, list)
+        result = list(self.connector.parse_isbn_search_data(search_data))
         self.assertEqual(len(result), 1)
 
-    def test_format_isbn_search_result(self):
-        """translate json from openlibrary into SearchResult"""
-        datafile = pathlib.Path(__file__).parent.joinpath("../data/ol_isbn_search.json")
-        search_data = json.loads(datafile.read_bytes())
-        results = self.connector.parse_isbn_search_data(search_data)
-        self.assertIsInstance(results, list)
-
-        result = self.connector.format_isbn_search_result(results[0])
+        result = result[0]
         self.assertIsInstance(result, SearchResult)
         self.assertEqual(result.title, "Les ombres errantes")
         self.assertEqual(result.key, "https://openlibrary.org/books/OL16262504M")
@@ -229,7 +211,7 @@ class Openlibrary(TestCase):
             status=200,
         )
         with patch(
-            "bookwyrm.connectors.openlibrary.Connector." "get_authors_from_data"
+            "bookwyrm.connectors.openlibrary.Connector.get_authors_from_data"
         ) as mock:
             mock.return_value = []
             result = self.connector.create_edition_from_data(work, self.edition_data)

bookwyrm/tests/lists_stream/test_signals.py

@@ -32,9 +32,10 @@ class ListsStreamSignals(TestCase):
 
     def test_add_list_on_create_command(self, _):
         """a new lists has entered"""
-        book_list = models.List.objects.create(
-            user=self.remote_user, name="hi", privacy="public"
-        )
+        with patch("bookwyrm.lists_stream.remove_list_task.delay"):
+            book_list = models.List.objects.create(
+                user=self.remote_user, name="hi", privacy="public"
+            )
         with patch("bookwyrm.lists_stream.add_list_task.delay") as mock:
             lists_stream.add_list_on_create_command(book_list.id)
         self.assertEqual(mock.call_count, 1)
@@ -43,9 +44,10 @@ class ListsStreamSignals(TestCase):
 
     def test_remove_list_on_delete(self, _):
         """delete a list"""
-        book_list = models.List.objects.create(
-            user=self.remote_user, name="hi", privacy="public"
-        )
+        with patch("bookwyrm.lists_stream.remove_list_task.delay"):
+            book_list = models.List.objects.create(
+                user=self.remote_user, name="hi", privacy="public"
+            )
         with patch("bookwyrm.lists_stream.remove_list_task.delay") as mock:
             lists_stream.remove_list_on_delete(models.List, book_list)
         args = mock.call_args[0]

bookwyrm/tests/lists_stream/test_stream.py

@@ -11,6 +11,7 @@ from bookwyrm import lists_stream, models
 @patch("bookwyrm.activitystreams.add_book_statuses_task.delay")
 @patch("bookwyrm.suggested_users.rerank_suggestions_task.delay")
 @patch("bookwyrm.activitystreams.populate_stream_task.delay")
+@patch("bookwyrm.lists_stream.remove_list_task.delay")
 class ListsStream(TestCase):
     """using redis to build activity streams"""
 

bookwyrm/tests/lists_stream/test_tasks.py

@@ -35,7 +35,9 @@ class Activitystreams(TestCase):
                 inbox="https://example.com/users/rat/inbox",
                 outbox="https://example.com/users/rat/outbox",
             )
-        with patch("bookwyrm.models.activitypub_mixin.broadcast_task.apply_async"):
+        with patch(
+            "bookwyrm.models.activitypub_mixin.broadcast_task.apply_async"
+        ), patch("bookwyrm.lists_stream.remove_list_task.delay"):
             self.list = models.List.objects.create(
                 user=self.local_user, name="hi", privacy="public"
             )