Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into emr_develop

.gitlab-ci.yml

@@ -24,6 +24,7 @@ stages:
   - docker
 
 before_script:
+  - echo $MIX_ENV
   - rm -rf _build/*/lib/pleroma
   - apt-get update && apt-get install -y cmake
   - mix local.hex --force
@@ -152,6 +153,20 @@ analysis:
   script:
     - mix credo --strict --only=warnings,todo,fixme,consistency,readability
 
+cycles:
+  stage: test
+  image: elixir:1.11
+  only:
+    changes:
+      - "**/*.ex"
+      - "**/*.exs"
+      - "mix.lock"
+  cache: {}
+  script:
+    - mix deps.get
+    - mix compile
+    - mix xref graph --format cycles --label compile | awk '{print $0} END{exit ($0 != "No cycles found")}'
+
 docs-deploy:
   stage: deploy
   cache: *testing_cache_policy

CHANGELOG.md

@@ -13,19 +13,24 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - The `application` metadata returned with statuses is no longer hardcoded. Apps that want to display these details will now have valid data for new posts after this change.
 - HTTPSecurityPlug now sends a response header to opt out of Google's FLoC (Federated Learning of Cohorts) targeted advertising.
 - Email address is now returned if requesting user is the owner of the user account so it can be exposed in client and FE user settings UIs.
+- Improved Twittercard and OpenGraph meta tag generation including thumbnails and image dimension metadata when available.
+- ActivityPub Client-to-Server(C2S): Limitation on the type of Activity/Object are lifted as they are now passed through ObjectValidators
 
 ### Added
 
 - MRF (`FollowBotPolicy`): New MRF Policy which makes a designated local Bot account attempt to follow all users in public Notes received by your instance. Users who require approving follower requests or have #nobot in their profile are excluded.
 - Return OAuth token `id` (primary key) in POST `/oauth/token`.
-- `AnalyzeMetadata` upload filter for extracting attachment dimensions and generating blurhashes.
+- `AnalyzeMetadata` upload filter for extracting image/video attachment dimensions and generating blurhashes for images. Blurhashes for videos are not generated at this time.
 - Attachment dimensions and blurhashes are federated when available.
 - Pinned posts federation
 
 ### Fixed
 - Don't crash so hard when email settings are invalid.
 - Checking activated Upload Filters for required commands.
+- Remote users can no longer reappear after being deleted.
+- Deactivated users may now be deleted.
 - Mix task `pleroma.database prune_objects`
+- Linkify: Parsing crash with URLs ending in unbalanced closed paren, no path separator, and no query parameters
 
 ### Removed
 - **Breaking**: Remove deprecated `/api/qvitter/statuses/notifications/read` (replaced by `/api/v1/pleroma/notifications/read`)

docs/installation/alpine_linux_en.md

@@ -5,25 +5,7 @@ This guide is a step-by-step installation guide for Alpine Linux. The instructio
 
 It assumes that you have administrative rights, either as root or a user with [sudo permissions](https://www.linode.com/docs/tools-reference/custom-kernels-distros/install-alpine-linux-on-your-linode/#configuration). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su -l <username> -s $SHELL -c 'command'` instead.
 
-### Required packages
+{! backend/installation/generic_dependencies.include !}
-
-* `postgresql`
-* `elixir`
-* `erlang`
-* `erlang-parsetools`
-* `erlang-xmerl`
-* `git`
-* `file-dev`
-* Development Tools
-* `cmake`
-
-#### Optional packages used in this guide
-
-* `nginx` (preferred, example configs for other reverse proxies can be found in the repo)
-* `certbot` (or any other ACME client for Let’s Encrypt certificates)
-* `ImageMagick`
-* `ffmpeg`
-* `exiftool`
 
 ### Prepare the system
 

docs/installation/debian_based_en.md

@@ -1,27 +1,9 @@
 # Installing on Debian Based Distributions
 ## Installation
 
-This guide will assume you are on Debian Stretch. This guide should also work with Ubuntu 16.04 and 18.04. It also assumes that you have administrative rights, either as root or a user with [sudo permissions](https://www.digitalocean.com/community/tutorials/how-to-add-delete-and-grant-sudo-privileges-to-users-on-a-debian-vps). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su <username> -s $SHELL -c 'command'` instead.
+This guide will assume you are on Debian 11 (“bullseye”) or later. This guide should also work with Ubuntu 18.04 (“Bionic Beaver”) and later. It also assumes that you have administrative rights, either as root or a user with [sudo permissions](https://www.digitalocean.com/community/tutorials/how-to-add-delete-and-grant-sudo-privileges-to-users-on-a-debian-vps). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su <username> -s $SHELL -c 'command'` instead.
 
-### Required packages
+{! backend/installation/generic_dependencies.include !}
-
-* `postgresql` (9.6+, Ubuntu 16.04 comes with 9.5, you can get a newer version from [here](https://www.postgresql.org/download/linux/ubuntu/))
-* `postgresql-contrib` (9.6+, same situtation as above)
-* `elixir` (1.8+, Follow the guide to install from the Erlang Solutions repo or use [asdf](https://github.com/asdf-vm/asdf) as the pleroma user)
-* `erlang-dev`
-* `erlang-nox`
-* `libmagic-dev`
-* `git`
-* `build-essential`
-* `cmake`
-
-#### Optional packages used in this guide
-
-* `nginx` (preferred, example configs for other reverse proxies can be found in the repo)
-* `certbot` (or any other ACME client for Let’s Encrypt certificates)
-* `ImageMagick`
-* `ffmpeg`
-* `exiftool`
 
 ### Prepare the system
 
@@ -40,20 +22,14 @@ sudo apt install git build-essential postgresql postgresql-contrib cmake libmagi
 
 ### Install Elixir and Erlang
 
-* Download and add the Erlang repository:
+* Install Elixir and Erlang (you might need to use backports or [asdf](https://github.com/asdf-vm/asdf) on old systems):
-
-```shell
-wget -P /tmp/ https://packages.erlang-solutions.com/erlang-solutions_2.0_all.deb
-sudo dpkg -i /tmp/erlang-solutions_2.0_all.deb
-```
-
-* Install Elixir and Erlang:
 
 ```shell
 sudo apt update
 sudo apt install elixir erlang-dev erlang-nox
 ```
 
+
 ### Optional packages: [`docs/installation/optional/media_graphics_packages.md`](../installation/optional/media_graphics_packages.md)
 
 ```shell

docs/installation/freebsd_en.md

@@ -2,7 +2,9 @@
 
 This document was written for FreeBSD 12.1, but should be work on future releases.
 
-## Required software
+{! backend/installation/generic_dependencies.include !}
+
+## Installing software used in this guide
 
 This assumes the target system has `pkg(8)`.
 

docs/installation/generic_dependencies.include

@@ -0,0 +1,16 @@
+## Required dependencies
+
+* PostgreSQL 9.6+
+* Elixir 1.9+
+* Erlang OTP 22.2+
+* git
+* file / libmagic
+* gcc (clang might also work)
+* GNU make
+* CMake
+
+## Optionnal dependencies
+
+* ImageMagick
+* FFmpeg
+* exiftool

docs/installation/gentoo_en.md

@@ -3,9 +3,7 @@
 
 This guide will assume that you have administrative rights, either as root or a user with [sudo permissions](https://wiki.gentoo.org/wiki/Sudo). Lines that begin with `#` indicate that they should be run as the superuser. Lines using `$` should be run as the indicated user, e.g. `pleroma$` should be run as the `pleroma` user.
 
-### Configuring your hostname (optional)
+{! backend/installation/generic_dependencies.include !}
-
-If you would like your prompt to permanently include your host/domain, change `/etc/conf.d/hostname` to your hostname. You can reboot or use the `hostname` command to make immediate changes.
 
 ### Your make.conf, package.use, and USE flags
 

docs/installation/netbsd_en.md

@@ -1,6 +1,8 @@
 # Installing on NetBSD
 
-## Required software
+{! backend/installation/generic_dependencies.include !}
+
+## Installing software used in this guide
 
 pkgin should have been installed by the NetBSD installer if you selected
 the right options. If it isn't installed, install it using pkg_add.

docs/installation/openbsd_en.md

@@ -4,19 +4,11 @@ This guide describes the installation and configuration of pleroma (and the requ
 
 For any additional information regarding commands and configuration files mentioned here, check the man pages [online](https://man.openbsd.org/) or directly on your server with the man command.
 
+{! backend/installation/generic_dependencies.include !}
+
+### Preparing the system
 #### Required software
 
-The following packages need to be installed:
-
-  * elixir
-  * gmake
-  * git
-  * postgresql-server
-  * postgresql-contrib
-  * cmake
-  * ffmpeg
-  * ImageMagick
-
 To install them, run the following command (with doas or as root):
 
 ```

docs/installation/otp_en.md

@@ -31,7 +31,7 @@ Other than things bundled in the OTP release Pleroma depends on:
 
 === "Alpine"
     ```
-    echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
+    awk 'NR==2' /etc/apk/repositories | sed 's/main/community/' | tee -a /etc/apk/repositories
     apk update
     apk add curl unzip ncurses postgresql postgresql-contrib nginx certbot file-dev
     ```
@@ -50,7 +50,6 @@ Per [`docs/installation/optional/media_graphics_packages.md`](optional/media_gra
 
 === "Alpine"
     ```
-    echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
     apk update
     apk add imagemagick ffmpeg exiftool
     ```

lib/pleroma/activity.ex

@@ -292,7 +292,8 @@ defmodule Pleroma.Activity do
     get_in_reply_to_activity_from_object(Object.normalize(activity, fetch: false))
   end
 
-  def normalize(obj) when is_map(obj), do: get_by_ap_id_with_object(obj["id"])
+  def normalize(%Activity{data: %{"id" => ap_id}}), do: get_by_ap_id_with_object(ap_id)
+  def normalize(%{"id" => ap_id}), do: get_by_ap_id_with_object(ap_id)
   def normalize(ap_id) when is_binary(ap_id), do: get_by_ap_id_with_object(ap_id)
   def normalize(_), do: nil
 
@@ -313,13 +314,15 @@ defmodule Pleroma.Activity do
 
   def delete_all_by_object_ap_id(_), do: nil
 
-  defp purge_web_resp_cache(%Activity{} = activity) do
+  defp purge_web_resp_cache(%Activity{data: %{"id" => id}} = activity) when is_binary(id) do
-    %{path: path} = URI.parse(activity.data["id"])
+    with %{path: path} <- URI.parse(id) do
       @cachex.del(:web_resp_cache, path)
+    end
+
     activity
   end
 
-  defp purge_web_resp_cache(nil), do: nil
+  defp purge_web_resp_cache(activity), do: activity
 
   def follow_accepted?(
         %Activity{data: %{"type" => "Follow", "object" => followed_ap_id}} = activity

lib/pleroma/application_requirements.ex

@@ -168,7 +168,8 @@ defmodule Pleroma.ApplicationRequirements do
       check_filter(Pleroma.Upload.Filter.Mogrify, "mogrify"),
       check_filter(Pleroma.Upload.Filter.Mogrifun, "mogrify"),
       check_filter(Pleroma.Upload.Filter.AnalyzeMetadata, "mogrify"),
-      check_filter(Pleroma.Upload.Filter.AnalyzeMetadata, "convert")
+      check_filter(Pleroma.Upload.Filter.AnalyzeMetadata, "convert"),
+      check_filter(Pleroma.Upload.Filter.AnalyzeMetadata, "ffprobe")
     ]
 
     preview_proxy_commands_status =

lib/pleroma/config/loader.ex

@@ -3,18 +3,18 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Config.Loader do
-  defp reject_keys,
+  # These modules are only being used as keys here (for equality check),
-    do: [
+  # so it's okay to use `Module.concat/1` to have the compiler ignore them.
-      Pleroma.Repo,
+  @reject_keys [
-      Pleroma.Web.Endpoint,
+    Module.concat(["Pleroma.Repo"]),
+    Module.concat(["Pleroma.Web.Endpoint"]),
     :env,
     :configurable_from_database,
     :database,
     :swarm
   ]
 
-  defp reject_groups,
+  @reject_groups [
-    do: [
     :postgrex,
     :tesla
   ]
@@ -54,7 +54,7 @@ defmodule Pleroma.Config.Loader do
   @spec filter_group(atom(), keyword()) :: keyword()
   def filter_group(group, configs) do
     Enum.reject(configs[group], fn {key, _v} ->
-      key in reject_keys() or group in reject_groups() or
+      key in @reject_keys or group in @reject_groups or
         (group == :phoenix and key == :serve_endpoints)
     end)
   end

lib/pleroma/instances.ex

@@ -5,13 +5,18 @@
 defmodule Pleroma.Instances do
   @moduledoc "Instances context."
 
-  @adapter Pleroma.Instances.Instance
+  alias Pleroma.Instances.Instance
 
-  defdelegate filter_reachable(urls_or_hosts), to: @adapter
+  def filter_reachable(urls_or_hosts), do: Instance.filter_reachable(urls_or_hosts)
-  defdelegate reachable?(url_or_host), to: @adapter
+
-  defdelegate set_reachable(url_or_host), to: @adapter
+  def reachable?(url_or_host), do: Instance.reachable?(url_or_host)
-  defdelegate set_unreachable(url_or_host, unreachable_since \\ nil), to: @adapter
+
-  defdelegate get_consistently_unreachable(), to: @adapter
+  def set_reachable(url_or_host), do: Instance.set_reachable(url_or_host)
+
+  def set_unreachable(url_or_host, unreachable_since \\ nil),
+    do: Instance.set_unreachable(url_or_host, unreachable_since)
+
+  def get_consistently_unreachable, do: Instance.get_consistently_unreachable()
 
   def set_consistently_unreachable(url_or_host),
     do: set_unreachable(url_or_host, reachability_datetime_threshold())

lib/pleroma/repo.ex

@@ -8,8 +8,6 @@ defmodule Pleroma.Repo do
     adapter: Ecto.Adapters.Postgres,
     migration_timestamps: [type: :naive_datetime_usec]
 
-  use Ecto.Explain
-
   import Ecto.Query
   require Logger
 

lib/pleroma/tests/auth_test_controller.ex

@@ -9,7 +9,6 @@ defmodule Pleroma.Tests.AuthTestController do
   use Pleroma.Web, :controller
 
   alias Pleroma.User
-  alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Web.Plugs.OAuthScopesPlug
 
   # Serves only with proper OAuth token (:api and :authenticated_api)
@@ -47,10 +46,7 @@ defmodule Pleroma.Tests.AuthTestController do
   # Via :authenticated_api, serves if token is present and has requested scopes
   #
   # Suggested use: as :fallback_oauth_check but open with nil :user for :api on private instances
-  plug(
+  plug(:skip_public_check when action == :fallback_oauth_skip_publicity_check)
-    :skip_plug,
-    EnsurePublicOrAuthenticatedPlug when action == :fallback_oauth_skip_publicity_check
-  )
 
   plug(
     OAuthScopesPlug,
@@ -62,11 +58,7 @@ defmodule Pleroma.Tests.AuthTestController do
   # Via :authenticated_api, serves if :user is set (regardless of token presence and its scopes)
   #
   # Suggested use: making an :api endpoint always accessible (e.g. email confirmation endpoint)
-  plug(
+  plug(:skip_auth when action == :skip_oauth_skip_publicity_check)
-    :skip_plug,
-    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug]
-    when action == :skip_oauth_skip_publicity_check
-  )
 
   # Via :authenticated_api, always fails with 403 (endpoint is insecure)
   # Via :api, drops :user if present and serves if public (private instance rejects on no user)

lib/pleroma/upload/filter.ex

@@ -15,13 +15,13 @@ defmodule Pleroma.Upload.Filter do
 
   require Logger
 
-  @callback filter(Pleroma.Upload.t()) ::
+  @callback filter(upload :: struct()) ::
               {:ok, :filtered}
               | {:ok, :noop}
-              | {:ok, :filtered, Pleroma.Upload.t()}
+              | {:ok, :filtered, upload :: struct()}
               | {:error, any()}
 
-  @spec filter([module()], Pleroma.Upload.t()) :: {:ok, Pleroma.Upload.t()} | {:error, any()}
+  @spec filter([module()], upload :: struct()) :: {:ok, upload :: struct()} | {:error, any()}
 
   def filter([], upload) do
     {:ok, upload}

lib/pleroma/upload/filter/analyze_metadata.ex

@@ -33,6 +33,23 @@ defmodule Pleroma.Upload.Filter.AnalyzeMetadata do
     end
   end
 
+  def filter(%Pleroma.Upload{tempfile: file, content_type: "video" <> _} = upload) do
+    try do
+      result = media_dimensions(file)
+
+      upload =
+        upload
+        |> Map.put(:width, result.width)
+        |> Map.put(:height, result.height)
+
+      {:ok, :filtered, upload}
+    rescue
+      e in ErlangError ->
+        Logger.warn("#{__MODULE__}: #{inspect(e)}")
+        {:ok, :noop}
+    end
+  end
+
   def filter(_), do: {:ok, :noop}
 
   defp get_blurhash(file) do
@@ -42,4 +59,25 @@ defmodule Pleroma.Upload.Filter.AnalyzeMetadata do
       _ -> nil
     end
   end
+
+  defp media_dimensions(file) do
+    with executable when is_binary(executable) <- System.find_executable("ffprobe"),
+         args = [
+           "-v",
+           "error",
+           "-show_entries",
+           "stream=width,height",
+           "-of",
+           "csv=p=0:s=x",
+           file
+         ],
+         {result, 0} <- System.cmd(executable, args),
+         [width, height] <-
+           String.split(String.trim(result), "x") |> Enum.map(&String.to_integer(&1)) do
+      %{width: width, height: height}
+    else
+      nil -> {:error, {:ffprobe, :command_not_found}}
+      {:error, _} = error -> error
+    end
+  end
 end

lib/pleroma/user.ex

@@ -1695,8 +1695,6 @@ defmodule Pleroma.User do
       email: nil,
       name: nil,
       password_hash: nil,
-      keys: nil,
-      public_key: nil,
       avatar: %{},
       tags: [],
       last_refreshed_at: nil,
@@ -1707,9 +1705,7 @@ defmodule Pleroma.User do
       follower_count: 0,
       following_count: 0,
       is_locked: false,
-      is_confirmed: true,
       password_reset_pending: false,
-      is_approved: true,
       registration_reason: nil,
       confirmation_token: nil,
       domain_blocks: [],
@@ -1725,45 +1721,53 @@ defmodule Pleroma.User do
       raw_fields: [],
       is_discoverable: false,
       also_known_as: []
+      # id: preserved
+      # ap_id: preserved
+      # nickname: preserved
     })
   end
 
+  # Purge doesn't delete the user from the database.
+  # It just nulls all its fields and deactivates it.
+  # See `User.purge_user_changeset/1` above.
+  defp purge(%User{} = user) do
+    user
+    |> purge_user_changeset()
+    |> update_and_set_cache()
+  end
+
   def delete(users) when is_list(users) do
     for user <- users, do: delete(user)
   end
 
   def delete(%User{} = user) do
+    # Purge the user immediately
+    purge(user)
     BackgroundWorker.enqueue("delete_user", %{"user_id" => user.id})
   end
 
-  defp delete_and_invalidate_cache(%User{} = user) do
+  # *Actually* delete the user from the DB
+  defp delete_from_db(%User{} = user) do
     invalidate_cache(user)
     Repo.delete(user)
   end
 
-  defp delete_or_deactivate(%User{local: false} = user), do: delete_and_invalidate_cache(user)
+  # If the user never finalized their account, it's safe to delete them.
+  defp maybe_delete_from_db(%User{local: true, is_confirmed: false} = user),
+    do: delete_from_db(user)
 
-  defp delete_or_deactivate(%User{local: true} = user) do
+  defp maybe_delete_from_db(%User{local: true, is_approved: false} = user),
-    status = account_status(user)
+    do: delete_from_db(user)
 
-    case status do
+  defp maybe_delete_from_db(user), do: {:ok, user}
-      :confirmation_pending ->
-        delete_and_invalidate_cache(user)
-
-      :approval_pending ->
-        delete_and_invalidate_cache(user)
-
-      _ ->
-        user
-        |> purge_user_changeset()
-        |> update_and_set_cache()
-    end
-  end
 
   def perform(:force_password_reset, user), do: force_password_reset(user)
 
   @spec perform(atom(), User.t()) :: {:ok, User.t()}
   def perform(:delete, %User{} = user) do
+    # Purge the user again, in case perform/2 is called directly
+    purge(user)
+
     # Remove all relationships
     user
     |> get_followers()
@@ -1781,10 +1785,9 @@ defmodule Pleroma.User do
 
     delete_user_activities(user)
     delete_notifications_from_user_activities(user)
-
     delete_outgoing_pending_follow_requests(user)
 
-    delete_or_deactivate(user)
+    maybe_delete_from_db(user)
   end
 
   def perform(:set_activation_async, user, status), do: set_activation(user, status)

lib/pleroma/web.ex

@@ -62,6 +62,14 @@ defmodule Pleroma.Web do
         )
       end
 
+      defp skip_auth(conn, _) do
+        skip_plug(conn, [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug])
+      end
+
+      defp skip_public_check(conn, _) do
+        skip_plug(conn, EnsurePublicOrAuthenticatedPlug)
+      end
+
       # Executed just before actual controller action, invokes before-action hooks (callbacks)
       defp action(conn, params) do
         with %{halted: false} = conn <-

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -53,15 +53,18 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
     {recipients, to, cc}
   end
 
-  defp check_actor_is_active(nil), do: true
+  defp check_actor_can_insert(%{"type" => "Delete"}), do: true
+  defp check_actor_can_insert(%{"type" => "Undo"}), do: true
 
-  defp check_actor_is_active(actor) when is_binary(actor) do
+  defp check_actor_can_insert(%{"actor" => actor}) when is_binary(actor) do
     case User.get_cached_by_ap_id(actor) do
       %User{is_active: true} -> true
       _ -> false
     end
   end
 
+  defp check_actor_can_insert(_), do: true
+
   defp check_remote_limit(%{"object" => %{"content" => content}}) when not is_nil(content) do
     limit = Config.get([:instance, :remote_limit])
     String.length(content) <= limit
@@ -88,7 +91,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
 
   defp increase_replies_count_if_reply(_create_data), do: :noop
 
-  @object_types ~w[ChatMessage Question Answer Audio Video Event Article Note]
+  @object_types ~w[ChatMessage Question Answer Audio Video Event Article Note Page]
   @impl true
   def persist(%{"type" => type} = object, meta) when type in @object_types do
     with {:ok, object} <- Object.create(object) do
@@ -117,7 +120,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when is_map(map) do
     with nil <- Activity.normalize(map),
          map <- lazy_put_activity_defaults(map, fake),
-         {_, true} <- {:actor_check, bypass_actor_check || check_actor_is_active(map["actor"])},
+         {_, true} <- {:actor_check, bypass_actor_check || check_actor_can_insert(map)},
          {_, true} <- {:remote_limit_pass, check_remote_limit(map)},
          {:ok, map} <- MRF.filter(map),
          {recipients, _, _} = get_recipients(map),

lib/pleroma/web/activity_pub/activity_pub_controller.ex

@@ -11,7 +11,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
   alias Pleroma.Object.Fetcher
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
-  alias Pleroma.Web.ActivityPub.Builder
   alias Pleroma.Web.ActivityPub.InternalFetchActor
   alias Pleroma.Web.ActivityPub.ObjectView
   alias Pleroma.Web.ActivityPub.Pipeline
@@ -403,83 +402,90 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
     |> json(err)
   end
 
-  defp handle_user_activity(
+  defp fix_user_message(%User{ap_id: actor}, %{"type" => "Create", "object" => object} = activity)
-         %User{} = user,
+       when is_map(object) do
-         %{"type" => "Create", "object" => %{"type" => "Note"} = object} = params
+    length =
-       ) do
+      [object["content"], object["summary"], object["name"]]
-    content = if is_binary(object["content"]), do: object["content"], else: ""
+      |> Enum.filter(&is_binary(&1))
-    name = if is_binary(object["name"]), do: object["name"], else: ""
+      |> Enum.join("")
-    summary = if is_binary(object["summary"]), do: object["summary"], else: ""
+      |> String.length()
-    length = String.length(content <> name <> summary)
 
-    if length > Pleroma.Config.get([:instance, :limit]) do
+    limit = Pleroma.Config.get([:instance, :limit])
-      {:error, dgettext("errors", "Note is over the character limit")}
+
-    else
+    if length < limit do
       object =
         object
-        |> Map.merge(Map.take(params, ["to", "cc"]))
+        |> Transmogrifier.strip_internal_fields()
-        |> Map.put("attributedTo", user.ap_id)
+        |> Map.put("attributedTo", actor)
-        |> Transmogrifier.fix_object()
+        |> Map.put("actor", actor)
+        |> Map.put("id", Utils.generate_object_id())
 
-      ActivityPub.create(%{
+      {:ok, Map.put(activity, "object", object)}
-        to: params["to"],
-        actor: user,
-        context: object["context"],
-        object: object,
-        additional: Map.take(params, ["cc"])
-      })
-    end
-  end
-
-  defp handle_user_activity(%User{} = user, %{"type" => "Delete"} = params) do
-    with %Object{} = object <- Object.normalize(params["object"], fetch: false),
-         true <- user.is_moderator || user.ap_id == object.data["actor"],
-         {:ok, delete_data, _} <- Builder.delete(user, object.data["id"]),
-         {:ok, delete, _} <- Pipeline.common_pipeline(delete_data, local: true) do
-      {:ok, delete}
     else
-      _ -> {:error, dgettext("errors", "Can't delete object")}
+      {:error,
+       dgettext(
+         "errors",
+         "Character limit (%{limit} characters) exceeded, contains %{length} characters",
+         limit: limit,
+         length: length
+       )}
     end
   end
 
-  defp handle_user_activity(%User{} = user, %{"type" => "Like"} = params) do
+  defp fix_user_message(
-    with %Object{} = object <- Object.normalize(params["object"], fetch: false),
+         %User{ap_id: actor} = user,
-         {_, {:ok, like_object, meta}} <- {:build_object, Builder.like(user, object)},
+         %{"type" => "Delete", "object" => object} = activity
-         {_, {:ok, %Activity{} = activity, _meta}} <-
+       ) do
-           {:common_pipeline,
+    with {_, %Object{data: object_data}} <- {:normalize, Object.normalize(object, fetch: false)},
-            Pipeline.common_pipeline(like_object, Keyword.put(meta, :local, true))} do
+         {_, true} <- {:permission, user.is_moderator || actor == object_data["actor"]} do
       {:ok, activity}
     else
-      _ -> {:error, dgettext("errors", "Can't like object")}
+      {:normalize, _} ->
+        {:error, "No such object found"}
+
+      {:permission, _} ->
+        {:forbidden, "You can't delete this object"}
     end
   end
 
-  defp handle_user_activity(_, _) do
+  defp fix_user_message(%User{}, activity) do
-    {:error, dgettext("errors", "Unhandled activity type")}
+    {:ok, activity}
   end
 
   def update_outbox(
-        %{assigns: %{user: %User{nickname: nickname} = user}} = conn,
+        %{assigns: %{user: %User{nickname: nickname, ap_id: actor} = user}} = conn,
         %{"nickname" => nickname} = params
       ) do
-    actor = user.ap_id
-
     params =
       params
-      |> Map.drop(["id"])
+      |> Map.drop(["nickname"])
+      |> Map.put("id", Utils.generate_activity_id())
       |> Map.put("actor", actor)
-      |> Transmogrifier.fix_addressing()
 
-    with {:ok, %Activity{} = activity} <- handle_user_activity(user, params) do
+    with {:ok, params} <- fix_user_message(user, params),
+         {:ok, activity, _} <- Pipeline.common_pipeline(params, local: true),
+         %Activity{data: activity_data} <- Activity.normalize(activity) do
       conn
       |> put_status(:created)
-      |> put_resp_header("location", activity.data["id"])
+      |> put_resp_header("location", activity_data["id"])
-      |> json(activity.data)
+      |> json(activity_data)
     else
+      {:forbidden, message} ->
+        conn
+        |> put_status(:forbidden)
+        |> json(message)
+
       {:error, message} ->
         conn
         |> put_status(:bad_request)
         |> json(message)
+
+      e ->
+        Logger.warn(fn -> "AP C2S: #{inspect(e)}" end)
+
+        conn
+        |> put_status(:bad_request)
+        |> json("Bad Request")
     end
   end
 

lib/pleroma/web/activity_pub/object_validator.ex

@@ -20,7 +20,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do
   alias Pleroma.Web.ActivityPub.ObjectValidators.AddRemoveValidator
   alias Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator
   alias Pleroma.Web.ActivityPub.ObjectValidators.AnswerValidator
-  alias Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.ArticleNotePageValidator
   alias Pleroma.Web.ActivityPub.ObjectValidators.AudioVideoValidator
   alias Pleroma.Web.ActivityPub.ObjectValidators.BlockValidator
   alias Pleroma.Web.ActivityPub.ObjectValidators.ChatMessageValidator
@@ -102,7 +102,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do
         %{"type" => "Create", "object" => %{"type" => objtype} = object} = create_activity,
         meta
       )
-      when objtype in ~w[Question Answer Audio Video Event Article Note] do
+      when objtype in ~w[Question Answer Audio Video Event Article Note Page] do
     with {:ok, object_data} <- cast_and_apply(object),
          meta = Keyword.put(meta, :object_data, object_data |> stringify_keys),
          {:ok, create_activity} <-
@@ -115,15 +115,16 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do
   end
 
   def validate(%{"type" => type} = object, meta)
-      when type in ~w[Event Question Audio Video Article Note] do
+      when type in ~w[Event Question Audio Video Article Note Page] do
     validator =
       case type do
         "Event" -> EventValidator
         "Question" -> QuestionValidator
         "Audio" -> AudioVideoValidator
         "Video" -> AudioVideoValidator
-        "Article" -> ArticleNoteValidator
+        "Article" -> ArticleNotePageValidator
-        "Note" -> ArticleNoteValidator
+        "Note" -> ArticleNotePageValidator
+        "Page" -> ArticleNotePageValidator
       end
 
     with {:ok, object} <-
@@ -175,6 +176,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do
     end
   end
 
+  def validate(o, m), do: {:error, {:validator_not_set, {o, m}}}
+
   def cast_and_apply(%{"type" => "ChatMessage"} = object) do
     ChatMessageValidator.cast_and_apply(object)
   end
@@ -195,8 +198,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do
     EventValidator.cast_and_apply(object)
   end
 
-  def cast_and_apply(%{"type" => type} = object) when type in ~w[Article Note] do
+  def cast_and_apply(%{"type" => type} = object) when type in ~w[Article Note Page] do
-    ArticleNoteValidator.cast_and_apply(object)
+    ArticleNotePageValidator.cast_and_apply(object)
   end
 
   def cast_and_apply(o), do: {:error, {:validator_not_set, o}}

lib/pleroma/web/activity_pub/object_validators/announce_validator.ex

@@ -8,6 +8,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator do
   alias Pleroma.EctoType.ActivityPub.ObjectValidators
   alias Pleroma.Object
   alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.ActivityPub.Visibility
 
@@ -23,7 +24,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator do
     field(:type, :string)
     field(:object, ObjectValidators.ObjectID)
     field(:actor, ObjectValidators.ObjectID)
-    field(:context, :string, autogenerate: {Utils, :generate_context_id, []})
+    field(:context, :string)
     field(:to, ObjectValidators.Recipients, default: [])
     field(:cc, ObjectValidators.Recipients, default: [])
     field(:published, ObjectValidators.DateTime)
@@ -36,6 +37,10 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator do
   end
 
   def cast_data(data) do
+    data =
+      data
+      |> fix()
+
     %__MODULE__{}
     |> changeset(data)
   end
@@ -43,11 +48,21 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator do
   def changeset(struct, data) do
     struct
     |> cast(data, __schema__(:fields))
-    |> fix_after_cast()
   end
 
-  def fix_after_cast(cng) do
+  defp fix(data) do
-    cng
+    data =
+      data
+      |> CommonFixes.fix_actor()
+      |> CommonFixes.fix_activity_addressing()
+
+    with %Object{} = object <- Object.normalize(data["object"]) do
+      data
+      |> CommonFixes.fix_activity_context(object)
+      |> CommonFixes.fix_object_action_recipients(object)
+    else
+      _ -> data
+    end
   end
 
   defp validate_data(data_cng) do
@@ -60,7 +75,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator do
     |> validate_announcable()
   end
 
-  def validate_announcable(cng) do
+  defp validate_announcable(cng) do
     with actor when is_binary(actor) <- get_field(cng, :actor),
          object when is_binary(object) <- get_field(cng, :object),
          %User{} = actor <- User.get_cached_by_ap_id(actor),
@@ -91,7 +106,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator do
     end
   end
 
-  def validate_existing_announce(cng) do
+  defp validate_existing_announce(cng) do
     actor = get_field(cng, :actor)
     object = get_field(cng, :object)
 

lib/pleroma/web/activity_pub/object_validators/article_note_page_validator.ex

@@ -2,7 +2,7 @@
 # Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only
 
-defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidator do
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNotePageValidator do
   use Ecto.Schema
 
   alias Pleroma.EctoType.ActivityPub.ObjectValidators
@@ -113,7 +113,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidator do
 
   defp validate_data(data_cng) do
     data_cng
-    |> validate_inclusion(:type, ["Article", "Note"])
+    |> validate_inclusion(:type, ["Article", "Note", "Page"])
     |> validate_required([:id, :actor, :attributedTo, :type, :context, :context_id])
     |> CommonValidations.validate_any_presence([:cc, :to])
     |> CommonValidations.validate_fields_match([:actor, :attributedTo])

lib/pleroma/web/activity_pub/object_validators/common_fixes.ex

@@ -4,6 +4,7 @@
 
 defmodule Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes do
   alias Pleroma.EctoType.ActivityPub.ObjectValidators
+  alias Pleroma.Object
   alias Pleroma.Object.Containment
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.Transmogrifier
@@ -36,7 +37,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes do
     |> Transmogrifier.fix_implicit_addressing(follower_collection)
   end
 
-  def fix_activity_addressing(activity, _meta) do
+  def fix_activity_addressing(activity) do
     %User{follower_address: follower_collection} = User.get_cached_by_ap_id(activity["actor"])
 
     activity
@@ -57,4 +58,21 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes do
     |> Map.put("actor", actor)
     |> Map.put("attributedTo", actor)
   end
+
+  def fix_activity_context(data, %Object{data: %{"context" => object_context}}) do
+    data
+    |> Map.put("context", object_context)
+  end
+
+  def fix_object_action_recipients(%{"actor" => actor} = data, %Object{data: %{"actor" => actor}}) do
+    to = ((data["to"] || []) -- [actor]) |> Enum.uniq()
+
+    Map.put(data, "to", to)
+  end
+
+  def fix_object_action_recipients(data, %Object{data: %{"actor" => actor}}) do
+    to = ((data["to"] || []) ++ [actor]) |> Enum.uniq()
+
+    Map.put(data, "to", to)
+  end
 end

lib/pleroma/web/activity_pub/object_validators/delete_validator.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator do
 
   alias Pleroma.Activity
   alias Pleroma.EctoType.ActivityPub.ObjectValidators
+  alias Pleroma.User
 
   import Ecto.Changeset
   import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
@@ -57,7 +58,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator do
     cng
     |> validate_required([:id, :type, :actor, :to, :cc, :object])
     |> validate_inclusion(:type, ["Delete"])
-    |> validate_actor_presence()
+    |> validate_delete_actor(:actor)
     |> validate_modification_rights()
     |> validate_object_or_user_presence(allowed_types: @deletable_types)
     |> add_deleted_activity_id()
@@ -72,4 +73,13 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator do
     |> cast_data
     |> validate_data
   end
+
+  defp validate_delete_actor(cng, field_name) do
+    validate_change(cng, field_name, fn field_name, actor ->
+      case User.get_cached_by_ap_id(actor) do
+        %User{} -> []
+        _ -> [{field_name, "can't find user"}]
+      end
+    end)
+  end
 end

lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator do
 
   alias Pleroma.EctoType.ActivityPub.ObjectValidators
   alias Pleroma.Object
+  alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes
 
   import Ecto.Changeset
   import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
@@ -31,6 +32,10 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator do
   end
 
   def cast_data(data) do
+    data =
+      data
+      |> fix()
+
     %__MODULE__{}
     |> changeset(data)
   end
@@ -38,28 +43,24 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator do
   def changeset(struct, data) do
     struct
     |> cast(data, __schema__(:fields))
-    |> fix_after_cast()
   end
 
-  def fix_after_cast(cng) do
+  defp fix(data) do
-    cng
+    data =
-    |> fix_context()
+      data
-  end
+      |> CommonFixes.fix_actor()
+      |> CommonFixes.fix_activity_addressing()
 
-  def fix_context(cng) do
+    with %Object{} = object <- Object.normalize(data["object"]) do
-    object = get_field(cng, :object)
+      data
-
+      |> CommonFixes.fix_activity_context(object)
-    with nil <- get_field(cng, :context),
+      |> CommonFixes.fix_object_action_recipients(object)
-         %Object{data: %{"context" => context}} <- Object.get_cached_by_ap_id(object) do
-      cng
-      |> put_change(:context, context)
     else
-      _ ->
+      _ -> data
-        cng
     end
   end
 
-  def validate_emoji(cng) do
+  defp validate_emoji(cng) do
     content = get_field(cng, :content)
 
     if Pleroma.Emoji.is_unicode_emoji?(content) do

lib/pleroma/web/activity_pub/object_validators/like_validator.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
 
   alias Pleroma.EctoType.ActivityPub.ObjectValidators
   alias Pleroma.Object
+  alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes
   alias Pleroma.Web.ActivityPub.Utils
 
   import Ecto.Changeset
@@ -31,6 +32,10 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
   end
 
   def cast_data(data) do
+    data =
+      data
+      |> fix()
+
     %__MODULE__{}
     |> changeset(data)
   end
@@ -38,41 +43,20 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
   def changeset(struct, data) do
     struct
     |> cast(data, __schema__(:fields))
-    |> fix_after_cast()
   end
 
-  def fix_after_cast(cng) do
+  defp fix(data) do
-    cng
+    data =
-    |> fix_recipients()
+      data
-    |> fix_context()
+      |> CommonFixes.fix_actor()
-  end
+      |> CommonFixes.fix_activity_addressing()
 
-  def fix_context(cng) do
+    with %Object{} = object <- Object.normalize(data["object"]) do
-    object = get_field(cng, :object)
+      data
-
+      |> CommonFixes.fix_activity_context(object)
-    with nil <- get_field(cng, :context),
+      |> CommonFixes.fix_object_action_recipients(object)
-         %Object{data: %{"context" => context}} <- Object.get_cached_by_ap_id(object) do
-      cng
-      |> put_change(:context, context)
     else
-      _ ->
+      _ -> data
-        cng
-    end
-  end
-
-  def fix_recipients(cng) do
-    to = get_field(cng, :to)
-    cc = get_field(cng, :cc)
-    object = get_field(cng, :object)
-
-    with {[], []} <- {to, cc},
-         %Object{data: %{"actor" => actor}} <- Object.get_cached_by_ap_id(object),
-         {:ok, actor} <- ObjectValidators.ObjectID.cast(actor) do
-      cng
-      |> put_change(:to, [actor])
-    else
-      _ ->
-        cng
     end
   end
 
@@ -85,7 +69,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
     |> validate_existing_like()
   end
 
-  def validate_existing_like(%{changes: %{actor: actor, object: object}} = cng) do
+  defp validate_existing_like(%{changes: %{actor: actor, object: object}} = cng) do
     if Utils.get_existing_like(actor, %{data: %{"id" => object}}) do
       cng
       |> add_error(:actor, "already liked this object")
@@ -95,5 +79,5 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
     end
   end
 
-  def validate_existing_like(cng), do: cng
+  defp validate_existing_like(cng), do: cng
 end

lib/pleroma/web/activity_pub/object_validators/undo_validator.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator do
 
   alias Pleroma.Activity
   alias Pleroma.EctoType.ActivityPub.ObjectValidators
+  alias Pleroma.User
 
   import Ecto.Changeset
   import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
@@ -42,7 +43,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator do
     data_cng
     |> validate_inclusion(:type, ["Undo"])
     |> validate_required([:id, :type, :object, :actor, :to, :cc])
-    |> validate_actor_presence()
+    |> validate_undo_actor(:actor)
     |> validate_object_presence()
     |> validate_undo_rights()
   end
@@ -59,4 +60,13 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator do
       _ -> cng
     end
   end
+
+  defp validate_undo_actor(cng, field_name) do
+    validate_change(cng, field_name, fn field_name, actor ->
+      case User.get_cached_by_ap_id(actor) do
+        %User{} -> []
+        _ -> [{field_name, "can't find user"}]
+      end
+    end)
+  end
 end

lib/pleroma/web/activity_pub/side_effects.ex

@@ -28,11 +28,12 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
   require Logger
 
   @cachex Pleroma.Config.get([:cachex, :provider], Cachex)
-  @ap_streamer Pleroma.Config.get([:side_effects, :ap_streamer], ActivityPub)
   @logger Pleroma.Config.get([:side_effects, :logger], Logger)
 
   @behaviour Pleroma.Web.ActivityPub.SideEffects.Handling
 
+  defp ap_streamer, do: Pleroma.Config.get([:side_effects, :ap_streamer], ActivityPub)
+
   @impl true
   def handle(object, meta \\ [])
 
@@ -302,8 +303,8 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
 
             MessageReference.delete_for_object(deleted_object)
 
-            @ap_streamer.stream_out(object)
+            ap_streamer().stream_out(object)
-            @ap_streamer.stream_out_participations(deleted_object, user)
+            ap_streamer().stream_out_participations(deleted_object, user)
             :ok
           else
             {:actor, _} ->
@@ -436,7 +437,7 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
   end
 
   def handle_object_creation(%{"type" => objtype} = object, meta)
-      when objtype in ~w[Audio Video Question Event Article Note] do
+      when objtype in ~w[Audio Video Question Event Article Note Page] do
     with {:ok, object, meta} <- Pipeline.common_pipeline(object, meta) do
       {:ok, object, meta}
     end

lib/pleroma/web/activity_pub/transmogrifier.ex

@@ -353,29 +353,6 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
     end)
   end
 
-  # Compatibility wrapper for Mastodon votes
-  defp handle_create(%{"object" => %{"type" => "Answer"}} = data, _user) do
-    handle_incoming(data)
-  end
-
-  defp handle_create(%{"object" => object} = data, user) do
-    %{
-      to: data["to"],
-      object: object,
-      actor: user,
-      context: object["context"],
-      local: false,
-      published: data["published"],
-      additional:
-        Map.take(data, [
-          "cc",
-          "directMessage",
-          "id"
-        ])
-    }
-    |> ActivityPub.create()
-  end
-
   def handle_incoming(data, options \\ [])
 
   # Flag objects are placed ahead of the ID check because Mastodon 2.8 and earlier send them
@@ -407,43 +384,6 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
   def handle_incoming(%{"id" => id}, _options) when is_binary(id) and byte_size(id) < 8,
     do: :error
 
-  # TODO: validate those with a Ecto scheme
-  # - tags
-  # - emoji
-  def handle_incoming(
-        %{"type" => "Create", "object" => %{"type" => "Page"} = object} = data,
-        options
-      ) do
-    actor = Containment.get_actor(data)
-
-    with nil <- Activity.get_create_by_object_ap_id(object["id"]),
-         {:ok, %User{} = user} <- User.get_or_fetch_by_ap_id(actor) do
-      data =
-        data
-        |> Map.put("object", fix_object(object, options))
-        |> Map.put("actor", actor)
-        |> fix_addressing()
-
-      with {:ok, created_activity} <- handle_create(data, user) do
-        reply_depth = (options[:depth] || 0) + 1
-
-        if Federator.allowed_thread_distance?(reply_depth) do
-          for reply_id <- replies(object) do
-            Pleroma.Workers.RemoteFetcherWorker.enqueue("fetch_remote", %{
-              "id" => reply_id,
-              "depth" => reply_depth
-            })
-          end
-        end
-
-        {:ok, created_activity}
-      end
-    else
-      %Activity{} = activity -> {:ok, activity}
-      _e -> :error
-    end
-  end
-
   def handle_incoming(
         %{"type" => "Listen", "object" => %{"type" => "Audio"} = object} = data,
         options
@@ -507,7 +447,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
         %{"type" => "Create", "object" => %{"type" => objtype, "id" => obj_id}} = data,
         options
       )
-      when objtype in ~w{Question Answer ChatMessage Audio Video Event Article Note} do
+      when objtype in ~w{Question Answer ChatMessage Audio Video Event Article Note Page} do
     fetch_options = Keyword.put(options, :depth, (options[:depth] || 0) + 1)
 
     object =

lib/pleroma/web/admin_api/controllers/user_controller.ex

@@ -45,8 +45,6 @@ defmodule Pleroma.Web.AdminAPI.UserController do
     when action in [:follow, :unfollow]
   )
 
-  plug(:put_view, Pleroma.Web.AdminAPI.AccountView)
-
   action_fallback(AdminAPI.FallbackController)
 
   defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.UserOperation

lib/pleroma/web/admin_api/views/user_view.ex

@@ -0,0 +1,10 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.UserView do
+  use Pleroma.Web, :view
+  alias Pleroma.Web.AdminAPI
+
+  def render(view, opts), do: AdminAPI.AccountView.render(view, opts)
+end

lib/pleroma/web/api_spec/operations/media_operation.ex

@@ -24,6 +24,7 @@ defmodule Pleroma.Web.ApiSpec.MediaOperation do
       requestBody: Helpers.request_body("Parameters", create_request()),
       responses: %{
         200 => Operation.response("Media", "application/json", Attachment),
+        400 => Operation.response("Media", "application/json", ApiError),
         401 => Operation.response("Media", "application/json", ApiError),
         422 => Operation.response("Media", "application/json", ApiError)
       }
@@ -121,6 +122,7 @@ defmodule Pleroma.Web.ApiSpec.MediaOperation do
       requestBody: Helpers.request_body("Parameters", create_request()),
       responses: %{
         202 => Operation.response("Media", "application/json", Attachment),
+        400 => Operation.response("Media", "application/json", ApiError),
         422 => Operation.response("Media", "application/json", ApiError),
         500 => Operation.response("Media", "application/json", ApiError)
       }

lib/pleroma/web/api_spec/schemas/boolean_like.ex

@@ -34,7 +34,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.BooleanLike do
 
   def cast(%Cast{value: value} = context) do
     context
-    |> Map.put(:value, Pleroma.Web.ControllerHelper.truthy_param?(value))
+    |> Map.put(:value, Pleroma.Web.Utils.Params.truthy_param?(value))
     |> Cast.ok()
   end
 end

lib/pleroma/web/auth/authenticator.ex

@@ -3,68 +3,11 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.Auth.Authenticator do
-  alias Pleroma.Registration
+  @callback get_user(Plug.Conn.t()) :: {:ok, user :: struct()} | {:error, any()}
-  alias Pleroma.User
+  @callback create_from_registration(Plug.Conn.t(), registration :: struct()) ::
-
-  def implementation do
-    Pleroma.Config.get(
-      Pleroma.Web.Auth.Authenticator,
-      Pleroma.Web.Auth.PleromaAuthenticator
-    )
-  end
-
-  @callback get_user(Plug.Conn.t()) :: {:ok, User.t()} | {:error, any()}
-  def get_user(plug), do: implementation().get_user(plug)
-
-  @callback create_from_registration(Plug.Conn.t(), Registration.t()) ::
               {:ok, User.t()} | {:error, any()}
-  def create_from_registration(plug, registration),
+  @callback get_registration(Plug.Conn.t()) :: {:ok, registration :: struct()} | {:error, any()}
-    do: implementation().create_from_registration(plug, registration)
-
-  @callback get_registration(Plug.Conn.t()) :: {:ok, Registration.t()} | {:error, any()}
-  def get_registration(plug), do: implementation().get_registration(plug)
-
   @callback handle_error(Plug.Conn.t(), any()) :: any()
-  def handle_error(plug, error),
-    do: implementation().handle_error(plug, error)
-
   @callback auth_template() :: String.t() | nil
-  def auth_template do
-    # Note: `config :pleroma, :auth_template, "..."` support is deprecated
-    implementation().auth_template() ||
-      Pleroma.Config.get([:auth, :auth_template], Pleroma.Config.get(:auth_template)) ||
-      "show.html"
-  end
-
   @callback oauth_consumer_template() :: String.t() | nil
-  def oauth_consumer_template do
-    implementation().oauth_consumer_template() ||
-      Pleroma.Config.get([:auth, :oauth_consumer_template], "consumer.html")
-  end
-
-  @doc "Gets user by nickname or email for auth."
-  @spec fetch_user(String.t()) :: User.t() | nil
-  def fetch_user(name) do
-    User.get_by_nickname_or_email(name)
-  end
-
-  # Gets name and password from conn
-  #
-  @spec fetch_credentials(Plug.Conn.t() | map()) ::
-          {:ok, {name :: any, password :: any}} | {:error, :invalid_credentials}
-  def fetch_credentials(%Plug.Conn{params: params} = _),
-    do: fetch_credentials(params)
-
-  def fetch_credentials(params) do
-    case params do
-      %{"authorization" => %{"name" => name, "password" => password}} ->
-        {:ok, {name, password}}
-
-      %{"grant_type" => "password", "username" => name, "password" => password} ->
-        {:ok, {name, password}}
-
-      _ ->
-        {:error, :invalid_credentials}
-    end
-  end
 end

lib/pleroma/web/auth/helpers.ex

@@ -0,0 +1,33 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Auth.Helpers do
+  alias Pleroma.User
+
+  @doc "Gets user by nickname or email for auth."
+  @spec fetch_user(String.t()) :: User.t() | nil
+  def fetch_user(name) do
+    User.get_by_nickname_or_email(name)
+  end
+
+  # Gets name and password from conn
+  #
+  @spec fetch_credentials(Plug.Conn.t() | map()) ::
+          {:ok, {name :: any, password :: any}} | {:error, :invalid_credentials}
+  def fetch_credentials(%Plug.Conn{params: params} = _),
+    do: fetch_credentials(params)
+
+  def fetch_credentials(params) do
+    case params do
+      %{"authorization" => %{"name" => name, "password" => password}} ->
+        {:ok, {name, password}}
+
+      %{"grant_type" => "password", "username" => name, "password" => password} ->
+        {:ok, {name, password}}
+
+      _ ->
+        {:error, :invalid_credentials}
+    end
+  end
+end

lib/pleroma/web/auth/ldap_authenticator.ex

@@ -7,8 +7,7 @@ defmodule Pleroma.Web.Auth.LDAPAuthenticator do
 
   require Logger
 
-  import Pleroma.Web.Auth.Authenticator,
+  import Pleroma.Web.Auth.Helpers, only: [fetch_credentials: 1, fetch_user: 1]
-    only: [fetch_credentials: 1, fetch_user: 1]
 
   @behaviour Pleroma.Web.Auth.Authenticator
   @base Pleroma.Web.Auth.PleromaAuthenticator

lib/pleroma/web/auth/pleroma_authenticator.ex

@@ -8,8 +8,7 @@ defmodule Pleroma.Web.Auth.PleromaAuthenticator do
   alias Pleroma.User
   alias Pleroma.Web.Plugs.AuthenticationPlug
 
-  import Pleroma.Web.Auth.Authenticator,
+  import Pleroma.Web.Auth.Helpers, only: [fetch_credentials: 1, fetch_user: 1]
-    only: [fetch_credentials: 1, fetch_user: 1]
 
   @behaviour Pleroma.Web.Auth.Authenticator
 

lib/pleroma/web/auth/wrapper_authenticator.ex

@@ -0,0 +1,42 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Auth.WrapperAuthenticator do
+  @behaviour Pleroma.Web.Auth.Authenticator
+
+  defp implementation do
+    Pleroma.Config.get(
+      Pleroma.Web.Auth.Authenticator,
+      Pleroma.Web.Auth.PleromaAuthenticator
+    )
+  end
+
+  @impl true
+  def get_user(plug), do: implementation().get_user(plug)
+
+  @impl true
+  def create_from_registration(plug, registration),
+    do: implementation().create_from_registration(plug, registration)
+
+  @impl true
+  def get_registration(plug), do: implementation().get_registration(plug)
+
+  @impl true
+  def handle_error(plug, error),
+    do: implementation().handle_error(plug, error)
+
+  @impl true
+  def auth_template do
+    # Note: `config :pleroma, :auth_template, "..."` support is deprecated
+    implementation().auth_template() ||
+      Pleroma.Config.get([:auth, :auth_template], Pleroma.Config.get(:auth_template)) ||
+      "show.html"
+  end
+
+  @impl true
+  def oauth_consumer_template do
+    implementation().oauth_consumer_template() ||
+      Pleroma.Config.get([:auth, :oauth_consumer_template], "consumer.html")
+  end
+end

lib/pleroma/web/common_api/activity_draft.ex

@@ -223,7 +223,7 @@ defmodule Pleroma.Web.CommonAPI.ActivityDraft do
   end
 
   defp preview?(draft) do
-    preview? = Pleroma.Web.ControllerHelper.truthy_param?(draft.params[:preview])
+    preview? = Pleroma.Web.Utils.Params.truthy_param?(draft.params[:preview])
     %__MODULE__{draft | preview?: preview?}
   end
 

lib/pleroma/web/common_api/utils.ex

@@ -4,7 +4,6 @@
 
 defmodule Pleroma.Web.CommonAPI.Utils do
   import Pleroma.Web.Gettext
-  import Pleroma.Web.ControllerHelper, only: [truthy_param?: 1]
 
   alias Calendar.Strftime
   alias Pleroma.Activity
@@ -19,6 +18,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
   alias Pleroma.Web.CommonAPI.ActivityDraft
   alias Pleroma.Web.MediaProxy
   alias Pleroma.Web.Plugs.AuthenticationPlug
+  alias Pleroma.Web.Utils.Params
 
   require Logger
   require Pleroma.Constants
@@ -160,7 +160,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
         |> DateTime.add(expires_in)
         |> DateTime.to_iso8601()
 
-      key = if truthy_param?(data.poll[:multiple]), do: "anyOf", else: "oneOf"
+      key = if Params.truthy_param?(data.poll[:multiple]), do: "anyOf", else: "oneOf"
       poll = %{"type" => "Question", key => option_notes, "closed" => end_time}
 
       {:ok, {poll, emoji}}
@@ -203,7 +203,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
     attachment_links =
       draft.params
       |> Map.get("attachment_links", Config.get([:instance, :attachment_links]))
-      |> truthy_param?()
+      |> Params.truthy_param?()
 
     content_type = get_content_type(draft.params[:content_type])
 

lib/pleroma/web/controller_helper.ex

@@ -6,17 +6,7 @@ defmodule Pleroma.Web.ControllerHelper do
   use Pleroma.Web, :controller
 
   alias Pleroma.Pagination
-
+  alias Pleroma.Web.Utils.Params
-  # As in Mastodon API, per https://api.rubyonrails.org/classes/ActiveModel/Type/Boolean.html
-  @falsy_param_values [false, 0, "0", "f", "F", "false", "False", "FALSE", "off", "OFF"]
-
-  def explicitly_falsy_param?(value), do: value in @falsy_param_values
-
-  # Note: `nil` and `""` are considered falsy values in Pleroma
-  def falsy_param?(value),
-    do: explicitly_falsy_param?(value) or value in [nil, ""]
-
-  def truthy_param?(value), do: not falsy_param?(value)
 
   def json_response(conn, status, _) when status in [204, :no_content] do
     conn
@@ -123,6 +113,6 @@ defmodule Pleroma.Web.ControllerHelper do
     # To do once OpenAPI transition mess is over: just `truthy_param?(params[:with_relationships])`
     params
     |> Map.get(:with_relationships, params["with_relationships"])
-    |> truthy_param?()
+    |> Params.truthy_param?()
   end
 end

lib/pleroma/web/endpoint.ex

@@ -102,7 +102,7 @@ defmodule Pleroma.Web.Endpoint do
   plug(Plug.Parsers,
     parsers: [
       :urlencoded,
-      {:multipart, length: Config.get([:instance, :upload_limit])},
+      {:multipart, length: {Config, :get, [[:instance, :upload_limit]]}},
       :json
     ],
     pass: ["*/*"],

lib/pleroma/web/masto_fe_controller.ex

@@ -8,13 +8,12 @@ defmodule Pleroma.Web.MastoFEController do
   alias Pleroma.User
   alias Pleroma.Web.MastodonAPI.AuthController
   alias Pleroma.Web.OAuth.Token
-  alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Web.Plugs.OAuthScopesPlug
 
   plug(OAuthScopesPlug, %{scopes: ["write:accounts"]} when action == :put_settings)
 
   # Note: :index action handles attempt of unauthenticated access to private instance with redirect
-  plug(:skip_plug, EnsurePublicOrAuthenticatedPlug when action == :index)
+  plug(:skip_public_check when action == :index)
 
   plug(
     OAuthScopesPlug,
@@ -22,10 +21,7 @@ defmodule Pleroma.Web.MastoFEController do
     when action == :index
   )
 
-  plug(
+  plug(:skip_auth when action == :manifest)
-    :skip_plug,
-    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :manifest
-  )
 
   @doc "GET /web/*path"
   def index(conn, _params) do

lib/pleroma/web/mastodon_api/controllers/account_controller.ex

@@ -8,7 +8,6 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
   import Pleroma.Web.ControllerHelper,
     only: [
       add_link_headers: 2,
-      truthy_param?: 1,
       assign_account_by_id: 2,
       embed_relationships?: 1,
       json_response: 3
@@ -25,16 +24,16 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
   alias Pleroma.Web.MastodonAPI.MastodonAPIController
   alias Pleroma.Web.MastodonAPI.StatusView
   alias Pleroma.Web.OAuth.OAuthController
-  alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Web.Plugs.OAuthScopesPlug
   alias Pleroma.Web.Plugs.RateLimiter
   alias Pleroma.Web.TwitterAPI.TwitterAPI
+  alias Pleroma.Web.Utils.Params
 
   plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
-  plug(:skip_plug, [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :create)
+  plug(:skip_auth when action == :create)
 
-  plug(:skip_plug, EnsurePublicOrAuthenticatedPlug when action in [:show, :statuses])
+  plug(:skip_public_check when action in [:show, :statuses])
 
   plug(
     OAuthScopesPlug,
@@ -188,7 +187,7 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
         :accepts_chat_messages
       ]
       |> Enum.reduce(%{}, fn key, acc ->
-        Maps.put_if_present(acc, key, params[key], &{:ok, truthy_param?(&1)})
+        Maps.put_if_present(acc, key, params[key], &{:ok, Params.truthy_param?(&1)})
       end)
       |> Maps.put_if_present(:name, params[:display_name])
       |> Maps.put_if_present(:bio, params[:note])

lib/pleroma/web/mastodon_api/controllers/app_controller.ex

@@ -14,16 +14,10 @@ defmodule Pleroma.Web.MastodonAPI.AppController do
   alias Pleroma.Web.OAuth.App
   alias Pleroma.Web.OAuth.Scopes
   alias Pleroma.Web.OAuth.Token
-  alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
-  alias Pleroma.Web.Plugs.OAuthScopesPlug
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
-  plug(
+  plug(:skip_auth when action in [:create, :verify_credentials])
-    :skip_plug,
-    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug]
-    when action in [:create, :verify_credentials]
-  )
 
   plug(Pleroma.Web.ApiSpec.CastAndValidate)
 

lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex

@@ -7,11 +7,7 @@ defmodule Pleroma.Web.MastodonAPI.CustomEmojiController do
 
   plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
-  plug(
+  plug(:skip_auth when action == :index)
-    :skip_plug,
-    [Pleroma.Web.Plugs.OAuthScopesPlug, Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug]
-    when action == :index
-  )
 
   defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.CustomEmojiOperation
 

lib/pleroma/web/mastodon_api/controllers/instance_controller.ex

@@ -7,11 +7,7 @@ defmodule Pleroma.Web.MastodonAPI.InstanceController do
 
   plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
-  plug(
+  plug(:skip_auth when action in [:show, :peers])
-    :skip_plug,
-    [Pleroma.Web.Plugs.OAuthScopesPlug, Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug]
-    when action in [:show, :peers]
-  )
 
   defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.InstanceOperation
 

lib/pleroma/web/mastodon_api/controllers/mastodon_api_controller.ex

@@ -15,11 +15,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
 
   require Logger
 
-  plug(
+  plug(:skip_auth when action in [:empty_array, :empty_object])
-    :skip_plug,
-    [Pleroma.Web.Plugs.OAuthScopesPlug, Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug]
-    when action in [:empty_array, :empty_object]
-  )
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 

lib/pleroma/web/mastodon_api/controllers/status_controller.ex

@@ -27,10 +27,7 @@ defmodule Pleroma.Web.MastodonAPI.StatusController do
 
   plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
-  plug(
+  plug(:skip_public_check when action in [:index, :show])
-    :skip_plug,
-    Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug when action in [:index, :show]
-  )
 
   @unauthenticated_access %{fallback: :proceed_unauthenticated, scopes: []}
 

lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex

@@ -12,12 +12,11 @@ defmodule Pleroma.Web.MastodonAPI.TimelineController do
   alias Pleroma.Pagination
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
-  alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Web.Plugs.OAuthScopesPlug
   alias Pleroma.Web.Plugs.RateLimiter
 
   plug(Pleroma.Web.ApiSpec.CastAndValidate)
-  plug(:skip_plug, EnsurePublicOrAuthenticatedPlug when action in [:public, :hashtag])
+  plug(:skip_public_check when action in [:public, :hashtag])
 
   # TODO: Replace with a macro when there is a Phoenix release with the following commit in it:
   # https://github.com/phoenixframework/phoenix/commit/2e8c63c01fec4dde5467dbbbf9705ff9e780735e

lib/pleroma/web/metadata/providers/open_graph.ex

@@ -4,6 +4,7 @@
 
 defmodule Pleroma.Web.Metadata.Providers.OpenGraph do
   alias Pleroma.User
+  alias Pleroma.Web.MediaProxy
   alias Pleroma.Web.Metadata
   alias Pleroma.Web.Metadata.Providers.Provider
   alias Pleroma.Web.Metadata.Utils
@@ -19,37 +20,24 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraph do
       }) do
     attachments = build_attachments(object)
     scrubbed_content = Utils.scrub_html_and_truncate(object)
-    # Zero width space
-    content =
-      if scrubbed_content != "" and scrubbed_content != "\u200B" do
-        ": “" <> scrubbed_content <> "”"
-      else
-        ""
-      end
 
-    # Most previews only show og:title which is inconvenient. Instagram
-    # hacks this by putting the description in the title and making the
-    # description longer prefixed by how many likes and shares the post
-    # has. Here we use the descriptive nickname in the title, and expand
-    # the full account & nickname in the description. We also use the cute^Wevil
-    # smart quotes around the status text like Instagram, too.
     [
       {:meta,
        [
          property: "og:title",
-         content: "#{user.name}" <> content
+         content: Utils.user_name_string(user)
        ], []},
       {:meta, [property: "og:url", content: url], []},
       {:meta,
        [
          property: "og:description",
-         content: "#{Utils.user_name_string(user)}" <> content
+         content: scrubbed_content
        ], []},
-      {:meta, [property: "og:type", content: "website"], []}
+      {:meta, [property: "og:type", content: "article"], []}
     ] ++
       if attachments == [] or Metadata.activity_nsfw?(object) do
         [
-          {:meta, [property: "og:image", content: Utils.attachment_url(User.avatar_url(user))],
+          {:meta, [property: "og:image", content: MediaProxy.preview_url(User.avatar_url(user))],
            []},
           {:meta, [property: "og:image:width", content: 150], []},
           {:meta, [property: "og:image:height", content: 150], []}
@@ -70,8 +58,9 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraph do
          ], []},
         {:meta, [property: "og:url", content: user.uri || user.ap_id], []},
         {:meta, [property: "og:description", content: truncated_bio], []},
-        {:meta, [property: "og:type", content: "website"], []},
+        {:meta, [property: "og:type", content: "article"], []},
-        {:meta, [property: "og:image", content: Utils.attachment_url(User.avatar_url(user))], []},
+        {:meta, [property: "og:image", content: MediaProxy.preview_url(User.avatar_url(user))],
+         []},
         {:meta, [property: "og:image:width", content: 150], []},
         {:meta, [property: "og:image:height", content: 150], []}
       ]
@@ -82,29 +71,35 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraph do
     Enum.reduce(attachments, [], fn attachment, acc ->
       rendered_tags =
         Enum.reduce(attachment["url"], [], fn url, acc ->
-          # TODO: Add additional properties to objects when we have the data available.
+          # TODO: Whatsapp only wants JPEG or PNGs. It seems that if we add a second og:image
-          # Also, Whatsapp only wants JPEG or PNGs. It seems that if we add a second og:image
           # object when a Video or GIF is attached it will display that in Whatsapp Rich Preview.
           case Utils.fetch_media_type(@media_types, url["mediaType"]) do
             "audio" ->
               [
-                {:meta, [property: "og:audio", content: Utils.attachment_url(url["href"])], []}
+                {:meta, [property: "og:audio", content: MediaProxy.url(url["href"])], []}
                 | acc
               ]
 
+            # Not using preview_url for this. It saves bandwidth, but the image dimensions will
+            # be wrong. We generate it on the fly and have no way to capture or analyze the
+            # image to get the dimensions. This can be an issue for apps/FEs rendering images
+            # in timelines too, but you can get clever with the aspect ratio metadata as a
+            # workaround.
             "image" ->
               [
-                {:meta, [property: "og:image", content: Utils.attachment_url(url["href"])], []},
+                {:meta, [property: "og:image", content: MediaProxy.url(url["href"])], []},
-                {:meta, [property: "og:image:width", content: 150], []},
+                {:meta, [property: "og:image:alt", content: attachment["name"]], []}
-                {:meta, [property: "og:image:height", content: 150], []}
                 | acc
               ]
+              |> maybe_add_dimensions(url)
 
             "video" ->
               [
-                {:meta, [property: "og:video", content: Utils.attachment_url(url["href"])], []}
+                {:meta, [property: "og:video", content: MediaProxy.url(url["href"])], []}
                 | acc
               ]
+              |> maybe_add_dimensions(url)
+              |> maybe_add_video_thumbnail(url)
 
             _ ->
               acc
@@ -116,4 +111,38 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraph do
   end
 
   defp build_attachments(_), do: []
+
+  # We can use url["mediaType"] to dynamically fill the metadata
+  defp maybe_add_dimensions(metadata, url) do
+    type = url["mediaType"] |> String.split("/") |> List.first()
+
+    cond do
+      !is_nil(url["height"]) && !is_nil(url["width"]) ->
+        metadata ++
+          [
+            {:meta, [property: "og:#{type}:width", content: "#{url["width"]}"], []},
+            {:meta, [property: "og:#{type}:height", content: "#{url["height"]}"], []}
+          ]
+
+      true ->
+        metadata
+    end
+  end
+
+  # Media Preview Proxy makes thumbnails of videos without resizing, so we can trust the
+  # width and height of the source video.
+  defp maybe_add_video_thumbnail(metadata, url) do
+    cond do
+      Pleroma.Config.get([:media_preview_proxy, :enabled], false) ->
+        metadata ++
+          [
+            {:meta, [property: "og:image:width", content: "#{url["width"]}"], []},
+            {:meta, [property: "og:image:height", content: "#{url["height"]}"], []},
+            {:meta, [property: "og:image", content: MediaProxy.preview_url(url["href"])], []}
+          ]
+
+      true ->
+        metadata
+    end
+  end
 end

lib/pleroma/web/metadata/providers/twitter_card.ex

@@ -5,6 +5,7 @@
 
 defmodule Pleroma.Web.Metadata.Providers.TwitterCard do
   alias Pleroma.User
+  alias Pleroma.Web.MediaProxy
   alias Pleroma.Web.Metadata
   alias Pleroma.Web.Metadata.Providers.Provider
   alias Pleroma.Web.Metadata.Utils
@@ -16,17 +17,10 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCard do
   def build_tags(%{activity_id: id, object: object, user: user}) do
     attachments = build_attachments(id, object)
     scrubbed_content = Utils.scrub_html_and_truncate(object)
-    # Zero width space
-    content =
-      if scrubbed_content != "" and scrubbed_content != "\u200B" do
-        "“" <> scrubbed_content <> "”"
-      else
-        ""
-      end
 
     [
       title_tag(user),
-      {:meta, [property: "twitter:description", content: content], []}
+      {:meta, [property: "twitter:description", content: scrubbed_content], []}
     ] ++
       if attachments == [] or Metadata.activity_nsfw?(object) do
         [
@@ -55,14 +49,14 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCard do
   end
 
   def image_tag(user) do
-    {:meta, [property: "twitter:image", content: Utils.attachment_url(User.avatar_url(user))], []}
+    {:meta, [property: "twitter:image", content: MediaProxy.preview_url(User.avatar_url(user))],
+     []}
   end
 
   defp build_attachments(id, %{data: %{"attachment" => attachments}}) do
     Enum.reduce(attachments, [], fn attachment, acc ->
       rendered_tags =
         Enum.reduce(attachment["url"], [], fn url, acc ->
-          # TODO: Add additional properties to objects when we have the data available.
           case Utils.fetch_media_type(@media_types, url["mediaType"]) do
             "audio" ->
               [
@@ -73,25 +67,37 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCard do
                 | acc
               ]
 
+            # Not using preview_url for this. It saves bandwidth, but the image dimensions will
+            # be wrong. We generate it on the fly and have no way to capture or analyze the
+            # image to get the dimensions. This can be an issue for apps/FEs rendering images
+            # in timelines too, but you can get clever with the aspect ratio metadata as a
+            # workaround.
             "image" ->
               [
                 {:meta, [property: "twitter:card", content: "summary_large_image"], []},
                 {:meta,
                  [
                    property: "twitter:player",
-                   content: Utils.attachment_url(url["href"])
+                   content: MediaProxy.url(url["href"])
                  ], []}
                 | acc
               ]
+              |> maybe_add_dimensions(url)
 
-            # TODO: Need the true width and height values here or Twitter renders an iFrame with
-            # a bad aspect ratio
             "video" ->
+              # fallback to old placeholder values
+              height = url["height"] || 480
+              width = url["width"] || 480
+
               [
                 {:meta, [property: "twitter:card", content: "player"], []},
                 {:meta, [property: "twitter:player", content: player_url(id)], []},
-                {:meta, [property: "twitter:player:width", content: "480"], []},
+                {:meta, [property: "twitter:player:width", content: "#{width}"], []},
-                {:meta, [property: "twitter:player:height", content: "480"], []}
+                {:meta, [property: "twitter:player:height", content: "#{height}"], []},
+                {:meta, [property: "twitter:player:stream", content: MediaProxy.url(url["href"])],
+                 []},
+                {:meta,
+                 [property: "twitter:player:stream:content_type", content: url["mediaType"]], []}
                 | acc
               ]
 
@@ -109,4 +115,20 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCard do
   defp player_url(id) do
     Pleroma.Web.Router.Helpers.o_status_url(Pleroma.Web.Endpoint, :notice_player, id)
   end
+
+  # Videos have problems without dimensions, but we used to not provide WxH for images.
+  # A default (read: incorrect) fallback for images is likely to cause rendering bugs.
+  defp maybe_add_dimensions(metadata, url) do
+    cond do
+      !is_nil(url["height"]) && !is_nil(url["width"]) ->
+        metadata ++
+          [
+            {:meta, [property: "twitter:player:width", content: "#{url["width"]}"], []},
+            {:meta, [property: "twitter:player:height", content: "#{url["height"]}"], []}
+          ]
+
+      true ->
+        metadata
+    end
+  end
 end

lib/pleroma/web/metadata/utils.ex

@@ -7,7 +7,6 @@ defmodule Pleroma.Web.Metadata.Utils do
   alias Pleroma.Emoji
   alias Pleroma.Formatter
   alias Pleroma.HTML
-  alias Pleroma.Web.MediaProxy
 
   def scrub_html_and_truncate(%{data: %{"content" => content}} = object) do
     content
@@ -38,10 +37,6 @@ defmodule Pleroma.Web.Metadata.Utils do
 
   def scrub_html(content), do: content
 
-  def attachment_url(url) do
-    MediaProxy.preview_url(url)
-  end
-
   def user_name_string(user) do
     "#{user.name} " <>
       if user.local do

lib/pleroma/web/o_auth/o_auth_controller.ex

@@ -12,8 +12,7 @@ defmodule Pleroma.Web.OAuth.OAuthController do
   alias Pleroma.Registration
   alias Pleroma.Repo
   alias Pleroma.User
-  alias Pleroma.Web.Auth.Authenticator
+  alias Pleroma.Web.Auth.WrapperAuthenticator, as: Authenticator
-  alias Pleroma.Web.ControllerHelper
   alias Pleroma.Web.OAuth.App
   alias Pleroma.Web.OAuth.Authorization
   alias Pleroma.Web.OAuth.MFAController
@@ -24,6 +23,7 @@ defmodule Pleroma.Web.OAuth.OAuthController do
   alias Pleroma.Web.OAuth.Token.Strategy.RefreshToken
   alias Pleroma.Web.OAuth.Token.Strategy.Revoke, as: RevokeToken
   alias Pleroma.Web.Plugs.RateLimiter
+  alias Pleroma.Web.Utils.Params
 
   require Logger
 
@@ -32,10 +32,7 @@ defmodule Pleroma.Web.OAuth.OAuthController do
   plug(:fetch_session)
   plug(:fetch_flash)
 
-  plug(:skip_plug, [
+  plug(:skip_auth)
-    Pleroma.Web.Plugs.OAuthScopesPlug,
-    Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
-  ])
 
   plug(RateLimiter, [name: :authentication] when action == :create_authorization)
 
@@ -50,7 +47,7 @@ defmodule Pleroma.Web.OAuth.OAuthController do
   end
 
   def authorize(%Plug.Conn{assigns: %{token: %Token{}}} = conn, %{"force_login" => _} = params) do
-    if ControllerHelper.truthy_param?(params["force_login"]) do
+    if Params.truthy_param?(params["force_login"]) do
       do_authorize(conn, params)
     else
       handle_existing_authorization(conn, params)

lib/pleroma/web/pleroma_api/controllers/account_controller.ex

@@ -11,7 +11,6 @@ defmodule Pleroma.Web.PleromaAPI.AccountController do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.MastodonAPI.StatusView
-  alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Web.Plugs.OAuthScopesPlug
   alias Pleroma.Web.Plugs.RateLimiter
 
@@ -29,10 +28,7 @@ defmodule Pleroma.Web.PleromaAPI.AccountController do
 
   plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
-  plug(
+  plug(:skip_auth when action == :confirmation_resend)
-    :skip_plug,
-    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :confirmation_resend
-  )
 
   plug(
     OAuthScopesPlug,

lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex

@@ -22,11 +22,7 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackController do
          ]
   )
 
-  @skip_plugs [
+  plug(:skip_auth when action in [:index, :archive, :show])
-    Pleroma.Web.Plugs.OAuthScopesPlug,
-    Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
-  ]
-  plug(:skip_plug, @skip_plugs when action in [:index, :archive, :show])
 
   defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.PleromaEmojiPackOperation
 

lib/pleroma/web/templates/o_auth/o_auth/show.html.eex

@@ -61,5 +61,5 @@
 <% end %>
 
 <%= if Pleroma.Config.oauth_consumer_enabled?() do %>
-  <%= render @view_module, Pleroma.Web.Auth.Authenticator.oauth_consumer_template(), assigns %>
+  <%= render @view_module, Pleroma.Web.Auth.WrapperAuthenticator.oauth_consumer_template(), assigns %>
 <% end %>

lib/pleroma/web/twitter_api/controller.ex

@@ -7,17 +7,12 @@ defmodule Pleroma.Web.TwitterAPI.Controller do
 
   alias Pleroma.User
   alias Pleroma.Web.OAuth.Token
-  alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Web.Plugs.OAuthScopesPlug
   alias Pleroma.Web.TwitterAPI.TokenView
 
   require Logger
 
-  plug(
+  plug(:skip_auth when action == :confirm_email)
-    :skip_plug,
-    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :confirm_email
-  )
-
   plug(:skip_plug, OAuthScopesPlug when action in [:oauth_tokens, :revoke_token])
 
   action_fallback(:errors)

lib/pleroma/web/twitter_api/controllers/remote_follow_controller.ex

@@ -11,8 +11,8 @@ defmodule Pleroma.Web.TwitterAPI.RemoteFollowController do
   alias Pleroma.MFA
   alias Pleroma.Object.Fetcher
   alias Pleroma.User
-  alias Pleroma.Web.Auth.Authenticator
   alias Pleroma.Web.Auth.TOTPAuthenticator
+  alias Pleroma.Web.Auth.WrapperAuthenticator
   alias Pleroma.Web.CommonAPI
 
   @status_types ["Article", "Event", "Note", "Video", "Page", "Question"]
@@ -88,7 +88,7 @@ defmodule Pleroma.Web.TwitterAPI.RemoteFollowController do
   #
   def do_follow(conn, %{"authorization" => %{"name" => _, "password" => _, "id" => id}}) do
     with {_, %User{} = followee} <- {:fetch_user, User.get_cached_by_id(id)},
-         {_, {:ok, user}, _} <- {:auth, Authenticator.get_user(conn), followee},
+         {_, {:ok, user}, _} <- {:auth, WrapperAuthenticator.get_user(conn), followee},
          {_, _, _, false} <- {:mfa_required, followee, user, MFA.require?(user)},
          {:ok, _, _, _} <- CommonAPI.follow(user, followee) do
       redirect(conn, to: "/users/#{followee.id}")

lib/pleroma/web/utils/params.ex

@@ -0,0 +1,16 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Utils.Params do
+  # As in Mastodon API, per https://api.rubyonrails.org/classes/ActiveModel/Type/Boolean.html
+  @falsy_param_values [false, 0, "0", "f", "F", "false", "False", "FALSE", "off", "OFF"]
+
+  defp explicitly_falsy_param?(value), do: value in @falsy_param_values
+
+  # Note: `nil` and `""` are considered falsy values in Pleroma
+  defp falsy_param?(value),
+    do: explicitly_falsy_param?(value) or value in [nil, ""]
+
+  def truthy_param?(value), do: not falsy_param?(value)
+end

mix.exs

@@ -123,8 +123,7 @@ defmodule Pleroma.Mixfile do
       {:phoenix_pubsub, "~> 2.0"},
       {:phoenix_ecto, "~> 4.0"},
       {:ecto_enum, "~> 1.4"},
-      {:ecto_explain, "~> 0.1.2"},
+      {:ecto_sql, "~> 3.6.2"},
-      {:ecto_sql, "~> 3.4.4"},
       {:postgrex, ">= 0.15.5"},
       {:oban, "~> 2.3.4"},
       {:gettext, "~> 0.18"},
@@ -160,7 +159,7 @@ defmodule Pleroma.Mixfile do
       {:floki, "~> 0.27"},
       {:timex, "~> 3.6"},
       {:ueberauth, "~> 0.4"},
-      {:linkify, "~> 0.5.0"},
+      {:linkify, "~> 0.5.1"},
       {:http_signatures, "~> 0.1.0"},
       {:telemetry, "~> 0.3"},
       {:poolboy, "~> 1.5"},
@@ -198,11 +197,12 @@ defmodule Pleroma.Mixfile do
       {:majic,
        git: "https://git.pleroma.social/pleroma/elixir-libraries/majic.git",
        ref: "289cda1b6d0d70ccb2ba508a2b0bd24638db2880"},
-      {:eblurhash,
+      {:eblurhash, "~> 1.1.0"},
-       git: "https://github.com/zotonic/eblurhash.git",
-       ref: "04a0b76eadf4de1be17726f39b6313b88708fd12"},
       {:open_api_spex, "~> 3.10"},
 
+      # indirect dependency version override
+      {:plug, "~> 1.10.4", override: true},
+
       ## dev & test
       {:ex_doc, "~> 0.22", only: :dev, runtime: false},
       {:ex_machina, "~> 2.4", only: :test},

mix.lock

@@ -29,11 +29,10 @@
   "deep_merge": {:hex, :deep_merge, "1.0.0", "b4aa1a0d1acac393bdf38b2291af38cb1d4a52806cf7a4906f718e1feb5ee961", [:mix], [], "hexpm", "ce708e5f094b9cd4e8f2be4f00d2f4250c4095be93f8cd6d018c753894885430"},
   "earmark": {:hex, :earmark, "1.4.15", "2c7f924bf495ec1f65bd144b355d0949a05a254d0ec561740308a54946a67888", [:mix], [{:earmark_parser, ">= 1.4.13", [hex: :earmark_parser, repo: "hexpm", optional: false]}], "hexpm", "3b1209b85bc9f3586f370f7c363f6533788fb4e51db23aa79565875e7f9999ee"},
   "earmark_parser": {:hex, :earmark_parser, "1.4.13", "0c98163e7d04a15feb62000e1a891489feb29f3d10cb57d4f845c405852bbef8", [:mix], [], "hexpm", "d602c26af3a0af43d2f2645613f65841657ad6efc9f0e361c3b6c06b578214ba"},
-  "eblurhash": {:git, "https://github.com/zotonic/eblurhash.git", "04a0b76eadf4de1be17726f39b6313b88708fd12", [ref: "04a0b76eadf4de1be17726f39b6313b88708fd12"]},
+  "eblurhash": {:hex, :eblurhash, "1.1.0", "e10ccae762598507ebfacf0b645ed49520f2afa3e7e9943e73a91117dffce415", [:rebar3], [], "hexpm", "2e6b889d09fddd374e3c5ac57c486138768763264e99ac1074ae5fa7fc9ab51d"},
-  "ecto": {:hex, :ecto, "3.4.6", "08f7afad3257d6eb8613309af31037e16c36808dfda5a3cd0cb4e9738db030e4", [:mix], [{:decimal, "~> 1.6 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "6f13a9e2a62e75c2dcfc7207bfc65645ab387af8360db4c89fee8b5a4bf3f70b"},
+  "ecto": {:hex, :ecto, "3.6.2", "efdf52acfc4ce29249bab5417415bd50abd62db7b0603b8bab0d7b996548c2bc", [:mix], [{:decimal, "~> 1.6 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "efad6dfb04e6f986b8a3047822b0f826d9affe8e4ebdd2aeedbfcb14fd48884e"},
   "ecto_enum": {:hex, :ecto_enum, "1.4.0", "d14b00e04b974afc69c251632d1e49594d899067ee2b376277efd8233027aec8", [:mix], [{:ecto, ">= 3.0.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:ecto_sql, "> 3.0.0", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:mariaex, ">= 0.0.0", [hex: :mariaex, repo: "hexpm", optional: true]}, {:postgrex, ">= 0.0.0", [hex: :postgrex, repo: "hexpm", optional: true]}], "hexpm", "8fb55c087181c2b15eee406519dc22578fa60dd82c088be376d0010172764ee4"},
-  "ecto_explain": {:hex, :ecto_explain, "0.1.2", "a9d504cbd4adc809911f796d5ef7ebb17a576a6d32286c3d464c015bd39d5541", [:mix], [], "hexpm", "1d0e7798ae30ecf4ce34e912e5354a0c1c832b7ebceba39298270b9a9f316330"},
+  "ecto_sql": {:hex, :ecto_sql, "3.6.2", "9526b5f691701a5181427634c30655ac33d11e17e4069eff3ae1176c764e0ba3", [:mix], [{:db_connection, "~> 2.2", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.6.2", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.4.0 or ~> 0.5.0", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.15.0 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.1", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "5ec9d7e6f742ea39b63aceaea9ac1d1773d574ea40df5a53ef8afbd9242fdb6b"},
-  "ecto_sql": {:hex, :ecto_sql, "3.4.5", "30161f81b167d561a9a2df4329c10ae05ff36eca7ccc84628f2c8b9fa1e43323", [:mix], [{:db_connection, "~> 2.2", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.4.3", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.3.0 or ~> 0.4.0", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.15.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.0", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "31990c6a3579b36a3c0841d34a94c275e727de8b84f58509da5f1b2032c98ac2"},
   "eimp": {:hex, :eimp, "1.0.14", "fc297f0c7e2700457a95a60c7010a5f1dcb768a083b6d53f49cd94ab95a28f22", [:rebar3], [{:p1_utils, "1.0.18", [hex: :p1_utils, repo: "hexpm", optional: false]}], "hexpm", "501133f3112079b92d9e22da8b88bf4f0e13d4d67ae9c15c42c30bd25ceb83b6"},
   "elixir_make": {:hex, :elixir_make, "0.6.2", "7dffacd77dec4c37b39af867cedaabb0b59f6a871f89722c25b28fcd4bd70530", [:mix], [], "hexpm", "03e49eadda22526a7e5279d53321d1cced6552f344ba4e03e619063de75348d9"},
   "esshd": {:hex, :esshd, "0.1.1", "d4dd4c46698093a40a56afecce8a46e246eb35463c457c246dacba2e056f31b5", [:mix], [], "hexpm", "d73e341e3009d390aa36387dc8862860bf9f874c94d9fd92ade2926376f49981"},
@@ -68,7 +67,7 @@
   "jose": {:hex, :jose, "1.11.1", "59da64010c69aad6cde2f5b9248b896b84472e99bd18f246085b7b9fe435dcdb", [:mix, :rebar3], [], "hexpm", "078f6c9fb3cd2f4cfafc972c814261a7d1e8d2b3685c0a76eb87e158efff1ac5"},
   "jumper": {:hex, :jumper, "1.0.1", "3c00542ef1a83532b72269fab9f0f0c82bf23a35e27d278bfd9ed0865cecabff", [:mix], [], "hexpm", "318c59078ac220e966d27af3646026db9b5a5e6703cb2aa3e26bcfaba65b7433"},
   "libring": {:hex, :libring, "1.4.0", "41246ba2f3fbc76b3971f6bce83119dfec1eee17e977a48d8a9cfaaf58c2a8d6", [:mix], [], "hexpm"},
-  "linkify": {:hex, :linkify, "0.5.0", "e0ea8de73ff44742d6a889721221f4c4eccaad5284957ee9832ffeb347602d54", [:mix], [], "hexpm", "4ccd958350aee7c51c89e21f05b15d30596ebbba707e051d21766be1809df2d7"},
+  "linkify": {:hex, :linkify, "0.5.1", "6dc415cbc948b2f6ecec7cb226aab7ba9d3a1815bb501ae33e042334d707ecee", [:mix], [], "hexpm", "a3128c7e22fada4aa7214009501d8131e1fa3faf2f0a68b33dba379dc84ff944"},
   "majic": {:git, "https://git.pleroma.social/pleroma/elixir-libraries/majic.git", "289cda1b6d0d70ccb2ba508a2b0bd24638db2880", [ref: "289cda1b6d0d70ccb2ba508a2b0bd24638db2880"]},
   "makeup": {:hex, :makeup, "1.0.5", "d5a830bc42c9800ce07dd97fa94669dfb93d3bf5fcf6ea7a0c67b2e0e4a7f26c", [:mix], [{:nimble_parsec, "~> 0.5 or ~> 1.0", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "cfa158c02d3f5c0c665d0af11512fed3fba0144cf1aadee0f2ce17747fba2ca9"},
   "makeup_elixir": {:hex, :makeup_elixir, "0.14.1", "4f0e96847c63c17841d42c08107405a005a2680eb9c7ccadfd757bd31dabccfb", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm", "f2438b1a80eaec9ede832b5c41cd4f373b38fd7aa33e3b22d9db79e640cbde11"},
@@ -95,7 +94,7 @@
   "phoenix_html": {:hex, :phoenix_html, "2.14.3", "51f720d0d543e4e157ff06b65de38e13303d5778a7919bcc696599e5934271b8", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "efd697a7fff35a13eeeb6b43db884705cba353a1a41d127d118fda5f90c8e80f"},
   "phoenix_pubsub": {:hex, :phoenix_pubsub, "2.0.0", "a1ae76717bb168cdeb10ec9d92d1480fec99e3080f011402c0a2d68d47395ffb", [:mix], [], "hexpm", "c52d948c4f261577b9c6fa804be91884b381a7f8f18450c5045975435350f771"},
   "phoenix_swoosh": {:hex, :phoenix_swoosh, "0.3.3", "039435dd975f7e55953525b88f1d596f26c6141412584c16f4db109708a8ee68", [:mix], [{:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: false]}, {:phoenix, "~> 1.4", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.14", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:swoosh, "~> 1.0", [hex: :swoosh, repo: "hexpm", optional: false]}], "hexpm", "4a540cea32e05356541737033d666ee7fea7700eb2101bf76783adbfe06601cd"},
-  "plug": {:hex, :plug, "1.11.1", "f2992bac66fdae679453c9e86134a4201f6f43a687d8ff1cd1b2862d53c80259", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.1.1 or ~> 1.2", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "23524e4fefbb587c11f0833b3910bfb414bf2e2534d61928e920f54e3a1b881f"},
+  "plug": {:hex, :plug, "1.10.4", "41eba7d1a2d671faaf531fa867645bd5a3dce0957d8e2a3f398ccff7d2ef017f", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.1.1 or ~> 1.2", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "ad1e233fe73d2eec56616568d260777b67f53148a999dc2d048f4eb9778fe4a0"},
   "plug_cowboy": {:hex, :plug_cowboy, "2.5.0", "51c998f788c4e68fc9f947a5eba8c215fbb1d63a520f7604134cab0270ea6513", [:mix], [{:cowboy, "~> 2.7", [hex: :cowboy, repo: "hexpm", optional: false]}, {:cowboy_telemetry, "~> 0.3", [hex: :cowboy_telemetry, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "5b2c8925a5e2587446f33810a58c01e66b3c345652eeec809b76ba007acde71a"},
   "plug_crypto": {:hex, :plug_crypto, "1.2.2", "05654514ac717ff3a1843204b424477d9e60c143406aa94daf2274fdd280794d", [:mix], [], "hexpm", "87631c7ad914a5a445f0a3809f99b079113ae4ed4b867348dd9eec288cecb6db"},
   "plug_static_index_html": {:hex, :plug_static_index_html, "1.0.0", "840123d4d3975585133485ea86af73cb2600afd7f2a976f9f5fd8b3808e636a0", [:mix], [{:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "79fd4fcf34d110605c26560cbae8f23c603ec4158c08298bd4360fdea90bb5cf"},

test/fixtures/tesla_mock/lemmy-page.json

@@ -0,0 +1,17 @@
+{
+  "commentsEnabled": true,
+  "sensitive": false,
+  "stickied": false,
+  "attributedTo": "https://enterprise.lemmy.ml/u/nutomic",
+  "summary": "Hello Federation!",
+  "url": "https://enterprise.lemmy.ml/pictrs/image/US52d9DPvf.jpg",
+  "image": {
+    "type": "Image",
+    "url": "https://enterprise.lemmy.ml/pictrs/image/lwFAcXHUjS.jpg"
+  },
+  "published": "2020-09-14T15:03:11.909105+00:00",
+  "to": "https://enterprise.lemmy.ml/c/main",
+  "@context": "https://www.w3.org/ns/activitystreams",
+  "id": "https://enterprise.lemmy.ml/post/3",
+  "type": "Page"
+}

test/fixtures/tesla_mock/lemmy-user.json

@@ -0,0 +1,27 @@
+{
+  "publicKey": {
+    "id": "https://enterprise.lemmy.ml/u/nutomic#main-key",
+    "owner": "https://enterprise.lemmy.ml/u/nutomic",
+    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvfwAYPxp1gOk2HcCRoUd\nupoecvmnpzRc5Gu6/N3YQyOyRsrYuiYLNQq2cgM3kcU80ZeEetkwkYgXkRJOKu/b\nBWb7i1zt2tdr5k6lUdW8dfCyjht8ooFPQdov8J3QYHfgBHyUYxuCNfSujryxx2wu\nLQcdjRQa5NIWcomSO8OXmCF5/Yhg2XWCbtnlxEq6Y+AFddr1mAlTOy5pBr5d+xZz\njLw/U3CioNJ79yGi/sJhgp6IyJqtUSoN3b4BgRIEts2QVvn44W1rQy9wCbRYQrO1\nBcB9Wel4k3rJJK8uHg+LpHVMaZppkNaWGkMBhMbzr8qmIlcNWNi7cbMK/p5vyviy\nSwIDAQAB\n-----END PUBLIC KEY-----\n"
+  },
+  "inbox": "https://enterprise.lemmy.ml/u/nutomic/inbox",
+  "preferredUsername": "Nutomic",
+  "endpoints": {
+    "sharedInbox": "https://enterprise.lemmy.ml/inbox"
+  },
+  "summary": "some bio",
+  "icon": {
+    "type": "Image",
+    "url": "https://enterprise.lemmy.ml/pictrs/image/F6Z7QcWZRJ.jpg"
+  },
+  "image": {
+    "type": "Image",
+    "url": "https://enterprise.lemmy.ml:/pictrs/image/Q79N9oCDEG.png"
+  },
+  "published": "2020-09-14T14:54:53.080949+00:00",
+  "updated": "2020-10-14T10:58:28.139178+00:00",
+  "@context": "https://www.w3.org/ns/activitystreams",
+  "id": "https://enterprise.lemmy.ml/u/nutomic",
+  "type": "Person",
+  "name": "nutomic"
+}

test/mix/tasks/pleroma/ecto/migrate_test.exs

@@ -13,7 +13,7 @@ defmodule Mix.Tasks.Pleroma.Ecto.MigrateTest do
 
     assert capture_log(fn ->
              Mix.Tasks.Pleroma.Ecto.Migrate.run()
-           end) =~ "[info] Already up"
+           end) =~ "[info] Migrations already up"
 
     Logger.configure(level: level)
   end

test/pleroma/upload/filter/analyze_metadata_test.exs

@@ -6,7 +6,7 @@ defmodule Pleroma.Upload.Filter.AnalyzeMetadataTest do
   use Pleroma.DataCase, async: true
   alias Pleroma.Upload.Filter.AnalyzeMetadata
 
-  test "adds the image dimensions" do
+  test "adds the dimensions and blurhash for images" do
     upload = %Pleroma.Upload{
       name: "an… image.jpg",
       content_type: "image/jpeg",
@@ -14,6 +14,20 @@ defmodule Pleroma.Upload.Filter.AnalyzeMetadataTest do
       tempfile: Path.absname("test/fixtures/image.jpg")
     }
 
-    assert {:ok, :filtered, %{width: 1024, height: 768}} = AnalyzeMetadata.filter(upload)
+    {:ok, :filtered, meta} = AnalyzeMetadata.filter(upload)
+
+    assert %{width: 1024, height: 768} = meta
+    assert meta.blurhash
+  end
+
+  test "adds the dimensions for videos" do
+    upload = %Pleroma.Upload{
+      name: "coolvideo.mp4",
+      content_type: "video/mp4",
+      path: Path.absname("test/fixtures/video.mp4"),
+      tempfile: Path.absname("test/fixtures/video.mp4")
+    }
+
+    assert {:ok, :filtered, %{width: 480, height: 480}} = AnalyzeMetadata.filter(upload)
   end
 end

test/pleroma/user_test.exs

@@ -1639,9 +1639,9 @@ defmodule Pleroma.UserTest do
         follower_count: 9,
         following_count: 9001,
         is_locked: true,
-        is_confirmed: false,
+        is_confirmed: true,
         password_reset_pending: true,
-        is_approved: false,
+        is_approved: true,
         registration_reason: "ahhhhh",
         confirmation_token: "qqqq",
         domain_blocks: ["lain.com"],
@@ -1669,8 +1669,8 @@ defmodule Pleroma.UserTest do
              email: nil,
              name: nil,
              password_hash: nil,
-             keys: nil,
+             keys: "RSA begin buplic key",
-             public_key: nil,
+             public_key: "--PRIVATE KEYE--",
              avatar: %{},
              tags: [],
              last_refreshed_at: nil,
@@ -1702,6 +1702,24 @@ defmodule Pleroma.UserTest do
            } = user
   end
 
+  test "delete/1 purges a remote user" do
+    user =
+      insert(:user, %{
+        name: "qqqqqqq",
+        avatar: %{"a" => "b"},
+        banner: %{"a" => "b"},
+        local: false
+      })
+
+    {:ok, job} = User.delete(user)
+    {:ok, _} = ObanHelpers.perform(job)
+    user = User.get_by_id(user.id)
+
+    assert user.name == nil
+    assert user.avatar == %{}
+    assert user.banner == %{}
+  end
+
   test "get_public_key_for_ap_id fetches a user that's not in the db" do
     assert {:ok, _key} = User.get_public_key_for_ap_id("http://mastodon.example.org/users/admin")
   end

test/pleroma/web/activity_pub/activity_pub_controller_test.exs

@@ -1334,10 +1334,13 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do
         activity: %{
           "@context" => "https://www.w3.org/ns/activitystreams",
           "type" => "Create",
-          "object" => %{"type" => "Note", "content" => "AP C2S test"},
+          "object" => %{
+            "type" => "Note",
+            "content" => "AP C2S test",
             "to" => "https://www.w3.org/ns/activitystreams#Public",
             "cc" => []
           }
+        }
       ]
     end
 
@@ -1442,19 +1445,19 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do
       user = User.get_cached_by_ap_id(note_activity.data["actor"])
 
       data = %{
-        type: "Delete",
+        "type" => "Delete",
-        object: %{
+        "object" => %{
-          id: note_object.data["id"]
+          "id" => note_object.data["id"]
         }
       }
 
-      conn =
+      result =
         conn
         |> assign(:user, user)
         |> put_req_header("content-type", "application/activity+json")
         |> post("/users/#{user.nickname}/outbox", data)
+        |> json_response(201)
 
-      result = json_response(conn, 201)
       assert Activity.get_by_ap_id(result["id"])
 
       assert object = Object.get_by_ap_id(note_object.data["id"])
@@ -1479,7 +1482,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do
         |> put_req_header("content-type", "application/activity+json")
         |> post("/users/#{user.nickname}/outbox", data)
 
-      assert json_response(conn, 400)
+      assert json_response(conn, 403)
     end
 
     test "it increases like count when receiving a like action", %{conn: conn} do
@@ -1557,7 +1560,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do
         |> post("/users/#{user.nickname}/outbox", activity)
         |> json_response(400)
 
-      assert result == "Note is over the character limit"
+      assert result == "Character limit (5 characters) exceeded, contains 11 characters"
     end
   end
 
@@ -1934,11 +1937,11 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do
         "object" => %{
           "type" => "Note",
           "content" => "AP C2S test, attachment",
-          "attachment" => [object]
+          "attachment" => [object],
-        },
           "to" => "https://www.w3.org/ns/activitystreams#Public",
           "cc" => []
         }
+      }
 
       activity_response =
         conn

test/pleroma/web/activity_pub/object_validators/announce_validation_test.exs

@@ -33,6 +33,18 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidationTest do
       assert {:ok, _object, _meta} = ObjectValidator.validate(valid_announce, [])
     end
 
+    test "keeps announced object context", %{valid_announce: valid_announce} do
+      assert %Object{data: %{"context" => object_context}} =
+               Object.get_cached_by_ap_id(valid_announce["object"])
+
+      {:ok, %{"context" => context}, _} =
+        valid_announce
+        |> Map.put("context", "https://example.org/invalid_context_id")
+        |> ObjectValidator.validate([])
+
+      assert context == object_context
+    end
+
     test "returns an error if the object can't be found", %{valid_announce: valid_announce} do
       without_object =
         valid_announce
@@ -51,16 +63,6 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidationTest do
       assert {:object, {"can't find object", []}} in cng.errors
     end
 
-    test "returns an error if we don't have the actor", %{valid_announce: valid_announce} do
-      nonexisting_actor =
-        valid_announce
-        |> Map.put("actor", "https://gensokyo.2hu/users/raymoo")
-
-      {:error, cng} = ObjectValidator.validate(nonexisting_actor, [])
-
-      assert {:actor, {"can't find user", []}} in cng.errors
-    end
-
     test "returns an error if the actor already announced the object", %{
       valid_announce: valid_announce,
       announcer: announcer,

test/pleroma/web/activity_pub/object_validators/article_note_page_validator_test.exs

@@ -2,10 +2,10 @@
 # Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only
 
-defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidatorTest do
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNotePageValidatorTest do
   use Pleroma.DataCase, async: true
 
-  alias Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.ArticleNotePageValidator
   alias Pleroma.Web.ActivityPub.Utils
 
   import Pleroma.Factory
@@ -29,7 +29,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidatorTest do
     end
 
     test "a basic note validates", %{note: note} do
-      %{valid?: true} = ArticleNoteValidator.cast_and_validate(note)
+      %{valid?: true} = ArticleNotePageValidator.cast_and_validate(note)
     end
   end
 end

test/pleroma/web/activity_pub/object_validators/like_validation_test.exs

@@ -40,17 +40,30 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidationTest do
       assert LikeValidator.cast_and_validate(valid_like).valid?
     end
 
-    test "sets the 'to' field to the object actor if no recipients are given", %{
+    test "Add object actor from 'to' field if it doesn't owns the like", %{valid_like: valid_like} do
+      user = insert(:user)
+
+      object_actor = valid_like["actor"]
+
+      valid_like =
+        valid_like
+        |> Map.put("actor", user.ap_id)
+        |> Map.put("to", [])
+
+      {:ok, object, _meta} = ObjectValidator.validate(valid_like, [])
+      assert object_actor in object["to"]
+    end
+
+    test "Removes object actor from 'to' field if it owns the like", %{
       valid_like: valid_like,
       user: user
     } do
-      without_recipients =
+      valid_like =
         valid_like
-        |> Map.delete("to")
+        |> Map.put("to", [user.ap_id])
 
-      {:ok, object, _meta} = ObjectValidator.validate(without_recipients, [])
+      {:ok, object, _meta} = ObjectValidator.validate(valid_like, [])
-
+      refute user.ap_id in object["to"]
-      assert object["to"] == [user.ap_id]
     end
 
     test "sets the context field to the context of the object if no context is given", %{
@@ -66,16 +79,6 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidationTest do
       assert object["context"] == post_activity.data["context"]
     end
 
-    test "it errors when the actor is missing or not known", %{valid_like: valid_like} do
-      without_actor = Map.delete(valid_like, "actor")
-
-      refute LikeValidator.cast_and_validate(without_actor).valid?
-
-      with_invalid_actor = Map.put(valid_like, "actor", "invalidactor")
-
-      refute LikeValidator.cast_and_validate(with_invalid_actor).valid?
-    end
-
     test "it errors when the object is missing or not known", %{valid_like: valid_like} do
       without_object = Map.delete(valid_like, "object")
 

test/pleroma/web/activity_pub/relay_test.exs

@@ -148,7 +148,7 @@ defmodule Pleroma.Web.ActivityPub.RelayTest do
       assert {:ok, %Activity{} = activity} = Relay.publish(note)
       assert activity.data["type"] == "Announce"
       assert activity.data["actor"] == service_actor.ap_id
-      assert activity.data["to"] == [service_actor.follower_address]
+      assert service_actor.follower_address in activity.data["to"]
       assert called(Pleroma.Web.Federator.publish(activity))
     end
 

test/pleroma/web/activity_pub/transmogrifier/announce_handling_test.exs

@@ -150,27 +150,4 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier.AnnounceHandlingTest do
 
     assert {:error, _e} = Transmogrifier.handle_incoming(data)
   end
-
-  test "it does not clobber the addressing on announce activities" do
-    user = insert(:user)
-    {:ok, activity} = CommonAPI.post(user, %{status: "hey"})
-
-    data =
-      File.read!("test/fixtures/mastodon-announce.json")
-      |> Jason.decode!()
-      |> Map.put("object", Object.normalize(activity, fetch: false).data["id"])
-      |> Map.put("to", ["http://mastodon.example.org/users/admin/followers"])
-      |> Map.put("cc", [])
-
-    _user =
-      insert(:user,
-        local: false,
-        ap_id: data["actor"],
-        follower_address: "http://mastodon.example.org/users/admin/followers"
-      )
-
-    {:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
-
-    assert data["to"] == ["http://mastodon.example.org/users/admin/followers"]
-  end
 end

test/pleroma/web/activity_pub/transmogrifier/page_handling_test.exs

@@ -0,0 +1,36 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.Transmogrifier.PageHandlingTest do
+  use Oban.Testing, repo: Pleroma.Repo
+  use Pleroma.DataCase
+
+  alias Pleroma.Object.Fetcher
+
+  test "Lemmy Page" do
+    Tesla.Mock.mock(fn
+      %{url: "https://enterprise.lemmy.ml/post/3"} ->
+        %Tesla.Env{
+          status: 200,
+          headers: [{"content-type", "application/activity+json"}],
+          body: File.read!("test/fixtures/tesla_mock/lemmy-page.json")
+        }
+
+      %{url: "https://enterprise.lemmy.ml/u/nutomic"} ->
+        %Tesla.Env{
+          status: 200,
+          headers: [{"content-type", "application/activity+json"}],
+          body: File.read!("test/fixtures/tesla_mock/lemmy-user.json")
+        }
+    end)
+
+    {:ok, object} = Fetcher.fetch_object_from_id("https://enterprise.lemmy.ml/post/3")
+
+    assert object.data["summary"] == "Hello Federation!"
+    assert object.data["published"] == "2020-09-14T15:03:11.909105Z"
+
+    # WAT
+    assert object.data["url"] == "https://enterprise.lemmy.ml/pictrs/image/US52d9DPvf.jpg"
+  end
+end

test/pleroma/web/auth/authenticator_test.exs

@@ -5,38 +5,38 @@
 defmodule Pleroma.Web.Auth.AuthenticatorTest do
   use Pleroma.Web.ConnCase, async: true
 
-  alias Pleroma.Web.Auth.Authenticator
+  alias Pleroma.Web.Auth.Helpers
   import Pleroma.Factory
 
   describe "fetch_user/1" do
     test "returns user by name" do
       user = insert(:user)
-      assert Authenticator.fetch_user(user.nickname) == user
+      assert Helpers.fetch_user(user.nickname) == user
     end
 
     test "returns user by email" do
       user = insert(:user)
-      assert Authenticator.fetch_user(user.email) == user
+      assert Helpers.fetch_user(user.email) == user
     end
 
     test "returns nil" do
-      assert Authenticator.fetch_user("email") == nil
+      assert Helpers.fetch_user("email") == nil
     end
   end
 
   describe "fetch_credentials/1" do
     test "returns name and password from authorization params" do
       params = %{"authorization" => %{"name" => "test", "password" => "test-pass"}}
-      assert Authenticator.fetch_credentials(params) == {:ok, {"test", "test-pass"}}
+      assert Helpers.fetch_credentials(params) == {:ok, {"test", "test-pass"}}
     end
 
     test "returns name and password with grant_type 'password'" do
       params = %{"grant_type" => "password", "username" => "test", "password" => "test-pass"}
-      assert Authenticator.fetch_credentials(params) == {:ok, {"test", "test-pass"}}
+      assert Helpers.fetch_credentials(params) == {:ok, {"test", "test-pass"}}
     end
 
     test "returns error" do
-      assert Authenticator.fetch_credentials(%{}) == {:error, :invalid_credentials}
+      assert Helpers.fetch_credentials(%{}) == {:error, :invalid_credentials}
     end
   end
 end

test/pleroma/web/mastodon_api/controllers/media_controller_test.exs

@@ -5,6 +5,8 @@
 defmodule Pleroma.Web.MastodonAPI.MediaControllerTest do
   use Pleroma.Web.ConnCase
 
+  import ExUnit.CaptureLog
+
   alias Pleroma.Object
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
@@ -67,6 +69,59 @@ defmodule Pleroma.Web.MastodonAPI.MediaControllerTest do
       object = Object.get_by_id(media["id"])
       assert object.data["actor"] == user.ap_id
     end
+
+    test "/api/v2/media, upload_limit", %{conn: conn, user: user} do
+      desc = "Description of the binary"
+
+      upload_limit = Config.get([:instance, :upload_limit]) * 8 + 8
+
+      assert :ok ==
+               File.write(Path.absname("test/tmp/large_binary.data"), <<0::size(upload_limit)>>)
+
+      large_binary = %Plug.Upload{
+        content_type: nil,
+        path: Path.absname("test/tmp/large_binary.data"),
+        filename: "large_binary.data"
+      }
+
+      assert capture_log(fn ->
+               assert %{"error" => "file_too_large"} =
+                        conn
+                        |> put_req_header("content-type", "multipart/form-data")
+                        |> post("/api/v2/media", %{
+                          "file" => large_binary,
+                          "description" => desc
+                        })
+                        |> json_response_and_validate_schema(400)
+             end) =~
+               "[error] Elixir.Pleroma.Upload store (using Pleroma.Uploaders.Local) failed: :file_too_large"
+
+      clear_config([:instance, :upload_limit], upload_limit)
+
+      assert response =
+               conn
+               |> put_req_header("content-type", "multipart/form-data")
+               |> post("/api/v2/media", %{
+                 "file" => large_binary,
+                 "description" => desc
+               })
+               |> json_response_and_validate_schema(202)
+
+      assert media_id = response["id"]
+
+      %{conn: conn} = oauth_access(["read:media"], user: user)
+
+      media =
+        conn
+        |> get("/api/v1/media/#{media_id}")
+        |> json_response_and_validate_schema(200)
+
+      assert media["type"] == "unknown"
+      assert media["description"] == desc
+      assert media["id"]
+
+      assert :ok == File.rm(Path.absname("test/tmp/large_binary.data"))
+    end
   end
 
   describe "Update media description" do

test/pleroma/web/metadata/providers/open_graph_test.exs

@@ -22,7 +22,12 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraphTest do
           "attachment" => [
             %{
               "url" => [
-                %{"mediaType" => "image/png", "href" => "https://pleroma.gov/tenshi.png"}
+                %{
+                  "mediaType" => "image/png",
+                  "href" => "https://pleroma.gov/tenshi.png",
+                  "height" => 1024,
+                  "width" => 1280
+                }
               ]
             },
             %{
@@ -35,7 +40,12 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraphTest do
             },
             %{
               "url" => [
-                %{"mediaType" => "video/webm", "href" => "https://pleroma.gov/about/juche.webm"}
+                %{
+                  "mediaType" => "video/webm",
+                  "href" => "https://pleroma.gov/about/juche.webm",
+                  "height" => 600,
+                  "width" => 800
+                }
               ]
             },
             %{
@@ -55,11 +65,15 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraphTest do
     assert Enum.all?(
              [
                {:meta, [property: "og:image", content: "https://pleroma.gov/tenshi.png"], []},
+               {:meta, [property: "og:image:width", content: "1280"], []},
+               {:meta, [property: "og:image:height", content: "1024"], []},
                {:meta,
                 [property: "og:audio", content: "http://www.gnu.org/music/free-software-song.au"],
                 []},
                {:meta, [property: "og:video", content: "https://pleroma.gov/about/juche.webm"],
-                []}
+                []},
+               {:meta, [property: "og:video:width", content: "800"], []},
+               {:meta, [property: "og:video:height", content: "600"], []}
              ],
              fn element -> element in result end
            )
@@ -93,4 +107,84 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraphTest do
 
     refute {:meta, [property: "og:image", content: "https://misskey.microsoft/corndog.png"], []} in result
   end
+
+  test "video attachments have image thumbnail with WxH metadata with Preview Proxy enabled" do
+    clear_config([:media_proxy, :enabled], true)
+    clear_config([:media_preview_proxy, :enabled], true)
+    user = insert(:user)
+
+    note =
+      insert(:note, %{
+        data: %{
+          "actor" => user.ap_id,
+          "id" => "https://pleroma.gov/objects/whatever",
+          "content" => "test video post",
+          "sensitive" => false,
+          "attachment" => [
+            %{
+              "url" => [
+                %{
+                  "mediaType" => "video/webm",
+                  "href" => "https://pleroma.gov/about/juche.webm",
+                  "height" => 600,
+                  "width" => 800
+                }
+              ]
+            }
+          ]
+        }
+      })
+
+    result = OpenGraph.build_tags(%{object: note, url: note.data["id"], user: user})
+
+    assert {:meta, [property: "og:image:width", content: "800"], []} in result
+    assert {:meta, [property: "og:image:height", content: "600"], []} in result
+
+    assert {:meta,
+            [
+              property: "og:image",
+              content:
+                "http://localhost:4001/proxy/preview/LzAnlke-l5oZbNzWsrHfprX1rGw/aHR0cHM6Ly9wbGVyb21hLmdvdi9hYm91dC9qdWNoZS53ZWJt/juche.webm"
+            ], []} in result
+  end
+
+  test "video attachments have no image thumbnail with Preview Proxy disabled" do
+    clear_config([:media_proxy, :enabled], true)
+    clear_config([:media_preview_proxy, :enabled], false)
+    user = insert(:user)
+
+    note =
+      insert(:note, %{
+        data: %{
+          "actor" => user.ap_id,
+          "id" => "https://pleroma.gov/objects/whatever",
+          "content" => "test video post",
+          "sensitive" => false,
+          "attachment" => [
+            %{
+              "url" => [
+                %{
+                  "mediaType" => "video/webm",
+                  "href" => "https://pleroma.gov/about/juche.webm",
+                  "height" => 600,
+                  "width" => 800
+                }
+              ]
+            }
+          ]
+        }
+      })
+
+    result = OpenGraph.build_tags(%{object: note, url: note.data["id"], user: user})
+
+    refute {:meta, [property: "og:image:width", content: "800"], []} in result
+    refute {:meta, [property: "og:image:height", content: "600"], []} in result
+
+    refute {:meta,
+            [
+              property: "og:image",
+              content:
+                "http://localhost:4001/proxy/preview/LzAnlke-l5oZbNzWsrHfprX1rGw/aHR0cHM6Ly9wbGVyb21hLmdvdi9hYm91dC9qdWNoZS53ZWJt/juche.webm"
+            ], []} in result
+  end
 end

test/pleroma/web/metadata/providers/twitter_card_test.exs

@@ -9,6 +9,7 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
   alias Pleroma.User
   alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.Endpoint
+  alias Pleroma.Web.MediaProxy
   alias Pleroma.Web.Metadata.Providers.TwitterCard
   alias Pleroma.Web.Metadata.Utils
   alias Pleroma.Web.Router
@@ -17,7 +18,7 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
 
   test "it renders twitter card for user info" do
     user = insert(:user, name: "Jimmy Hendriks", bio: "born 19 March 1994")
-    avatar_url = Utils.attachment_url(User.avatar_url(user))
+    avatar_url = MediaProxy.preview_url(User.avatar_url(user))
     res = TwitterCard.build_tags(%{user: user})
 
     assert res == [
@@ -46,7 +47,7 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
 
     assert [
              {:meta, [property: "twitter:title", content: Utils.user_name_string(user)], []},
-             {:meta, [property: "twitter:description", content: "“pleroma in a nutshell”"], []},
+             {:meta, [property: "twitter:description", content: "pleroma in a nutshell"], []},
              {:meta, [property: "twitter:image", content: "http://localhost:4001/images/avi.png"],
               []},
              {:meta, [property: "twitter:card", content: "summary"], []}
@@ -91,7 +92,7 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
 
     assert [
              {:meta, [property: "twitter:title", content: Utils.user_name_string(user)], []},
-             {:meta, [property: "twitter:description", content: "“pleroma in a nutshell”"], []},
+             {:meta, [property: "twitter:description", content: "pleroma in a nutshell"], []},
              {:meta, [property: "twitter:image", content: "http://localhost:4001/images/avi.png"],
               []},
              {:meta, [property: "twitter:card", content: "summary"], []}
@@ -111,7 +112,14 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
           "content" => "pleroma in a nutshell",
           "attachment" => [
             %{
-              "url" => [%{"mediaType" => "image/png", "href" => "https://pleroma.gov/tenshi.png"}]
+              "url" => [
+                %{
+                  "mediaType" => "image/png",
+                  "href" => "https://pleroma.gov/tenshi.png",
+                  "height" => 1024,
+                  "width" => 1280
+                }
+              ]
             },
             %{
               "url" => [
@@ -123,7 +131,12 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
             },
             %{
               "url" => [
-                %{"mediaType" => "video/webm", "href" => "https://pleroma.gov/about/juche.webm"}
+                %{
+                  "mediaType" => "video/webm",
+                  "href" => "https://pleroma.gov/about/juche.webm",
+                  "height" => 600,
+                  "width" => 800
+                }
               ]
             }
           ]
@@ -134,17 +147,25 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
 
     assert [
              {:meta, [property: "twitter:title", content: Utils.user_name_string(user)], []},
-             {:meta, [property: "twitter:description", content: "“pleroma in a nutshell”"], []},
+             {:meta, [property: "twitter:description", content: "pleroma in a nutshell"], []},
              {:meta, [property: "twitter:card", content: "summary_large_image"], []},
              {:meta, [property: "twitter:player", content: "https://pleroma.gov/tenshi.png"], []},
+             {:meta, [property: "twitter:player:width", content: "1280"], []},
+             {:meta, [property: "twitter:player:height", content: "1024"], []},
              {:meta, [property: "twitter:card", content: "player"], []},
              {:meta,
               [
                 property: "twitter:player",
                 content: Router.Helpers.o_status_url(Endpoint, :notice_player, activity.id)
               ], []},
-             {:meta, [property: "twitter:player:width", content: "480"], []},
+             {:meta, [property: "twitter:player:width", content: "800"], []},
-             {:meta, [property: "twitter:player:height", content: "480"], []}
+             {:meta, [property: "twitter:player:height", content: "600"], []},
+             {:meta,
+              [
+                property: "twitter:player:stream",
+                content: "https://pleroma.gov/about/juche.webm"
+              ], []},
+             {:meta, [property: "twitter:player:stream:content_type", content: "video/webm"], []}
            ] == result
   end
 end