Commit Graph

208 Commits

Author SHA1 Message Date
floatingghost 28626eafc1 Allow higher amount of restarts for Pleroma.Repo during testing
This was done by floatingghost as part of a bigger commit in Akkoma.
See <37ae047e16/lib/pleroma/application.ex (L83)>.

As explained in <https://ihatebeinga.live/objects/860d23e1-dc64-4b07-8b4d-020b9c56cff6>

> there are so many caches that clearing them all can nuke the supervisor, which by default will become an hero if it gets more than 3 restarts in <5 seconds

And further down the thread

> essentially we've got like 11 caches (37ae047e16/lib/pleroma/application.ex (L165))
> then in test we fetch them all (https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/test/support/data_case.ex#L50) and call clear on them
> so if this clear fails on any 3 of them, the pleroma supervisor itself will die

How it fails?

> idk maybe cachex dies, maybe :ets does a weird thing
> it doesn't log anything, it just consistently dies during cache clearing so i figured it had to be that

> honestly my best bet is locksmith and queuing
> https://github.com/whitfin/cachex/blob/master/lib/cachex/actions/clear.ex#L26
> clear is thrown into a locksmith transaction

> locksmith says
> >If the process is already in a transactional context, the provided function will be executed immediately. Otherwise the required keys will be locked until the provided function has finished executing.

> so if we get 2 clears too close together, maybe it locks, then doesn't like the next clear?
2022-07-14 13:50:44 +02:00
Sean King 17aa3644be
Copyright bump for 2022 2022-02-25 23:11:42 -07:00
Lain Soykaf 4e98ba3c3a Application: Actually start finch if it's needed 2021-12-15 15:42:37 -05:00
Lain Soykaf abb62dd886 Application, dependencies: prepare for finch 2021-12-15 13:53:09 -05:00
Mark Felder 4a181982c3 More confusingly named legacy chat code renamed to shout 2021-06-01 11:49:16 -05:00
Mark Felder a3cff59659 Ensure we actually start ShoutChannel 2021-06-01 11:49:16 -05:00
Alex Gleason f2134e605b
Merge remote-tracking branch 'pleroma/develop' into cycles-base-url 2021-05-31 16:49:46 -05:00
Alex Gleason 51a9f97e87
Deprecate Pleroma.Web.base_url/0
Use Pleroma.Web.Endpoint.url/0 directly instead. Reduces compiler cycles.
2021-05-31 16:48:03 -05:00
Ivan Tashkinov 3edf45021e [#3213] Background migration infrastructure refactoring.
Extracted BaseMigrator and BaseMigratorState.
2021-03-12 12:18:11 +03:00
Ivan Tashkinov 8f88a90ca3 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	lib/pleroma/application.ex
#	lib/pleroma/config.ex
2021-02-23 13:58:35 +03:00
Ivan Tashkinov 0ef783baa1 [#2534] Earlier init of Pleroma.Web.Endpoint (must be started prior to Pleroma.Web.Streamer). 2021-02-22 23:09:41 +03:00
Ivan Tashkinov 5992382cf8 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
#	lib/mix/tasks/pleroma/database.ex
#	lib/pleroma/web/templates/feed/feed/_activity.rss.eex
2021-02-11 19:31:57 +03:00
Mark Felder bf9cd4a0e2 Standardize the way we capture and use Mix.env() 2021-02-04 10:13:03 -06:00
Ivan Tashkinov 2634a16b4c Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
2021-01-21 20:20:35 +03:00
Alexander Strizhakov 6d48144a9d
use proper naming
for MediaProxyWarmingPolicy in ConcurrentLimiter
2021-01-21 09:50:18 +03:00
Alexander Strizhakov b4ff63d020
configurable limits for ConcurrentLimiter
Pleroma.Web.RichMedia.Helpers & Pleroma.Web.MediaProxy
2021-01-21 09:47:21 +03:00
Ivan Tashkinov e350898828 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-13 22:11:16 +03:00
Ivan Tashkinov 3e4d84729a [#3213] Prototype of data migrations functionality / HashtagsTableMigrator. 2021-01-13 22:07:38 +03:00
Haelwenn (lanodan) Monnier c4439c630f
Bump Copyright to 2021
grep -rl '# Copyright © .* Pleroma' * | xargs sed -i 's;Copyright © .* Pleroma .*;Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>;'
2021-01-13 07:49:50 +01:00
lain 67b15cc033 Search: Save detected pg version in a persistent term. 2020-11-23 15:29:55 +01:00
lain f9a4cf2097 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into websearch 2020-11-23 15:18:19 +01:00
lain e4289792d2 Linting. 2020-11-20 16:38:05 +01:00
lain a407e33c78 Application: Save postgres version in the environment 2020-11-20 16:26:22 +01:00
Alexander Strizhakov 0a5b22bc3b
start limiters in mix tasks 2020-11-20 11:37:01 +03:00
feld cd1b4155d5 Merge branch 'oban-jobs-to-simple-tasks' into 'develop'
Moving some background jobs into simple tasks

Closes #1790

See merge request pleroma/pleroma!3129
2020-11-19 20:32:32 +00:00
rinpatch 2c55f7d7cb Remove FedSockets
Current FedSocket implementation has a bunch of problems. It doesn't
have proper error handling (in case of an error the server just doesn't
respond until the connection is closed, while the client doesn't match
any error messages and just assumes there has been an error after 15s)
and the code is full of bad descisions (see: fetch registry which uses
uuids for no reason and waits for a response by recursively querying a
 ets table until the value changes, or double JSON encoding).

Sometime ago I almost completed rewriting fedsockets from scrach to
adress these issues. However, while doing so, I realized that fedsockets
 are just too overkill for what they were trying to accomplish, which is
 reduce the overhead of federation by not signing every message.
This could be done without reimplementing failure states and endpoint
 logic we already have with HTTP by, for example, using TLS cert auth,
or switching to a more performant signature algorithm. I opened
https://git.pleroma.social/pleroma/pleroma/-/issues/2262 for further
discussion on alternatives to fedsockets.

From discussions I had with other Pleroma developers it seems like they
 would approve the descision to remove them as well,
therefore I am submitting this patch.
2020-11-17 17:28:30 +03:00
Alexander Strizhakov 8d218ebaf5
Moving some background jobs into simple tasks
- fetching activity data
- attachment prefetching
- using limiter to prevent overload
2020-11-11 13:39:49 +03:00
eugenijm 8e41baff40 Add idempotency_key to the chat_message entity. 2020-10-31 05:50:59 +03:00
Ivan Tashkinov 8a55de1d78 [#3059] Fixed Phoenix 1.5 telemetry processing. 2020-10-22 13:54:15 +03:00
Ivan Tashkinov 9ea31b373f Merge remote-tracking branch 'remotes/origin/develop' into chore/elixir-1.11 2020-10-17 17:53:47 +03:00
Mark Felder b738f70953 Merge branch 'develop' into feature/gen-magic 2020-10-13 10:08:38 -05:00
Mark Felder 64553ebae2 Merge branch 'develop' into chore/elixir-1.11 2020-10-13 09:54:53 -05:00
Alexander Strizhakov abc3c7689b
HTTPSecurityPlug module name and filename 2020-10-13 16:43:55 +03:00
Alexander Strizhakov 6a87f94ee2
renaming ratelimiter supervisor 2020-10-13 16:38:48 +03:00
Mark Felder 04b514c567 Merge branch 'develop' into feature/gen-magic 2020-10-09 11:26:10 -05:00
Mark Felder 8156940a49 Compatibility with phoenix_pubsub 2.0.0 2020-10-07 13:28:39 -05:00
Mark Felder 70880d54f8 @env is not used 2020-10-07 12:55:16 -05:00
Ivan Tashkinov 7d5c3883ac [#3031] Refactoring: moved system commands checks to ApplicationRequirements. 2020-09-29 16:28:06 +03:00
Ivan Tashkinov 4e4f771082 Adjusted MediaProxyControllerTest to gracefully fail on missing dependencies. Installation docs update.
Added ffmpeg/imagemagick checks to launch checks (if media preview proxy is enabled). Added documentation on installing optional media / graphics packages (imagemagick, ffmpeg, exiftool).
2020-09-26 19:32:16 +03:00
Steven Fuchs f2ef9735c5 Federate data through persistent websocket connections 2020-09-18 11:58:22 +00:00
Mark Felder 55562ca936 Merge branch 'develop' into feature/gen-magic 2020-09-10 16:05:22 -05:00
Alexander Strizhakov ee67c98e55
removing Stats worker from Oban cron jobs 2020-09-07 19:16:14 +03:00
Alexander Strizhakov a11f23c130
user agent if Endpoint is not started yet 2020-09-02 15:45:47 +03:00
rinpatch dc3a418c27 application.ex: disable warnings_as_errors at runtime
see changed files for rationale
2020-09-01 09:08:54 +03:00
Roman Chvanikov 7569f225f1 Move checks to application startup 2020-08-05 19:38:55 +03:00
rinpatch 4128e3a84a HTTP: Implement max request limits 2020-07-15 15:26:35 +03:00
rinpatch 7738fbbaf5 Connection pool: implement logging and telemetry events 2020-07-15 15:26:25 +03:00
rinpatch ec9d0d146b Connection pool: Fix race conditions in limit enforcement
Fixes race conditions in limit enforcement by putting worker processes
in a DynamicSupervisor
2020-07-15 15:17:27 +03:00
rinpatch 58a4f350a8 Refactor gun pooling and simplify adapter option insertion
This patch refactors gun pooling to use Elixir process registry and
simplifies adapter option insertion.

Having the pool use process registry instead of a GenServer has a number of advantages:
- Simpler code: the initial implementation adds about half the lines of code it deletes
- Concurrency: unlike a GenServer, ETS-based registry can handle multiple checkout/checkin
requests at the same time
- Precise and easy idle connection clousure: current proposal for closing idle connections in
the GenServer-based pool needs to filter through all connections once a minute and compare their
last active time with closing time. With Elixir process registry this can be done
by just using `Process.send_after`/`Process.cancel_timer` in the worker process.
- Lower memory footprint: In my tests `gun-memory-leak` branch uses about 290mb on peak load (250 connections)
and 235mb on idle (5-10 connections). Registry-based pool uses 210mb on idle and 240mb on peak load
2020-07-15 15:17:27 +03:00
rinpatch e6ccc25565 Fix in-db configuration in dev environment
Previously, in-db configuration only worked when `warnings_as_errors`
was disabled because re-compiling scrubbers on application restart
created a warning about module conflicts. This patch fixes that
by enabling `ignore_module_conflict` option of the compiler at runtime,
and enables `warnings_as_errors` in prod since there is no reason
to keep it disabled anymore.
2020-07-14 13:16:54 +03:00