woem.men/forkey
4
2
Fork 3
Code Issues 34 Pull requests 16 Projects Releases Packages Wiki Activity Actions

Delay crawling links #62

New issue
Open
opened 2025-02-05 15:14:21 +00:00 by leah · 2 comments
leah commented 2025-02-05 15:14:21 +00:00
Owner
Copy link

Summary

When every server crawls a link as soon as stuff federates it can cause a lot of stress on servers. We should crawl the link a bit after it federates like mastodon does to try spread out the amount of hits they get on their website.

1be5269151/app/lib/activitypub/activity/create.rb (L69)

Purpose

To stop sites from getting ddossed by mistake

Do you want to implement this feature yourself?

  • Yes, I will implement this by myself and send a pull request
### Summary When every server crawls a link as soon as stuff federates it can cause a lot of stress on servers. We should crawl the link a bit after it federates like mastodon does to try spread out the amount of hits they get on their website. https://github.com/mastodon/mastodon/blob/1be526915161687366e31fdf3456bba644e2ac0b/app/lib/activitypub/activity/create.rb#L69 ### Purpose To stop sites from getting ddossed by mistake ### Do you want to implement this feature yourself? - [ ] Yes, I will implement this by myself and send a pull request
sugar commented 2025-02-07 21:49:34 +00:00
Owner
Copy link

fae imagines it's not a big problem the way it was implemented on misskey, as misskey only crawls links when somebody sees a post with the link (as opposed to mastodon which always crawls links no matter if anyone sees them or not)

fae imagines it's not a big problem the way it was implemented on misskey, as misskey only crawls links when somebody sees a post with the link (as opposed to mastodon which always crawls links no matter if anyone sees them or not)
leah commented 2025-02-08 01:01:48 +00:00
Author
Owner
Copy link

Ah ok I wasnt sure how it was handled in misskeys code. That sounds way better yea

Ah ok I wasnt sure how it was handled in misskeys code. That sounds way better yea
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
fix/add-missing-translations
feature/mark-incoming-cwed-post-media-as-sensitive
mute-fix
cherrypick/small-and-center
feature/post-languages
feature/custom-email-template
cherrypick/refresh-polls
cherrypick/listenbrainz
feature/alt-text-achivements
cherrypick/choosable-search-engine
feature/indicate-block-mute
cherrypick/wildcard-antenna-accts
fix/scroll-alt-text
feature/alt-text-badge
feature/friendly-captcha
forkey-rebrand
bug/workflows
feature/approval
feature/meta-mfm
feature/note-editing
No results found.
Labels
Clear labels   
bug

Something is not working

  
customization

Change something to be more suitable for forkey

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
feature parity

Another fork has this feature, and it should be pulled into this fork.

  
help wanted

Need some help

  
high priority

   
invalid

Something is wrong

  
low priority

   
question

More information is needed

  
wontfix

This won't be fixed

No labels
bug
customization
duplicate
enhancement
feature parity
help wanted
high priority
invalid
low priority
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: woem.men/forkey#62
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?