What’s the difference between ceddit and removeddit? Also meaning of comments scanned.

43    25 Aug 2018 18:16 by u/slimuser98 *

So I was looking at this thread [here. ](https://www.google.com/amp/s/amp.reddit.com/r/AskReddit/comments/8pks1u/suicide_prevention_megathread/) Ceddit and removeddit are obviously showing different comments. Removeddit shows more. I’m also curious why they aren’t able to scan all 16000 comments vs 10000. Is this because of Reddit gold limits? Is the Pushshift not up to date? But aren’t they accessing reddit API so they shouldn’t be limited? Edit: Removeddit [link ](https://www.removeddit.com/r/AskReddit/comments/8pks1u/suicide_prevention_megathread/?utm_source=amp&utm_medium=comment_header) Ceddit [link](https://snew.github.io/r/AskReddit/comments/8pks1u/suicide_prevention_megathread/)

3 comments

17 u/Stuck_In_the_Matrix * 25 Aug 2018 22:12
I just wanted to update you (and everyone else) on this issue. The problem stems from both ceddit and removeddit sending a limit/size with their requests. Both sites are limiting to 10k. I have reached out to both maintainers and have asked that ceddit drops the limit completely and that removeddit increase their limit to 25k for now. The change for ceddit should happen later this evening. Eventually removeddit could remove the limit entirely, but they are using a different endpoint and I need to confirm a few things first before they remove their limit entirely. I hope this helps you /u/slimuser98! Thanks so much for bringing this to my attention. (I am the creator / maintainer of Pushshift.io) **Edit:** Why did they have a 10k limit? If I remember correctly, at some point I had a 10k limit for the number of objects that could be returned from a request. However, with more aggressive caching, that 10k limit is no longer necessary on my end as my system can (hopefully) handle the load without issue now. **Edit 2:** Removeddit now has a 20k limit. There was an issue with 25k for some reason -- still investigating. But the link above (the removeddit one) should have them all now.
6 u/Lucifirius 26 Aug 2018 01:01
Nice work
0 u/[deleted] 25 Aug 2018 18:16
[removed]