Quasar wrote:Can zdoom.org please unblock my web browser? Otherwise as the maintainer of the the Doom Wiki I may be forced to archive all links to the wiki since I can no longer personally verify that any of them work. Thanks.
zdoom.org cannot be accessed by other sites maintainers
zdoom.org cannot be accessed by other sites maintainers
In short: https://www.doomworld.com/forum/topic/1 ... nt=2984386
-
SanyaWaffles
- Posts: 889
- Joined: Thu Apr 25, 2013 12:21 pm
- Preferred Pronouns: They/Them
- Operating System Version (Optional): Windows 11 for the Motorola Powerstack II
- Graphics Processor: nVidia with Vulkan support
- Location: The Corn Fields
- Contact:
Re: zdoom.org cannot be accessed by other sites maintainers
I'll say this as politely as I can: I dunno what is going on with zdoom.org's protective measures against spam and such, but given spam occasionally gets through anyway and a sizable amount of people have been having issues getting on here, we need to get to the bottom of what's causing these issues with the legit users and fix it.
Re: zdoom.org cannot be accessed by other sites maintainers
It isn't about spam, it's about letting the site run at all. When the measures were first put in place the site was getting around 6000 hits per minute, it was being bombarded by web scrapers often pretending to be old browsers. This site is not equipped to deal with that kind of traffic, and we can't afford the kind of web hosting that can. With that volume of requests, the site was frequently crashing and timing out, it would max its RAM usage much too quickly, and in some cases even kernel panic. And even now, those hits have never stopped, but the only reason the site can even be accessed at all was because a lot of the attempts to access it had to simply be denied right at the gate. You think the complaints are bad now? You should've been there back then.
The protective measures that are currently in place are made with a very simple calculation: Anyone with a decent (and updated) web browser can, should, and often does have access to the site. Almost all automated clients are completely blocked.
One thing that was completely beyond our control was the number of people who refuse to update their browsers, or insist on allowing their virus scanner to stand MITM every access to the site (this mangles your request headers and fools the server into thinking you are a bot - and there's no way to whitelist through this or I would have done so). Those people are unfortunately the most vocal, but they are still a minority. Do not allow your virus scanner to access the site (or only allow it to read the page *AFTER* it is loaded), and make sure you use a supported web browser. An *updated* version of Firefox, Chrome, Opera, Edge, and Brave all are able to access the site with no troubles, this is tested very often. If he must use another browser not listed, then I need to know what it is to add to my tests.
Another thing that can happen is if he is on a subnet that is used frequently by bot or node operators, that can also get him blacklisted - the site only allows a certain number of connections (very high) before it issues a temporary ban on the subnet. Those last about an hour. Again, this was another thing we had to do to block the immeasurable volume of bots that did actually manage to crash the site several times.
If he still has issues he can come to the ZD Discord and communicate with the moderation team - they'll give me his IP and browser string and I will try and see what I can do to help.
Some further reading about the kind of shit we have been a target of for a good while now (from a perspective of a company with completely different goals than our's, but they still suffered the same kind of attack none the less): https://whitewidget.com/insights/cybers ... what-to-do
The protective measures that are currently in place are made with a very simple calculation: Anyone with a decent (and updated) web browser can, should, and often does have access to the site. Almost all automated clients are completely blocked.
One thing that was completely beyond our control was the number of people who refuse to update their browsers, or insist on allowing their virus scanner to stand MITM every access to the site (this mangles your request headers and fools the server into thinking you are a bot - and there's no way to whitelist through this or I would have done so). Those people are unfortunately the most vocal, but they are still a minority. Do not allow your virus scanner to access the site (or only allow it to read the page *AFTER* it is loaded), and make sure you use a supported web browser. An *updated* version of Firefox, Chrome, Opera, Edge, and Brave all are able to access the site with no troubles, this is tested very often. If he must use another browser not listed, then I need to know what it is to add to my tests.
Another thing that can happen is if he is on a subnet that is used frequently by bot or node operators, that can also get him blacklisted - the site only allows a certain number of connections (very high) before it issues a temporary ban on the subnet. Those last about an hour. Again, this was another thing we had to do to block the immeasurable volume of bots that did actually manage to crash the site several times.
If he still has issues he can come to the ZD Discord and communicate with the moderation team - they'll give me his IP and browser string and I will try and see what I can do to help.
Some further reading about the kind of shit we have been a target of for a good while now (from a perspective of a company with completely different goals than our's, but they still suffered the same kind of attack none the less): https://whitewidget.com/insights/cybers ... what-to-do
Re: zdoom.org cannot be accessed by other sites maintainers
If scrapers are your problem, then your solution is using Anubis: https://anubis.techaro.lol/