Proposition: CAPTCHA changes

[Rachael]

One thing that has been a pretty sore spot for some people trying to access this forum is that the CAPTCHA system as it is, is so tough. If you aren't an absolute fan of the classic Doom series there's a good chance you won't be able to answer most of them on your own.

The idea has always been that you could go to live chats like IRC, Discord, or Matrix, and ask the question there and someone knowledgeable would be able to help you. But some people are just shy and don't want to use that option.

On the counter end of this though, we've been having actual humans or very intelligent bots (probably a bit of both) bypass the CAPTCHA system with simple questions that are more common knowledge.

The questions we have right now are very tough. They've managed to weed out all but 3 spam-bot incidents in the past few months. That's actually very good. The problem is, it's too tough sometimes for actual legitimate users - as it stands, the forum can really only grow with people who know a lot about the original Doom series. That's not very conducive to a community that encourages the sharing of ideas and works for others to enjoy and learn from.

So the idea I had was this: What if the CAPTCHA system as it is was completely revamped to require you to download a version of ZDoom or GZDoom along with shareware versions of Doom and/or Heretic, use cheat codes to freeze the level and warp to certain points, and describe what you see on your screen, instead? It'd be a simple question like "What is the monster shown in E1Mx of Shareware Doom at point (xxx, yyy) when the player is facing north?" It could include a link to the bestiary of either game so that prospective users would know what to type and how to type it to pass the CAPTCHA.

I am not really sure if I like this system, though: First, it requires running an external program, second it could be easily defeated by a specially written bot that can look up the level data on its own.

I am open for ideas and discussion on this topic - however, I will not entertain any ideas which have the potential to greatly increase the workload of the site's moderators.

[dpJudas]

Given that the Doom community is pretty small and therefore unlikely that anyone would directly code a bot to bypass a custom CAPTCHA system (they'd be able to only spam zdoom.org), why not just write the answer on the page where it asks the question?

Basically the same system as when people type their email addresses in different forms to avoid a bot grabbing it for spamming. For example, put a big picture on it with some arrows or similar that any human should be able to figure out where a bot would fail. As long as the word isn't directly spelled so a primitive "grab all words on the page and try them" or something an OCR tool succeeds with I think it would do the trick.

[Jimmy]

So the idea I had was this: What if the CAPTCHA system as it is was completely revamped to require you to download a version of ZDoom or GZDoom along with shareware versions of Doom and/or Heretic, use cheat codes to freeze the level and warp to certain points, and describe what you see on your screen, instead? It'd be a simple question like "What is the monster shown in E1Mx of Shareware Doom at point (xxx, yyy) when the player is facing north?" It could include a link to the bestiary of either game so that prospective users would know what to type and how to type it to pass the CAPTCHA.

I am not really sure if I like this system, though: First, it requires running an external program, second it could be easily defeated by a specially written bot that can look up the level data on its own.

Yeeeahhh, that system seems really overly complicated from the user's point of view. What if the user was having trouble getting GZDoom to work, but couldn't register on the forum to ask about doing so because they couldn't launch the game to begin with?

Surely showing images of those levels and asking the user to identify them would be a better option? Keep them to E1M1, MAP01, etc - all the iconic opening shots.

What are the questions at the moment? If they're easy and broad enough, chances are someone signing up to the ZDoom forums has played Doom before, and knows at least roughly what it's about.

[Gez]

So the idea I had was this: What if the CAPTCHA system as it is was completely revamped to require you to download a version of ZDoom or GZDoom along with shareware versions of Doom and/or Heretic, use cheat codes to freeze the level and warp to certain points, and describe what you see on your screen, instead? It'd be a simple question like "What is the monster shown in E1Mx of Shareware Doom at point (xxx, yyy) when the player is facing north?" It could include a link to the bestiary of either game so that prospective users would know what to type and how to type it to pass the CAPTCHA.

That seems tougher than the existing system, and more likely to discourage people. Would you want to download and run a program in order to be able to register on a forum? As a general rule it'd be a pretty big red flag to me, honestly. Besides, who says that the system people use to go to the forum is even able to run GZDoom? Lots of people use their phones for Internet-stuff nowadays. Someone who is asked to download and run a PC exe while they're on their morning commute in a subway may be understandably irritated.

What I could see to make it simpler is to provide a link to the Doom wiki on the question. Something like a boilerplate sentence could be appended to all questions, such as "To help you find the answer, you can look it up in the Doom Wiki at DoomWiki.org." after a newline.

[Zhs2]

Personally I think the current system can be okay - the questions themselves could do with some access to less ambiguous answers. The question I'm thinking of as the most guilty of this, paraphrased, is "What is the hitscan enemy infamous for populating the Plutonia IWAD?" Besides being one of those questions that requires classic Doom knowledge, there are three, count 'em, three whole bolded possible answers in the first paragraph on the Doomwiki.org page for Heavy Weapon Dude. And what if you're someone who works with the code? Tying down the questions to one possible answer each would go a long way to alleviating the frustration. Note that this includes informing captcha answerers of the casing of said answer because I'm sure case sensitivity throws otherwise perfectly valid answers out too. dpJudas' idea makes a lot of sense to me.

[Rachael]

Actually, the CAPTCHA is only case sensitive if you tell it to be for that specific question. Otherwise, I think you can even mess up punctuation and spaces in your answer if you wanted to. Also there are multiple valid answers to most of the questions.

Nevertheless, I've added "Heavy Weapon Dude" as a valid answer to that question.

[Matt]

It would be a lot less daunting if the page actually told you that the answers were not case sensitive nor (ideally) space sensitive.

[kuemmel]

Yay! Just got past it via a question to a game I never played. Google was my friend leading me to the Heretic Doom Wiki walkthrough page of level 1. There, from two options, I selected the answer that had the less demanding orthography.

While I like that 'takes some brain approach', here is a suggestion: Why not add the link to the relevant doom wiki page (target='_blank' to make it open a new tab if you like) to the question? That way advertises the wiki, solves the orthography problem, and shortens the search for genuine people and really sophisticated bots.

[Graf Zahl]

It definitely needs to be easier, but the main question is, how can that be done without paving an entryway for the bots? I'm sure that any system that purely copies text will fail. How about this?

Ask questions like "How many monsters can be found on E1M1 on the hardest difficulty in single player mode? You will find the answer here.
For a bot to bypass this question they'd need some sophisticated ability to parse Doom Wiki pages which is not that likely to happen for a single forum. Important here is to use terminology that cannot be 1:1 evaluated to the linked page's information, e.g. in this case the user still needs to add up the number of the different bad guys and "hardest difficulty" never appears on the page, but anyone with even remote interest in the game should be able to make the connection. And swap out the questions regularly so that the bots cannot accumulate and share knowledge about the questions used.

[Rachael]

Bots of 2020 are much smarter than the trash crapware we had back in 2005.

The old tricks simply don't work anymore. They are able to conceptualize questions and at the very least answer some basic ones. Have you ever seen a typical Discord chat bot? It's scary how self-aware they seem to be - they almost do seem human.

Also the questions get rotated out when they start to fail - this doesn't happen often, but it does happen, and each rotation they get slightly harder.

[Graf Zahl]

You have to decide if it is more important to keep the bots out or new users in. The fact remains that the current questions are too hard, the last time I tried the system just to see how it goes the question was ambiguous enough that I wasn't sure what to reply. That's a total turn-off for more casual users. Do you have any numbers of how many daily registrations get stuck in the CAPTCHA system?


Regarding Discord, no, I haven't. I'm not using any social media, there's nothing useful to be found there as you just confirmed.

[Rachael]

If I am to be perfectly honest, in the last month we had about 145 successful registrations, which is between 4 to 5 users per day. And the numbers of how many get stuck in the CAPTCHA system are totally useless - we don't even know how many of them are bots. I can tell you, however, that there are more than 5000 hits per day to the registration page. Considering my past experience with DRD Team and ZDoom combined, I can tell you with absolute certainty that over 90% of those are bots, anyhow, and even that's a conservative figure. If I were to turn off the flood gates for one day, I can guarantee you the moderators would be spending 2-3 days cleaning up the mess. How do I know this? Because I've gotten stuck doing it before.

The only other "viable" system is actual human-to-human verification, and I don't know anyone who wants to stick around 24/7 to verify 5000 registrations per day.

(for a scalar reference - the forum section of the site gets half a million pageviews per day)

[Redneckerz]

I had to do a re-captcha myself when i had to add this site to my password vault. Safety, first eh?

Had a Heretic themed question so with some guided searches you can still relatively quickly find yourself, but i agree, it should be slightly more logical.

Perhaps rephrase words as such that they are hard to guess for a bot (with random numbers and all) but easy to solve for a human? I reckon this is a limited idea given the rist of AI networks to train bots to recognize words.

Its partially why i look hesitant at Nvidia's and others that invest heavily in AI. Yes, it has its advantages, but as with every new development - It will get misused to train far smarter bots than even what we have today. Bots that not only recognize a word or picture, but also know what it means and references. Neural networks are already halfway there.

[Gez]

Its partially why i look hesitant at Nvidia's and others that invest heavily in AI. Yes, it has its advantages, but as with every new development - It will get misused to train far smarter bots than even what we have today. Bots that not only recognize a word or picture, but also know what it means and references. Neural networks are already halfway there.

It's quite likely that we'll soon get to a point where bots are more likely to correctly answer the CAPTCHA than humans.

[Rachael]

It's quite likely that we'll soon get to a point where bots are more likely to correctly answer the CAPTCHA than humans.

That is one of my biggest fears.