New user registration issues.

Have questions? Just saying hello? This is the place.
No explicit, hateful, or hurtful language. Nothing illegal.
Post Reply
Posts: 38
Location: Vermont

Post by st5555 » 03-12-19 4:43 pm


Could you explain just a bit more how having spammer's emails rejected gets your email blacklisted? That doesn't seem fair. Is the only viable plan to let things cool down or are there plans to revamp the registration process somehow.

I'm just a bit curious about what's going on behind the scenes.


Posts: 12329

Post by Nicholas » 03-12-19 11:17 pm

I agree it's not fair! ;)

So, the "Junk Mail" button in the web interface of websites like Gmail doesn't just hide the message from you. It actually registers a sort of digital "complaint". Once the receiving server sees enough of those complaints from its own users, it simply assumes the sending server is in the hands of spammers and begins a series of increasingly-long bans (usually starting at 24 hours and escalating from there), blocking ALL email from that server. Another way to erode the trust of a remote server is if they keep sending messages to accounts that don't exist.

Fast-forward to the way spam bots attack a forum like this one: because we require email registration, they use publicly available (usually stolen/hacked) lists of email addresses to try opening accounts. That hits both of those detectors: many real people will see the automatic forum registration messages, not recognize them, and click their "Junk Mail" button. And many of those email lists are stale with many of the addresses no longer active, so from the receiving server's point of view, is trying to send out lots of bogus-looking email.

There was a separate issue, too: we were using Gmail's servers before, which are supposed to be used by a human instead of automated, "transactional" email like purchase receipts and automatic forum registrations. Because of the intended use-case, Google included a hard cap at 250 messages sent per 24 hour period. That was actually worse than the trust/ban issues: around the holidays spammers ramp things way up, so the forum was getting (much) more than 250 bad registrations a day, so at some point we just started receiving returned messages to ourselves stating "you've sent too many messages today; please wait before sending more".

For both of those reasons, just for having the forum registration enabled, real users weren't getting their unlock keys.

Step one was to switch email services for our transactional email. We now use the SES service that's part of AWS (with a comically ridiculous sending limit of 50,000 messages per day).

For the server trust issue, I do have a workaround in mind. I haven't updated the forum software in a long time (because our spam plugin---which I like a lot---is no longer available for the newest version), but there is a new feature that can be enabled that might help. During new user registration we can add a box that requires a little explanation for why someone wants to join. Then, we get to manually approve or deny the request. Requiring a bit of human input (and having a human review it on our side) should eliminate enough spam accounts to offset the loss of our spam plugin from the upgrade. (There wouldn't be a filter anymore, but presumably close to 100% of the people invited would be real people.) Adding a human to the loop means a little more work on our side, but at best this forum might see a half-dozen real registrations a week, so it shouldn't be that big of a productivity drain. Especially because it's a snap decision: if the explanation appears at all fishy, we can just ignore it. That's just the time it takes to read a sentence or two, which isn't so bad.

I had considered an alternative, but it would take more effort with a little custom code: if you enter your short code in the same explanation box, it would count as proof enough and circumvent the process of waiting for us to approve someone. Sort of a "you already spent money here, you get the premium treatment." :lol:

All of that is waiting until after the next Synthesia 11 preview. I recently promised a release date for the first time in years, so we're working extra hard on that with no distractions. (That deadline is going to be a close one!) :D

... all of that said, it sure has been awfully quiet here since I disabled the registrations! I've manually added three or four users since November (upon emailed request), but I don't think I really appreciated just how much of the forum's activity was from brand new users.

User avatar
Posts: 1792
Location: Illinois

Post by jimhenry » 03-13-19 4:44 pm

If you are going to add a human review step, I suggest requiring a user's first post to be approved. I use that at my Miditzer Forum. A large percentage of users never post. If spammers just want to read, well OK. When a spammer tries to post, I just summarily delete their account without notice. Pretty simple and very effective.
Jim Henry
Author of the Miditzer, a free virtual theatre pipe organ

Posts: 12329

Post by Nicholas » 03-13-19 9:01 pm

I agree. That's effectively what the "why are you registering?" question will be. I may reword it to something like "what's your first post going to be about?"

That makes it even more succinct to audit: a sentence or two instead of a potentially longer post. (Though, really, it only takes 1-3 seconds to evaluate something like that no matter the post length. The only spammers that take longer than that are the ones that start by posting snippets copy-pasted from existing, older posts from real users just to start building up their post history. That's another reason I like the during-registration question because it's private, out-of-context text that a spam bot wouldn't be able to replicate from other users.)

Post Reply