Experts Round Table Network

Community Affairs => Propose and Consult => Topic started by: nicholassolutions on May 25, 2009, 05:16:09 PM



Title: spam prevention
Post by: nicholassolutions on May 25, 2009, 05:16:09 PM
I'd like to come up with a reasonable way to prevent spam on the site, while allowing legitimate new users to sign up. Here's what I am thinking. Please let me know your thoughts:

I. Principles:
1. The most important thing is that the limited time people spend here is productive. That means improving the site in meaningful ways, not just removing spam. This is even more important than getting new legitimate members. So, within reason, we have to prevent spammers from signing up, and remove the ones who are here.

2. It should still be possible for new users to sign up somehow.

3. #2 should not be too time consuming either, for the same reasons as in #1

II. Practice (my suggestions for actions to take)
1. Delete all accounts that are >3 months old and have no posts.
2. Disable automatic sign up.
3. Create an email address to write to in order to request an account.  Uses will have to include their emails, proposed username, and a brief description of their interests and why they want to join. There is no review process, this just insures that actual human beings are signing up. Unless things change, I predict we will get very few of these emails. The mail can fwd to me, huntress, and coral, who will make the account manually.
4. I will look into CAPTCHA options instead of #3, which may do a good job preventing bot signups, and then we will not need to manually make accounts.


Title: Re: spam prevention
Post by: coral1 on May 25, 2009, 05:46:09 PM
I was going to ask if #4 worked with our software.  ; )

I am looking at IP addys, and one I have up now, list signup attempts back to April 13, 2009, with Sender Verify fail. And 2 attempts today, with No Such User Here.

NARF! 
Somebody is yelling at me they can't get their grill going.  : (
I'll be back...


Title: Re: spam prevention
Post by: nicholassolutions on May 25, 2009, 06:10:56 PM
I was going to ask if #4 worked with our software.  ; )
In theory, yes, but in practice it may be a PITA. In general, our software is kind of patchy right now. I need to look around and see what is available, but it may be that the best thing to do is scrap what we have now, and rebuild with a forum, wiki, and articles. Everything else seems to have been pretty much unsuccessful anyhow. The idea would be to leave what we have now in the forum as an archive, and move everything else over to a new platform (which will no doubt get scrapped eventually itself). The main issue is whether I can migrate the users over to new software/databases...

For the time being, I'm going to disable new signups...



Title: Re: spam prevention
Post by: GrandSchtroumpf on May 26, 2009, 10:08:46 AM
how about using "member approval" in registration settings?
that's pretty easy to manage... i guess.


Title: Re: spam prevention
Post by: coral1 on May 26, 2009, 09:30:09 PM
Maybe a reply email with a site generated password before they can logon.


Title: Re: spam prevention
Post by: Huntress on May 27, 2009, 01:06:57 AM
I think CAPTCHA or re-CAPTCHA is probably the best way to go.  It does well on many other sites I'm on.

http://recaptcha.net/whyrecaptcha.html (http://recaptcha.net/whyrecaptcha.html)


Title: Re: spam prevention
Post by: nicholassolutions on May 28, 2009, 12:48:26 AM
Quote
how about using "member approval" in registration settings?
that's pretty easy to manage... i guess.
Yeah, that is what I turned on a couple days ago. The problem is, then we get tons of "spam" from the site every time a bot signs up (once every few minutes). Iit's hard to tell if the sign up is legit or a bot, so for any given signup it's difficult to decide whether to activate or not. The only reliable way would be to email the person and only activate if you get a response. But that could get to be a PITA very quickly.

Quote
Maybe a reply email with a site generated password before they can logon.
That's an interesting idea. There's no easy way to do that using the CMS we're running, so I think I'd have to write my own sign-up script, but it shouldn't be that hard. I've also got scripts from previous work that would be more or less drop-in solutions for the hard parts of this. I'll get back to you ;)

Quote
I think CAPTCHA or re-CAPTCHA is probably the best way to go. 
I like reCAPTCHA a lot, especially since it's pretty accessible, and it uses the user input to do something useful (digitize old books). I've had a pretty easy time setting it up on other sites. The problem is, hooking it into our current software is not super easy, and I don't want to spend time figuring it out. If I upgrade the forum, there are pre-existing solutions to link it in, though. This is one of many reasons to upgrade the forum and wiki, and ditch the current CMS, I think. I just need to make sure I can port at least the user info.


Title: Re: spam prevention
Post by: coral1 on May 28, 2009, 01:20:44 AM
Just had a thought, how much load will the password email put on the system?
Sending out passwords to bots seems like a waste.
Doing IP checks last night, I saw several with 6-8 hits on them. And I only did about half a dozen, before you killed the signup  (thank you).

Anyway we can add a text box for them to type in the SUBMIT link, instead of them just clicking the button?
Maybe spell it in haxor, with instructions to type it correctly.


Title: Re: spam prevention
Post by: Huntress on May 28, 2009, 01:30:14 AM
Just keep us updated on what you decide and need from us and we'll help in any way we can.  I've just been through the userlist and cleaned it up a bit.  Not sure why the ban filters aren't working right but that's moot since most of these are signed up via proxy accounts with different IPs.  Anyway....we're here.


Title: Re: spam prevention
Post by: GrandSchtroumpf on May 30, 2009, 03:17:56 AM
Are you saying that there is no email verification at all?
All the CMS i have looked at have that...
The system sends either the initial password (like drupal) or an activation link that needs to be clicked?
No wonder we have problems with spambots.


Title: Re: spam prevention
Post by: coral1 on May 30, 2009, 10:31:16 PM
Hmmm... looks like the notifs are down also. I didn't get one for GrandSchtroumpfs post.


Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 10:55:07 AM
How odd.  I didn't get one for GS either but I did get one for coral1?  Must be an intermittent problem?


Found a bug...Tried to upload an avatar and got this--> The attachments upload directory is not writable. Your attachment or avatar cannot be saved.   Just a heads up.  Was wondering why I couldn't see any avatars.


Title: Re: spam prevention
Post by: coral1 on June 01, 2009, 09:10:41 PM
I think there is a problem with the Avatar directory.
They stopped showing up after the move, just the place holder was there.
And after a couple of days, I think Nick turned them off, because even that stopped showing.


Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 09:14:31 PM
That's what I thought.


Title: Re: spam prevention
Post by: nicholassolutions on June 01, 2009, 10:10:09 PM
it was a server setting I forgot to update. should be fixed now ;)


Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 10:15:33 PM
Sweet!  Thanks hun.


Title: Re: spam prevention
Post by: nicholassolutions on June 01, 2009, 10:27:27 PM
Are you saying that there is no email verification at all?
All the CMS i have looked at have that...
The system sends either the initial password (like drupal) or an activation link that needs to be clicked?
No wonder we have problems with spambots.

We have an email verification...but it's not that hard to write a program to sign up for a gmail account, sign up at a forum, and then check mail frequently and follow activation links (especially easy if it's a standard email hard-coded in the CMS). Not all of the spam bots seem to be doing that, but I think some do. That's why I'd like to come up with something that is a little harder for a computer to manage.


Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 10:51:33 PM
I have an idea.  Can you setup an email account from the server that only the admins can access then we can post in the news section that all registrations have to be made through that email.  That way we can filter the baddy's and register the good ones manually.  Maybe require them to write a little note or something?


Title: Re: spam prevention
Post by: nicholassolutions on June 01, 2009, 11:06:12 PM
yeah that is sort of what I was thinking with #3 in my original post
Quote
3. Create an email address to write to in order to request an account.  Uses will have to include their emails, proposed username, and a brief description of their interests and why they want to join. There is no review process, this just insures that actual human beings are signing up. Unless things change, I predict we will get very few of these emails. The mail can fwd to me, huntress, and coral, who will make the account manually.

I'm a little worried it could be a PITA though. I guess it never hurts to try. There are two ways to do it:
1) Have an account on the server that you can check via webmail or your email program (thunderbird etc)
2) just set up an alias so that mail to that addy fwds to the admins (at whatever account you want)

which one works better for you guys?


Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 11:09:57 PM
I, for one, have been doing these for quite sometime so it kind of should fall on me.  I'd rather setup a pop3 in TBird like I used to have on the old servers when we were still .com.  I only work part-time now so have plenty of time to deal with it.


Title: Re: spam prevention
Post by: nicholassolutions on June 01, 2009, 11:12:07 PM
It's a deal :)
I'll set up an account and email you with the credentials.



Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 11:15:07 PM
Great!  :-)  After it's done I'll post instructions in the news section.


Title: Re: spam prevention
Post by: nicholassolutions on June 01, 2009, 11:17:30 PM
sounds good -- I'll also try to look through the forum code and see if I can insert a message to replace the one that says "registration disabled." If I can, I'll put whatever message you post over there also.


Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 11:18:36 PM
Excellent.  :-)  I'm ready when you are.


Title: Re: spam prevention
Post by: coral1 on June 01, 2009, 11:19:30 PM
Ditto on the SWEET !   : )

HAHAHAHA... while I was typing out something like that, you 2 already did it.  : )

Yell if you need a hand.


Title: Re: spam prevention
Post by: Huntress on June 01, 2009, 11:21:00 PM
You know it!  ;-)  Thanks Craig.


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 12:27:07 AM
Please check the news on the main page.  :-)  It's all set now.  I will be getting the messages on my machine but they will also be left on the server for up to 7 days so that if anyone wants to give a hand and has access to the webmail then have at it!  ;-)


Title: Re: spam prevention
Post by: nicholassolutions on June 02, 2009, 12:29:05 AM
Looks good! I'll try to paste it into the register link too.


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 12:32:02 AM
Sweet!


Title: Re: spam prevention
Post by: coral1 on June 02, 2009, 12:40:58 AM
Do you have some way to mark them so we don't double up?

I am assuming you are already logged on there, since I can't get in, yet.
I will set up a pop tonight for OE, and try again tomorrow. It's past my bedtime.  : )


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 12:42:53 AM
WAIT!  If you set up the POP account you have to leave the mssages on the server!  Make sure of that.


Title: Re: spam prevention
Post by: nicholassolutions on June 02, 2009, 12:49:36 AM
heheh, I can see this is stirring up trouble already =)

One (or both) of you can also use IMAP to prevent a conflict. Credentials/servers are the same.

I updated the text on the registration page. Let's see how it works out!


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 12:57:12 AM
I'll give it a test right now...Username=Muse


Title: Re: spam prevention
Post by: coral1 on June 02, 2009, 12:58:48 AM
Leave messages -- check

I was just going to do a test run. I won't do anything until we can sync our efforts. : D

I really need to get to bed now.


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 12:59:28 AM
LOL  It's giving an error but still gives the info at least.  ;-)


Title: Re: spam prevention
Post by: nicholassolutions on June 02, 2009, 01:03:25 AM
dang I forgot to fix that. Lemme try


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 01:07:34 AM
Just got my email for resistering at least.  I think this will work well to deter a lot of the crap.  And as far as registrations go...I'll need to change the message to include "desired username"  password will be generated by whichever admin does the reg then sent to the user.  This way, all we have to do is search the usernames to see if they've already been done by another admin.


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 01:09:41 AM
News edited.

Quote
To register as a member here you will have to send an email with the subject "Register" then include your desired username and a brief message in the body to register@expertsrt.net Thank you!


Title: Re: spam prevention
Post by: nicholassolutions on June 02, 2009, 01:10:41 AM
Sounds like a plan. I fixed the registration screen message so it doesn't look like an error. I'll update the message in a second. Thanks for the help sorting this out. Now it's past my bedtime too :)

Talk to you guys soon. Probably not before the weekend - I got a bunch of junk to do this week.


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 01:11:59 AM
Ok hun.  ttys!  Sleep well both of you.


Title: Re: spam prevention
Post by: Huntress on June 02, 2009, 01:34:47 AM
GOT IT!  If you login through Horde you can mark messages as "Answered".  That won't work if we all use POP3 though.  :-/


Title: Re: spam prevention
Post by: coral1 on June 02, 2009, 09:40:40 PM
OK. I got in through Horde, and looked around a bit.
Inbox was empty, and I saw the 2 tests in Sent.

What are you using to generate passwords?


Title: Re: spam prevention
Post by: Huntress on June 03, 2009, 12:16:00 AM
Haven't had to yet.  I think we may have deterred the worst of it now.  It might be awhile before we get any new members again.

Besides, it doesn't really matter what the password is since they should change it as soon as they login.


Title: Re: spam prevention
Post by: coral1 on June 03, 2009, 08:49:22 PM
>> it doesn't really matter what the password is since...

That's kind of what I thought, but wanted to double check.

I don't want to furar things TOO soon.   ; )


Title: Re: spam prevention
Post by: coral1 on June 03, 2009, 09:44:07 PM
OK, what did I do wrong to get this:

[error message removed by matt for security]

I was trying to register a new user, and that's what I got when I hit the Register Button.
The email is from Nick Wilson, just in case it went out despite the error message.

update:

I just checked again, seems it did register him (Kestrel). I will go ahead and mark it Answered.


Title: Re: spam prevention
Post by: GrandSchtroumpf on June 03, 2009, 11:31:15 PM
i also got this message when posting a simple message...
you did not do anything wrong.
it's just the absolute path to the website that has changed.
a typical problem you encounter when migrating.


Title: Re: spam prevention
Post by: coral1 on June 03, 2009, 11:51:43 PM
OK.
Something else I noticed, when looking at his profile, my IP addy showed up in it.
May I assume this will 'fix' itself when he logs on the first time?


Title: Re: spam prevention
Post by: Huntress on June 04, 2009, 12:14:32 AM
Just got the same error.  Nick or Matt will have to look at it.  I's something in the coding.


Title: Re: spam prevention
Post by: coral1 on June 04, 2009, 12:31:57 AM
That's a relief.   : )

wow. moving Threads is mind numbingly tedious.   8 |


Title: Re: spam prevention
Post by: Huntress on June 04, 2009, 12:33:26 AM
Don't I know it!


Title: Re: spam prevention
Post by: coral1 on June 04, 2009, 12:42:23 AM
LOL.   Thanks. I needed a good laugh to "break the daze".   : )


Title: Re: spam prevention
Post by: nicholassolutions on June 04, 2009, 09:17:30 PM
The error should be fixed now -- lemme know if anything else pops up.


Title: Re: spam prevention
Post by: coral1 on June 04, 2009, 09:21:58 PM
You can count on it.  ; )


Title: Re: spam prevention
Post by: Huntress on June 24, 2009, 05:03:21 AM
grrr, now I can't get into the horde mailbox.  Did someone change the password or is it just plain down?  I haven't been chacking my email like I used to and saw a new user registration.  Just wanted to remove it.

And the ban filters don't appear to be working properly either.  I added *@mail.ru and I don't think it's working.


Title: Re: spam prevention
Post by: coral1 on June 24, 2009, 05:59:06 PM
Horde is working for me. I've been checking it almost every night, and I just looked now.

There have been 2 requests that I put through, but it doesn't look like they have come back yet. I even sent a Welcome email from one of my gmail accounts and told them to yell if they had problems.

You want me to PM you the details?


Title: Re: spam prevention
Post by: Huntress on June 24, 2009, 07:18:24 PM
I saw the last one but you can send the one before that.