I was hit with another spam flood tonight (Akismet did catch it) so I decided to give Bad Behavior another shot. I’m wondering how much traffic coming from MySpace is actually just spammers following links, so it’ll be interesting to see what this does to the referrers.
Anyway, if you have issues accessing my pages or commenting on posts, let me know (hopefully you know my email address by now).
The DreamHost admin panel has been taunting me for many weeks now with a shiny link that would allow me to instantly update my WordPress installation. I’ve been meaning to upgrade to 2.0.2 for some time, so tonight I made a backup of my database and then asked DreamHost to do their thing.
Disappointment. I got email a few minutes later telling me the upgrade failed because DreamHost couldn’t download the necessary files. First, what happened? I was able to download WordPress just fine. Second, why is the upgrade for 2.0.2 pulling from wordpress.org directly? They don’t even make 2.0.2 available directly, they just have a version titled “latest”. Third, why isn’t DreamHost caching the download? I suppose it makes some sense to give WordPress a better idea of how many times it’s been downloaded, but there are better ways of figuring that out.
Anyway, I ended up downloading WordPress on my own and installed it by hand. Let me know if you see any issues.
You probably didn’t notice, but the machine at DreamHost that holds all of my data (including this blog) went down around 8:00am Pacific time this morning.
The server you’re on (yoda) has suffered a hard drive failure on its
system drive at approximately 8am PST. Services and data are being moved
over to new hardware, and it should be back online in under an hour.
It took them quite a while to bring everything back up, but things seem to be back to normal now. The database actually took quite a while to come back. Between this, a recent mail outage and a recent DoS attack…DreamHost has been having quite a lot of problems lately.
I upgraded my WordPress installation to the 2.0 Release Candidate tonight. The admin UI certainly is more…busy now. I haven’t decided if I like that or not yet. What I do like is that I unzip’d the file from wordpress.org, copied in my configs, my plugins and my tweaked theme and swapped out the old WordPress for the new one and it just worked. There’s something to be said for how simple it is to upgrade WordPress.
As always, let me know if you notice any issues.
So now that I’ve been running Akismet for a little while, I’m pretty impressed. It’s just survived it’s second 30+ spam storm and in all I think it’s only let a single piece of spam through (even that might not have been classified spam, just someone typing gibberish with no hyperlinks).
The UI is working much more to my liking after my tweaks to it.
It’s a glorious day to blog, once again.
I spent about 15 minutes tweaking Akismet tonight so that it wouldn’t do those things that piss me off. In particular, I changed the management page to show all spam messages, not just one message per spammer IP address. I also changed the sort order of the spam messages. It was displaying the newest at the top. I wanted to scroll the list chronologically. Not for any particular reason, just because that’s the way I read dated things.
While looking through the PHP, I noticed a bunch of dead code. In particular, inside the loop where it displays the currently held spam messages, it goes to through the trouble of formatting a date from the database for output:
$comment_date = mysql2date(get_settings(”date_format”) . ” @ ” . get_settings(”time_format”), $comment->comment_date);
Then it never uses the variable. Instead it calls the comment_date() function, which only gives you the date (if it’s not clear from the code above, $comment_date has the date and time). I switched the template to use $comment_date instead so I could see what time the message was posted. Again, not because I care that much but mostly because I hated seeing the information go to waste.
Before I make it sound like Akismet is the worst piece of software ever, there are two things I want to point out:
So rejoice, Akismet is back on the job. Hopefully for good this time.
Well, it’s always too good to be true. Earlier I gave Akismet another chance. Shortly after, I deactivated it once again. This time for trying to deceive me. I can’t tell if it’s actively trying to make me mad or if it’s just bad at math.
In the screenshot above you can see where I’ve circled something in red. It says “There are currently 2 comments identified as spam.” If you look immediately below that, you’ll only see one piece of spam. That was actually an older screenshot. Some time after I saw that Akismet told me there were currently 8 or 9 pieces of spam and yet it only displayed 3.
I don’t know what gives, but when I had already only placed tentative trust in the software, this doesn’t instill any additional confidence. So, once again, comments are being moderated by yours truly. If I get some spare time I might pop over to the Akismet web site or do some searches and see if I can find out whether other people have seen this issue.
Update: I’ve found why the counts are different. The count displayed comes from this query:
SELECT COUNT(comment_ID) FROM $wpdb->comments WHERE comment_approved = ’spam’
While the list of spam comments comes from this query:
SELECT *, COUNT(*) AS ccount FROM $wpdb->comments WHERE comment_approved = ’spam’ GROUP BY comment_author_IP ORDER BY comment_date DESC LIMIT 150
That “GROUP BY” is the big differentiator. The spam count query does an absolute count of all of the individual pieces of spam in the system. The comment list query (if I’m reading it correctly) groups all of the comments by the IP of the comment author. So if a spammer sends me two pieces of comment spam from the same IP, it shows up as +2 in the count but only one of the comments shows up in the list.
It’s probably still doing what I want it to do, but it’s confusing as hell. I’m not sure why on earth you would ever design it this way. Looks like it’s time to use the “Contact Us” link on the Akismet home page.
A while ago, I had to give up on Akismet because of some issues I was having. Today I reenabled it and things have gone pretty well so far. It’s let through two legitimate comments and it’s stopped two pieces of spam.
I won’t really declare it a victory until it stops the next quick spam flood (my spam seems to come in waves for the most part). But this is definitely looking much better. It’s looking so good that I went ahead and deleted all of the old spam I still had in the comments table (I was saving it in case I could use it for some Bayesian filtering).
I already don’t like Akismet so I’ve deactivated it. I didn’t even get to the point where it misbehaved in handling a comment. The admin panel is what pisses me off. It tells me there’s 440 pieces of spam it’s flagged when it hasn’t been active long enough. As it turns out, anything marked “spam” in the comments database is fair game. And Akismet tells me that after 15 days it’s going to trash anything marked “spam”. I actually prefer to keep that in the database for historical reasons.
*sigh* Another one bites the dust. I wonder how difficult it would be to hack out the bits that just do an “is it spam” check from the plugin. That’s all I’m really interested in.
I heard about Akismet today. It’s yet another anti comment spam plugin for WordPress. It requires an API key that you can only get by having a wordpress.com account, which you can get now by downloading flock. Hopefully this plugin works better than some of the others I’ve used in the past (really…it can’t be any worse).
So, as it turns out, the rules Bad Behaviour uses aren’t very good. I noticed it began blacklisting an IP I was coming from at work. As it turns out, that IP was hitting my feed URL without an “Accept” HTTP header. That pissed off Bad Behaviour and so Bad Behaviour rejected the request and threw an entry in the bad request database table. Later, I sent a perfectly valid HTTP request to fetch my blog and it too was rejected. Not because I didn’t send an “Accept” header (because I did), but because “I know you and I don’t like you, dirty spammer.”
As it turns out, Bad Behaviour checks requests for the usual fingerprinting. If none of those flag the request as a spammer, it makes one more check. It looks to see if the IP making the request has ever had any infractions logged against it. In my case, an errant request for my feed got an entire IP address blacklisted.
Why is that bad? Well…for starters…a whole slew of people in the world make their requests from the same IP address. People who use proxy servers or who use a NAT-enabled network all share the same public IP address. That means one person can ruin everything for the rest of the users on that network. LAME.
So, I’m pretty sure Bad Behaviour’s days are over on my site. It was a fun experiment. It was nice to see what good software could accomplish. Now all I have to do is find that software.
While monitoring my Bad Behaviour installation during the day today, I wanted to know how much comment spam I had been receiving. So I hopped into the database and ran a little mojo on it:
SELECT comment_approved, count(*)
FROM wp_comments
GROUP BY comment_approved
The number returned was staggering. As it turns out, my blog has received 2,726 comments…of which 2,602 are spam (I flag comments as spam rather than delete them so I can do just this sort of analysis). That’s more than 95% spam! I can only imagine how mad I’d get with my email if spam rates there were as high.
Anyway, I just figured I’d share just how much comment spam I’ve moderated since I turned comment moderation on back in May (a little over 5 months ago). There’s a bunch of old spam that I did delete back before I had moderation turned on, so the total amount of spam is actually higher. But ~500 spam entries a month seems about right. *sigh*
I’ve installed the Bad Behaviour WordPress plugin to see how well it deals with comment spam. Bad Behaviour looks at user agent traffic profiles and compares it to the behavior of known spambots. I’ll give it a try and see how it goes. Please let me know if you experience any issues accessing my site over the next week or two.
Depending on how Bad Behaviour does or doesn’t work out, I may give Spam Karma 2.0 a try as well.
This morning I woke up to see I have 187 comments awaiting moderation. With a number that high, I instantly know it must be at least 99% spam. In fact, it turned out to be 100% spam. That’s 187 pieces of comment spam in about 8-9 hours. I don’t get that many pieces of email spam in a month.
I’ve been hesitant to put any of the WordPress spamblocking plugins on my site, mostly because I haven’t had a lot of comment spam but also because I haven’t had enough time to see how they operate (do they drop stuff on the floor or do I get a chance to review it?). And I refuse to put anything on my site that will require user action, such as a CAPTCHA.
Is anyone out there using any non-invasive spamblocking technology for WordPress that they really like?
I’ve become bored with the default theme and more than one person has told me that I need to switch off of it (you know who you are, now look what you’ve done). So I’ve switched to the Blix theme by Sebastian Schmieg. I’m not sure if I’ll stick with it long term, but it’s a refreshing change.
Weary of having a broken comments form, I dug into the problem tonight. After some thrashing about in the PHP, I started doing some searching. First I tried searching on the WordPress Support site. That was a total fiasco. It’s really hard to find anything in that mess.
So I turned to Yahoo! Search. Thankfully, my bretheren on the search team didn’t let me down. The second result turned up this gem. As far as I can tell, a TON of people have had this problem. It turns out, at some point WordPress kindly created a user for me with no username and no password. So when people were seeing the “Logged in as . Logout” text in my comments form, they really were logged in as someone. I have no idea how that account got there and I don’t see any way of being able to remove it. But the WordPress forum document indicated I could make the problem go away by renaming the user and changing their password.
Miracle of miracles, IT WORKED! I was able to successfully submit a comment while not logged in as myself (an anonymous comment) and I was able to submit a comment while logged in as myself. If you happen to notice something not working with the comments, drop me mail (ryan AT unclehulka.com).
I really hate the bullshit WordPress makes you go through sometimes. The upgrade path totally sucks, which I constantly complain about, and now I’ve got to go digging because at some point it added an empty user to my blog that I now can’t remove (though I can rename it).
In related news, I’m making a new voodoo doll.
A kind reader pointed out that my comments form is messed up. I wonder how long it’s been like that, probably since I did my WordPress 1.5.2 upgrade. I’ll look into it.
I think this sums up why I’ll be getting off of WordPress at some point. I’ve posted before about how concerning it has been to me that they’ve had so many security updates lately. Now it comes out that they released 1.5.2, were informed of a security issue in 1.5.2 and overwrote the old 1.5.2 tarball with a new one containing a patch.
This is just an example of irresponsible engineering and makes me wary of what’s really lurking under the hood. If they can’t be bothered to have good versioning practices, how can I be sure they’ve taken any time to engineer a quality piece of software.
Don’t get me wrong, I’ve enjoyed using WordPress and I think it’s a great program. Shame it seems to be a bit of a mess behind the scenes.
Update: If this is true then I’ll be a bit happier about what happened with regard to 1.5.2’s versioning. I’m still wary of all the security updating going on, though. There just seems to be way too much of that lately.
I’m starting to get tired of all the WordPress upgrades made necessary by recurring security vulnerabilities. Both of the last two minor releases of WordPress have been for security updates. That doesn’t install a lot of faith in them, especially when I work with someone who had their WP installation hacked.
Anyway, I’m up to WP 1.5.1.3 now. I wish the upgrade involved less “copy files here, remove these, untar that, copy to here.” Would it be so difficult to include a script that manages all that crap for you? They’ve got the process well documented enough that it shouldn’t be hard to script.
Lars mentioned something to me today that reminded me to check my access lots. Fortunately, Dreamhost runs them all through Analog for me.
What’d I find? Not a whole lot of interest, except for my user agent log. For the current month (June), the MSN search bot (user agent “msnbot”) has outpaced the Google search bot (user agent “Googlebot”) 1252 requests to 409! I just have to wonder what on earth the MSN bot is looking for. It’s not like this site has a ton of content and what little content it does have rarely changes.
Is anyone else having a similar msnbot experience?
