Free Technology Newsletters
» All 33 InfoWorld Newsletters
Technology & Business Daily
 
InfoWorld
 
   

Botnet Spam Getting Out of Hand

By Ed Foster, Section The Gripelog
Posted on Fri Nov 03, 2006 at 01:07:36 AM PDT

Well, at least I'm not the only one who feels like the spammers are winning. Since I wrote last week about our struggles with link spam here, the evidence has been mounting that spam in general has been increasing at an alarming rate in just the last month. And the cause of all these woes is the growing menace of botnet attacks.


E-mail security vendor Postini reports the amount of spam it intercepted in October was up 59 percent over September and that as of yesterday 91 percent of all e-mail traffic consists of unwanted messages. Last week it was reported that spam blacklist maintainer Total Quality Management Cubed has seen 450 percent more spam in the last two months.

And there's no question where all this additional spam is coming from. "We can see it's coming from the bots," says Daniel Druker, executive vice president of marketing for Postini. "The part of our system that tracks this type of attack are just off the charts over the last six weeks. It's gotten to the point now where in any 24-hour period we'll see a million different IP addresses being used in coordinated attacks, and 50,000 operating at any given instant. We're starting to get emergency calls from large organizations that are finding that they simply can't handle the spam problem on their own anymore."

Of course, if everyone had effective security software on their computers, there wouldn't even be a botnet problem because there'd be no zombies for the bots to control. Since that day isn't likely to dawn anytime soon, though, how do we keep the botnets from making e-mail worthless for us all? One reader pointed to an interesting discussion in this regard by Ed Felton. Part of the problem, he says, is there simply isn't enough discussion about botnets outside the security business. The more people are made aware of the bots are doing to us, the more likely they will defend their computers against being taken over.

So, by all means, let's discuss botnets, because seeing that I'm not alone in suffering from their attacks doesn't make me feel all that much better. What do you think can be done to rescue the Internet from this rapidly growing scourge?

Post your comments about this column below, write me at Foster@gripe2ed.com or phone my voice mail at 1 888 875-7916.

< Cox Clobbers Competitor's Customers | Sun Never Sets on Java Security Updates >


Display: Sort:
Botnet Spam Getting Out of Hand | 69 comments (69 topical) | Post A Comment
???? Please explain[ Reply to This ] (none / 0) (#1)
by Anonymous User on Fri Nov 03, 2006 at 01:20:36 AM PDT

Can someone first explain what a Botnet is and how it works?

[ Reply to This ]


If only...[ Parent | Reply to This ] (none / 0) (#2)
by wantobe on Fri Nov 03, 2006 at 03:46:42 AM PDT

If only there were a website where you could enter a word or two about some subject and it would search all the web pages and return links that relate to that subject. Man, I bet I could make millions if I were to invent something like that. I could call it a search site (or something like that).

Try this link: Google
Rob Miles
--
There are 10 kinds of people in the world; those who understand binary and those who don't.
[ Parent | Reply to This ]



and it's...[ Parent | Reply to This ] (none / 0) (#3)
by Anonymous User on Fri Nov 03, 2006 at 05:50:17 AM PDT

...people like you why people don't even want to learn abuot this kind of thing.  Arrogant Jerks with horrible attitudes who respond to simple questions with A-hole responses, exactly like yours, is what convinces people not to even bother.

Try being part of the solution rather than the problem.

[ Parent | Reply to This ]



Insert subject here[ Parent | Reply to This ] (none / 0) (#9)
by Anonymous User on Fri Nov 03, 2006 at 11:19:50 AM PDT

Yeah, it's so hard to google what you're curious about. It's much harder than blaming your ignorance and laziness on the other person while tossing in a brisk personal attack and assuming a tone of unwarranted moral superiority.

Being hypersensitive about "face" online Does Not Work. Especially when you're the one who caused your own loss of face.

[ Parent | Reply to This ]



Grow up people[ Parent | Reply to This ] (none / 0) (#12)
by Anonymous User on Fri Nov 03, 2006 at 03:05:55 PM PDT

I love how people like to spend more time ripping someone for asking a question than they would if they just answered the question.

[ Parent | Reply to This ]


Didn't you notice?[ Parent | Reply to This ] (none / 0) (#15)
by wantobe on Fri Nov 03, 2006 at 06:19:20 PM PDT

Maybe I'm an asshole (though, really, didn't the gently chiding tone come through?), but I did provide the answer for the original poster too. Well, I provided a link to the Google search.

Lighten up, guys. I got to be sarcastic, the original poster got his answer. Everyone's happy!


Rob Miles
--
There are 10 kinds of people in the world; those who understand binary and those who don't.
[ Parent | Reply to This ]



Actually,[ Parent | Reply to This ] (none / 0) (#27)
by Anonymous User on Sun Nov 05, 2006 at 04:35:50 AM PDT

No maybe about it

[ Parent | Reply to This ]


Please go away[ Parent | Reply to This ] (none / 0) (#29)
by Anonymous User on Mon Nov 06, 2006 at 09:27:53 AM PDT

The time you put into being a jerk could have instead been used to answer the question and not make people avoid posting messages in this forum out of fear of being attacked by people like you.

[ Parent | Reply to This ]


The Silent Majority Did...[ Parent | Reply to This ] (none / 0) (#32)
by In my humble opinion on Mon Nov 06, 2006 at 05:13:09 PM PDT

...but a vocal minority took issue with your sharp wit. Perhaps they only have half of your wit...

I thought your response was quite crisp. Green light for the next wave of complaints. Apologies for the use of free speech.

[ Parent | Reply to This ]


Great Article, i agreee with you[ Parent | Reply to This ] (none / 0) (#78)
by Anonymous User on Mon Sep 01, 2008 at 10:48:10 PM PDT

dis j'ai jamais vu de poisson sans ouies........et avec une forme pareille.......Internet Marketing 迷你倉 護膚 .

[ Parent | Reply to This ]


22[ Parent | Reply to This ] (none / 0) (#75)
by Anonymous User on Thu Aug 07, 2008 at 06:45:31 AM PDT

Free YouTube Downloader | YouTube to iPod | YouTube on PSP | YouTube to MP3 | YouTube to MP4 | YouTube to 3GP | YouTube to AVI | YouTube to MPEG | YouTube to WMV | YouTube to DivX | YouTube to XviD | YouTube to MOV | YouTube to WMA | YouTube Ripper YouTube to iPod | YouTube to iPhone | YouTube to PSP | YouTube to Zune | YouTube to MP4 | YouTube to Apple TV | YouTube to H.264 | YouTube to 3GP

[ Parent | Reply to This ]


Rob Miles - Try This Link[ Parent | Reply to This ] (none / 0) (#33)
by Anonymous User on Mon Nov 06, 2006 at 09:44:43 PM PDT

Rob Miles - Try This Link

http://www.google.com/search?q=ButtHead&btnG=Google+Search

Results: 1 found

Did you mean Rob Miles is a Butt Head? (1,523,846)?

All meaningful results displayed. Caching unnecessary.


[ Parent | Reply to This ]


so many Anons out there[ Parent | Reply to This ] (none / 0) (#43)
by Anonymous User on Thu Nov 09, 2006 at 08:52:30 AM PDT

Amazing that only one person had the guts to use his name in the previous posts. Hats off to Rob for fearlessness ... the rest of you whiners ... could any one of you have explained a botnet? Guess not.

[ Parent | Reply to This ]


Eh?[ Parent | Reply to This ] (none / 0) (#51)
by Anonymous User on Thu Nov 09, 2006 at 08:00:20 PM PDT

You're one to talk, "Anonymous User"!

[ Parent | Reply to This ]


Spam solution?[ Parent | Reply to This ] (none / 0) (#24)
by Anonymous User on Sat Nov 04, 2006 at 02:21:23 PM PDT

Perhaps Ed should have gone into greater detail; but it seems to me he made it fairly clear that the link to Felten's blog post leads to more info.

Botnets are actually just about peaking. Consumers are, increasingly, locking down their machines (though prompted by more visible infections of more overt viruses and spyware), and newer computers and operating systems have had firewall capabilities and the like shipping with them and enabled by default for a while. As older, infected machines are replaced botnets will start weakening. Common malicious outbound traffic patterns might become targets for filtering by ISPs (hopefully without impacting user-desired traffic, such as p2p, but we know how likely the ISPs are to disrupt that intentionally and then blame it on spam filtering...) and e-mail itself might be superseded by a pseudonymous, sender-machine-address-authenticated form of mail.

Or we might simply see a rise in webmails and forwarding services that will do something like this:

Alice sends Bob an email, to his address at Gmail or a forwarding service or whatever. The service has no record of Alice as one of Bob's contacts, and as a result, the message isn't simply sent on for Bob to read yet. Instead, an autoreply is sent to Alice that directs her to the service's Web site. Once there she encounters a captcha. If she proves she's human, the message is released from quarantine and future messages from Alice to Bob encounter no obstacles. Unless of course Bob tells the service to block Alice or put her back on "probation".

This requires some way to identify Alice, so the system would require the captcha authentication for every message with any irregularity in the Received: header fields or no reverse lookup on the source IP. Otherwise, after one message is allowed through, future messages coming from the same IP sharing the same From: are passed through.

The effects on spam are as follows:
1. Spam sent directly from a consumer broadband address, generally from bot-infected PCs, produces a reply email that probably bounces (the errors-to or whatever address is phony), and nobody shows up and passes the captcha within the requisite time (say, one week). After a few such failures from the same source IP, it's blanket-blocked by the forwarding service and the spam doesn't even cost that service much anymore. None of it reaches a potential buyer/scam-victim/whatever.
Some of this spam may also be blocked by an ISP blocking outbound traffic whose destination port is 25 and source IP is a customer's machine (regardless of source port).

2. Spam sent via an ISP's mail exchange gateway is susceptible to being blocked readily by the ISP, which can terminate accounts misused to send spam through its gateway in the traditional manner. "Spamhaus" ISPs (ISPs that let users spam with impunity) get blacklisted as usual on the wider 'net. The forwarding services and webmails block them entirely, or filter most of the spam with the captcha and the few where the spammer actually deigns to authenticate as a human get someone mad who then has that sender blocked from sending to them. If too many combinations of from and IP with the same IP are blocked or captcha'd the ISP may be told to shape up or be blanket-blocked altogether.

Notice that none of the above depends in any way, shape, or form on filtering (bayesian or otherwise) that might produce false positives. The only "false positives" occur when people send mail but either use a bogus reply-to or don't bother to verify they're human the one time. The mail can't have been that important, then, can it?

(For the record, mail sent to report spam using "abuse.net" did an authentication thing like this for a long time, and perhaps still does. It didn't use a captcha; it just authenticated that the message had a valid reply-to before whitelisting that sender from then on. Otherwise the service could have been abused to spam sysadmins. Adding a captcha makes it even more robust.)

The long term result is especially interesting. Spammers have to either jump through hoops (per infected source machine!) to get their spam to its recipient, or the spam is blackholed without ever being seen by human eyes. Spam doesn't pay once it gets very little visibility and costs more time and effort to send at all. Spam itself would then decrease, and botnets devalue as this particular use decreased, and the costs to the mail filtering places would further decrease. In the long run, everybody wins -- except, of course, the spammers.

[ Parent | Reply to This ]



UGH! - Another Challenge-response idea[ Parent | Reply to This ] (none / 0) (#36)
by Anonymous User on Tue Nov 07, 2006 at 02:44:23 PM PDT

C-R does not work. In fact, it is a form of spam and increases network traffic. 1. Most spam is sent with a forged return address. Upon hitting a C-R ("captcha") system, a challenge will be sent back to the forged address, not to the spammer. This can result in hundreds or thousands of "challenges" send to someone whose e-mail was forged into a spam run. This makes C-R, itself, spam. 2. What happens when a C-R request is sent to another machine using C-R? Perhaps as a result of a forgery? Answer should be obvious.

[ Parent | Reply to This ]


Eh[ Parent | Reply to This ] (none / 0) (#38)
by Anonymous User on Tue Nov 07, 2006 at 08:19:26 PM PDT

A naive implementation will certainly have problems. No disagreement there. But that doesn't mean that no implementation will ever work.

[ Parent | Reply to This ]


A Botnet is . . . [ Reply to This ] (none / 0) (#4)
by Anonymous User on Fri Nov 03, 2006 at 06:18:55 AM PDT

Botnet is a jargon term for a collection of software robots, or bots, which run autonomously. This can also refer to the network of computers using distributed computing software.

While the term "botnet" can be used to refer to any group of bots, such as IRC bots, the word is generally used to refer to a collection of compromised machines running programs, usually referred to as worms, Trojan horses, or backdoors, under a common command and control infrastructure.

Complete definition at http://en.wikipedia.org/wiki/Botnet

[ Reply to This ]



part of the solution[ Parent | Reply to This ] (none / 0) (#14)
by Anonymous User on Fri Nov 03, 2006 at 04:23:39 PM PDT

What I'd like to know is how can I find out if I'm part of a botnet. Just using myself as an example; I have XP and Norton firewall and antivirus. I keep my Norton updates current. I *think* I'm in good shape with a clean system. But... how do I know for sure? Is there a way to know if there's an intruder smtp loaded on my system that's sending out spam? I'm an experienced IT person who understands the issues and technologies, and if *I'm* wondering this, can you imagine how non-IT people get along?

[ Parent | Reply to This ]


Watching that traffic[ Parent | Reply to This ] (none / 0) (#17)
by Anonymous User on Fri Nov 03, 2006 at 07:06:22 PM PDT

On my FreeBSD machine I use tcpdump to watch what goes in and out when it matters. There's something like this for Windows, and I've seen good reviews of this one: http://www.winpcap.org/windump/ . Note I have absolutely no connection to them.

Ed Hurst <jehurst at gmail dot com>

[ Parent | Reply to This ]



A Solution Won't be Allowed[ Reply to This ] (none / 0) (#5)
by srynas on Fri Nov 03, 2006 at 06:31:37 AM PDT

This is one of those problems were corporate pressure on our political system will not allow a solution. We are a "marketing society".  By that, I mean that everything is about making the NEXT sale.  Any law that would limit a corporation from bombarding a consumer with come-ons simply can't be allowed.

Corporate marketing mania is similar to that of a drug addict.  No matter how numb the customer may become from the deluge of spam, the marketers spew even more spam in the hope of getting the next sale.

Of course, there are other aspects to this.  The internet is a global community.  Consequently spam can originate from areas where the US would have no jurisdiction. So, if a solution would somehow be allowed, it would have to be a global one.

[ Reply to This ]



Firewall ACL's[ Reply to This ] (none / 0) (#6)
by tscoff on Fri Nov 03, 2006 at 07:46:11 AM PDT

I don't have major problems with spam because I have all of Asia, Russia, and Africa plus parts of Europe and South America blocked by my firewall.  My mail server and web server can not be accessed at all by computers in significant portions of the world.

It's a drastic step, and I miss about 1 e-mail a month because of it, but it also keeps my spam problem mostly under control.  It only works because I only care about receiving e-mail from people in America.

[ Reply to This ]



Not just email spam[ Reply to This ] (none / 0) (#7)
by sconeu on Fri Nov 03, 2006 at 08:16:00 AM PDT

Link spam too.

What we've been seeing here is merely a drop in the bucket.

I'm a member of a long running site (since 1998) that hosts discussion boards for "nitpicking" TV shows and movies, and recently we've been hit by a linkspam botnet, which floods the boards, to the point where they've been useless.

Previously, we had a pretty much open policy, with no registration required, allowing anyone to post under any name, which led to some quite amusing long-running threads/stories/interactive-fiction.

We have pretty much come to the consensus that to defeat the spambot and keep the site useful, we have to introduce mandatory registration.

These spamming jerks have destroyed the character of our site, and death is too good for them.

--
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the United States of America.
[ Reply to This ]



Captcha[ Parent | Reply to This ] (none / 0) (#21)
by Anonymous User on Sat Nov 04, 2006 at 02:16:32 PM PDT

Just make posting require a captcha. Or optional registration, with posting anonymous requiring passing a captcha per post, and registering requiring passing a captcha the one time.

[ Parent | Reply to This ]


Windows botnets? Make Micro$oft solve the problem![ Reply to This ] (none / 0) (#8)
by Anonymous User on Fri Nov 03, 2006 at 08:29:29 AM PDT

Since most of the botnets are a result of Micro$oft's horrendous security in Windows, THEY should be charged with cleaning up this mess.

Every Windows PC should be required to have a FREE anti-virus solution installed, automatically. This should happen as part of a Windows update. This should NOT be a profit making opportunity for Micro$oft. This should be required of them since their insecure software is what allows the infection of PCs with viruses (I use the term "virus" generically to cover bots, trojans, etc.)

The reality is that most people already have too much to do in their everyday lives and don't WANT to be educated about this issue. They want a solution that WORKS, and which they don't have to think about, NOR pay for.

This must be done for EVERY version of Windows out there! It's "not supported anymore" is NOT an excuse for Micro$oft's past negligence being allowed to plague us into the future.

Yes, I know, this solution is going to be considered "unrealistic" since many companies provide paid-for anti-virus solutions. But, until we stand up and take care of the SOURCE of the problem, we have no one to blame but ourselves. (We are to blame for the SPAM because we don't FORCE Micro$oft to create a solution that makes ALL their operating systems (relatively) safe from exploitation. We are too worried about preserving a market for a few companies.)

Are we going to preserve a market for a few paid anti-virus companies, or are we going to make a dent in the problem of SPAM (phishing, etc.?)

Until Micro$oft is REQUIRED to do this, the problem won't be solved.


[ Reply to This ]



Windows botnets? Make Micro$oft solve the problem![ Parent | Reply to This ] (none / 0) (#10)
by Carl on Fri Nov 03, 2006 at 12:19:00 PM PDT

Microsoft already does distribute antivirus/antispyware stuff with Vista, and they have antispyware with XP SP2.

[ Parent | Reply to This ]


Windows botnets? Make Micro$oft solve the problem![ Parent | Reply to This ] (none / 0) (#11)
by Anonymous User on Fri Nov 03, 2006 at 02:12:15 PM PDT

Actually the easiest method would be just to simply de-activate all the non-supported or non-updated copies of windows. Would be quite simple with XP...

[ Parent | Reply to This ]


And limit legitmate users?[ Parent | Reply to This ] (none / 0) (#30)
by TonyK on Mon Nov 06, 2006 at 11:21:52 AM PDT

That is what would happen. My XP SP2 was routinely updated until WGA. As a result of WGA I no longer even try.

WGA does nothing but hassle the legal user while priates and thieves find ways around the system. There are numerous problems and issues with WGA flagging a legal copy of Win XP as non-compliant and the poor user has little to no recourse

Let's not victimize the victims again.

Peace,

[ Parent | Reply to This ]



Not just microsoft.[ Parent | Reply to This ] (none / 0) (#18)
by foxyshadis1 on Fri Nov 03, 2006 at 09:52:04 PM PDT

If every consumer-level router had come with automatic block of outbound smtp beyond one per minute (with an initial burst of 5, resetting every hour or two), the spam problem would probably be only a tenth what it is now. Linksys (Cisco), Netgear, D-Link, and the smaller companies that ISPs use could have published and applied guidelines for the curbing of spam with minimal interruption, and prevented the need for widespread ISP-level blocking of port 25.

Updating what's already out there can never "fix" the problem, because it would never be installed by the vast number of non-updating systems. After all, trojans disable autoupdate, and before XP SP2, it didn't run entirely by default at all. And while free OneCare for all is a nice idea, most viruses and trojans are difficult to remove without offline recovery. Shipping everyone a CD that runs a scan outside windows and cleans up would be a great idea, aside from self-updating viruses, and you can actually make a CD like that with AVG and BartPE.

It's all nice to say that Microsoft should have this and that, but the only way to make a lot of things truly safe would be class action suit against microsoft that actually wins, for vouchers for those stuck on old OSes to upgrade to vista and/or unix with some tech support, or get a large-scale antivirus program for their entire organisation. If you're willing to start that suit, be my guest, but pontificating that they should do this and that on an obscure blog won't get it to happen.

[ Parent | Reply to This ]



This is going to interfere with legit use[ Parent | Reply to This ] (none / 0) (#28)
by LasVegan on Sun Nov 05, 2006 at 10:33:14 AM PDT

I on occasion send more than 5 messages in a quick burst and I'm sure there are other people in the same boat. I wouldn't mind a filter like that as a default on the router so long as it could be overridden, but don't force it.

[ Parent | Reply to This ]


Won't work[ Parent | Reply to This ] (none / 0) (#59)
by Anonymous User on Tue Nov 14, 2006 at 08:46:56 AM PDT

If broadband consumers' routers widely deploy a feature like that, spammers will widely deploy botnet agents that use UPnP to turn it off.

[ Parent | Reply to This ]


It's a little worse than this[ Parent | Reply to This ] (none / 0) (#45)
by Anonymous User on Thu Nov 09, 2006 at 11:47:57 AM PDT

True, but it is more than just their typical sloppy code.

With the release of XP M$ added full support for the TCP/IP standards for raw sockets - full total support (for an Open Standard, no less!).  Sounds good, but the only part of the TCP/IP stadards that wasn't already in Win2K was the raw socket commands.  These are _only_ used - very sparingly - by Sys Admins/deveoplers for testing.  They are fully implemented on Unix/Linux, and while "nice to have" on Windows, they are not required, and there was no major push from customers for it.  Reason is they are very dangerous in that they permit remote execution.

Add to that the wilful/bizarre/short-sighted/pick-your-own-adjective decision by M$ to essentially force all but the most tech-savvy users of XP to run in Admin privilege mode all the time, and bingo, M$ made every run of the mill new PC as attractive and useful to bot herders as the most powerful Unix servers.  In fact often more attractive, as they are both less likely to be discovered, and more unused CPU cycles.

This scenario was predicted, and brought to M$ attention before release of XP in case it was an oversight, but they went ahead and released it anyway.  They have now finally put in restrictions to this access of raw sockets, but only for those systems running XP SP2 _AND_ who've installed a required later patch _AND_ who weren't infected before that, or who aren't infected while doing their initial installation and patch updates!  (See http://www.grc.com/dos/intro.htm for the entire history, including a posting from a bot herder of the time rejoicing about all the new targets.  This is also a good link to learn about bots if you have the time, Steve Gibson infiltrated the IRC bot networks at one time, and exposed many of their conversations.)

Why did M$ do it?  Who knows?  But little M$ does is without a reason.

[ Parent | Reply to This ]



Raw Sockets[ Parent | Reply to This ] (none / 0) (#47)
by Anonymous User on Thu Nov 09, 2006 at 03:42:03 PM PDT

Raw Sockets were removed at part of XP SP2. Gibson was right!

[ Parent | Reply to This ]


Easy[ Reply to This ] (none / 0) (#13)
by tcsbiz on Fri Nov 03, 2006 at 03:17:19 PM PDT

Do not allow anonymous posting...

-or-

Use a captcha on anonymous postings...

[ Reply to This ]



Captchas.[ Parent | Reply to This ] (none / 0) (#16)
by foxyshadis1 on Fri Nov 03, 2006 at 06:44:05 PM PDT

Captchas aren't magic, see this link: http://sam.zoy.org/pwntcha/

Notice that most of the "very good" captchas have the caveat "not always human-solvable", which is damning for a site like this. The ones that don't do make excellent references, but even more than that, the problem boils down to: what percentage of the current spammers are going to see my site as worth the trouble of OCR? Given that most of the current spam is random link spam, probably generated by bots trawling the web for submission forms. So the question is how much can you stop, without costing too much time or money.

For now, relying on botnets being stupid should be sufficient. They don't generate monthly reports, so sites that implement captchas may be entirely off their radar for months or more. That means even a constant word can cut marketedly into the spam! In fact, I bet the subject requirement already helps. They also don't run javascript, so very basic javascript-based captchas (where the server generates both an image and the javascript to solve it) will be sufficient to stop them cold for the time being. I got interested in it so I wrote a proof of concept or two, it isn't hard at all.

The trouble with captchas is, we've done so much research into the way the eyes (and ears) work, eventually spammers will be able to use stolen tools to model them near-perfectly. Maybe at that point we'll pop open a videoconference to make sure they're real. =p

[ Parent | Reply to This ]



No answer?[ Parent | Reply to This ] (none / 0) (#19)
by tcsbiz on Sat Nov 04, 2006 at 06:45:39 AM PDT

OK, so there is no real answer?

Just shut down all the web sites! To quote John Wayne (I think), "That'll fix their wagon!"

No websites, nowhere for 'bots to go. Why do we need 'em anyway? Nothin' but time wasters!

[ Parent | Reply to This ]



Magic[ Parent | Reply to This ] (none / 0) (#25)
by tcsbiz on Sat Nov 04, 2006 at 04:04:00 PM PDT

If we had magic we wouldn't have the problem.

[ Parent | Reply to This ]


WARNING, GOATSE ON REF'D PAGE PAGE[ Parent | Reply to This ] (none / 0) (#31)
by Anonymous User on Mon Nov 06, 2006 at 01:46:09 PM PDT

which, if you don't know, is an image that you do NOT want to see. Check out wikipedia if you must know; do NOT search using Google/GIS.

[ Parent | Reply to This ]


And as a user, I hate captchas[ Parent | Reply to This ] (none / 0) (#46)
by Reziac on Thu Nov 09, 2006 at 12:00:12 PM PDT

One of the big problems with captchas, is that the damned things don't always load. Reload the page or "R-Click/Load Image" to try to force it, and about half the time the login/register page will lose your info and you have to start over... only to discover that now you're seeing the captcha that went with the PREVIOUS page, so it doesn't match the CURRENT page -- so your "answer" is rejected.

A "hidden input fields" approach was discussed on slashdot a couple days ago, where an ordinary input box is simply flagged "never display" -- so if something is input there, you know a bot did it, not a human. This is transparent to human users, since the bot-trap input box is completely invisible. It occurs to me that while this approach is no doubt easy enough for a well-trained bot to defeat, it's also flexible and could be scripted to change randomly in whatever ways some creative type can think up.

All without annoying or perplexing legit users, and without requiring that the user load images or have javascript active -- so it works with ANY browser or device.

.

~REZ~
[ Parent | Reply to This ]



Eh[ Parent | Reply to This ] (none / 0) (#50)
by Anonymous User on Thu Nov 09, 2006 at 07:57:31 PM PDT

Bots will just ignore all input fields with "never display" set.

[ Parent | Reply to This ]


This is old -- very old -- news[ Reply to This ] (none / 0) (#20)
by Anonymous User on Sat Nov 04, 2006 at 07:05:41 AM PDT

1. It's nice that the mere newbies at Postini have finally caught on, but this problem was discussed in copious detail (in places like Spam-L, the Internet's most important list for issues) years ago. 2. Captchas? Captchas are only used by the stupid and those who hate blind people. Nobody of even minimal intelligence or compassion would even consider them. 3. The problem is that -- to a very large extent -- in 2006, "the spam problem" equates to "the Windows security problem". And the latter is not solved, nor is it likely to be solved -- EVER -- because it's clearly not in the financial self-interest of several large players to solve it. 4. Outbound port 25 blocking by ISPs would of course do a lot to control this -- in the short term. (Long-term is another story: there are many ways around such blocks.) However, the irresponsible, spam-supporting jerks running Comcast and Verizon and Charter and Roadrunner and Adelphia and Wanadoo and Versatel and and and...all refuse to do anything about the problem. 5. So for the moment, the best course of action appears to be to utilize passive OS fingerprinting and curtail the service provided to systems that are running Windows. See (for example) this clever hack: http://use.perl.org/~merlyn/journal/17094 which shows how to restrict the bandwidth available to Windows systems connecting as SMTP clients. I've used this for quite some time now, and have found that it works beautifully.

[ Reply to This ]


Captchas[ Parent | Reply to This ] (none / 0) (#23)
by Anonymous User on Sat Nov 04, 2006 at 02:20:56 PM PDT

"Captchas are only used by the stupid and those that hate blind people"

Early implementations of the concept have tended to be plagued with accessibility problems; but not some modern ones. (Check out blogspot comment posting. Note the wheelchair button next to the captcha? Try it.)

Nothing about captchas makes them inherently discriminatory--except against automation, and maybe people with a single-digit IQ. Such people usually spend all day trying to figure out how to get out of bed without ever getting near a computer anyway, I figure; they have much, much bigger problems than needing an assistant to help them post to a blog, anyway.

[ Parent | Reply to This ]



Hate the blind[ Parent | Reply to This ] (none / 0) (#26)
by tcsbiz on Sat Nov 04, 2006 at 04:16:48 PM PDT

2. If I only have two choices, I will choose hating the blind.

5. Cool. So, if I already hate blind people (see 2), I can now hate 95% of the internet's users. I am so ready I can already taste the hatred. I'm salivating just thinking about all those souls I can hate. Oh wait... they have no souls because they use Windows.

I'm not sure, though, how an internet business can survive by ignoring 95% of the entire market. But like true haters, we only like those who are like ourselves and don't use Windows, so the hate comes easy.

[ Parent | Reply to This ]



Touch of ego there?[ Parent | Reply to This ] (none / 0) (#34)
by foxyshadis1 on Mon Nov 06, 2006 at 11:00:03 PM PDT

I think I stopped taking this message seriously here: "1. It's nice that the mere newbies at Postini have finally caught on," given that Postini has been banging on this particular drum for quite a few years now. (See http://www.itbusinessedge.com/item/?ci=1950 for instance, which is just the first google hit.)

And let's get a list of ISPs that block port 25, either totally or until request:
AT&T
BellSouth
CableOne
Charter
Comcast ATTBI
Cox
EarthLink
Flashnet
MediaOne
MindSpring
MSN
NetZero
People PC
Sprynet
Sympatico.ca
Verio
Verizon

Funny that half the names you listed are there. Too bad eastern hosts have started making up for the void that cutting homegrown spam has caused, and then some.

[ Parent | Reply to This ]



No....[ Reply to This ] (none / 0) (#35)
by Anonymous User on Tue Nov 07, 2006 at 11:51:53 AM PDT

"The more people are made aware of the bots are doing to us, the more likely they will defend their computers against being taken over."

Well, no.  Most people, given more awareness, will simply sigh and wonder what they're supposed to do about a problem they have no idea how to solve.  And why should they solve this?  Why aren't computers sold with a simple and secure OS that doesn't require lots of "awareness" and fiddling to make it work, securely, like it should?

The real problem is that software producers are insulated from warranty and product liability laws.  They should be held accountable for their broken products just like any company.

"We reserve the right to remove any posts our users find offensive..."

Would that it were so!!!  How about in addition to reserving the right, you exercise that right?  Just take a look at some of the posts prior to this one.

[ Reply to This ]



Agreement with prior post about broken products[ Parent | Reply to This ] (none / 0) (#37)
by Anonymous User on Tue Nov 07, 2006 at 03:29:27 PM PDT

much like when the U-Shaped bicycle locks were shown to be easily foiled using a Bic pen. That company recalled the warranty period locks for replacement. Microsoft should do the same with their buggy software to add a free anti-virus capability. I've actually heard of folks trashing their computers because the pc ran so slow due to viruses and spam bots but the owners were clueless of even free anti-virus and anti-spam solutions.

[ Parent | Reply to This ]


What People Want[ Parent | Reply to This ] (none / 0) (#39)
by partan on Wed Nov 08, 2006 at 07:51:02 AM PDT

I completely agree.  Most people want to buy a computer (or any product for that matter) that just works.  It's crazy that all this is going on.

Attempting to control or protect computers on the user side, or front end, doesn't work because not everyone knows how or is aware that it can be helped.  I'm not smart enough to know how to solve the the problem on the back end, the bad people who produce the spam, viruses, etc., but it seems that would be a better solution.  Extremely inteligent hard working people built the Internet.  I'm sure that there are some people that can come up with a way to make it more secure.

I know it's probably a pipe dream, but expecting each individual to do the right thing and know how to do the right thing doesn't seem to be working.

[ Parent | Reply to This ]



Invasion of the TCG shills?[ Parent | Reply to This ] (none / 0) (#49)
by Anonymous User on Thu Nov 09, 2006 at 07:54:50 PM PDT

"Attempting to control or protect computers on the user side, or front end, doesn't work..."

You may think that. It may even be true. But there is no palatable alternative.

The only other choice would be controlling or protecting computers centrally. In other words, that "trusted computing" BS.

That's selling all computer owners into serfdom. What they used to personally own, they would now be effectively renting -- no user modification permitted. And, no doubt, they'd soon be renting their hardware and their software's functionality in all the other ways too, and paying through the nose for the privilege.

Today, you can buy one copy of Word (or get one free copy of OpenOffice Writer!) and churn out as many word processing documents as you wish, for decades if you so desire. (Of course, if you don't upgrade you'll start receiving office documents that don't display properly, but that's a separate issue.)

Tomorrow, if people like you and Bill G. have their way, you will only be able to rent it (and the thing it runs on -- I hesitate to call it a "computer"; let's call it a "terminal"). You'll have to pay to renew every month to do any kind of word processing. Or maybe pay for every page of output, perhaps even every page you write even counting stuff you later delete or revise.

Of course, you also won't be able to play some music on some devices, do some things with some data at all (such as incorporate particular images into art of your own design, say, or successfully print a Word document that badmouths Microsoft, or something), and there'll be other arbitrary restrictions that you can't work around, that make no logical sense as necessary for the functioning of the software, and that just so happen to be limitations desirable from the point of view of various government agencies, large corporations, and special interests.

All of a sudden, you'll be paying a whole lot more for tools that do a whole lot less.

That isn't just.

Just say no!


[ Parent | Reply to This ]



More SMTP filtering at the ISP.[ Reply to This ] (none / 0) (#40)
by Anonymous User on Wed Nov 08, 2006 at 02:28:40 PM PDT

Companies & ISPs need to do what I do: restrict SMTP traffic (port 25) to specific hosts. If every end-node is permitted to send SMTP anywhere in the world, you make it easy for the botnets to become spambotnets. If you're forcing end-nodes to use a local SMTP host--one that does IP validation, etc, too, not just some stupid open relay--then you eliminate the problem of spambots. It doesn't fix the general 'bot problem, but it does address the spambot portion.

[ Reply to This ]


This is not a technical problem.[ Reply to This ] (none / 0) (#41)
by Hasai on Thu Nov 09, 2006 at 08:02:48 AM PDT

When your favorite tool is a hammer, all problems look like nails.

Folks, I know it's terribly tempting to view this as a technical problem, but in doing so we limit ourselves to passive defense. In the long run this is just as effective as employing no countermeasures to thievery other than a lock on the front door: the thieves just go get a bigger crowbar.

The spam problem will continue to worsen, because it is not a technical problem. It is a criminal problem, with real live people behind the entire mess. These people are enjoying substantial profit for very little personal risk, and as long as they continue to do so, spamming will continue to increase with little regard to the passive defenses we continue to erect.

People, the only thing that will permanently dent this garbage is an active defense, and this means bringing in the cops, dragging them in kicking and screaming if we have to. The risk-to-reward ratio for these activities has to be slewed over to the other end of the spectrum, and as we live in a 'civilized' society, the only effective way to do this is via an officer's truncheon.

[ Reply to This ]



Use both[ Parent | Reply to This ] (none / 0) (#42)
by tcsbiz on Thu Nov 09, 2006 at 08:20:02 AM PDT

It is important to legally punish the offenders. But, even with an effective police force and enforcement, I would still lock my doors, use outdoor lighting, and take other measures.

Tom.


[ Parent | Reply to This ]



A-men and don't feed them either.[ Parent | Reply to This ] (none / 0) (#48)
by Anonymous User on Thu Nov 09, 2006 at 07:03:24 PM PDT

Fully agree that time in the joint with some broom handles for those found and convicted of violating a REAL anti-spam law will indeed serve the rest of us well.

However, I must point out to some of us here in the discussion that the VAST number of users are not capable of cleanly rebooting let alone understanding this issue IMHO. If everyone was forced to understand these "toasters" as would be required in order to self service their bot-ready boxes, there would certainly be fewer doctors, lawyers and Indian chiefs.

This IT profession requires 28 hour days amd 9 day weeks. Whom are we kidding? And now, you actually want M$ to give us a free cure for their oversights and laziness and greed? Oh, did I forget to mention Automatic Update - Over My Dead Body. EU has the right idea AFAIAC. Burn them (M$ and spammers) and burn them bad. Clean the product up, burn your monopoly down, or find another market to sell your stuff too. Woo Hoo!

[ Parent | Reply to This ]



Here's something for Ed[ Reply to This ] (none / 0) (#52)
by Anonymous User on Sat Nov 11, 2006 at 09:31:10 AM PDT

http://www.squarefree.com/securitytips/web-developers.html

"* Make sure form submissions use your own forms by including a hidden field that is an MD5 hash of the login cookie and a secret on the server. Then only accept the form if the hidden field is correct."

This will work for non-anonymous postings (and most of the linkspam bots seem to register themselves) except where they actually use the real form, if they ever even do.

The "form key" hidden field you already use to prevent duplicate posts can be used in lieu of the login cookie to cover anonymous posts (or both).

[ Reply to This ]



More...[ Parent | Reply to This ] (none / 0) (#53)
by Anonymous User on Sat Nov 11, 2006 at 09:33:07 AM PDT

And here's another from the same source:

"* Optional added paranoia: Add a timestamp as a hidden field and include it in the hash. Make the form expire if the timestamp is too old. Give users a way to submit the form again when the form expires, such as by returning the form pre-filled with the data they entered last time but with a fresh hash."

This can be done here via that "form key invalid" page we all hate to get. Embed the timestamp in the "form key"; too-old or from-the-future ones are simply treated as invalid. Use the timestamp in the hash for the other, currently nonexistent header field suggested in the parent.

[ Parent | Reply to This ]



Another sort of web traffic surge problem?[ Reply to This ] (none / 0) (#54)
by Anonymous User on Sat Nov 11, 2006 at 05:02:04 PM PDT

Some research on sites getting DoS-level traffic surges reveals that spiders are sometimes the cause. Including badly behaved "rapid-fire" ones that don't slowly crawl the site but retrieve everything as fast as automation makes possible.

Even slower crawlers add up when there's apparently zillions of them proliferating, from sources ranging from fledgling Google wannabes to dubious operations like "copyright compliance monitors", "school test plagiarism detection services", and schemes to clone large numbers of sites and substitute different ads, which generate revenue for the cloners instead of the original author. (These are generally termed "scrapers", and divert some human bandwidth resource usage as well as some ad revenue. But they add bot bandwidth resource usage...)

The "robots.txt" protocol is ignored or misused by the more nefarious agents and even the benevolent ones can pile up in sheer numbers.

This post at a blog got me thinking:

http://tinyurl.com/yxm3sq (Warning: rated PG-13 for language)

Pardon his French, but the mention of his site's search (funny, I didn't actually see one and used Google to trawl it for more info) leads to an idea.

Maybe one thing Web 2.0 needs is a standardized interface for site search to be driven by automation? The easiest way to do that is to make an XML schema for search results, and make it standard for http://www.site.com/search.cgi?q=query with a standard query format to a) work and b) output the results in a machine-readable format. Normal browsers would render this as a hit parade, perhaps formatted using the style sheet of the site's choice (supplied in the XML-formatted response) so it can have a consistent "look and feel" to the rest of the site. Polite bots would retrieve either a targeted query or a null one (which generates the full site map), and search engines would no longer even need to bother much of the time. Consider a search engine that as a condition of using it expected users to run a certain browser extension, which would be free of nefarious behavior (e.g. spyware). This would periodically send them a statistic of what links to what, aggregated and lacking anything personalized. Queries at the engine use the resulting statistics to prioritize sites, and then pass the query to those sites' internal search engines to generate individual hits to aggregate. Hits whose relevance scores might even be, as a matter of protocol, comparable across sites.

Better yet, sites could provide an interface to retrieve their referrer logs to generate the statistics. This would enable PageRank type algorithms for who's-most-linked-to which could guide aggregation of local site searches (hit score = originating site score times site's own engine's score for the hit?) and minimize the amount of automated traffic from legitimate search activities.

More dubious search activities, some research activities, and referrer spam would need dealing with of course. Bot trapping could be more aggressive if legitimate indexing didn't tend to use traditional crawlers anymore, though. Referrer spam would actually boost the spammed site's search rankings, not the spammer's, by increasing the apparent number and diversity of inbound links to it. Research activities tend to be low-volume, so no problem.

As for robot exclusion, when you can more aggressively ban spiders and regular search uses your site's built-in engine as its bottom tier, you can exclude whatever you damn well please by controlling what the built-in engine indexes and what it ignores. Far more flexibly than the whole-directories approach of robots.txt exclusion. (User-agent discrimination is bad, mmkay? and can still be done in various ways anyway. Best to just 30-minute blanket-block any IP that hammers the site with too many requests in a few seconds, bot or not. And don't have too slow a server, as humans will hammer the "reload" button or a link to your site if simply clicking it once doesn't appear to work. Then get fed up and browse elsewhere, especially if they start getting 403 errors.)

[ Reply to This ]



Greylisting[ Parent | Reply to This ] (none / 0) (#55)
by Anonymous User on Sun Nov 12, 2006 at 11:13:48 PM PDT

I'm a sysadmin for a small company, and also was overwhelmed by the deluge of spam. We've taken and implemented steps to reduce the amount of spam. Greylisting works quite well - and needs no specialized hard/software. Basically it works like this : The originating email server sends out an email, which is received by the recipient. The recipient then takes that specific email's particulars (originating IP) and put it in a "blocked" queue. Because the originating server never received an acknow