The New Gmail Spam Filter

The Gmail new spam filter is really Google Friend Connect, disguised as a friends list the levels of permission now available in Gmail is a very effective spam filter.

This is an excerpt from my social marketing members only site. This is the kind of thinking and leading edge use of GFC you will find there.

This one should really blow your socks off. Google Friend Connect and the Google friends list in Gmail is not a friends list. It is a spam filter. Stay with me here this is really good stuff…

There are four levels of spam filtering in Gmail.

  • Sites we have joined thru the Google Friend Connect widget.
  •  

  • Sites you have not joined.

  • Gmail users not listed in your contacts.

  • Gmail users who are listed in contacts and friends lists in Gmail.

  • Those that are listed in Chat.

  • Those listed in Google Latitude.

Thinking about Google the other day and the fact that I get so little spam that this just rolled right in. Many of you have followed me for a while now and you remember back to when a mistaken Trend Micro false positive destroyed my business.

What I never explained is that my Google friends list as well as my social contacts in Facebook are what saved me. The only reason I continued to deliver email to anyone was thru my Gmail friends list. Because I was a very early adopter of Google Friend Connect and was out to build a list in Gmail early on, I had many mutual friends in the eyes of Google.

When you become mutual friends in any of the Google applications: GFC widgets, Gchat, Google Lattitude, you are automatically added, your Gmail address that is, to each other’s Gmail contacts list.

Now as a social marketer, I think that is cool, however from an email standpoint, getting your email to the Inbox is how you make money. What easier way is there than getting you visitors to friend you thru a GFC widget?

Then Google does you work for you! There is also no whitelisting process for email delivery in Gmail and no public FBL (feed back loop) for receiving spam button click complaints. These are the basic necessities we use as emailers to keep getting our email delivered.

Google has replaced this process by using levels of trust. I am just theorizing here so I cannot say this is fact and we will never know if it is.

Google levels of trust

Level of trust #1.Let’s say you have joined my site thru the GFC widget on your right. Then Google would definitely deliver email from that site to your Gmail inbox. Since it was YOU who established the relationship.

Level of trust #2. Let’s say that you and I are mutual friends made through GFC and now have each others address in our Gmail contacts. I would say our emails to each other would definitely get delivered.

Level of trust #3. Let’s say that I have sent you a chat invitation and you have accepted and we now both show in each others chat widget. I would say this is an even higher indicator that our emails to each other should go directly to the Inbox.

Level of trust #4. Let’s say you have sent me a Google Latitude request and you and I now share our locations at what ever level in Google Latitude. I would hazard to say that our emails are DEFINITELY going to land in each others Inbox at this point.

Can you see how Google is using what normally is a social site to better deliver what we want all want? I don’t know about you but I want NO SPAM and I want my EMAIL NOW, not after searching the spam folder for an hour.

This only happens when you use a Gmail address. You only get the advantage of these social features when you use a Gmail address to log into Google applications like Google Lattitude. So go get one. If you do not understand how to set up Google Friend Connect properly you can get my FREE Google Friend Connect Tutorial here and watch my Google Friend Connect YouTube Channel here.

Gmail Breaks AWeber Tracking Links

AWeber links are breaking in Gmail due to an unknown line break

The reason I caught this was one of my subscribers replied to my email on an unrealted issue after I sent an email to my Google Friend Connect list with a new update on what Google is really doing.

UPDATE!

Just got off the phone with an AWeber admin and we could not replicate the broken URL from emails from my AWeber list to my Gmail address.

AWeber assured me that they go to extreme lengths to make sure the tracking links never get truncated. We believe that it was a keystroke on the part of my Gmail subscriber. We shall see and we will be monitoring this closely.

If any of you can replicate the broken string from your accounts then please let me know and AWeber as well. I will publish any screen shots you care to send and link to any of your own blog posts.

Tom Kulzer just emailed me and assured me that this was a line break introduced by the sender. For the moment we cannot replicate the string truncation.

More tomorrow and I will keep you up to date. This is one time I am glad to very possibly be wrong. – Chris Lang

I checked it at all possible resolutions down to 800 X 600 and it breaks in the same place.

I am sure it is due to a querystring length because no matter the screen resolution the AWeber click tracking link breaks in the same place ever time.

This screen shot is taken at 1280 X 1024 and is full size as it would be in my browser. You can click the image to see the full size screen shot of the Gmail email body center column.



Here it is at 800 pixels wide and it breaks in the same place.



I am sending Tom Kulzer a link to this today and I will publish his response to me if he allows me to.

If AWeber responds on their own site I will link to it and send you all an email to the fix (hopefully).

For now I am not going to be using the redirect links and simply linking without tracking to my landing pages.

Facebook Blocked as Phishing Site in Firefox and Chrome

Facebook listed as a phishing site. Why?

The following domain is now listed as a phishing site by Google chrome and Firefox. http://fbcdn.net

Facebook blocked as phishing

UPDATE:Just in, TechCrunch just reported that Safari is now listing Facebook as a suspicious site too. You can read their article in the trackback in comments where they linked to this artilce. (Cheers! Robin Wauters, thanks for the link…)

This is actually a Facebook domain. By adding this domain to your list of trusted sites in Internet Explorer you will no longer have problems with Facebook. Currently Internet Explorer is not listing it as a phishing site in it’s database. Could this be because Microsoft has $260,000,000 invested in Facebook?

Basically the only way to use Facebook is to use Internet Explorer because they do not seem to be blocking it right now.

I am trying to get to the bottom of the Phishing RBL now, I am already taking this up with Trend Micro to make sure this does not spread. Trend Micro blocking my site made me quite an authority on Phishing filters and URL RBLs.

Currently the phishing filter used by Firefox does not list fbdcn.net as a phishing site and neither does Internet Explorer anti Phishing filter. Don’t know where this blocking is coming from but I want to know as someone who cannot wait to implement Facebook Connect.

After a Google search here is why Facebook runs JavaScript on another domain:

There are a few reasons; one of the posts added the additional domain lookup, but that only needs to happen once. What happens instead, is that a browser has a built in limit of 2 connections per domain. This means that if you have a lot of external content (CSS style sheets, Javascript, images, etc) you’re limited to load two at any given time. By adding an extra domain, you double that amount of items you can be downloading concurrently. There were some various experiments done, and it was found that 2-3 was generally the best you could squeeze out.

The link to the information about CDNs was provided — companies like Akamai have servers placed strategically around the world and use some DNS voodoo to have localized requests directed to their servers nearest to the request. This means that you can download the data faster.

Another reason is about bandwidth. Facebook (and all sorts of other sites where you have to login) track all sorts of information about you in cookies. Every request you make to a site, all of those cookies get passed around. This can add up very quickly. By having a domain that isn’t just a subdomain (ie, x.facebook.com) like fbcdn.net, each request isn’t burdened with the additional cookies and thus minimizes the bandwidth required on the request (you transmit less, so it’s faster; they receive less so it’s faster and cheaper [since they pay for all that in and out bandwidth])

I’m sure there could be even more reasons, but those are the main ones AFAIK.

This puts a real damper of the launch of Facebook Connect.

I will keep updating this post as news breaks on this. Either the Facebook domain fbcdn.net was hacked or this is something odd, I have not seen this occur before.

UPDATE:Facebook says the blocking has been cleared in the Facebook Forum. This is the same place I tried to warn Facebook that this was a possibility. The other developers said:

“This is some pretty tinfoil hat sounding stuff.”

I happen to like the way my monitor reflects off my tin foil hat, what’s it to ya?

Microsoft IE 7 Anti Phishing Filter – Is Internet Explorer Blocking Your Site?

Microsoft Anti Phishing Filter in Internet Explorer 7 Can Display False Positive Warnings. Here is how you can tell if you are being blocked by a Internet Explorer Anti Phishing False Positive Warning

In a further effort to help legitimate businesses, here is how you can tell you have a Microsoft Internet Explorer 7 or 8 blocking your site issue.

I am going to do a series of articles on exact terms that someone that was in my position with a anti virus false positive phishing entry so you can hopefully find this article and begin to clear your name.

This is about the only way you will be able to tell that an anti virus is blocking your site. The real problem occurs when your listing in the AV database spreads to other major networks.

Google Analytics Saved my Butt

You can tell that Internet Explorer is blocking your site if you have goal conversions set up in Google analytics. Check to see if you are selling any products to visitors using Internet Explorer 7.

If you have some type of block in a database it will be possibly consumed by the IE 7 anti phishing filter database and you will have almost 0 sales occurring in Internet Explorer. At the same time you will have a way higher sales success rate in Firefox.

That is the only thing that saved me was finding that FF had a 14% sales success rate and IE had a 0.26% rate.

This is not guessing on my part. This is the findings of the Trend Micro investigation.

Sudden drops in newsletter subscribe rates, abandoned payments, email click thru rates and sales in general are all indicative of this kind of blocking.

The only reason I got a few emails delivered was that I was highly whitelisted in Gmail due to my Google social network friends list.

Please tell anyone you know to look into this so that this does not happen to them. If I can help anyone do not hesitate to call me.

Still alive and kicking,
– Chris Lang

Trend Micro blocking my site, my PayPal forms, my URLs, Emails & Trend Micro Spam Filters

Due to Trend Micro blocking my site, my PalPal redirect forms, my emails in Trend Micro spam filters and my URLs in emails and web traffic requests I can now advise you on how not to make this mistake. Don’t let this happen to you.

Trend Micro will block your site in the browser, block your web traffic at the server level, your URLs in any emails and send your emails to the Trend Micro spam folder if you make the mistakes that I did. It was not Trend Micro’s fault, my payment software or anything I did. Trend Micro blocked me because I was at the wrong place at the wrong time.

NOTE: This is NEW information that no one else has. Read this thourghly!

When Trend Micro blocked my site

In September Jennifer Horowitz sent a recommendation of my product, Social Marketing book to her email list. We had a very poor response rate and and I began to think her email was not getting past spam filters.

Then one of her subscribers sent me this screen shot of Trend Micro wrongly blocking my site.

Click the image to see it full size, this will shock you.



image of trend micro blocking my site in the browser

I was stupid enough to ignore this and simply believed it was a very minor issue due to the redirect to PayPal in my payment software. Big mistake. More like a $20,000 mistake.

Trend Micro is currently the most purchased Window software there is as reported by Cnet just last week.

As time went by it got WAY worse. More and more payment attempts were being abandoned and comments on my blog and email click thru rates plummeted. Two three week periods passed without a sale and I began to really worry. At the same time my mind really went to work on this.

Meanwhile I was using all the skills I have to track down what I believed was a new spam filter blocking my emails. I enlisted the aid of even more experienced email pros to help me and they could not find the source. Neither could my GoDaddy host or AWeber, my email list provider. No bounce messages, no FBL reports, no nothing.

That was when I began to think it was a client side spam filter and something new to boot.

Then about a week ago I remembered the screen shot, in the end that Trend Micro blocking screen shot was my savoir.

Trend Micro says keywebdata.com and Chris Lang are innocent of any phishing, undesirable, dangerous or malicious activity or wrong doing.

Chris,

Just saw you called sorry I am in a meeting on various things at the moment. Either way I figured I would email you as I saw your email this morning when I logged on. I quickly read over the questions but will go over in more detail later and answer those that I can for you.

As for our analyses of your URLs we found no malicious activity so it looks to just be a False Positive.

The two entries that we found for being blocked where from the web traffic on our side the weird part is we only see the 2 and without the logs from an actual user we can’t determine what exactly happened.

For your blog you can put that it was a false positive by Trend Micro and that we have verified that no malicious activity was found.

Again I will go through each question a little later and reply to those that I can.


This is the email I received back today.

Below is the synopsis of what the next phone conversation produced. I want to stress that these are facts as told to me by my Trend Micro contact in charge of the team that investigated the keywebdata.com blocking.

Why did Trend Micro block my site and how can you avoid it

First off the best indicator of something going on are subscribers clicking the spam button in web email. Why are they doing that? Because they just saw a big freakin huge banner like the screen shot above. What else are they going to do when their anti virus software just screamed and yelled at them to never go to your site again?

After 8 straight days of working 20 hours a day, today, October 27th, I have gotten to the bottom of why Pc Cillin warnings were displayed and Trend Micro Internet Security and Trend Micro anti virus blocked my site.

I just got off the phone with my Trend Micro contact.

Keywebdata.com was flagged in the browser, in Internet Explorer 6 or 7 and blocking began.

At that point my site and payment forms were reviewed by Trend Micro.

Due to a unsecured form, submitting to my server, then redirecting to a PayPal CGI bin URL my site was deemed indicative of a Phishing site and blocking occurred.

This did create global blocking of my domain at both the server level and the end users of all Trend Micro Internet Security products.

Any email with my URL in it was blocked by TM and possibly Yahoo, Gmail and Microsoft, both at the browser and email server level.

Also any web traffic passing through a data center with Trend Micro filtering software installed would have blocked the traffic right there too. This may be why so many of my emails never arrived.

Any http request from a browser resulted in the screen shot image being displayed.

Any traffic crossing the Trend Micro server level software resulted in the request not being passed and the request to my server was not completed. In other words if a link to my site was clicked it would not result in the request either going thru to my server and no HTML would be displayed.

No footprint of this blocking is visible in server logs because the http request (http link click) was denied at either the home user end client or at the server leval. So, even if you do pull your server logs there is not footprint to show that you are serving lots of pages and getting no browser page views.

It would do no good to pull server logs because no request for a page would ever make it to the server.

Trend Micro is looking into if they will release numbers as to the end user blocking numbers for keywebdata.com

Basically I am going to have to pick myself up, dust myself off, learn from this and move the heck on.

TM does agree and is willing to suggest that using any HTML form that is submitting to any payment provider without using HTTPS protocols will draw the ire of anti phishing software. This also means Microsoft Internet Explorer anti phishing filters.

What can you do if Trend Micro PC Cillin blocks your site?

Step #1

First of all, I now have extreme knowledge of this and can check your site to see if it is blocked. To learn how to get your site unblocked and protect your new sites from being blocked, I have detailed instructions at this link.

If you are being blocked I can navigate thru the process of getting and investigation started and if you are innocent, get the blocking removed. Do not email me to death just to see if the site is blocked. If you experiencing problems I will help you.

Step #2

First of all, stop sending any emails with your domain URLs in it, don’t send any email to your email lists and pull access to any payment forms.

If someone tries to buy and gets a phishing warning you have just lost any possibility of a future sale.

If your list gets an email from you and anyone has seen the phishing warning on a previous visit you can bet your last dollar they are going to mark your email as spam and any others they may get.

Lost sales now are nothing compared to losing most of your list and further email blocking. I did not catch this early enough on and lost a list with 2500 hard earned subscribers on it.

Currently there is no feedback loop from Trend Micro and my contacts tell me they are considering adding this so that we are directly contacted should a new script on our site suddenly trip a Trend Micro red flag.

You will also want to keep your hosting provider in the loop so that there are no issues with your domain or site. Call them and create a help desk case number. Then email the abuse@YOUR HOSTING address and document every step of the way using the case number. It can come in handy if anyone sends false accusations their way.

Save every email, document everything and keep records of everything that occurs.

Step #3

If you are using payment forms generated by IPN scripts that submit to PayPal for payment then get them changed to using HTTPS protocols in the form tag. Don’t worry about forms created by PayPal, they are sending to PayPal URLs. The problem lies in forms that submit to your domain and then the server redirects to PayPal. That is what started my tale of woe.

You will need to add a SSL certificate to your site and your host can take care of this for you. Just give them a call.

Step #4

Then we will need to get Microsoft, Yahoo and Gmail to remove any blocking as well.

I can get this underway for you as well.

Step #5

Once you get this cleared up and your site free of any negative indicators you can resume mailing to your list and you should be in good shape. I am.

I do want to thank Paul Myers who kept me calm and supplied great ideas to get through out this. Above all he told me to handle myself professionally and calmly.

I want to thank John McGowan who dealt with the yelling and screaming portion of this little odyssey.

Lyris Add DKIM, About Time

DKIM authentication now supported by Lyris HQ online marketing suite, one step closer to wide adoption

Lyris UK www.lyris.co.uk , today announced the availability of DomainKeys Identified Mail DKIM technology within the Lyris HQ online marketing suite. By making DKIM authentication more widely accessible to organizations of any size, Lyris is helping marketers build and sustain their online reputations with Internet Service Providers ISPs. Now if shared hosting would just provide us with the abiliaty maybe my shared SMTP server would deliver something besides spam folder fodder.

I Just Marked My Own Email As Spam and Why

I was just in Gmail and went to click the delete button on one of my own emails that arrives there as a test.

Since the delete button is only a quarter inch from the spam button, I, in a hurry I clicked the wrong one. I had just marked my own message as spam.

I wonder how many times that happens by mistake to our emails. What do you think?

Virginia Court Declares Anti-Spam Law Unconstitutional

The court unanimously agreed with Jeremy Jaynes’ argument that the law violates free-speech and overturns spammer conviction, setting the stage for a Supreme Court spam laws showdown!

RICHMOND, Va. (AP) — The Virginia Supreme Court declared the state’s anti-spam law unconstitutional Friday and reversed the conviction of a man once considered one of the world’s most prolific spammers.

The court unanimously agreed with Jeremy Jaynes’ argument that the law violates the free-speech protections of the First Amendment because it does not just restrict commercial e-mails — it restricts other unsolicited messages as well. Most other states also have anti-spam laws, and there is a federal CAN-SPAM Act as well, but those laws apply only to commercial e-mail pitches.

The Virginia law ”is unconstitutionally overbroad on its face because it prohibits the anonymous transmission of all unsolicited bulk e-mails, including those containing political, religious or other speech protected by the First Amendment to the U.S. Constitution,” Justice G. Steven Agee wrote.

Agee wrote that ”were the Federalist Papers just being published today via e-mail, that transmission by Publius would violate the statute.” Publius was the pseudonym used by Alexander Hamilton, James Madison and John Jay in essays urging ratification of the Constitution.

”In my view, the case was never about Jeremy Jaynes — it was about the First Amendment,” said Jaynes’ attorney, Thomas M. Wolf. ”The argument was never that there’s a constitutional right to send commercial spam. It was that the government cannot criminalize the sending of noncommercial e-mail for political and religious purposes, and that is what this statute did.”

Lawyers for the state had argued that the First Amendment doesn’t apply because the Virginia law bars trespassing on privately owned e-mail servers through phony e-mail routing and transmission information. The court rejected that characterization of the law.

Attorney General Bob McDonnell said he was ”deeply disappointed” and vowed to take the issue to the U.S. Supreme Court.

”Jeremy Jaynes used the private property of Internet service providers to defraud individuals worldwide,” McDonnell said. ”This was not a matter of free speech, it was fraud. Virginia acted appropriately to use this new law to put an end to this criminal behavior.”

John Levine, a board member of the Coalition Against Unsolicited Commercial E-mail and one of the state’s expert witnesses in the Jaynes case, said he too was disappointed, but added that the ruling won’t have broad repercussions because Virginia is the only state that prohibits noncommercial spam.

”I don’t see it as a fatal setback for anti-spam law,” Levine said.

In 2004, Jaynes became the first person in the country to be convicted of a felony for sending unsolicited bulk e-mail. Authorities claimed Jaynes sent up to 10 million e-mails a day from his home in Raleigh, N.C. He was sentenced to nine years but is currently serving time in federal prison for an unrelated securities fraud conviction unrelated to the Virginia case, Wolf said.

Jaynes was charged in the spam case in Virginia because the e-mails went through an AOL server there.

The Virginia Supreme Court last February affirmed Jaynes’ conviction on several grounds but later agreed, without explanation, to reconsider the First Amendment issue. Jaynes was allowed to argue that the law unconstitutionally infringed on political and religious speech even though all his spam was commercial.

Wolf said sending commercial spam is still illegal in Virginia under the federal CAN-SPAM Act. However, he said the federal law does not apply to Jaynes because it was adopted after he sent the e-mails that were the basis for the state charges.

How To Increase Email Delivery From Return Path

New Advice From Return Path and PDF

Return Path’s Q2 2008 Reputation Benchmark Report (pdf) found e-mails sent from “legitimate” e-mail servers averaged a delivery rate of 56 percent. 20 percent were rejected; 8 percent filtered out of the inbox. The rest — 16 percent — were bounces.

So nearly half of the time, e-mail marketers’ messages don’t get through. But there are ways to increase deliverability, insists George Bilbrey, Return Path’s general manager of delivery assurance. Here are five:

Do I Need to Include My Autoresponders SPF Record in My DNS?

Email Delivery: Should I Add My Autoresponder’s SPF Records to My Domain?

I recently ran across a post that claimed that by adding the SPF records of the writers autoresponder (in this case AWeber) that he expected to get past being blocked by a Canadian ISP. This is complete crap because any authentication technology associates the email in question with the sending domain and IP address not the email from address or the return email address.

I even contacted AWeber’s CEO Tom Kulzer with this scenario just to be absolutely sure and here is the email excerpt.

Chris Lang wrote:

Let’s say that an ISP receives my email from AWeber with my from address and reply address in the header. Do they look at my SPF record to see if I have a SPF DNS entry associated with your (my autoresponder) email servers?

Tom Kulzer said

They look at aweber.com SPF records.

Chris:

Also is all email sent from AWeber under the address keywebdata (at) aweber.com sent from the same IP address everytime?

Tom Kulzer:

It’s not sent from the same single IP, but load balanced across the same range of IP addresses. Those ranges can all be found in our SPF record directly or in our FAQ on the website.

Chris:

In other words does it matter if authentication records associate my domain and from address with yours?

Tom:

Does sending from the same single IP matter? No.

Does sending from the same group of IP’s matter that have an excellent reputation and reliable volume of mail built over a long period of time matter? Absolutely, yes.

Chris:

Also is there any data to support a higher delivery rate due to the use of SPF, Sender Id and DKIM?

Tom:

Not that I’ve seen which clearly shows this, but general industry knowledge of how various ISP’s build reputations
of senders and make delivery choices tells me it does help support higher delivery rates.

Chris:

Tom you have been a wealth of information on email delivery to us all many times, I just want to thank you again for taking time away from your business to set us straight.