Google’s belligerent Panda: how Google’s algorithm updates hurt small sites and give others an unfair advantage

Google’s last Panda 3.8 update which happened in late June 2012 took away two thirds of my traffic, reducing my readership to the same level it was more than 18 months ago, and making the site essentially unsustainable, a personal cash drain that cannot be maintained in the long run.

Mind you, it is not that I disagree with their policy; however, the opaque way they go about it, treating good sites like spammers and changing the entire environment in which we have operated for years without notice, is damaging and unprofessional.

But, more to the point, this article argues that these unexpected, sudden changes in the environment have this unintended consequence: that they greatly harm small sites in particular, giving an unfair advantage to larger sites with more resources.

Here’s a quick table of contents:

1. Some history:this site and it’s SEO ‘Strategy’ prior to Panda 3.8
2. Damage control:our initial reaction to the fall from Google’s grace
3. Figuring it out:identifying the reason(s) for what happened
4. My relationship with Adsense
5. Recommendations:how they should have gone about this
6. Why this hurts small sites more than big ones: and is this really what Google wants?

Note on the illustration above: it was not easy to find a menacing-looking Panda, I’ll tell you that 😉

---

1. Some history: this site and it’s SEO ‘Strategy’ prior to Panda 3.8

This site has been around since 2006. It started as a joke or at best an experiment, but somehow I kept it up. After the first year, in which I nearly quit about a dozen times, I realized that I had a good thing going, that there were people actually logging into the site first thing in the morning, that big sites like Lifehacker and Download Squad were linking to my small blog. I kept it up, writing posts during my lunch hour at work and deep into the night. Finally, in January of 2011, I started doing it full time, and shortly thereafter took on another blogger, Benjamin T, and a research assistant, Alaa K, whom you might have occasionally encountered in these pages. The one overriding vision was to write ‘quality’ articles that were entertaining and visual, and to post at least once a day, 10 times a week. The formula was working: between Jan 2011 Jun 2012 daily visitors almost doubled. My personal goal was to have a site to be proud of, but also to eventually make as much money doing this blog as I was making before at my corporate job. I liked telling people I just met that I was a ‘professional blogger’.

Then, in the last week of Jun and the first week of July 2012, my traffic suddenly crashed. I didn’t quite understand what happened at first. June had started out to be a spectacular month, albeit one of the weirdest ever. In retrospect I now believe that the algorithm updates which Google had implemented in the beginning of June had initially benefited my site greatly. (Moreover, one developer was apparently buying ads to send traffic to my review of their software, which was very strange). Then suddenly, the rug was pulled from underneath us. The last week of June and beginning of July was abysmal, and I truly thought to myself that maybe it was a combination of the 4th of July holiday and the wildfires across the Midwest that left large swaths of population without electricity that was responsible. It wasn’t. What actually happened was that my site had fell out of favor with Google, and I had no clue why.

2. Damage control: our initial reaction to the fall from Google’s grace

I am not an SEO, but I thought I knew a thing or two about the subject. (Note: SEO= ‘Search Engine Optimization’, but it also denotes someone whose profession is to understand and manipulate how sites get ranked higher on search engines). Our SEO strategy was simply to write ‘good’ original articles and posts that people were likely to link into. I thought I had avoided the typical pitfalls that hurt sites with Google, such as selling links or writing paid reviews or cultivating spammy inbound links. I thought that daily updates were also a good thing for search engine ranking, which I now think may not actually be a very important factor, and I knew that the site is generally slow, which Google considers a negative factor when it considers rankings.

I wondered if the reason may have been that some of my highest trafficked articles (so called ‘evergreen’ posts in bloggingspeak) had not been updated in over one or two years. Were there newer sites that came along with better information on the same subjects? But I also knew that my site lived off the ‘long tail’ of random or rare searches, and the loss in traffic seemed global, across the board. I googled some terms and found that where I was listed anywhere between 3rd and 7th on some highly trafficked terms I was now in the bottom of the second page. Google didn’t like my site anymore and I didn’t know why. I was at such a loss to explain it that my only thought was that I needed to make my site faster. I use cloud hosting, a caching plugin, and a CDN service, and blamed the slowness of my site on something being wrong with the theme that I was using. So I set out to replace it, and after more of a week of nonstop work had a new WordPress theme that your are looking at right now (more on that here). I also set about changing my permalinks, getting rid of the year/month/day/postname URLs in favor or postnames only. In short, I did everything that came to mind, even though I suspected that none of it was the ‘real’ reason.

3. Figuring it out: identifying the reason(s) for what happened

After doing some research, I think I finally found it out the reason this might have happened. A recent, post-Panda update video of the Google ‘Webspam team’ (which I found here) gave some good information. Apparently, Google has raters that go on sites and evaluate the user experience, and one of the things Google decided they didn’t like was ads above the fold (the ‘fold’ is an area roughly 1024×786 that approximates what a website would look like on, say, the screen of an average netbook). If you had lots of ads above the fold, they didn’t like you any more. They now want to measure just how much of the area above the fold is taken up by ads vs. content, and if it’s a high percentage, your site will suffer. More on this here.

Here are some screenshots that illustrate this. Before and after screenshots that show how we changed the theme and ad layout on this site.

 
Update 9/2012: you may have noticed that the ad layout as you read this is different from what is described above. This is because I am still experimenting with an ad layout that can offer a decent clickthrough rate AND allow for a good ratio of ads-to-content above the fold. Two of the layouts I am testing are displayed below.

The question I have is: now that we’ve ‘complied’, did it really make too much of a difference? Enough to warrant that the site lose so much of its Google ranking, and that we go into each and every post to adapt it to the new layout? (which will take weeks). Of course, it would be even better if the one ad was in the sidebar, but sidebar ads have never worked on this site, and would not allow us to survive.

There are two other possibilities: if this is not the correct explanation of what happened, there are two others (#1) that an ad I had in the body of the text, which was wider than the content area and slightly truncated on one end, was in violation of the Adsense ToS and responsible for the downgrade of my site. But that to me seems like it should be the purview of Adsense rather than the Google search team. Alternately, it could be (#2) that I simply have not found the answer, and that there is something here that I am not seeing at all. However, I doubt that this is the case, and my gut tells me that the ads situation above the fold is the answer. (And how ridiculous is this situation?, where you know you’re a ‘sinner’ but cannot figure out your ‘sin’).

How I feel about the policy: in principle, I do not disagree with it. I think definitely when a user opens a web page, they should be able to see the beginning of the content. From a purely financial standpoint in the long term, I don’t really think it will make much of a difference (I believe that by making clicks more scarce, it will merely drive the price per click upwards, making it on balance more or less the same for us content creators).

However, I do have a major grievance about how they want about implementing this, pulling the rug from under unsuspecting sites without any communication or care for the uncertainty and disruption that this might cause. I understand that they do this to spammy sites and sites that want to game the system all the time, but we are not spammers, and we never played games to get ranking. For a company whose (ridiculous) motto is ‘do no evil’, this is a major fail, IMHO.

In any case, Google was an accomplice in the old ad layout; see the ‘My relationship with AdSense’ below. I also have a few recommendations on how they SHOULD have done this (in the ‘Recommendations’ section below).

4. My relationship with Adsense

Adsense is Google’s online ad network, and the reason Google is, in my opinion, the world’s biggest ad distribution network primarily, and only secondarily a search engine. However, though a lot of people are not aware of it, Adsense is a large part of the reason why there is so much great content on the internet, and why thoughtful, entrepreneurial people are encouraged and rewarded for creating it.

In any case, I remember about two years ago I got an email regarding an ‘Adsense in your city’ event, which was being held at a trendy café in Seattle’s Capitol Hill neighborhood.  The one thing I wanted to know is this: how to display ads on my site for best effect, in a way that gets good results but also looks professional and attractive. I printed out full page screenshots from my site and several other tech sites and took them with me. At the meeting itself there was free coffee and pastries, and about five or six Google reps with Macbooks. I met with a very friendly guy and asked about the best layouts, what he might recommend that I use, which of the half a dozen ad layouts I showed him might work best. He didn’t seem to know. He asked about my clickthrough rate and I logged into my Adsense account and showed him. He seemed impressed, and told me that my current layout is apparently working very well. Despite my disappointment that we were not going to have an intelligent conversation about the merits and demerits of various approaches to ad placement, I came out of that café feeling good.

The said ad layout was the same one I had up until July 10th, shown in the first screenshot to the left in the ‘Figuring it out’ section above.So whether they like it or not, Google and Adsense were accomplices in why the ads above the fold were what they were (not that anyone cares, mind you).

Once my traffic and revenue had all but evaporated in the beginning of July, I used the  ‘send us feedback’ link on the Adsense page to ask if they had any information as to what might have happened, but got no response.

5. Recommendations: how they should have gone about this

Here’s a revolutionary idea: why not warn people? Why punish people first?  Why not consider that there are situations that call for forewarning rather than opacity, that maybe, just maybe, opacity is NOT always a virtue.

I know that frequently, it is considered a good thing that the right arm (Adsense) does not know what the left arm is up to (Search/Webspam teams), but in this case it would have been better if they did. Adsense could have sent a notification, alerting publishers that the ratio of ads vs. content above the fold will now be taken into consideration in Google rankings. This would have allowed sites to act, and the desired or hoped-for increase in the user experience accomplished.

Even better still, Google could have simply published this new policy before implementing it. (There is nothing whatsoever about Ads or ad layouts in Google’s webmaster guidelines).

Doing any of the above would have also enhanced Google’s reputation as a responsible agent, rather than an unpredictable Godzilla that has no compunction stepping on good people when it decides to change course (nothing personal, mind, just collateral damage).

Here’s another idea: if Google wants a better user experience, how about having Adsense ban those ridiculous, misleading ads with a big button that says ‘download’ on them. That, in my opinion, would have been much more direct and effective.

6. Why this hurts small sites more than big ones: and is this really what Google wants?

The one good thing is that Google states that it is possible to recover from Panda. I put in a reconsideration request, stating in a somewhat rambling paragraph what I think happened and what I had done about it.

But here’s the thing: aside from there being no guarantees that anything at all would ever be done to change the situation, it could take weeks or months for an update that might ‘fix’ this. Now, while big sites or corporations might be able to take a hit for one, three, six, twelve months or what have you, a blogger like myself cannot. I provide content for ‘free’ and today as things stand it is a net cash drain (despite the fact that May and June were both excellent revenue months for us). When I decided to blog rather than continue working I had no idea that this sort of disruption to the environment that I exist in could happen, and today I feel like a chump, like two years of work were taken away at one fell swoop. I seriously wonder if I should stop wasting my time and start looking for a job. And the sad thing is, I feel like it was so needless and unnecessary.

Moreover, the more complicated and opaque SEO becomes, the more of an edge bigger sites which have SEOs and developers have, such that they would be better able to figure out what went wrong and implement remedies (while smaller sites, like this one, languish in a state of uncertainty.)