Google's form filling bot a benefit to some, scares others.

Google's form filling bot a benefit to some, scares others. Kevin Heisler's article in SEW points out a dilemma that Google faces; in an attempt to homogenize the desires and intents of the masses, they will please some while angering, annoying or frightening others. I am not nearly as bothered by this as Kevin. As this question popped up in some communications in my company, my response was...

 

"Google has been inundated with questions as to why pages are not showing up in the index, only to explore the issue and find out that the only way to get to the pages in question is to submit a form of some type. The most obvious is  corporate home pages where the user has to select the country / region in a drop  down (Matt's example). Until this new release, Google couldn't crawl the pages from the home page. Other examples include product selection, category information where you  have to tell the site, via a form, what you want. Web masters and publishers  have be frustrated by their in ability to get a lot of content indexed because  managing it requires data driven applications and the use of forms. This is  Google's attempt to rectify the problem."

For those really worried about this, blocking the bot from sub pages can be done." 

Matt Cutts has a good post on this.

I think another aspect of blocking the bot is the robot.txt. As Matt says, "If you’d prefer that Google not crawl urls like this, you can use robots.txt to block the urls that would be discovered by crawling through a form." These URLs should probably be part of the robot.txt file anyway. But if not, this should not be too arduous a task to add them.
Any way, like so many other "things" Google, this seems bigger at first than it will in hind sight.

Enamored with Technology... the Google - ization of us all.

At AdTech last week, I was going to meet some folks for dinner. I knew the
name of the restaurant and the street name, that's it. No address. So,
I pulled out my blackberry, went to Google, and wham! nothing. There
were some reviews, but not a listing. Next Yahoo! Go!. nothing. Again,
some web sites with reviews. Then Live. Bingo. No websites, no links.
just Name, Address and Phone number. Then click, a map. Oh, and I was
probably just a few feet from a yellow pages directory in the room. But, I wanted to use the technology. To me, this would seem like an obvious search. A mobile device and a specific restaurant name. Live knew (or guessed) exactly what I wanted. The other two were clueless. 

But, I wanted it to work. I wanted technology to provide the answer. So, while it took a bit longer than I'd like, 1 of the 3 did work for me. But this got me thinking, 'are we too enamored with technology?' I could have picked up the phone, talked to concierge and had my directions faster. But, I didn't. 

I see this take place in SEM all the time. Bid management tools, algorithms that can tell you (so they say) when someone is ready to buy, or can optimize your media program. I was on a call the other week with an agency that appeared to rely nearly 100% on statistically driven bid management programs. I wish I could say these things worked. But they don't. Sure, they can do what you tell them, adjusting bids based on historical inputs and manage to your parameters. But they can not 'read' the market. Adjusting to the unexpected is too cumbersome, and anticipating the new is impossible. If 'it' is not in the historical data, whatever 'it' is can not be considered by the technology. 

People, however are very good at this. We know how our competition and consumers respond. We know our clients and their marketing calender. We can anticipate, and adjust and optimize. We can also take risks. This is where the rewards come from. Try something you've never done and see what happens. Algorithms can't do this SEO suffers from the same problem (but I think they get more feisty about it). SEO is a very manual service. No two SEO experts will agree on every 'best' way to do things. Computer programs that analyze your site are useless. A good SEO person will admit and adjust to stumbles. SEO programs will keep blundering along. 

In a world where we really want technology to solve problems (and it does have its place among our tools), sometimes it is hard to accept that the real answer is not a technological one. Its human. 

Experience, perception, anticipation, risk taking and hard work. These are the hallmarks of a good SEM shop.