by Paul Graham
Traffic from general-purpose Web searches will become less important
as the search engines set up special shopping
areas, but for the moment, at least, stores still get significant
traffic from ordinary Web searches.
There are hundreds of search engines and indices, but only eight
that matter: Yahoo!, AltaVista, Excite, WebCrawler, MSN,
Infoseek, Lycos, and HotBot.
All the others might account for 1% of your traffic, combined.
So if you are in the top eight, you are doing fine.
Although Yahoo! is often called a search engine, it is fundamentally
different from the others. In fact, it is not really a search
engine at all.
A search engine works by sending a program, called a "crawler" or
"spider" to your site. It retrieves your pages and stores them in
a database. Later, when someone comes to the search engine and
searches for "chocolate", the search engine will tell them about
all the pages in its database that contain that phrase, possibly
Yahoo! is different. It does not have a crawler. When you submit
your site to Yahoo!, you can supply a description of up to 200
characters. And when someone does a search of Yahoo!, those
descriptions are all that get searched.
So while searching for "chocolate" at a true search engine like
AltaVista or Excite could find you if the word "chocolate" occurs
anywhere in your site, a Yahoo! search for "chocolate" will only
find you if "chocolate" was included in the description you supplied
when you submitted your site to Yahoo!.
Yahoo! searches are narrower. If you search for "chocolate" in a true search
engine, you'll get thousands upon thousands of results, most of
them merely recipes. If you search for it in Yahoo!, at least you'll
only get those sites where chocolate was important enough to be
mentioned in the description.
What does this mean for you? It means that you should be very
careful when you write your site description for Yahoo! You should
use all 200 characters, and if there is any text in your description
that you couldn't imagine someone searching for, it is wasted.
You probably can't just supply a list of keywords as your Yahoo!
description, because Yahoo! will reject it, but the closer you can
get, the better.
Here is an example of a ineffective Yahoo!
We have been selling the highest quality domestic and imported
gourmet foods since 1927.
There is only one word in this description that your customers
might search for: "gourmet". Your customers are not going to be
searching for "quality" or "1927".
You'll get a lot more traffic if you say:
Selling cookware and gourmet foods including cheese, caviar,
sturgeon, lox, salmon, foie gras, truffles, herbs, spices, olive
oil, vinegar, chutney, mustard, tea, coffee, rugelach and chocolate.
This description uses almost all 200 characters, and almost every
word is a possible search keyword.
It's worth taking the time to write a good Yahoo! description,
because Yahoo gets the most visitors of all the Internet hubs.
When you submit your URL to search engines, don't expect your site
to show up immediately. Your front page will show up in AltaVista
and Excite the next day, but the others
are slow to add new listings.
In some search engines, it can be a month or more before even
your first page is indexed, and much longer before your site is
That's because most search engine crawlers typically retrieve
only a few pages from each site on each visit, and visits can
be weeks apart.
Since it can take months for your site to be fully indexed, you
should have your pages online months before you expect people to
search for what's on them.
So if you want consumers to find your site when
they search for gifts in November and December, your site should
be up and running in August or September.
When search engines return results, they are sorted by "relevance".
Most users don't look beyond the first couple pages of search
results, so it is a big advantage to come up first. How do you
get your site to appear first in the search engines?
There is no easy trick that will work in all cases, because (a)
all the search engines are different, and (b), if there were a
trick, everyone would use it, and it would be just as hard to come
Generally, someone searching for "chocolate" is more likely to get
a page in your site if the word "chocolate" appears often on that
page, especially if it appears in the title. But it will not work
simply to have your page begin with the word "chocolate" repeated
100 times. Most search engines filter out sites that try that.
The best approach is to use key words frequently in your site, but
not in an unnatural way. In general, the more text in
your site, the bigger a target you present to search engines.
It may also help to put keywords in special tags, called "meta
tags", within your pages. If your software supports meta tag
keywords (Yahoo! Store does), it can't hurt to use them,
but if not, don't worry. As
long as you use the important keywords in your pages, meta tags
won't add much.
Is there anything you shouldn't do, if you want traffic from search
engines? There are two major don'ts. Don't use frames, and don't
use software that generates your pages dynamically.
Frames are a way of dividing your pages into separate "panes".
Many search engine crawlers ignore sites that use frames. If you
use frames in your site, you will only make it into about half the
Even more important, if you want traffic from search engines, is
not to use software that generates your pages
dynamically, as the
consumer is looking at them. Currently, all search engine crawlers
ignore dynamically generated pages. So any part of your site
that is dynamically generated
will not exist, as far as search engines are concerned.
To maximize your traffic from search engines:
- Submit your site to the top eight (or use
software that does it for you - like Yahoo! Store).
- Use search keywords in your Yahoo! description.
- Use keywords frequently in your site, but not in an unnatural way.
- Allow search engines several months to index your site.
- Use frames.
- Use software that generates your pages dynamically.
Text copyright © 1999 Paul Graham. Feel free to
reproduce any of this text on your own Web site, so long as you reproduce it verbatim, and include this message. For any other use, please contact the author. Yahoo! Store is a trademark of Yahoo Inc. All other trademarks are the property of their respective owners.