This post is part of an ongoing series: How Search Really Works.
Previously: Relevance (2)
Instead of painstakingly grabbing the absolute best matches for your query to then rank those with infinite precision, one time saving strategy has search engines go for "close enough".
Painstaking Precision

Given all the time, money and resources in the world, here's what we'd [...]

3 clever comments | Keep reading »

This post is part of an ongoing series: How Search Really Works.
Previously: Relevance (1)
Another way we can assess the relevance of a document is by term weighting.
From the keyword density myth we know that true term weighting is done collection wide.
By looking at the number of documents in the index that a term appears in [...]

7 captivating comments | Keep reading »

This post is part of an ongoing series: How Search Really Works.
Previously: Simple Query Optimization.
Search is always boolean: yes or no. True or false.
Either the words are in the document or not.

But as you see, not all documents are "born alike". Some are about our topic, some just mention it.
What we need, what we want, [...]

13 riveting comments | Keep reading »

This post is part of an ongoing series: How Search Really Works. Last week: The Compressed Index.
While human beings can scan a page and see if the whole phrase "a grandiloquent dictionary" appears on it, a search engine can't.
A search engine needs to:

Lookup the occurrences for each word in [...]

7 gripping comments | Keep reading »

This post is part of an ongoing series: How Search Really Works. Last week: Recognize this index?
Memory is much faster than looking things up.
In order for a search engine in high demand to serve its users efficiently it should keep things in memory instead of looking it up on [...]

8 fascinating comments | Keep reading »

This post is part of an ongoing series: How Search Really Works. Last week: "The" Index (2).
Oversimplified: we have at least a few pages in our index, have extracted every single word from those pages and have written down in an index where in which pages those words occur.
Want [...]

9 riveting comments | Keep reading »

This post is part of an ongoing series: How Search Really Works. Last week: "The" Index (1).
Last week we saw how an inverted index (where a list of words points to a list of documents in which they appear) is insanely useful for doing AND queries.

But what if [...]

5 astute comments | Keep reading »

This post is part of an ongoing series: How Search Really Works. Previous Instalment: The Keyword Density Myth.
If a search engine would search "live" through the documents it knows about for the occurrence of the word we're looking for it could take its time and then simply report where [...]

4 brilliant comments | Keep reading »

This post is part of an ongoing series: How Search Really Works. Last week: Keyword Stuffing.
What is Keyword Density?
Keyword Density is a function, a calculation, of keyword frequency.
It's calculated as number of occurrences divided by number of words and is usually expressed as a percentage.
 
What is Keyword Density Used [...]

25 riveting comments | Keep reading »

This post is part of an ongoing series: How Search Really Works. Last week: Keyword Links.
Left to their own devices, people will assign keywords (tag or link) as they please.
They paint a rich picture of the linked content.

Keyword stuffing is the unnatural repetitive use of a specific [...]

3 smart comments | Keep reading »