How to Make Money Online As a Freelance Search Engine Result Evaluator


If you are looking for home-based employment or self-employment, becoming a search engine results evaluator or search engine evaluator might be a good option for you if you have the right skill set.

Search engine result evaluators look at the results that they get when they enter all sorts of search queries into search engines like Google and Yahoo. They evaluate the results in terms of accuracy and usefulness. They record their findings and they are paid for helping search engines improve their users’ experience.

Why would companies pay for Search Engine Evaluation work?

Search engines are not perfect. You probably already know this if you have tried searching for something that is kind of off the beaten path or pretty obscure, all bets are off. Chances are you are not going to get the exact result that you are looking for. In fact, a lot of searchers settle for “second best” results. As you probably already know, when it comes to your personal life, second best is never good enough.

That is why search engines are always looking to tighten up their game. They are looking at tweaking their algorithms or the sophisticated pieces of software that enable them to index, as well as rank search results based on the queries consumers enter into their engines. This is not a perfect science. There is always a room for improvement. As the engineers for search engines like Google make algorithm changes and tweaks on an almost weekly basis, somebody has to monitor the effect of these changes.

Search Engine Automization and Machine Learning versus Real Human Beings

You might be thinking, considering how sophisticated and powerful Google’s technology is why don’t they just create software that monitors how human users interact with the results they get. They already are, this is called the Google RankBrain update. RankBrain uses a form of artificial intelligence called Machine Learning.

Machine Learning essentially learns from how users interact with the software. It makes predictions regarding their behavior. It makes projections regarding the kind of judgement calls people make as they use the software. They are not intentionally trying to program the software, but the software is being programmed nonetheless because it automatically learns from people’s actual search behavior.

How does RankBrain perform Search Engine Result Quality Evaluation?

When a user enters a keyword or a key phrase into Google, they get a list of results. They can click whatever listed result or link they want. Apparently, they pick one link over another. Once they get to that page, they can do one or two things. They can read page after page because the content that they found actually address their needs or provided a solution to their problem.

Alternatively, they can click the back button. This happens when people find themselves on a page that does not have the answer that they are looking for. Now, this one is a little tricky. The page can actually have the answer, but the problem is whoever published the page did not highlight the answer so it would be easy to see or scan. People no longer read web pages, they scan or they skim. If their eyeballs do not hit the right keyword or the right phrase or piece of content within a few seconds, they click the back button or they tap away.

Google’s RankBrain pays attention to this bounce pattern. If the software notices that most of the users that end up on a certain page bounce out, this is called Pogo-sticking. Sooner or later, the software demotes that page. For whatever reason, people do not like the results, so Google promotes another page that has a lower bounce rate. Using this technology, the users of Google itself end up training Google’s central software.

Given the rise of RankBrain, is there still a job for Search Engine Evaluators?

You are probably thinking to yourself, considering how sophisticated and seemingly scary RankBrain is, is there still a job for search engine evaluators? The answer is a surprising yes. While RankBrain does do a great job of detecting user patterns, it does a lousy job of paying attention to qualitative factors.

The truth is, different people behave differently on different web pages based on different situations. Maybe the reason why people bounced out immediately is because they found the answer so quickly. They did not need to stick around. They found the answer that they are looking for, why should they bother clicking page after page.

Similarly, a lot of web pages are very sneaky. They promise the answer, but the user has to click on page after page until they get the full answer. They tease users. These web pages have really enticing and appealing titles; this maximizes their click-through. Once the user gets on the page, they are then teased to click link after link and page after page.

In the eyes of RankBrain, this is a successful search. After all, the user stayed on that page for a very long time. They did not bounce out. What RankBrain does not know and could not know is the fact that whoever put the page together is actually playing all sorts of tricks to increase page dwell time. In other words, this is a sophisticated and clever scheme to make RankBrain rank that page higher than higher quality page.

Search Engine Result Evaluators will always have a job

If you paid attention to what I just said in the previous sections, my conclusions should not shock you. It should not come as a surprise to conclude that flesh and blood human search engine evaluators will always have a job. How come? You have a brain. RankBrain despite its name, is still a piece of software; it is a collection of codes that run on several computers throughout the world, but at the end of the day, it is not human.

Human beings have the following capabilities that are crucial to ensuring high-quality search engine results. It is going to be a long time until human beings come up with software that does a great job mimicking, simulating or otherwise compensating for the following factors:

Factor #1: Judgement Calls

Have you ever come across a piece of information and it is obvious to you that it does not address your question directly but you still kept reading it? The short answer to this is a judgement call. You are already there so you make a judgement call as to the relative value of the content. Most of the time, given your experience, your gamble pays off. It seems that for a lot of web pages, if you are able to connect the dots and read between the lines, you are able to get the answer that you are looking for.

In fact, compared to other pages that seem to give you the answer directly, these are the pages that seem to require a lot more of your brainpower work actually delivers a far greater value. They deliver context, they give you a very deep understanding of the problem that you are trying to solve, you also get to see the big picture as far as the different options you have available to you.

Factor #2: Determining Context Trough Nuance

One of the most powerful human thinking capabilities we have is our ability to notice nuance. Put simply, we understand that the world is not black and white. We know fully well that the world cannot be reduced to some sort of simple formula or some sort of neat and tidy equation. Instead, we know that in many cases, answers tend to bleed or tend to present themselves as gray instead of black and white.

Knowing this, our brains are configured to pay attention to nuance and make relative quality judgements based on those nuances. This enables us to make better decisions. The world cannot be reduced to some sort of mathematical equation as much as we would like it to. Life would definitely be much more convenient and comfortable if that were the case. Still, people being people, there is always room for nuance.

For example, if you see somebody cross the room that you somewhat know, but you really cannot quite place, you probably would smile at that person and that person would smile back at you or send signals. It takes quite a bit of human experience as well as attention to detail to get the nuances right to make the right judgement. Good luck getting a computer to do that.

Computers are great when the signals that it perceives are black and white. The closer you can get data points to raw numbers or better yet, binary code like zero and one, the easier it would be for a computer to take on that job, but for everything else that requires nuance and judgement calls, computers are going to have a tough time. When people search on the internet, there is a tremendous amount of nuance involved.

Factor #3: Collective and Comprehensive Learning

Did you know that every time you learn something, your brain actually makes a physical connection among nuance? This happens because your brain is very lazy. I know it hurts to hear that, but it is true. Your brain is always looking for a shortcut. When you keep doing the same things day after day and you end up thinking about the same things routinely, your brain no longer devotes a tremendous amount of its resources and fire power to whatever it is you are doing. Instead, it would make hardware shortcuts or physical shortcuts in your neurons and neural pathways to make its job easier.

This is why drug addiction is such a big problem. When addicts get their fix, the brain makes powerful physical connections among neurons to maximize and speed up pleasure reception and pleasure perception. This physically rewires the brain. Believe it or not, coffee does the same thing as well as pornography or gossiping. Whatever mental habit you get into, your brain will automatically make these associations.

When it comes to processing information, your brain also has these associations that it makes. Believe it or not, these are more efficient than the most complicated software. When it comes to data processing, a straight line will always be the shortest path; it will also be the most efficient. This is very hard to do with sophisticated software because the more data it has to crunch and the more things it has to remember, the longer it takes. It becomes more resource intensive, the chances of it getting stuck in a loop increases.

If your mobile phone or laptop computer has ever been stuck or got caught in a glitch, you know exactly how this works. Not so with the brain. Because if you search Google extensively, sooner or later the brain is going to want to create a shortcut. It is going to want to establish certain habitual patterns of data recognition, data processing and decision-making. Computers do not do that, they are going to only look at the numbers.

How do you qualify to become an Evaluator of search engine results?

Believe it or not, you do not need a fancy degree. You do not need a diploma. You do not even need many years of experience evaluating search engine results. Thankfully, a lot of companies that hire home based freelance search engine evaluators do not care about formal training or formal experience. They couldn’t care less. Instead, they focus on your skills. If you do not have all of the skills below, tough luck. You have to have all of these skills. Not one, not some, but all of them.

#1: You must have excellent English skills

This is hard to fake when applying for any kind of online freelance job because you will be tested. Worse yet, you will be compared to everybody who was applying for that same position, so this means that you have to rank really high. Even if you are a native English speaker who was born in Middle America like Kansas, but your English fluency is weak compared to other applicants regardless of where they were born, you are not going to get the job. English fluency is tested in terms of reading, comprehension and writing. You have to have excellent English reading, writing and comprehension skills.

#2: You need to impress your interviewer

A lot of search engine evaluation jobs do not just include an online test. These are a series of forms that you fill out or a series of online boxes that you fill in. In addition to that, multiple choice as well as essay tests, you have to have an interview. Usually, face-to-face interview is over the internet aim for one thing. They are intended to see your temperament.

Are you a flaky person? Are you the kind of person who starts a project, but cannot seem to quite finish it? Are you the type of person that folds easily under pressure? These are the kinds of things interviewers are trained to detect. If you exhibit any of these, you will not get the job.

#3: You might need a college degree for higher-level work

As I mentioned above, if you just want an entry-level search engine evaluator freelance job, you probably already qualify even though you do not have a college degree. But if you want higher-level evaluation or data analysis jobs, you might have to have some sort of college degree. The great thing about companies like Google is they are not really obsessing about the specific course you have a diploma in. You can be a music major and still be hired for higher-level search engine evaluation work.

#4: You must be able to meet minimum work hours

If you are reading this, you want to enjoy the freelance lifestyle. What is not to love? You work only a few hours in a week and you earn a full time income. It would be great if a lot of people had the same lifestyle. The problem is when it comes to search engine evaluation work, a lot of this work requires continuity, and a lot of this work requires a tremendous amount of materials that need to be processed.

Accordingly, they require freelancers to work a set minimum number of hours per week. Currently, the industry standard is 10 hours or so. If you cannot commit to 10 hours every week of freelance work, you probably would be better off doing something else besides search engine evaluation work.

#5: Contemporary knowledge of global pop culture and current events

A lot of search engine queries involve current events as well as pop culture. This is not a problem for people living in the United States because most people are aware of what is going on as well as the latest and greatest pop culture icons and celebrities. But if you live on the other side of the planet and the search queries that you have been assigned to research and evaluate involve American pop culture and current events, you might have a problem.

Fortunately, this is very easy to fix. You just need to soak up the latest and greatest celebrity news, gossip as well as political news, opinion and current events. There is no shortage of websites that dish out this kind of information. Ideally, you should brush up on your exposure and knowledge of this type of information before applying for search engine evaluator freelance job.

#6: Top-notch research skills

I saved the best for last. You have to have amazing research skills. A lot of people think that Google research is all about just typing in keywords, looking at the results and calling it a day. It does not work that way. You have to understand that different people from different backgrounds in different circumstances type in different keywords and phrases into Google. They all mean the same thing, they are all looking for those same solutions to their common problems, but the problem is they use different ways to get there.

This is the biggest challenge, and this is how your research skills will be challenged, because you have not just been tasked with looking at the results produced by a certain key phrase or keyword. Instead, you are supposed to look at the results, analyze it and then pay attention to the intent behind the many different keywords that could possibly be used to find that same information. This requires quite a bit of research. This also requires a tremendous amount of intuition and intelligence.

What will you do as a Search Engine Evaluator?

Generally, when search engine evaluators get accepted to a program they will be given a special piece of software that they can add on to their browser, or is loaded as a standalone application on their computer. Regardless, you will be sent a list of search terms that you are going to use on Google to pull up all sorts of results. You will have clear instructions on how to evaluate the results based on the search terms.

Depending on your understanding of the search terms and your prior research on the search terms, your quality may suffer. Because if you are very literal, very few web pages will probably directly answer the wide range of questions a particular key phrase brings to the table. This is where your intelligence comes in. You have to make a clear case when filling out your report, why you think a specific result is a better fit than another result.

If you are unable to back up your claims or explain in clear terms why you think there is a better result, then your report is not all that valuable. It is not going to be all that useful for Google, Yahoo, Bing or whatever other company.

What is the most important factor in Search Engine Evaluation work?

If you want to be a search engine evaluator, you have to understand that you are an independent contractor. This means that the company that hires you is usually not Google itself. It is usually the other company of the other and is usually not the ultimate search engine that will be using your work product.

When you are doing work, you know that the results come from Yahoo so your work is probably going to influence Yahoo or Bing search results, but your actual contact is not Yahoo, Bing or Google. It may be a contractor company. Also, you are an independent contractor, meaning your project can expire anytime, they can also let you go anytime. In fact, they do not even have to let you go, they just need to stop sending you work.

If you are reading this, you are probably already aware of how freelancing works and this is not a big deal to you, but I have highlighted this just to make sure that you are under no illusions. This is a very easy trap to fall into because in the beginning you probably will get a lot of work. In fact, you will get nice big blocks of work and this can give you the impression that you are pretty much working on a more or less permanent job.

Get those ideas out of your head. You are an independent contractor, so just as the work can come one day, it can easily disappear the next day. That is the name of the game, unless of course you get invited to interview for a higher-level position, which can be either more stable or semi-permanent or might even be an actual permanent position.

How much money can I make evaluating search engine results?

Generally, search engine evaluators annual earnings in the US are slightly south of $40,000 a year. This is a survey figure, obviously search engine companies as well as their contractors and sub-contractors do not release this information to the general public. But according to the employment website Glassdoor and its survey of people who were hired as search engine evaluators, the average annual income for people with that job title is less than $40,000.


Do not get too excited too excited about the figure. Please understand that this sub $40,000 figure is based on people who do search engine evaluation work in the United States. If you live outside the United States, you will be earning far less than that figure. I’m not saying that you may be or you could be earning less, I’m saying you will be earning less.

For example, in India or Bangladesh, people should expect to earn half the US figure, the same applies to the Philippines. But due to foreign exchange rates as well as different costs of living, even if you were paid only $20,000 a year, that still translates to 1 million pesos or north of 80,000 pesos per month, which is generally not a bad salary for somebody who is single and still living with their parents.

The advantages of working as a Search Engine Evaluation Freelancer

You get to work from home, you don’t really need to get advanced skills. Depending on where you live­ and your living arrangement, the pay can actually be quite attractive. Also, since this is freelance work with fixed deadlines, you have a lot of control over how you work, when you work and when you send in your report.

The disadvantages of working as a Search Engine Results Evaluator

The disadvantages of this type of freelance works are really not much different from other types of freelance work. Whether you are working as a transcriber, a typist, a rewriter or copy and paste data encoder, the downsides are all the same.

First of all, work is not steady. You may have work for several days on end and then it would dry up, weeks can pass and then you get an email and you have work again. Also, you have to be a self-starter. When the sub-contractor gives you a deadline, you have to meet the deadline. The problem is, unlike an office where you are on the clock and people putting pressure on you, you do not have any of that structure when you are drinking coffee at Starbucks working on your laptop.

Maybe you can work on evaluating search engine results there or you can take a walk in the park with your kids, or catch a movie or fall asleep on the couch. A lot of people view this as one of the perks of the freelance lifestyle. It can easily become a disadvantage if you are not a self-starter.

Finally, search engine evaluation work can be quite erratic because as you can probably already tell, the need for it changes from time to time.

The biggest drawback to Search Engine Evaluator jobs

As awesome as the job description may be, the biggest drawback to this type of freelance work is the fact that search engines are not always updating their algorithm. Even if they were, a lot of the times they use in-house software to track these changes.

From a corporate perspective, it makes more sense for search engine companies to use the services of search engine evaluators only when they are rolling out very big or a potentially disruptive update to their software. Unfortunately, those are not all that frequent. So keep this in mind, search engine evaluator jobs are often heavily promoted by recruitment companies, but after the update has been completed and all the evaluations have been set in and the software has essentially been tweaked to a good enough workable condition, the search engine evaluator jobs disappear.

Where can I find Search Engine Evaluator freelance opportunities?


Appen is the 800-pound gorilla of the global search engine evaluator service space. It is the biggest provider of this type of service to the major search engines. It essentially is just a middleman. Basically, you sign up to Appen and then they will farm out the work they get to you. They also have a distributed network of freelancers located all over the world. The great thing about Appen is they focus on language specific work. If your English skills are not all that good because you speak Spanish as your first language, no problem. Appen has a lot of Spanish language work they can assign you.

The downside to Appen is that it tends to only take in freelance workers from a shortlist of countries. There are only 43 countries on its list. Also, Appen is very tough on language proficiency. If English is your second language and your English skills are spotty, you might have a tough time passing Appen’s screening procedures.


Lionbridge’s term for search engine evaluators is search engine internet assessors. The great thing about Lionbridge is that they have openings throughout the world. They require people to send in their resumes and CVs when applying. They also want you to pass certain tests using your mobile phone. Payouts at Lionbridge usually range between $13-$16 per evaluation. Generally speaking, if you are going to average it out, the typical Lionbridge search engine evaluator makes $15.

As I mentioned earlier, do not get too excited about this $15 figure. This only applies for people who live in Western Europe or the United States. For the rest of the world, Lionbridge has a sliding scale or may not have jobs available at all.


Believe it or not Google’s career page highlights search engine evaluation jobs periodically. Again, it all depends on when Google is working on a major algorithm update. The problem with relying on Google’s career page or hanging out there and checking week after week or day after day, involves business strategy.

As you probably would suspect, a lot of people would like to know when Google is working on their algorithm. One giveaway is when Google advertises search engine evaluator jobs. It doesn’t take a Sherlock Holmes to figure this out. Not surprisingly, Google is very reluctant to let the public know that it is working on an algorithm update. Usually it would use a third party agent or some sort of subcontractor.

The hardest part about Search Engine Evaluator freelance positions

The biggest hassle of getting a search engine evaluation job is the screening process. A lot of people simply don’t have the English skill. This is why they get filtered out on stage one. This is all for the best because the company that is hiring you is not going to be served if your English is very spotty. On the other hand, you are not going to be doing yourself any favors if you can barely understand the tasks in front of you. Your work output is not going to be all that great. The second stage gets even harder because oftentimes this requires a real live testing of your skills.

Finally, you have to pass the interview. This is why it is a good idea to apply for online courses that can help you crack the search engine evaluation screening process.

Is it worth your time and money getting trained as a Search Engine Evaluator?

The training involved really is just all about passing the screening. That is really all the training you need because once you get search engine keywords and instructions, it’s all pretty self-explanatory if you have solid English skills.

There is a reason why many people think it is worth getting trained as a search engine evaluator. It is the fact that once you got approved as a search engine evaluator, you qualify for jobs when they become available. In other words, compared to all the other people out there who want to be paid to analyze Google results, you get first priority. But as I have mentioned earlier, it all depends on when jobs are available. But if they are available, you get first priority.

Keep this in mind when trying to decide if you should get search engine evaluation skills training so you can pass the screening test.

If you’d like to earn extra dollars online without spending any money, try these 24 other online money making opportunities

Leave a Comment