This topic has been archived, and won't accept reply postings.
With all the talk about 'experts' I went looking for any long term studies on the subject, and I found this twenty year study.
( other 'expert' studies are available
" political experts’ predicitons are only slightly better than a random guess, and way worse than a statistical model.
In fact, so called experts were better at predicting events OUTSIDE their own field. ...a study from the 1980′s, when Philip Tetlock had 284 political “experts” make about a hundred predictions.
For each of these predictions, Tetlock insisted that the experts specify which of two outcomes they expected and also assign a probability to their prediction. He did so in a way that confident predictions scored more points when correct, but also lost more points when mistaken.
With those predictions in hand, he then sat back and waited for the events themselves to play out. TWENTY years later, he published his results, and what he found was striking...
the experts did slightly better when operating outside their area of expertise than within it."
http://www.smithsonianmag.com/smart-news/why-experts-are-almost-always-wron...
( other 'expert' studies are available
" political experts’ predicitons are only slightly better than a random guess, and way worse than a statistical model.
In fact, so called experts were better at predicting events OUTSIDE their own field. ...a study from the 1980′s, when Philip Tetlock had 284 political “experts” make about a hundred predictions.
For each of these predictions, Tetlock insisted that the experts specify which of two outcomes they expected and also assign a probability to their prediction. He did so in a way that confident predictions scored more points when correct, but also lost more points when mistaken.
With those predictions in hand, he then sat back and waited for the events themselves to play out. TWENTY years later, he published his results, and what he found was striking...
the experts did slightly better when operating outside their area of expertise than within it."
http://www.smithsonianmag.com/smart-news/why-experts-are-almost-always-wron...
7
In reply to Jim C:
It's an interesting study. But have a look at this recent piece:
https://hbr.org/2015/02/what-research-tells-us-about-making-accurate-predic...
That has some interesting points about open-mindedness, biases and the value of team-work.
I would also suggest that there is more to expertise than prediction. Fetlock was only looking at political experts, and prediction is probably one of their key roles. When you start looking at other domains (medicine, flying, structural engineering, almost anything) it's clear that being an expert isn't only about prediction. It also involves the application of skills, the ability to conduct procedures, advising others on how to do things, checking for and find faults, and many other things.
It's an interesting study. But have a look at this recent piece:
https://hbr.org/2015/02/what-research-tells-us-about-making-accurate-predic...
That has some interesting points about open-mindedness, biases and the value of team-work.
I would also suggest that there is more to expertise than prediction. Fetlock was only looking at political experts, and prediction is probably one of their key roles. When you start looking at other domains (medicine, flying, structural engineering, almost anything) it's clear that being an expert isn't only about prediction. It also involves the application of skills, the ability to conduct procedures, advising others on how to do things, checking for and find faults, and many other things.
12
In reply to Jim C:
Having previously been sort of an "expert" in an area of political research I would say that not making predictions is always a good strategy. From that article I have absolutely no idea what is meant by the word "expert" anyway.
Having previously been sort of an "expert" in an area of political research I would say that not making predictions is always a good strategy. From that article I have absolutely no idea what is meant by the word "expert" anyway.
Post edited at 12:16
5
In reply to Jim C:
You read that article, so that makes you an expert, I guess
Which means that you're wrong. ...
You read that article, so that makes you an expert, I guess
Which means that you're wrong. ...
1
In reply to TobyA:
As Niels Boehr is supposed to have said, prediction is difficult, especially about the future.
As Niels Boehr is supposed to have said, prediction is difficult, especially about the future.
4
16
In reply to Jim C:
The best paid 'expert' jobs aren't about making the most accurate prediction of the future. They are about collating evidence in support of whatever the client wants to do for personal, emotional or political reasons. If you are the 'Go To' expert for the Tories or Goldman Sachs and are asked for an opinion about what will happen politically you are always going to come up with one that helps the Tories or Goldman Sachs. You aren't getting paid for accuracy you are getting paid for loyalty.
The best paid 'expert' jobs aren't about making the most accurate prediction of the future. They are about collating evidence in support of whatever the client wants to do for personal, emotional or political reasons. If you are the 'Go To' expert for the Tories or Goldman Sachs and are asked for an opinion about what will happen politically you are always going to come up with one that helps the Tories or Goldman Sachs. You aren't getting paid for accuracy you are getting paid for loyalty.
4
2
In reply to tom_in_edinburgh:
Any legal opinion I've taken has been provided on the same basis in my experience. But always with a eye to the maximising of their fee. *
* To be fair, that observation could be levied at me when in professional advisory mode.
> The best paid 'expert' jobs aren't about making the most accurate prediction of the future. They are about collating evidence in support of whatever the client wants to do for personal, emotional or political reasons. If you are the 'Go To' expert for the Tories or Goldman Sachs and are asked for an opinion about what will happen politically you are always going to come up with one that helps the Tories or Goldman Sachs. You aren't getting paid for accuracy you are getting paid for loyalty.
Any legal opinion I've taken has been provided on the same basis in my experience. But always with a eye to the maximising of their fee. *
* To be fair, that observation could be levied at me when in professional advisory mode.
Post edited at 12:46
1
In reply to Jim C:
The use of evidence based medicine has tried to get away from the role of the favoured expert or trends/fashions in treatment. It recognises that expert opinion is weak compared to unbiased trials. However the system is far from perfect, particularly as negative and problematic trials are not published and here the expert may have a valuable role in challenging and designing trials to improve treatments.
In my discipline almost anyone can call themselves an expert whereas you have to be assessed to be a specialist which is protected.
The use of evidence based medicine has tried to get away from the role of the favoured expert or trends/fashions in treatment. It recognises that expert opinion is weak compared to unbiased trials. However the system is far from perfect, particularly as negative and problematic trials are not published and here the expert may have a valuable role in challenging and designing trials to improve treatments.
In my discipline almost anyone can call themselves an expert whereas you have to be assessed to be a specialist which is protected.
In reply to FactorXXX:
"Outstanding in his field" I heard!
Chris
> Worzel Gummidge is an expert in his field.
"Outstanding in his field" I heard!
Chris
8
In reply to SouthernSteve:
When information is available from unbiased trials an expert is not necessarily required.
Usually that is not the case. Someone with a wide knowledge and good track record is needed to make decisions if the problem is a completely new one.
The miss named "expert system" is however not generally useful for new problems.
When information is available from unbiased trials an expert is not necessarily required.
Usually that is not the case. Someone with a wide knowledge and good track record is needed to make decisions if the problem is a completely new one.
The miss named "expert system" is however not generally useful for new problems.
In reply to Jim C:
According to my old Dad - Expert = X is an unknown quantity and a Spurt is a drip under pressure. I rest my case m'lud.
According to my old Dad - Expert = X is an unknown quantity and a Spurt is a drip under pressure. I rest my case m'lud.
2
In reply to no_more_scotch_eggs:
I work with , rely on, and respect lots of Engineering experts all day, IT,chemical , electrical, metalurgical , mechanical etc.
The experts that appear to come a cropper most often, are those who's job it is to make predictions.
So the sales department experts, who predict a record sales year and profits , and we make losses, the Shipping experts who say things will arrive on time, and get are late, I'm not saying too much about weather 'experts' , as my mate was one
So financial, and political'experts' involved in predictiing markets ,and voting , are for sure not to be trusted.
I work with , rely on, and respect lots of Engineering experts all day, IT,chemical , electrical, metalurgical , mechanical etc.
The experts that appear to come a cropper most often, are those who's job it is to make predictions.
So the sales department experts, who predict a record sales year and profits , and we make losses, the Shipping experts who say things will arrive on time, and get are late, I'm not saying too much about weather 'experts' , as my mate was one
So financial, and political'experts' involved in predictiing markets ,and voting , are for sure not to be trusted.
1
In reply to Jim C:
Yes, because politics and economics are not 'hard' sciences; they are more like branches of psychology.
Prediction is hard; I'm with Bohr.
I have, though, given my expert opinion on certain issues. Sometimes that opinion has been ignored, but my opinion has subsequently been shown to be correct. The opinions are predictions based on fact, though, and are short-term, using firm bases for prediction.
I was not able to predict the growth in demand for mobile phones, despite starting my career working on the GSM standard. Had I done so, I would have been substantially wealthier than I am...
> So financial, and political'experts' involved in predictiing markets ,and voting , are for sure not to be trusted.
Yes, because politics and economics are not 'hard' sciences; they are more like branches of psychology.
Prediction is hard; I'm with Bohr.
I have, though, given my expert opinion on certain issues. Sometimes that opinion has been ignored, but my opinion has subsequently been shown to be correct. The opinions are predictions based on fact, though, and are short-term, using firm bases for prediction.
I was not able to predict the growth in demand for mobile phones, despite starting my career working on the GSM standard. Had I done so, I would have been substantially wealthier than I am...
2
In reply to Bob Kemp:
Einstein said insanity was trying the same thing over and over and expecting a different result, but Rutherfords gold foil experiment did just that , and eventually his predicted result happened. ( just as well he never listened to that particular expert
> As Niels Boehr is supposed to have said, prediction is difficult, especially about the future.
Einstein said insanity was trying the same thing over and over and expecting a different result, but Rutherfords gold foil experiment did just that , and eventually his predicted result happened. ( just as well he never listened to that particular expert
2
In reply to FactorXXX:
I just got that, 6 hours after it was posted. Am I thick or what.
> Worzel Gummidge is an expert in his field.
I just got that, 6 hours after it was posted. Am I thick or what.
1
In reply to abseil: see Chris at 13:08 then - very good
Honestly, to the OP, the examples you give of sales and delivery experts getting it wrong - they are not good examples of unbiased prediction are they? What they really believe may well not be what they say in public if they want to keep their job
Honestly, to the OP, the examples you give of sales and delivery experts getting it wrong - they are not good examples of unbiased prediction are they? What they really believe may well not be what they say in public if they want to keep their job
1
In reply to wbo:
Just like politicians, and commissioned pollsters then.
I went to an young eye expert the other week, she told me some bad news, I have the signs of the start of cataracts, however in the plus side, she predicted it may never bother me in my lifetime !
Really cheered me up.
> see Chris at 13:08 then - very good
> Honestly, to the OP, the examples you give of sales and delivery experts getting it wrong - they are not good examples of unbiased prediction are they? What they really believe may well not be what they say in public if they want to keep their job
Just like politicians, and commissioned pollsters then.
I went to an young eye expert the other week, she told me some bad news, I have the signs of the start of cataracts, however in the plus side, she predicted it may never bother me in my lifetime !
Really cheered me up.
1
In reply to tom_in_edinburgh:
Telling people what they want to hear - is that loyalty?
Mick
> You aren't getting paid for accuracy you are getting paid for loyalty.
Telling people what they want to hear - is that loyalty?
Mick
1
In reply to wbo:
Thanks - bit I missed that too. I'm going to have a beer and a bag of chips now
> see Chris at 13:08 then - very good
Thanks - bit I missed that too. I'm going to have a beer and a bag of chips now
In reply to Jim C:
But Rutherford expected the 'hit' to be a very unlikely event, so the result was exactly as expected...
> Rutherfords gold foil experiment did just that , and eventually his predicted result happened.
But Rutherford expected the 'hit' to be a very unlikely event, so the result was exactly as expected...
2
In reply to captain paranoia:
Ah ! I get you.
Like a future labour government.
> But Rutherford expected the 'hit' to be a very unlikely event, so the result was exactly as expected...
Ah ! I get you.
Like a future labour government.
In reply to captain paranoia:
that's one big brush and bucket of tar you have there !
Are you an expert on what makes hard science
> Yes, because politics and economics are not 'hard' sciences; they are more like branches of psychology.
that's one big brush and bucket of tar you have there !
Are you an expert on what makes hard science
In reply to lithos:
I'm with the captain, however deluded he is in his paranoia. 'Politics' is no science at all and economics as taught is nonsensical drivel.
I'm with the captain, however deluded he is in his paranoia. 'Politics' is no science at all and economics as taught is nonsensical drivel.
1
1
2
In reply to Bob Kemp:
Yes, I bet he never predicted that you would spell his name incorrectly.
> As Niels Boehr is supposed to have said, prediction is difficult, especially about the future.
Yes, I bet he never predicted that you would spell his name incorrectly.
In reply to Jim C:
I think most experts are well aware that the only certainty is uncertainty, that change is both normal and accelerating, and that predictions have a nasty habit of being derailed by unpredictable events.
i.e. I think most experts in a field would avoid giving any predictions. They know they are worthless.
Therefore it's not really a surprise that expert predictions, when given, turn out to be as fallible as non-expert predictions.
I think most experts are well aware that the only certainty is uncertainty, that change is both normal and accelerating, and that predictions have a nasty habit of being derailed by unpredictable events.
i.e. I think most experts in a field would avoid giving any predictions. They know they are worthless.
Therefore it's not really a surprise that expert predictions, when given, turn out to be as fallible as non-expert predictions.
1
In reply to Jim C:
What's an engineering expert? Presumably a scientist, or do you just mean someone who read the manual?
What's an engineering expert? Presumably a scientist, or do you just mean someone who read the manual?
> I work with , rely on, and respect lots of Engineering experts all day, IT,chemical , electrical, metalurgical , mechanical etc.
3
In reply to Jim C:
>Are you an 'expert' in your field?
I'm theoretically the world's leading expert in one narrow field, having conducted (with input from others), the pre-eminent piece of research into the topic.
However, my research is published in full online. So, in theory, anyone who reads it knows as much as I do.
>Are you an 'expert' in your field?
I'm theoretically the world's leading expert in one narrow field, having conducted (with input from others), the pre-eminent piece of research into the topic.
However, my research is published in full online. So, in theory, anyone who reads it knows as much as I do.
5
In reply to Philip:
some are scientists , others Chartered Engineers, some Professors, I have not asked for their CVs.
> What's an engineering expert? Presumably a scientist, or do you just mean someone who read the manual?
some are scientists , others Chartered Engineers, some Professors, I have not asked for their CVs.
In reply to Jim C:
He probably never said that. http://www.news.hypercrit.net/2012/11/13/einstein-on-misattribution-i-proba...
> Einstein said insanity was trying the same thing over and over and expecting a different result
He probably never said that. http://www.news.hypercrit.net/2012/11/13/einstein-on-misattribution-i-proba...
In reply to Philip:
Someone suitably qualified and experienced in their field of engineering? If I have an issue with a reciprocating compressor or have concerns over a surge issue in a normally dry pipeline i know who, within my company, to talk to as they are experts with these issues?
Is it really that difficult to understand?
> What's an engineering expert? Presumably a scientist, or do you just mean someone who read the manual?
Someone suitably qualified and experienced in their field of engineering? If I have an issue with a reciprocating compressor or have concerns over a surge issue in a normally dry pipeline i know who, within my company, to talk to as they are experts with these issues?
Is it really that difficult to understand?
2
1
In reply to Jim C:
Political experts are usually snake oil salesman... a classic employment definition oxymoron. The crooks who sell their political views in disguise do real experts a diservice. Even with the dreaded economists and polsters, the honest ones get a bad rap because crooked political experts misuse their results and never state the error levels and assumptions. As others have said predictions can be hard even in physical science and engineering.
Political experts are usually snake oil salesman... a classic employment definition oxymoron. The crooks who sell their political views in disguise do real experts a diservice. Even with the dreaded economists and polsters, the honest ones get a bad rap because crooked political experts misuse their results and never state the error levels and assumptions. As others have said predictions can be hard even in physical science and engineering.
1
In reply to Philip:
I suppose I am an engineering expert as in just the last few week three companies have asked for my help in solving problems their own well qualified and experienced engineers are having difficulty with. Everything is relative though as I would I am sure be able to find people with more expertise (or experience) than me if I looked to a company who's specific business it was.
Cheers
Toby
I suppose I am an engineering expert as in just the last few week three companies have asked for my help in solving problems their own well qualified and experienced engineers are having difficulty with. Everything is relative though as I would I am sure be able to find people with more expertise (or experience) than me if I looked to a company who's specific business it was.
Cheers
Toby
In reply to Chris Harris:
Probably too busy concentrating on remembering which way round the 'i' and the 'e' were in his first name...
Probably too busy concentrating on remembering which way round the 'i' and the 'e' were in his first name...
In reply to galpinos:
No wonder we're so far behind Germany in manufacturing and exports if people think an engineer is someone who's read the manual............
> Someone suitably qualified and experienced in their field of engineering? If I have an issue with a reciprocating compressor or have concerns over a surge issue in a normally dry pipeline i know who, within my company, to talk to as they are experts with these issues?
> Is it really that difficult to understand?
No wonder we're so far behind Germany in manufacturing and exports if people think an engineer is someone who's read the manual............
2
In reply to George Ormerod:
Yeah; everyone knows an engineer is the bloke who comes to fix the washing machine...
Yeah; everyone knows an engineer is the bloke who comes to fix the washing machine...
2
In reply to Philip:
I am one hell of a Bullshitter, thats like an expert isnt it?
Arent we all just winging it, just on different levels?
I am one hell of a Bullshitter, thats like an expert isnt it?
Arent we all just winging it, just on different levels?
2
2
In reply to Jim C:
I sort of am.
In fact I'd fairly confidently say I am the top expert in my field in Europe and possibly the world outside US.
My very niche and unimpressive field (despite facilitating hundreds of millions turnover for my clients).
Doesn't help me land a new, less shit job though!
I sort of am.
In fact I'd fairly confidently say I am the top expert in my field in Europe and possibly the world outside US.
My very niche and unimpressive field (despite facilitating hundreds of millions turnover for my clients).
Doesn't help me land a new, less shit job though!
1
In reply to Jim C:
My mum is an ecologist with a special talent for identifying grasses and sedges. I think she is genuinely an expert in most fields.
(This is actually true, and not, for once, a cheap contrived pun)
My mum is an ecologist with a special talent for identifying grasses and sedges. I think she is genuinely an expert in most fields.
(This is actually true, and not, for once, a cheap contrived pun)
Post edited at 17:16
5
In reply to Jim C:
The best definition of an expert that i've come across is that X = an Unknown Quantity, and 'Spurt' = a Pressurised drip
Oh Bugger! -- I just read through the earlier posts and someone has beaten me to it!
The best definition of an expert that i've come across is that X = an Unknown Quantity, and 'Spurt' = a Pressurised drip
Oh Bugger! -- I just read through the earlier posts and someone has beaten me to it!
Post edited at 17:48
1
In reply to captain paranoia:
Watch it ! We like to call ourselves Domestic Appliance Service engineers - note the lower case on engineer. A few years ago on our trade forum we had a guy who took umbrage at us calling ourselves engineers as he was a proper Engineer with many degrees and numerous letters after his name. It was agreed that he could proudly call himself an Engineer (with a capital letter). I assume he was an'expert' but he new bugger all about washing machines.
> Yeah; everyone knows an engineer is the bloke who comes to fix the washing machine...
Watch it ! We like to call ourselves Domestic Appliance Service engineers - note the lower case on engineer. A few years ago on our trade forum we had a guy who took umbrage at us calling ourselves engineers as he was a proper Engineer with many degrees and numerous letters after his name. It was agreed that he could proudly call himself an Engineer (with a capital letter). I assume he was an'expert' but he new bugger all about washing machines.
In reply to Removed Userjess13:
A typical bloke then
> Watch it ! I assume he was an'expert' but he new bugger all about washing machines.
A typical bloke then
In reply to Removed Userjess13:
Well, in that case, I'm both an Engineer and an engineer; I design electronic stuff for a living, and fix my washing machine at home...
Well, in that case, I'm both an Engineer and an engineer; I design electronic stuff for a living, and fix my washing machine at home...
In reply to Jim C:
Expert is a much abused term. I get called an expert in my field, though certainly don't feel I'm one, not when I compare myself with those who I would regard as genuine experts in my field.
I think the term should be reserved for the very few people who really are expert. The rest of us are just journeymen. How one differentiates between the expert and journeymen is another question.
Expert is a much abused term. I get called an expert in my field, though certainly don't feel I'm one, not when I compare myself with those who I would regard as genuine experts in my field.
I think the term should be reserved for the very few people who really are expert. The rest of us are just journeymen. How one differentiates between the expert and journeymen is another question.
In reply to kipper12:
Easy, anyone who agrees with us is an expert, and everyone else is an idiot
> I think the term should be reserved for the very few people who really are expert........ How one differentiates between the expert and journeymen is another question.
Easy, anyone who agrees with us is an expert, and everyone else is an idiot
In reply to Jim C:
Years ago I used to work for an ex army chap (and I use the word advisedly) with a background in military intelligence in Cyprus. He offered this about experts: "x" is conventional notation for something unknown; a spurt is a jet of fluid (it's helpful to hear it in the full Glasgow Academy plummy accent) therefore an expert is an emission of an unknown fluid.
What did he do in Cyprus? Another of his old army sayings was if you don't think you're going to like the answer, don't ask the question......
Years ago I used to work for an ex army chap (and I use the word advisedly) with a background in military intelligence in Cyprus. He offered this about experts: "x" is conventional notation for something unknown; a spurt is a jet of fluid (it's helpful to hear it in the full Glasgow Academy plummy accent) therefore an expert is an emission of an unknown fluid.
What did he do in Cyprus? Another of his old army sayings was if you don't think you're going to like the answer, don't ask the question......
In reply to colinakmc:
>Another of his old army sayings was if you don't think you're going to like the answer, don't ask the question......
Similar to the lawyer's saying, never ask a question( in court) that you don't already know the answer to.
>Another of his old army sayings was if you don't think you're going to like the answer, don't ask the question......
Similar to the lawyer's saying, never ask a question( in court) that you don't already know the answer to.
This topic has been archived, and won't accept reply postings.
Elsewhere on the site
Fri Night Vid Finding Focus - Life Behind The Lens of a Climbing Photographer
This week's Friday Night Video is a portrait of a prolific climbing photographer from Wedge Climbing. Sam Pratt is well known in both the outdoor and competition scene but if you haven't heard of him, you've likely seen...
26 Apr
Loading Notifications...