UKC

Grade/stars voting on UKC

New Topic
This topic has been archived, and won't accept reply postings.
 The Pylon King 14 Jun 2011
Do people think that the votes shown on the UKC routes database are worth taking into account when preparing a new (non Rockfax) guide?
In reply to Pylon King:

No, 95% of people are idiots whose opinions should be disregarded.
 Monk 14 Jun 2011
In reply to Pylon King:

I would always consider any source of data, but some sources carry more weight than others. I think that the UKC databases can be quite useful, especially on less popular routes or newer areas where a consensus has not developed in the same way as it might at Stanage. I would also think that comments on higher grade routes may carry more weight than comments on easier routes - people climbing higher grade routes are more likely to have more experiene allowing them to rationally comment on a grade.

I suspect that the UKC databases are actually a very good source of data when treated cauiously - certainly better than asking your mates who have been climbing in an area for years and have done a route 20 times before - it'll always feel easy to them.
 Tom Last 14 Jun 2011
In reply to Pylon King:

Not in so far as new routes are concerned. People nearly always seem to star their FAs on here, but then since few get early repeats I suppose there won't be many votes anyway.
In reply to Southern Man:

no not new routes but relatively recent (last 10 years)
 Simon Caldwell 14 Jun 2011
In reply to Pylon King:
> Do people think that the votes shown on the UKC routes database are worth taking into account when preparing a new (non Rockfax) guide?

Yes, if only to highlight routes where there is a marked disagreement with the current grade/stars, so that these can be double-checked.
 Offwidth 14 Jun 2011
In reply to Pylon King:

I find the average grade voting a bit random to be honest in my areas of interest as a guidebook worker (mainly peak grit). Also where the help is needed most (on obscurities) there are often no votes. I think there are some clear tendencies to over-grade popular easier (in the grade) classics compared to well established consensus (maybe inexperienced climbers weighting the votes up?) and sometimes the opposite on unpopular routes which are hard for the grade (maybe hard core climbers making a point?). A really good feature is that some common posters have a clear, consistent view and make useful free text comments on routes where something is wrong (grade, description, hazards etc.): this is a very useful resource which I've used often to recheck routes or occasionally to try and help resolve a close team argument (not with much luck unfortunately but it was nice to have the opportunity to look).

Star votes are even less reliable IMHO. Even on the Rockfax star rating system (good routes normally get one star) people are giving one star routes (really nice climbs) far too many 3 star votes (should in some respects have national significance at the grade).

In reply to Offwidth:
>
> Star votes are even less reliable IMHO. Even on the Rockfax star rating system (good routes normally get one star) people are giving one star routes (really nice climbs) far too many 3 star votes (should be in some repects having national significance at the grade).

Agreed! It used to be that all routes were considered worthy unless stated otherwise in the description, then the really good routes got a star, the brilliant routes got two and those that were as good as any in the country got three. Now unless a route has at least a star it's considered unworthy and not worth bothering with. This also means that instead of having five grades of quality (black spot; no star; *; ** & ***) we now effectively have just four.

ALC



In reply to Pylon King:
> Do people think that the votes shown on the UKC routes database are worth taking into account when preparing a new (non Rockfax) guide?

The voting is incredibly useful and should certainly be taking into consideration.

You need to exercise a bit of caution though as others have pointed out, but a bell curve with 50+ votes is a solid indicator of public opinion.

We assume people tend to vote routes harder if the find them tough, and don't vote them easier with quite the same diligence if they find them easy. So our rule is 2/3 votes on a reasonable number (20+) votes for an upgrade, and just a simple consensus for a downgrade. Even then we don't always go with it since there are other factors at work.

The bounce effect (AKA the Three Pebble Slab effect) is also worth watching. When a route changes grades it tends to attract a lot of votes nudging it back the way it has come initially which can distort opinion. This would probably be lessen if we instigate a system to remember previous voting when a vote changes (which we have plans to do).

Alan
 Offwidth 14 Jun 2011
In reply to Alan James - UKC:

"but a bell curve with 50+ votes is a solid indicator of public opinion."

You need to go back to school as thats very unscientific at best and almost certainly often wrong. Firstly the sampling on a pure population would give significant errors in itself. Secondly the votes are not a pure sample due to several possible bias factors: as I've said before, easy routes in a grade get proportionally more inexperienced climbers (who usually won't vote down their first VS!) and brutes get more burly climbers. Thirdly, your vote increments are uneven: you can't vote for the position in the grade above and below (or vote two or more grades out!). Finally, I think a small minority of votors either lie or don't understand grading (some votes on routes with grade errors just don't make sense).

If I was you I'd look for 3/4 votes for an upgrade and 2/3 for a downgrade on a 50 vote sample or maybe use your current values when the votes reach a few hundred (The error scales approximately with the square route of n with no bias) .

Your 'bounce effect' is a well known factor in psychology. The norm reference value can be easily shown to influence votes. Take a borderline VS/HVS route and get two roughly equivalent sets of climbers to vote - one set where the route is given VS, the other set where the route is given HVS - the set with the VS grade norm WILL get more VS votes than the one that gets HVS as the norm.

In reply to Offwidth:

It is fairly safe to assume that voting patterns like this one - http://www.rockfax.com/databases/r.php?i=754 - are pretty solid endorsements of the grade. The RF and UKC databases are very good at giving a guidebook editor or writer feedback, especially for routes they themselves my not have climbed or did climb but it was a long time ago. Bel curves point to routes which are high in the grade and middle of the grade and are very useful

Of course you could choose to ignore this because it isn't scientific enough and give Mississippi Buttress HVS if you wanted to.

Alan
 Simon Caldwell 14 Jun 2011
In reply to Alan James - UKC:

This is where I normally point out that Maud's Garden was similarly endorsed at VDiff before it was upgraded to Severe in the last Western Grit
 Offwidth 14 Jun 2011
In reply to Alan James - UKC:

"Of course you could choose to ignore this because it isn't scientific enough" I think you are missing the point, if its not scientificly significant its not meaningful!

You said "a bell curve with 50+ votes is a solid indicator of public opinion" it's simply not, even without any bias. The 300 voted Mississippi Buttress is better but still may hide some problems.

A point I forgot to mention before relates to distributions: what is a 'bell curve' like, with variable standard deviation, on a voting stucture with 5 uneven quantised slots : (higher grade), high, mid, easy, (lower grade). Let's consider an idealised route as an easy HVS mean with a slightly wider standard deviation (than Mississippi) of about half a grade, on a bell curve, with your 300 votes (from Mississippi), with perfect votes (no psycholgical bias from having a stated norm). Hence, this could be distributed something roughly like 1 9 40 60 80 60 40 9 1, with each notch representing a third of a grade movement. Now place this honestly on your voting structure. With an HVS norm this would fit to votes of (10), 40 60, 80, (110). For a (not far) mistaken VS norm the distribution would be: (190), 60, 40, 9, (1). Neither are very bell like now (and the average shifts by about a three quarters of a notch or a quarter grade)!

Hence because Mississippi Buttress with its 300 votes and low standard deviation looks right and does match approximate consensus it doesn't mean other voting patterns will or do (it will however be likely more than twice as accurate as a sample set of 50 with the same standard deviation based on population issues). Mid grade routes 'sit' better on your low quantised structure they won't get the shape distortion/ distribution bias I described above. Mid grade routes also won't get a low or high grade bias due to the set of climbers voting (soft or hard respectively).

Your system IS a really good start and contains data not available elsewhere. Add a few more notches to the grades above and below, and the ability to keep votes when the grade changes, and it could start to give real accurate indications of the population view over time.




 Monk 14 Jun 2011
In reply to Offwidth:

> (In reply to Alan James - UKC)
>
> With an HVS norm this would fit to votes of (10), 40 60, 80, (110). For a (not far) mistaken VS norm the distribution would be: (190), 60, 40, 9, (1). Neither are very bell like now (and the average shifts by about a three quarters of a notch or a quarter grade)!
>

I'm slightly confused as to what you are getting at here - don't both these distributions end up with most people thinking that the route is HVS, hence you have a consensus, however you look at it? I'm pretty sure most of us can live with a quarter grade of fuzziness. In fact, I'm pretty sure I have no idea what a quarter of a grade feels like.
 Offwidth 14 Jun 2011
In reply to Monk:

That example shows the distortions possible with large voting numbers for the site (300) for the same route, just from being placed in two adjacent possitions, with no norm based psychological bias, with a sample assumed to match exactly the view of the wider population (ie ignoring sample errors), a nice idealised distribution and no-one voting in an inexperinced or silly way on a subjective quantity. The real combined errors could be obviously be much larger. My point is, that people look at these distributions without thinking properly about the statistics, subjectivity and psychology of the voting and get convinced they are OK. I posted to illustrate some reasons why you can never say a 50 vote sample is solid. I also suggested some ways Rockfax could improve things to get round some of the issues and allow a more reliable sample to build.

As for easily judging a quarter grade: compare Mississippi Buttress with the similarly popular neighbour Inverted V, according to large numbers of Rockfax votes; it feels harder to me! (cheating I know .

http://www.rockfax.com/databases/r.php?i=823
 Offwidth 14 Jun 2011
In reply to Monk:

....Or the quarter grade difference between Inverted V and Count's Crack being even sillier:

http://www.rockfax.com/databases/r.php?i=529

Those two show the extent of the softie climbers vs knarly climbers effect. What would you expect the distributions to be if exactly the same population set voted for Inverted V and Counts Crack. I think the 'true' seperation is anything from 3/4 to a whole grade.
 Monk 14 Jun 2011
In reply to Offwidth:

I'm still not entirely sure what you are getting at (but I haven't done Counts crack, and did both your other examples about 8 years ago). Of the two routes I've done, I would agree with the voting for grades (but not stars - both are vastly over-rated in my opinion). Missippi is just above average VS, and inverted V is easy but the polish on the lower crack adds to 'slip-off-ability'.

So basically, the routes are all within the VS spectrum, the voters mostly think that they are VS, and they are in the guidebooks as VS. I fail to see a major issue with the voting sytem, unless you are trying to suggest that we use 95% confidence intervals for every grade.



(For what it's worth, I think that I agree with what you are saying if we are talking about the results of medical trials or something important.)
 Offwidth 14 Jun 2011
In reply to Monk:

So you can judge between Inverted V and Mississippi Buttress and I suspect if you did Count's Crack you'd think it was a hardish climb for VS. These are on 50+ votes so the problems with smaller vote numbers would be larger and some full grade errors would be likely even on an 'honest' voting perspective. Hence a smallish group of experienced climbers that have some sensitivity at VS and have done the routes and average their opinions will still currently produce a more useful graded list than the popular Rockfax vote. Does this matter compared to the accuracy of medical trials? No. Does the Rockfax graded list always match the votes? No. Would I like to see the Rockfax voting improved? Yes.
 JimboWizbo 14 Jun 2011
In reply to Pylon King: We climbed a route graded VDiff in the guidebook, on here it's Severe, personally I wouldn't call it anything close to a severe!
 Monk 14 Jun 2011
In reply to Offwidth:

I think that I finally get it, and you've very nicely argued me into a corner.
 remus Global Crag Moderator 14 Jun 2011
In reply to Pylon King: Id say yes, its worth considering UKC votes in your appraisal of grades and quality, just bear in mind that they're likely to show a large sampling bias (people are more likely to vote if they think a route is hard/soft and less likely to vote if they think the grade is correct, so the votes are likely to be skewed in the direction of an upgrade/downgrade.)
 Offwidth 14 Jun 2011
In reply to JimboWizbo:

Route??
 JimboWizbo 14 Jun 2011
In reply to Offwidth: Beckermet slab at The Roaches upper tier. Main problem was rope drag as we were only using a single rope
 Ramblin dave 14 Jun 2011
In reply to remus:
Presumably the voting's also going to be weird if you have a route that's '4c if you know the trick' or a low grade route that relies on jamming or some other slightly nonobvious bit of technique...
 Rog Wilko 14 Jun 2011
In reply to Pylon King: Although I'm not very experienced in guidebook work I'd have thought that while the voting numbers would give you a nudge towards re-considering (not necessarily changing) the grade, a few well-reasoned comments from climbers with more than a couple of pages in their logbook would carry more weight. Personally speaking, if I consider a route wrongly graded I won't just vote, but will give my reasons, and I'd like to think that is of some value.
I wonder how far the nature of the techniques required can influence people's views? The obvious example would be something like The File, which will seem very hard to someone fresh off a climbing wall.
 James Moyle 14 Jun 2011
In reply to Pylon King: I think the grade votes are frequently right if enough people have voted. Having said that, this climb http://www.ukclimbing.com/logbook/c.php?i=32105 is never HVS 5a (Soft VS 4c IMO) but as you can see from the comments, its lots of peoples' first HVS who a) won't have anything to compare it and b) are unlikely to give it VS if it is their first HVS. Therefore, it still has a number of votes for HVS.
In reply to James Moyle:

The voting on that one shows that most people think it is borderline VS/HVS which it probably is - a bit like the HVS's at Central area Shorn cliff.
When i did it recently i thought it was VS and would put it in as such if i was writing a new guide for Portishead.
 dave frost 14 Jun 2011
In reply to Pylon King: you may also want to remove all the stars the shorncliff routes get, its silly. Motion pictures in particular is not one of the best e1's in the south west now is it.

Cheers
Dave
 Ramblin dave 14 Jun 2011
In reply to Rog Wilko:
> (In reply to Pylon King) Although I'm not very experienced in guidebook work I'd have thought that while the voting numbers would give you a nudge towards re-considering (not necessarily changing) the grade, a few well-reasoned comments from climbers with more than a couple of pages in their logbook would carry more weight.

Yeah, I would have assumed that too. Or even just making an executive decision that no, I can see why people might think that but they're wrong.
In reply to dave frost:

I agree theres a lot of 1 and 2 star routes there but not many 3 stars.
 James Moyle 14 Jun 2011
In reply to Pylon King: I count 7 3* routes which I don't think is all that much. The problem is that the HVS routes on the Giants Cave Area are all quite similar on superb rock so if you think one is worth 3*, then they all should be. IMO these routes are correct. Not done Tigers, Lundy Calling or Motion Pictures so can't comment on these. I think Shorn is the best crag in the Wye Valley by some way and so should have a decent number of 3 star routes.
In reply to James Moyle:

None of the 3 star routes i have done at Shorn Cliff are worth three stars, two and a half maybe but not classics of the country. Generally in the UK I think three stars are given out to easily and there are not enough one stars given out.
 Monk 15 Jun 2011
In reply to Pylon King:

I haven't done Lundy Calling or Motion Pictures, but I have done all the other 3 star routes at Shorn Cliff. Tigers is a strong line, but may not be worth 3 stars but is worth 2 (but I cut my teeth on grit, so a limestone climber might have a more memorable experience on it). I think that all the routes on the Musketeers slab are top quality routes and definitely deserve 3 stars each - I couldn't pick a favourite from them as they all have something different to offer. I don't think that there are an unreasonable number of starred routes at Shorn Cliff - it is a great crag, after all.
 Rog Wilko 15 Jun 2011
In reply to Pylon King:
> (In reply to James Moyle)
>
Generally in the UK I think three stars are given out to easily and there are not enough one stars given out.

I probably agree with that. I think a good case can be made for handing out one star for any route which is reasonably enjoyable, as too many no-star routes never get done, unjustly, and to the detriment of the routes. The only thing worse than routes getting too much traffic is getting too little and reverting to nature.
Then you'd probably need to introduce 4 stars for the national classics (of which most crags wouldn't have any).
This would leave no-star routes of no real quality only to be done by jaded locals or those who've done everything else loads of times.
 Offwidth 15 Jun 2011
In reply to Rog Wilko:

One distinction here is that Rockfax use a system of good, very good and excellent whereas most definitives use a system of very good,excellent outstanding or a significant route for the crag, area, country. This is an issue as its unclear to some if the UKC stars provide an independent definitive record (using the relevant definitive guide system) or a Rockfax database (using the Rockfax system).

Most definitive no star routes are very worthwhile and this is clearly indicated in the introductory text. Rockfax seem more consistent with star ratings than some definitives (especially some older ones where a crag author can be very mean or confetti like with stars)
 Ramblin dave 15 Jun 2011
In reply to Offwidth:
And I suppose Rockfax then have their top 50 routes as (roughly) equivalent to the 'definitve' three star routes? Although that depends on the area that the guide covers, obviously.

Although it seems nicer to assume that all routes are worth a go until proven otherwise - the Rockfax system does kind of encourage you to ignore anything without a star...
 john arran 15 Jun 2011
In reply to Ramblin dave:

In my ideal world every route would have a quality rating just like every route has a grade. We've become used to using stars for quality rating but the rockfax dynamite stick is a kind of rating too, as is a lack of stars on particular routes and also as are lists of recommended routes - albeit less usefully in a separate part of the book from the routes themselves. Rating only a small number of the best routes misses an opportunity to help people decide which routes to do, which is what the guidebook is there for.

By this logic the most helpful strategy would be to have a roughly equal number of routes with each quality rating, although of course this cannot be possible for all guidebooks without losing the ability to compare quality ratings from different guidebooks
 Simon Caldwell 15 Jun 2011
In reply to Rog Wilko:
> Then you'd probably need to introduce 4 stars for the national classics

The latest Scottish guidebooks all do this.

> too many no-star routes never get done, unjustly, and to the detriment of the routes

That's always going to be the way though. It's the same with crags, some never get visited, unjustly, to the detriment of the crags.

Sometimes people have tried to address this by missing out stars (Tremadog) or by allocating too many (Langdale). Neither approach seems to have made much difference, people mostly visit the same crags and do the same routes.

At least the no-star routes mean that there's usually something to do without a queue on it.
In reply to Rog Wilko:

I think three stars is plenty:

0 stars = nothing special but ok

1 star = a route i would recommend to somebody else

2 stars = a route of local significance

3 stars = a route of national significance
In reply to Pylon King:

That's the way I see it too. But it seems that we are in a minority since the majority assume that unless something is marked as being good (i.e. one star or more) then it isn't worth doing.

These are often the same people who defend stars as helping them use their precious leisure time to queue behind all the other people using their leisure time on the same few routes

I think the Tremadog "experiment" was fine, just the subjects needed sorting out!!

ALC
 PaulHermes 15 Jun 2011
In reply to Pylon King:
I think it should be extended to 5XXXs just like the hotel system
 Tom Last 15 Jun 2011
In reply to Pylon King:

Out of interest, is 'Definitive E0' about to hit the shelves?
 Simon Caldwell 15 Jun 2011
In reply to Pylon King:
plus a black spot for poor climbs presumably.

Most guides have this, but it's rarely used, so all the 'nothing special but OK' routes are given the same rating as all the dross. Perhaps this partly explains the perception that no stars = bad?
 Rog Wilko 15 Jun 2011
In reply to Toreador: Agree there should be more black spot routes. The purpose would be to say "you'll almost certainly not enjoy this route, but we're listing it just to save having people claiming a new route". This might help people to choose to do more no-star routes, if they know there might be worse.
 Rog Wilko 15 Jun 2011
In reply to Toreador:
> (In reply to Rog Wilko)
> [...]
>
> >
> Sometimes people have tried to address this by missing out stars (Tremadog) or by allocating too many (Langdale). >

Personally, I think the "excessive" starring in Langdale has encouraged me to do routes I'd not seriously considered previously. It certainly got me on to a number of routes on Gimmer's "North West Frontier" which I have thoroughly enjoyed.
In reply to Toreador:
> (In reply to Pylon King)
> plus a black spot for poor climbs presumably.

Yes good point, there should be more black spots for crap and dangerous climbs ( i know somewhere that could do with quite a few of these!)

In reply to Pylon King: Is this a good time to mention those starred routes on Cleeve Hill again.

Al
In reply to Southern Man:
> (In reply to Pylon King)
>
> Out of interest, is 'Definitive E0' about to hit the shelves?

I presume you mean Classic E0?

It's been a bit delayed as we are working on Classic Esoteria as well.
In reply to Gaston Rubberpants:
> (In reply to Pylon King) Is this a good time to mention those starred routes on Cleeve Hill again.
>
> Al

Nowt to do with me - the problem of having too many people working on the same guidebook!
 Andy Moles 15 Jun 2011
In reply to Pylon King:

I wonder do guidebook writers other than Rockfax look at the voting and comments on here? If I knew they did I would probably leave more comments rather than just voting. Some routes I've done recently in quieter spots in Scotland have been way off.
In reply to Andy Moles:
> (In reply to Pylon King)
>
> I wonder do guidebook writers other than Rockfax look at the voting and comments on here? If I knew they did I would probably leave more comments rather than just voting. Some routes I've done recently in quieter spots in Scotland have been way off.


Yes they do!

And comments both here and in the guide are more use than zero to three little symbols

In reply to Andy Moles:

I do
 Ramblin dave 15 Jun 2011
In reply to Pylon King:
How long before physical guidebooks are supplemented by phone apps that plug in to the UKC database and let you see the state of the voting and the dozens of comments saying "awful" and "S 4a? More like HVS 5b!" before you get on a route?
 Simon Caldwell 15 Jun 2011
In reply to Andy Moles:
> I wonder do guidebook writers other than Rockfax look at the voting and comments on here?

I do. Sadly, none of the routes on the crags I'm writing up have had more than one or two ascents logged, most have had none at all.
 sammycolon 15 Jun 2011
Stars seem to be for routes within the crag a lot of the time.

A 3 star route at a really good crag is close to proper 3 stars.
A 1 star route at a pants crag rarely deserves a mention with respect to quality of route, it's just a little better than the other routes at the pants crag so stands out and hence gets 1 star.

Trouble is people don't look at a climb and star it in comparison to the whole country (or dare I say world?). They compare it to a narrower field of vision. Whether that be just rock type, geographic location, crag or pesonal experience.

And for the record, flying butress direct is E0. And a classic, for stanage popular on the right day.


 James Moyle 15 Jun 2011
In reply to Pylon King:
> (In reply to James Moyle)
>
> None of the 3 star routes i have done at Shorn Cliff are worth three stars, two and a half maybe but not classics of the country. Generally in the UK I think three stars are given out to easily and there are not enough one stars given out.

The problem is that starring routes is even more subjective than grading the difficulty. Should Skeleton Ridge be given 3 stars or a black spot?

http://www.ukclimbing.com/logbook/c.php?i=38248

I think more attention should be given to the star voting than the grade voting. Unless people are grading a climb that is within a grade or two of thier limit, their is a good chance of them making a mistake. But saying how much pleasure they gained from a route is much more straightforward. Maybe the criteria should be something like:

3* - One of the best climbs I have ever done
2* - One of my favourite climbs of the year
1* - A pleasant climb
0 - OK
Black Spot - repuslive
 sammycolon 15 Jun 2011
In reply to Toreador:

Is that in the UK?

I see your a yorkshire mod, does attendance to the crag correspond to logged climbs in your opinion or are the people who log climbs biased in one way or location or something?

Ta
 Ramblin dave 15 Jun 2011
In reply to James Moyle:
> (In reply to Pylon King)
> [...]
>
> The problem is that starring routes is even more subjective than grading the difficulty. Should Skeleton Ridge be given 3 stars or a black spot?
>
> http://www.ukclimbing.com/logbook/c.php?i=38248
>
> I think more attention should be given to the star voting than the grade voting. Unless people are grading a climb that is within a grade or two of thier limit, their is a good chance of them making a mistake. But saying how much pleasure they gained from a route is much more straightforward. Maybe the criteria should be something like:
>
> 3* - One of the best climbs I have ever done
> 2* - One of my favourite climbs of the year
> 1* - A pleasant climb
> 0 - OK
> Black Spot - repuslive

I'd go for something like the michelin stars approach - it wouldn't be irrational to:
go to the area because you want to do a three star route
go to that crag if you're in the area because you want to do a two star route
make sure you do a one star route if you're at the crag.
knock off the no-star routes if you've run out of one star routes

I suppose this is basically the old idea of national and local 'significance'...
 mark20 15 Jun 2011
3* - polished and over rated
2* - over rated
1* - good route
0* - usually worthwhile

I think it's a dangerous game giving routes 3* and hyping them up, they rarely live up to expectations. I can only think of one 3* route that I can place above the majority of 2* routes, but if I was to tell you which route then that would ruin it!
 Simon Caldwell 16 Jun 2011
In reply to sammycolon:
> Is that in the UK?

Yorkshire

> does attendance to the crag correspond to logged climbs in your opinion

Certainly on the crags that I'm writing up. Very few climbs logged (on one of the crags there were none logged until we started route checking), and little or no sign of anyone climbing there (eg no tracks, no polished or even marked rock, etc).
 Stuart S 16 Jun 2011
In reply to Andy Moles:

> I wonder do guidebook writers other than Rockfax look at the voting and comments on here? If I knew they did I would probably leave more comments rather than just voting. Some routes I've done recently in quieter spots in Scotland have been way off.

I've got a chapter to update for the next edition of the SMC Northeast Outcrops guide and will definitely be taking the voting on here into account, so if you've any thoughts on routes in our quiet corner, please make your opinions known!
 Tom Last 16 Jun 2011
In reply to Pylon King:

Classic E0

That's the one - looking forward to it!
 Chris Craggs Global Crag Moderator 16 Jun 2011
In reply to Pylon King:
> Do people think that the votes shown on the UKC routes database are worth taking into account when preparing a new (non Rockfax) guide?

Precisely what to do with the information would be up to any individual crag writer but to ignore the huge wealth of data available would be perverse.

Chris
 Mick Ward 16 Jun 2011
In reply to mark20:

> 3* - polished and over rated
>
> I can only think of one 3* route that I can place above the majority of 2* routes...

Of course there are genuine three star classics. For instance, a few days ago, I did King Kong in the Wye Valley. A slippery start, loose bits, vegetation, etc, etc. And an undying three-star classic. A route I'll always remember.

Who could forget the poised rib on Astra? Or launching out across the flakes of Sloth or Quietus? (I've no idea what stars these routes get but they seem three star to me.)

Three star classics are with you forever . The memories become part of you.

Mick



 Rog Wilko 16 Jun 2011
In reply to Stuart S: Noticed that the lengths of some of the routes in Glen Clova are greatly exaggerated.
 Jon Stewart 16 Jun 2011
In reply to Pylon King: My view here:

http://www.rockfax.com/databases/r.php?i=22340

I think star ratings are good. On lesser climbed of vegetated crags, they often translate to "climbed often [because we gave it 3*] - clean - worth doing" to "no longer detectable as a route". Concentrating the traffic on a few lines can be a good thing.

I don't generally enjoy 0* routes, they're often ledgey, overly crux, dirty. I have to admit that if a route's hard, I like the holds to be chalked and the gear placements a bit scratched...
 Glen 16 Jun 2011
In reply to Pylon King:

I think it's pretty clear from a statistical point of view that voting on grades is unlikely to be accurate.

As I think Offwidth was saying, there are a wide range of likely biases that mean that a simple mean of the votes won't be a very good measure of how hard the route actually is.

There are two major sources of bias I can think of:

1. The 'ego bias' - people are unlikely to vote down grades that are close to their limit. This is likely to mean that the mid grades where most people are operating get inflated.
2. The 'experience bias' - the higher (and possibly lower) the grade of a climb relative to a climbers maximum grade, the fewer climbs they will have experienced at that grade (probably), although what this leads to in terms of voting behaviour isn't clear.


There are also problems with accuracy of course, particularly higher in the grades, or a unfrequented crags, simply due to the small sample size, but others have mentioned this above.



That all said, the above assumes a linear grade system. Who ever said it was linear? As long as most VDs are harder than most diffs, most E3s harder than most E2s etc, does it really matter what the step size is?
I suppose any grade inflationary issues will affect popular climbs more than others, leading to grading differences with popularity, which obviously shouldn't happen.
 Jon Stewart 16 Jun 2011
In reply to Glen: Well said. I like the fact that a hard HVS is harder than a soft E2. It's all part of trying to being objective with something as subjective as rating the difficulty of trad climbs.
 Offwidth 16 Jun 2011
In reply to Chris Craggs:

As I've pointed out above, to believe the vote data given sampling errors, quantisation distortion, voter bias and different climbers voting would also be statistically perverse. Do you seriously think most climbers voting on Inverted V as borderline low/mid-grade VS would think Counts Crack is mid VS?? These are not obscurities with a handful of votes. As an eastern edges editor, (where you have more votes than anywhere else) when our small but very experienced team view disgreed with the votes (we only knew because many of us are information obsessives) we always trusted our instincts.

The 'huge wealth' is also largely on known knowns, they don't help the likes of Toreador or the routes most likely to be misgraded.

To improve the voting reliability you need to add more and uniform steps (ie allow votes for easy mid high for grades above and below). Allow old data to stay when the grade shifts. An even better feature would be to 'Tag' voters (anonomously) so you can see those voting mis VS for Counts also show up as voting high HS for Inverted V.

The free text comments on UKC and Rockfax on the other hand have helped me many times. Please, please UKC contributors continue to use this facility anytime, anything looks odd.
 Chris Craggs Global Crag Moderator 16 Jun 2011
In reply to Offwidth:

Eh?


Chris
 remus Global Crag Moderator 16 Jun 2011
In reply to Chris Craggs: I think he's trying to say the voting is pretty much worthless because there are too many biases present.
 Andy Moles 16 Jun 2011
In reply to Stuart S:

It was more the NW than the NE I'm afraid!
 Offwidth 16 Jun 2011
In reply to remus:

"I think he's trying to say the voting is pretty much worthless because there are too many biases present."

Not quite. The average on large vote numbers gives you a good idea of a grade to within say a third of a grade either way. This tells you an approximate position of popular (mainly lower grade) climbs that you know are already well graded. For placement of these climbs on a graded list, I'm saying a small team sensitive to differences at that grade will give you a better list than the votes will at present. If the voting system improves significantly the balance may change.

The voting is only a useful guide when you already know the answer! When you dont know the answer the errors make the vote information unreliable.
 Bulls Crack 16 Jun 2011
In reply to remus:
> (In reply to Chris Craggs) I think he's trying to say the voting is pretty much worthless because there are too many biases present.

Or maybe he's taking it all far too seriosuly?!
 Jon Stewart 16 Jun 2011
In reply to Offwidth: I've never understood the point of a graded list, it just makes more plain than ever how silly the grades are, e.g. Pot Black harder than The Tippler, The Toy and Smoke ont' Watter. Does anyone really think that's right?
 Offwidth 16 Jun 2011
In reply to Bulls Crack:

I'm a scientist and an editor whaddya expect???

My seriousness is only in response to Alan's original point that 50 votes gives a solid grade (where I'd stick a 1/3 grade error bar on it). I can even predict from the nature of the route where grade biases will affect routes I've not done. Knarly moorland VS with 30 votes for top end VS with a big walk in will end up as HVS on Offwidth. Popular accesible starter route with 200 votes given low HS will be top end S on Offwidth.
 Offwidth 16 Jun 2011
In reply to Jon Stewart:

I'm pretty sure I'd agree with you from talking to many climbers. On Offwidth any I've done will be the right way round; but sadly knarly E1s let alone E2s are too hard for me at the moment. To be fair comparing bold routes with safe brutes is always tricky and it's unwise to underestimate consequencies of people at their limit on bold onsights.

As Chris rightly pointed out earlier, top end brutes (like Sentinal Crack) are still prominent in the definitives. Its easier as a punter editor to argue playing safe on bold routes you haven't done than safe ones, when you suspect stiff grading. Bottom line you raise the issues but trust the team decision and accept any subsequent flack as part of the game.
 Stuart S 16 Jun 2011
In reply to Rog Wilko:
> (In reply to Stuart S) Noticed that the lengths of some of the routes in Glen Clova are greatly exaggerated.

You can always click on the "Edit climb" button and submit a correct route length for any you think are particularly far out. Anything that helps improve accuracy would be appreciated!
 Monk 16 Jun 2011
In reply to all:

Is anyone else enjoying the irony of Rockfax resources being used by definitive guidebook editors, given the accusations of plagiarism that Rockfax often receive?
 Offwidth 16 Jun 2011
In reply to Monk:

Very cheeky! Plagiarism is copying of others written material without permission or acknowledgement. That's very different from looking at the public information: why would you compare the two?
 Simon Caldwell 16 Jun 2011
In reply to Offwidth:
> The voting is only a useful guide when you already know the answer! When you dont know the answer the errors make the vote information unreliable.

which is why (as I said at the top) I'd probably only use the voting to highlight routes that could do with another look.
Though that's academic when the only votes for a crag were made be me
 Chris Craggs Global Crag Moderator 16 Jun 2011
In reply to Pylon King:
> Do people think that the votes shown on the UKC routes database are worth taking into account when preparing a new (non Rockfax) guide?

All I said before was that the info it is a resource - to ignore it would be silly, as would to take it as Gospel - no more no less.


Chris

PS Loving the way such a body of info is it is so easily dismissed by some - the 'we know better' brigade maybe!
 Ramblin dave 16 Jun 2011
In reply to Pylon King:
Slightly tangential - one example of the sort of route where a guidebook writer might reasonably ignore the popular vote is a 'traditional' thrutchy low grade gritstone offwidth - the rationale being that it might seem near impossible for a beefy plastic pusher who expects a series of small but well defined holds to crank between, but to a well rounded climber who knows their 'traditional techniques' it's a walk in the park.

But is there a point where you say that almost everyone who tries it finds it hard, so maybe the people who can get up it easily are the weird specialists and it actually does need an upgrade?
 Offwidth 16 Jun 2011
In reply to Chris Craggs:

"Loving the way such a body of info is it is so easily dismissed by some the 'we know better' brigade maybe!"

I'm saddened that you use pejorative rhetoric to dismiss logical argument. You may as well ignore the votes if it almost never helps (currently if it tells you anything significant you should know already but with more accuracy). I admit my weakness is that I can't help but look anyway and that I do look forward to a time when the data improves.
 Monk 16 Jun 2011
In reply to Offwidth:
> (In reply to Monk)
>
> Very cheeky! Plagiarism is copying of others written material without permission or acknowledgement. That's very different from looking at the public information: why would you compare the two?

Obviously, I am only being lighthearted. I realise that this is not plagiarism, but there are always accusations of Rockfax being unable to exist without the hard work done by the definitive teams etc etc. bandied about every time they release a new guide. It tickles me to think that they are now the holders of such a wealth of information which provides a useful resource for us all (including definitive guide writers).
 Chris Craggs Global Crag Moderator 16 Jun 2011
In reply to Offwidth:
> You may as well ignore the votes if it almost never helps (currently if it tells you anything significant you should know already but with more accuracy).

See you are doing it again.

if a route gets 10 votes for HVS 5a, you can bet it isn't VS 4c or E1 5b so the votes do help. Your dismissiveness of the system and its users is laughable.


Chris
In reply to Pylon King:
> Do people think that the votes shown on the UKC routes database are worth taking into account when preparing a new (non Rockfax) guide?

So is that a yes or no then?
 Offwidth 18 Jun 2011
In reply to Chris Craggs:

If a route gets 10 votes and 10 votes only for HVS 5a it could well be VS 4c or E1 5a as a consensus, even though its most probably HVS 5a

Just in mathematical terms this is a sample of 10 values of a very large distribution (the views of all climbers who have climbed the route). If you can define the distribution you can even calculate the probabilitity of a mis-grade happening. In addition to this, we are dealing with a subjective assesment where norm bias is clear (the known psychological influence of the grade given) and other likely bias is possible (as an example most people I know climbing hard-for-the-grade moorland grit classics are simply not grading in the same way as the mean of the large sample that put Inverted V a quartile up the VS band).

Your apparent misunderstanding of the nature of statistics, the foibles of subjective grading and the point that I'm making that these are probabilities (rather than certainties) baffles me.
 Offwidth 18 Jun 2011
In reply to Monk:

Rockfax did piggy-back on the guidebook work of others; this didn't stop their existence but it did make producing some guides a lot easier. They argue they have as much a right to the information provided by past climbers as the definitive producers do. Rockfax were hardly inconsistent in this argument when they put their databases in the public domain.

The future will tell if a free market in guidebook production affects the long-term availability of good definitive information. At present the definitives have got better, not worse and Rockfax provide a good alternative source of information (that could be improved further if they listen). On the other hand the production costs and market pressures for competitive definitive guidebook producers have increased and there are some very unhappy hard working definitive volunteers in some parts of the UK and Europe.
 Chris Craggs Global Crag Moderator 18 Jun 2011
In reply to Offwidth:
>
>
> Your apparent misunderstanding of the nature of statistics, the foibles of subjective grading and the point that I'm making that these are probabilities (rather than certainties) baffles me.

But probably less than your belief in 'the one true grade' baffles me.


Chris
 Offwidth 18 Jun 2011
In reply to Chris Craggs:

Seems to me Alan's original point and your view that 10 votes for HVS 5a means HVS 5a is more "one true grade" than anything I said.

My main arguments were around how much new useful information the UKC/Rockfax votes currently provide to the a local expert writing up a crag for a definitive guide. I stated that this is normally very little, gave my reasons and provided examples. I will repeat them briefly to be clear: obscure routes get few or no votes; the grades given influence the votes; bottom half grade popular routes get graded easier than top half of the grade unpopular brutes; the quantised vote system you use distorts the results of borderline routes; any clear vote message that is in the votes any definitive author should already know.
 Offwidth 18 Jun 2011
In reply to Chris Craggs:

Ive just found a route that fits your criteria...as I'd give it VS 5a even though I think its spot on the border of HVS. Pedestal Crack: 35 out of 37 votes for mid HVS and only 2 for low HVS. This has good gear and a rest before you enter the crux upper groove. Compared with other tough VS climbs its the same for me as something like The Grazer or Lone Tree Groove, easier than Broken Crack. Two Pitch Route, Birthday Crack.

Just for you here are a few more of my favorites:

http://www.rockfax.com/databases/r.php?i=825 mid grade HS but should be HS/S border IMHO.

http://www.rockfax.com/databases/r.php?i=1706 votes mid grade VS; should be borderline VS/HS

http://www.rockfax.com/databases/r.php?i=1704 Votes M should be Hard Diff.

http://www.rockfax.com/databases/r.php?i=1719 Easy Severe should be easy HVD.

 Chris Craggs Global Crag Moderator 18 Jun 2011
In reply to Offwidth:
>
>
> Ive just found a route that fits your criteria...as I'd give it VS 5a even though I think its spot on the border of HVS. Pedestal Crack: 35 out of 37 votes for mid HVS and only 2 for low HVS. This has good gear and a rest before you enter the crux upper groove.

I think Pedestal Crack is middling HVS, so I agree wholeheartedly with the voting. Did it ever occur that maybe it is your 'system' that is broke?


Chris
 Offwidth 18 Jun 2011
In reply to Chris Craggs:

Yes it could be my view thats wrong (based on my ascents, the folk I watch and those I work with on the guides). I doubt it though; even on grade logic somethings up with Pedestal at mid grade HVS: a protected mid-grade HVS with a short crux would be 5b.

You're also neatly avoiding the comparitive question again... you really think Pedestal is half a grade harder than Birthday Crack, Two Pitch Route and Broken Crack? Most solid VS leaders I've known or seen appear to find find Pedestal comparable or easier.

You also still haven't answered the similar question about if you believe the quarter grade between the vote avareges of Inverted V and Count's Crack.
 Monk 18 Jun 2011
In reply to Offwidth:
> (In reply to Chris Craggs)
>
> Pedestal Crack... Compared with other tough VS climbs its the same for me as something like The Grazer or Lone Tree Groove, easier than Broken Crack. Two Pitch Route, Birthday Crack.
>

I guess your list shows how difficult it is to pin down grades. I wouldn't say the grazer is tough for a VS!
 Chris Craggs Global Crag Moderator 18 Jun 2011
In reply to Offwidth:

>
> Yes it could be my view thats wrong (based on my ascents, the folk I watch and those I work with on the guides). I doubt it though; even on grade logic somethings up with Pedestal at mid grade HVS: a protected mid-grade HVS with a short crux would be 5b.
>

Well either you (and your cabal) are 'wrong' or the 70+ folks who have voted on UKC for mid-HVS plus the other 70+ on RockFax (let me guess they are the same 70?) are 'wrong'.

> You're also neatly avoiding the comparitive question again... you really think Pedestal is half a grade harder than Birthday Crack, Two Pitch Route and Broken Crack? Most solid VS leaders I've known or seen appear to find find Pedestal comparable or easier.

Maybe the grades are wider than you believe?

> You also still haven't answered the similar question about if you believe the quarter grade between the vote avareges of Inverted V and Count's Crack.

Quarter grade? Tell me you are not serious.


Chris
 Bulls Crack 18 Jun 2011
In reply to Offwidth:
> (In reply to Chris Craggs)
>

>
> Just for you here are a few more of my favorites:
>
> http://www.rockfax.com/databases/r.php?i=825 mid grade HS but should be HS/S border IMHO.
>
> http://www.rockfax.com/databases/r.php?i=1706 votes mid grade VS; should be borderline VS/HS
>
> http://www.rockfax.com/databases/r.php?i=1704 Votes M should be Hard Diff.
>
> http://www.rockfax.com/databases/r.php?i=1719 Easy Severe should be easy HVD.

Apart from the diff is there any ppint in trying to differentiate the tiny differences in grade in these these routes?
 Offwidth 18 Jun 2011
In reply to Chris Craggs:

Your numbers are 35/37 on Rockfax and 78/84 on Rockfax (and Im sure there is a good overlap). The vote average for a route with good pro placed from a good half-rest position that then involves a crux with a few metres of easing layback moves, should not end up from grade logic as mid HVS, mid 5a. That to me is the layback on Great North Road territory. Pedestal is however a route that gets a lot of traffic from less experienced HVS leaders, being easy for the grade and protectable where it counts on a popular edge. The psychological grade bias from those using your guide will also push to mid HVS (thats the graded list position). I've naughtily told a few improving leaders it was a VS and they though: yep, fairly tough but not unreasonable. So I'm sticking with my grade view as low HVS (just above the border) and mid 5a (plus or minus a bit on each). It honestly feels VS to me but most of the views Ive had seem to average out at around easy HVS. Broken Crack just across the way often leaves the same climbers exasperated and saying easy to mid HVS, 5a/5b border.

Of course I'm not serious about a quarter grade. I think Inverted V is an easy VS (feels HS to me but again taking into account others I trust) and Counts Crack is a top end VS. so the true separation is close to a full grade (and at least 3/4).
 Offwidth 18 Jun 2011
In reply to Bulls Crack:

Half grade; half grade; (ignore); full grade: all on popular classics. Tiny differences? These votes should be spot on according to some.

New Topic
This topic has been archived, and won't accept reply postings.
Loading Notifications...