/ Logbook - Changes to the Voting System
The old system deleted all old votes for a route whenever the grade was changed. The new system doesn't delete these votes, it just slides the scale to be centred on the new grade and keeps all the old votes. This system has been running on the Rockfax route database for several weeks already.
This is stage one of some improvements and changes to the UKC Logbooks system that we are planning in the next few months. The next stage will be to link the UKC Logbook and Rockfax databases together so that their votes are combined where a route is listed on both databases. This will also enable us to upload a lot of new routes to the UKC Logbooks database from the Rockfax database which is very extensive for many French and Spanish crags.
Excellent news, keep up the good work at Rockfax / UKC / UKH HQ.
Any sign of a useful way of actually getting data out? As has also been promised for the last few years ;-)
I know, I know ...you deliver one long awaited improvement and what do you get? Thanks, not a bit of it; instead a deluge of demands for further changes!
I appreciate the work you're doing, honest!
Nope. That Things like (off the top of my head)
- what routes were climbed on a specific crag on a specific date?
- when was the last time someone climbed on a particular crag?
- what VSs have I climbed on limestone?
- what routes on north facing gritstone were climbed this week?
Sport/bouldering grades are horrible now, Sport routes should still be graded 'F7a' and boulder problems 'font 7a' or just '7a'
Looks really messy.
When are you going to do a 'ground up' option on the style of ascent?
Alan James, Franco Cookson here. I have to say, this new grade-voting beauspised by your fair hand be a move of equision. A reet good move, and one that should be well celebrated.
The incresed range on vote a-possible, be a step a-forward to providing a quickened consensusisation. Good work.
I'd say there is a greater need for a DWS option than a ground up one which, let's face it can cover all manner of ills
> Nope. That Things like (off the top of my head)
> - what routes were climbed on a specific crag on a specific date?
> - when was the last time someone climbed on a particular crag?
> - what VSs have I climbed on limestone?
> - what routes on north facing gritstone were climbed this week?
So you want to be able to download your own SQL database?
One problem with the new system: It makes the old votes the lower end of what the maximum grade used to be. So an E1, that had loads of voted for 'E2 or harder', now has a load of votes for low in the grade E2, when in actuality all those people thought it E3...
> One problem with the new system: It makes the old votes the lower end of what the maximum grade used to be. So an E1, that had loads of voted for 'E2 or harder', now has a load of votes for low in the grade E2, when in actuality all those people thought it E3...
Yep, noticed that myself in the winter logbooks.
Over time I guess this will iron out and the real sandbags and soft touches will emerge.
It is still an improvement though. We didn't have data for the grade below beyond it being 'the grade below'. We allocated all these votes to the top of the grade for want of a better place to put them. I can't think of any other way we could have dealt with them.
As said above, this will iron out as more votes are registered.
> I'd say there is a greater need for a DWS option than a ground up one which, let's face it can cover all manner of ills
or a 'clean aid' option. I guess you can make a note of any of these in your notes on the climb though so none are really essential
That's true and it is something I think we can add quite easily.
It's OK, I was going to vote E1 for Wrecker's Slab, so I'll tone it down to HVS to balance your vote out!
Can I ask why you didn't just allow people to vote on the grade without restricting it to a range from the current grade?
> Can I ask why you didn't just allow people to vote on the grade without restricting it to a range from the current grade?
I am not quite sure what you mean.
The old system stored grade values in one table and then, in a separate table, it assigned a weighting to that grade which had 5 divisions - next grade up, high, mid, low, next grade down. Votes went into the second table hence each great only had a weighting factor that related to the grade it was at any one time. Change the grade and the weighting became nonsense hence we deleted the votes.
It wasn't a very good system but we did create it a long time ago and it has worked ok since then accumulating nearly a million votes.
So why can I not vote a route currently voted VS is E1? I think restricting the range to HS -> HVS causes all sorts of issues.
Not really. The situation where a route is that under/overgraded is very rare. Adding two grades up/down would make for a messy form, plus the possibility for people to deliberately 'over vote' simply to make a point.
Grade changes are best done in small increments not in huge switches.
So what about where the route IS more than 1 grade out, this could have been for all sorts of reasons such as historical misgrading, changes in protection or rock fall. This just means that lots of people will vote as far as they can and when you change the grade of the route it will incorrectly get listed as closer to the previous grade.
Have you ever tried allowing people to vote more than one grade either side of the current grade or are you assuming "Grade changes are best done in small increments"?
I think if you use the mode grade voted rather than the mean grade (for the actual guide book grade) the few people that that deliberately incorrectly vote will not matter.
> the few people that that deliberately incorrectly vote will not matter.
Using the median (instead of the mode or mean) is likely best in such situations. Thus if one person votes 2 grades harder it has no more effect than if that person voted slightly harder, but if everyone votes 2 grades harder then that determines it.
You could email the crag moderator?
This would allow users to give an indication of routes which they feel are seriously over/undergraded without making the voting form any more messy. So:
I do think routes that are misgraded by 2 grades are a rare thing these days and if it feels that hard/easy you are doing it wrong or possibly off route!
... IS one grade out, ... in your opinion?
If the general consensus is that a route is more than one grade out, then that will show itself in the voting pattern and comments. Votes will be significantly weighted to the top of the next grade up and we can act on that. It may require a re-set of the grade after an upgrade but it really is only likely to happen in a tiny minority of cases and it would seem stupid to base our voting system and page layout on this rare eventuality.
Why would median be better than the mode? The mode also works in manner you describe:
"Thus if one person votes 2 grades harder it has no more effect than if that person voted slightly harder"
I personally think the grade to take is the one most people think it is not a grade skewed by votes for other grades.
No not in my opinion I actually mean what if it IS more than one grade out.
Also you never addressed my question :
Have you ever tried allowing people to vote more than one grade either side of the current grade or are you assuming "Grade changes are best done in small increments"?
No. We had a system, we changed it last week. That makes two systems we have tried. I've told you why I think our current system is adequate.
For example, if the voting were:
9 for E1
8 for E2
5 for E3
then the median is likely a better grade than the mode, since it is more robust in giving the middle of a range, particularly with small numbers of votes. The above is a somewhat unlikely. A more likely example is:
12 for hard-E1
11 for soft-E2
5 for mid-E2
The mode would give "hard-E1" and thus a grade of E1, whereas the median would give "soft E2" and thus E2. (Of course you might add the above to 12 for E1 and 16 for E2 and then take the mode, and also get E2.)
And if the votes were...
16 for hard E1
2 for soft E2
1 for mid E2
...the mode would give "hard E1" and the median would give "soft E2".
But then, you can prove anything by making up examples to suit ;-)
OK well as mentioned I think the new system is still flawed for the reasons mentioned. I think what you've done is a step in the right direction but but quite a small one. Saying your current system is adequate (which I guess the old one kind was also) is very different from stating "Grade changes are best done in small increments not in huge switches." and dismissing my idea (for which I have still not heard a robust reason why it wouldn't work).
By the way I wasn't suggesting just adding 2 grades up and down, I believe people should be able to vote on what grade they think it is (by opening a voting window) and that you should by default display only a subset of those results on the logbook page.
In your example the median and mode are both E1. For the median you find the middle value with the set ordered e1,e1,e1,e1,e1,e1,e1,e1,e1,-> e1,e1 <- ,e1,e1,e1,e1,e1,e2,e2,e2
which would be the average of E1 and E1 so E1.
I can see arguments for both mode and median but I still think the grade given should be the one the majority of people think it is. IMO when calculating this all the votes for a given grade (soft medium hard) should be collated.
Using the mode has the advantage that people can not tactically vote to skew the results (in fact this will be counter productive) whilst the median and mean can both be affected by this.
> So what about where the route IS more than 1 grade out, >
Example - South East Buttress at Roche Rock (http://www.ukclimbing.com/logbook/c.php?i=36622). Graded D 4a, when it is probably Severe 4a, as commented on by several climbers of a far better standard than myself.
How many "Diff" votes do you think Indian face would get?
the fewer the votes the more any system is open to abuse at the end of the day they are all flawed and some amount of interpretation of the data will be necessary in this case.
> Example - South East Buttress at Roche Rock (http://www.ukclimbing.com/logbook/c.php?i=36622). Graded D 4a, when it is probably Severe 4a, as commented on by several climbers of a far better standard than myself.
How do you explain that this isn't a single vote above HD then?
> 2 for soft E2
> 1 for mid E2
No it wouldn't! In that case both the mode and median would be E1.
No you can't. Contrary to popular supposition, statistics is not an arbitrary subject.
Not possible to vote above HD at the time people voted, i.e before the voting system was changed. Check out some of the comments to see several opinions that this route is more like a severe.
If possible i would change my vote.
Easy, the system didn't allow them to vote above HD. That's the point Skip is making.
Possibly because people were unable to vote on the actual grade they thought it was, the current system of only being able to vote HD when you think its VD or Severe doesn't make sense!.
There are a lot of good reasons why grade changes should only be done in small changes and also only done very occasionally. All sorts of strange things happen with grade votes when you change them and the more you change them (both in grade step and frequency) the more strange the voting becomes.
Effect 1 - Deliberate over-voting
People have a really hard time on a route and go home and vote for the highest grade the voting system gives them access to. THis doesn't mean that the route is 2, 3 or even 4 times the grade given, it means that they had a hard time and want to express this in their voting. WHich of us hasn't come off a route and screamed at the guidebook for having a ridiculous grade in it, only to sober up a bit later on after an ascent by a mate put the route (and our performance) in perspective. In the absence of this perspective, many probably would still vote for their inflated grade. This alone is reason to channel voting to a smaller band since such votes would not be as rare as you think and would have disproportionate effect on the average voting.
Effect 2 - Idiot prank voting
No need to explain this one. I can assure you that Indian Face would receive plenty of VDiff votes.
Effect 3 - Grade PIng Pong
This happens when grades are changed too frequently. A grade becomes settled usually. If you change it then the result is that you will get an increase in votes trying to return it to what it was before. This is especially true for downgrades. So you change it back to what it was before and the reverse happens. I hesitate to mention 3PS but that is a classic ping-pong grade route that should stick at what it is for the next 10 years (HVS BTW).
Grades that are significantly out (+/- 2 or more grades) will show themselves using our current system as I have already described. Votes crammed in the upper and lower voting bands will clearly indicate these problems and they can be dealt with on a case by case basis. It really isn't a major problem and the adaptation we have made to the current system will work very well in this respect and prevent the problems of Effects 1 and 2 above.
No offence Allan but you are basing all this on assumption as you haven't tried what I suggested.
Also as mentioned keeping the current votes when you change the grade outside the band you currently use is flawed as often people will have voted for the highest / lowest they can.
if you use the mode for the average they would have no disproportionate effect on most routes with a reasonable number of votes
you get this in your current system. Some kind of common sense is required here!
Grades aren't (and really shouldn't be) decided by votes IMO. Its far too skewed by votes from the fresh faced converts to climbing on grit with limited experience across the different styles and rock types needed to make a balanced grade judgement. The best a voting system can do is to alert an experienced guidebook team there might be anomolly in the published grade. Debates as to the best way to analyse votes are spurious at best.
Actually I kind of agree with that. However I still would prefer a system where we can vote on the actual grade we think a route is.
> No offence Allan but you are basing all this on assumption as you haven't tried what I suggested.
No offence Curlo, but I am basing it on the fact that I have been running two online databases for 12 years and know how to run one from a technical point of view and how to manage the voting to produce stable grades in guidebooks.
We don't chop and change systems since that costs time, money and resources as well as causes instability. We put thought into it and come up with solutions that work. They may not be perfect but, under the circumstances - which have many more things putting pressures on the decisions then the votes of extremely under-grade routes - we do a pretty good job.
I know that we have some anomalies after the system switch that people have brought out above but these will disappear with time. For the long term system I don't believe your suggestion is worth trying for the reasons I have already given and I have better things to spend my development money on anyway.
I do appreciate your feedback though.
yeah sure ;)
Quite right. And while we are at it, let's get rid of political voting as well after all look at where allowing the plebs to vote has got us.
I suspect that when votes sit consistently at the very top of the scale it would tend to indicate a route that may be harder than the voting allows you to indicate; and if you use the database voting as an aid in route selection (guilty) then routes where that kind of voting pattern is established should be approached with that in mind.
One related issue is routes that have out of date or incorrect grades in the database, if the centre of the grading scale is off, then it limits the voters ability to redress the balance. This is largely a crag moderator issue as I understand it though. An example of this is Rocking Chair on Clogwyn y Tawr, VS 4c in the current Ground Up guide, HVS 5a in CC Ogwen, but HS 4b on UKC, which means that votes for HVS are not possible.
The route that Skip highlighted further up the thread is another case in point, and for a while the votes may be quite misleading on these anomalies ...it seems one person who responded to him had interpreted the fact that the new system had converted the previous votes into the lower end of the grade above meant that voters felt that the route was only slightly misgraded, when in reality the comments indicate that most feel the route to be 3 grades above its database entry.
Giving logbook users the ability to revise votes (as discussed on another recent thread) would allow some of the conscientious sorts out there to redistribute their votes in the new system somewhere closer to where they feel they belong - for me this would probably only mean changing a handful of votes, but I'd feel better knowing that I hadn't contributed to sandbagging someone!
> Giving logbook users the ability to revise votes (as discussed on another recent thread) would allow some of the conscientious sorts out there to redistribute their votes in the new system somewhere closer to where they feel they belong - for me this would probably only mean changing a handful of votes, but I'd feel better knowing that I hadn't contributed to sandbagging someone!>
Agree. In my case it would only apply to 2 or 3 routes.
changes sound good but at the moment the crags look even more of a mess than before. Take Cratcliffe Tor for example a popular crag in the Peak district:
There now seems to be several ways of showing grades within the same grading system if mixing Font, V grades & Trad wasn't enough.
harder bolder problem that now look like sport routes, example: Jerry's Traverse f7B ***
easier boulder problems that still have 'font' in the grade, example: ArÍte font 4+ ***
Add a poorly edited crag and everything is a real mess. Sorry to single out one person/editor but i had to use an example. And yes i would take on editing a few more crags. Personally the way i see it is that too many crags have been to too few people to edit with not enough time to sort them out all out.
"let's get rid of political voting as well after all look at where allowing the plebs to vote has got us."
Ah but democracy isn't that simple is it? In our democracy most people think we should bring back hanging but the democratic system of representation in parliament prevents this. With guidebooks, no one elects us so we are a benign dictatorship at best and claiming democratic principle is a bit silly. In this I'll stick with a benign representation that listens but doesn't always slavishly follow the wishes of its people. Your books are no different than the BMC or CC or YMC in that ... you often simply don't have enough votes to listen to the people and where you do (Stanage soft touch VS) you ignore them as you know they are wrong.
"How do you explain that this isn't a single vote above HD then?"
That would be funny if it wasn't worrying... don't you understand your own voting system? There is also a phenomenum in Psychology called cognitive bias. This means that people who climb a lot harder (most of the traffic) will likley tend to agree with the grade allocated if its not way off; it also means some people who have made their mind up won't shift even if they are obviously wrong; there are other consequences as well. Its also possible to misread the guide and climb the route incorrectly (an easier variant or even the wrong route). So many answers for such a simple question on a number that is subjective in any case!
We had a discussion the other day on how bad grades could stay so long in guides and you asserted such things were often not checked one guide to the next. Whereas I thought the routes were climbed but not sytematically (recording the experience on the day) with someone who had good 'feel' at that grade. So in response I asked about Stanage and Birchen (that you had BMC author responsibility for). Are you still sticking to the fact that the routes were not climbed by anyone in the team or was I correct in these cases (most routes were climbed but by better climbers likely affected by confirmation biases)?
New Dawn 7c
Brad Pit 7C+ etc
The 'f' just makes it all very messy and i can't get used to it. This is how you guys at UKC differentiate on news articles and seems pretty consistent around Europe
Europe don't have UK tech grades to get confused with... you need an F for sports routes at least on a mixed database.
OK but currently all the sport grades on UKC are now just a letter and a number. Is this going to confuse everyone?
> "How do you explain that this isn't a single vote above HD then?"
> That would be funny if it wasn't worrying... don't you understand your own voting system? There is also a phenomenum in Psychology called cognitive bias. This means that people who climb a lot harder (most of the traffic) will likley tend to agree with the grade allocated if its not way off;
I also think that inexperienced climbers might be led by the previous votes that have been in some cases inaccurately redistributed by the introduction of the new voting system, this might mean the correction you would expect to happen over time might not occur to the extent it should.
I know when I do a route at a grade I have limited experience of (so E grades basically) I tend to look at the consensus opinion before I add my vote. There may be a reluctance to vote at the top or bottom of the scale when the new system has compressed votes into the middle of the scale.
Having said all that I do regard the new system as an improvement and think that where the "outside the given grade" votes have been placed is the most sensible option. It is only a handful of routes that are truly effected by this, and mostly this could be corrected by some conscientious crag moderation.
...Looking back through my contributions to this thread I feel a slight sense of embarrassment about what a grades anorak I have become! It's just rock after all and there are only two grades in reality: "I can climb it" & "Too damn hard" ;-)
"I can climb it" & "Too damn hard" Trite nonsense... you can breeze it; it feels pleasnt; it makes you think a bit; it feels a little tricky; it feels hard; you get it by the skin of your teeth; you can work it; you could work it when stronger.... this is how grades are made.
Sorry the two grade thing just annoys me. You don't.
Of course there are still a multitude of routes that will always remain "Too damn hard" for my modest abilities, such is life.
The reality of course is that I'm just one of many experienced guidebook workers now and in the past who has given up years of a life for free trying to make a difference in narrating climbing areas I love. I'm far from perfect (grading or otherwise) but I have with Moff hopefully made a difference in the Peak and elsewhere by actually climbing things (or listening to others we trust to do so where we are not good enough) and adding our bit, alongside the teams we are part of, to what came before. Call me old fashioned but isn't that what its about... anyone can copy the last version and change a few words but where is the soul and what is the future in that?
As for grading every climb in the UK: we will never be leading much above low extreme and dont tend to hitch a lift on harder clinbs that often, we are very much lower grade climbers, however we are well over 10.000 routes and problems down and still going strong and we nearly always take notes... for us it's the experience that matters most... grades just help climbers tailor their climbing better to maximise that experience (at least they do when they are reasonably accurate). Part of the joy of climbing is whatever grade you lead there are almost endless challenges and adventures to be had.
come off it - don't try to pretend you don't enjoy it!
Elsewhere on the site
Skiing Baffin’s couloirs has been on my to do list ever since I saw Andrew McLean and Brad Barlage’s inspirational... Read more
Backpackers want an extremely liveable and lightweight tent at good price. MSR answers the call with the Elixir 2 tent and... Read more
2012 saw the release of the beautiful first volume of definitive Yorkshire Gritstone climbing, produced by the YMC with Robin... Read more
The Women's Mountain Equipment Cho Oyu Jacket is the perfect choice for female mountaineers an explorers who... Read more
The Kendal Mountain Festival 2014 proved once again to be a busy and inspiring four days of films, photos, music, art... Read more