UKC

One consequence of the work we are doing to develop a Rockfax App is that we are going to need to change the way that UKC Logbook moderators deal with grades. We also want to undertake the long-overdue job of amalgamating the two different sets of votes.

At present we have a UKC Logbook listing for thousands of routes and we also have the Rockfax database listing for many of the same routes with separate voting records. We have spent a lot of time linking these routes with an ID number across the two databases over the last few years in preparation for the Rockfax App.

The grades in these two databases are often different. In order for the Rockfax App to work, they need to be the same since we will be accessing various aspects of the UKC logbook information from the App, but the main route data comes from the Rockfax database. This also has to happen for us to amalgamate the voting systems for obvious reasons.

In order to do this we will have to select a grade for every route that features in the App (that is the routes in most Rockfax guides since 2006) and stick with it and remove the ability of crag moderators to alter grades for these routes. We will make this grade selection based on voting received and we will be able to find anomalies before the change, so this doesn't mean that we will be going with the grade in the current printed Rockfax.

After this grade changes will have to be dealt with centrally via a master database since they will need to change in two places at once, one of which isn't public.

This doesn't mean that moderators can't change grades, in fact we will welcome their input to point out problem grades, however it does mean that they will no longer have the ability to change the grade via their moderator access - they will just need to report them to us.

Note - this is only for routes that feature in Rockfax guidebooks.

Alan

In reply to Alan James - Rockfax:

As I understand it, most of the grades on the UKC database come from definitive guidebooks, the RF grades are from the volume editor.
Often the two will be the same, but if they are different I would like to think that the RF grade would be the more accurate (personal experience and using the databases for much needed feedback) and should be the one in the new database and for the App.


Chris

In reply to Alan James - Rockfax:

Will the 2006 Lakes Bouldering guide be covered by this? The grades on the UKC database are now far more accurate than those on the RF database. For example, the grade for pretty much every problem on the ladder face at the Bowderstone (possibly the most significant bouldering venue in the Lake District) is wrong on the RF database, by several grades in a few cases.

In reply to Chris Craggs:




There's a danger of conflating current logbook grades with latest definitive guidebook grades. I have no idea why you think that RF grades would be 'more accurate' than the latter. Whether than they are better than the former is the real issue though. Plenty of crags in the database currently have grades taken from old definitive guides or apparently just plucked out of thin air. These crags would benefit from a blanket switch to the current RF grades, with input from moderators to pick up any typos or anomalies (say where a route has changed through rockfall or gear placements breaking). Crags which are currently well maintained and kept in line with latest definitive guide, where that is fairly recent, would probably not benefit from arbitrarily shifting around a few borderline routes.

I don't know what the best option is, but I don't see that it's clear that this change is unilaterally good, except for those who want to use your app. A market you have to appreciate is not the same as the group of people who actively use the logbooks on here.

In reply to GPN:


That guidebook is probably too old to be added to the App. It is also a bouldering guide and we are struggling a bit with the format for bouldering guides. So much so in fact that I don't think the forthcoming Peak Bouldering guidebook will be on the App straight away since we will need to do a lot more work specific for bouldering guides.

Alan

In reply to Alan James - Rockfax:

Cool. I can imagine the bouldering might be a bit more problematic than routes....

George.

In reply to Alan James - Rockfax:

will this also affect star ratings?

In reply to victim of mathematics:


Because quite a few of them are from his guidebooks?

Anyway, how will the 'split' grades be determined? I can think of several routes (which I will not name to prevent this descending into a petty grade debate) which get different grades in Rockfax books to on here, and the voting is rather warped by peoples desire to get a higher grade (especially around the E1 border) tick. And others which get different grades in definitive guidebooks - will they be picked solely on votes, the already existing grades in either database, the definitive guidebook or by the crag moderator? Or a combination of these?

In reply to ianstevens:

But it'd take a fair degree of arrogance to assume that RFs selective grades are 'better' than the current definitive guide, surely...

In reply to ianstevens:


At the moment I don't know the exact procedure we will use but I suspect it will be to make an assessment based on all available sources of information. Whether we do that for every route is probably unlikely but the whole process is ongoing and the votes on UKC are not going to be thrown away.

Alan

In reply to victim of mathematics:


I am not sure what an 'RF selective grade' is.

It would be arrogant to assume that a single author's grade opinion trumped everything but that isn't what we do. We use existing grades, logbook votes and comments and experience to choose them. Often the first two will outweigh the last one meaning we have to go with grades that we don't agree with ourselves.

Alan

In reply to Graeme Hammond:


Yes, we can only have one star rating per route.

Alan

In reply to Alan James - Rockfax:




Why?

This seems to go to heart of the problem touched on in other respones.

In some contexts computerised systems, of necessity, do no more than establish a partial view of reality. For the sake of creating a neat view (or product) that 'works', important aspects of every day life are ironed out. The results may look smart and give the appearance greater objectivity, but experience suggests that this can be misleading and comes at a cost. Binary systems are not good at doing grey; they can only deal in black and white.

One of the attractions of climbing is that has a lot of grey areas; not least in relation to grades. Essentially grades are subjective judgements made by first ascenionists at a particular moment in time. Judgements which are then open to scrutiny and debate by others that follow. To some extent, a democratic process which is not controlled by a centralised committee or governing body.

I've got no problem at all with the idea of an App; but it will have some downsides, downsides which in the longer term may unintentionally alter some of the things which currently set climbing apart.

Lets not get carried away with the idea that reality is neat and tidy and I do hope that those developing the App. give this some thought.

In reply to John Postlethwaite:


There are technical reasons for the App needing a set grade but the main reason is so that we can have a single voting database rather than two as we have at present.

Also, whilst you may cherish grey areas, we do get a lot of complaints about grade differences. At present we can have up to 4 different books covering one route, plus at least two databases (more actually). Reducing this to one database synced with one of the other information sources I think would be a good thing in most people's opinions.

Alan

In reply to Alan James - Rockfax:

I'm afraid you may have missed my point Alan.

To me it's not a question of 'cherishing' or holding onto grey area's; it's simple a matter of fact; they are an important constituent part of reality. This seems to be especialy true in the context of the grading system within climbing; indeed may be an important part of what makes climbing different. Moreover, experience in other (more serious) settings suggests we iron out ambiguity and contingency at our peril.

Now it clearly makes good sense for you to sync your database so that you get fewer complaints. All I am trying to suggest is that :

(i) some thought is given to the lessons learnt in other contexts
(ii) more importantly, that we are mindful of broader climbing context.

At a minimum I would hope that some of these important messages are worked through and made clear in the marketing of the App.

History such not be confused with nostalia and if people loose sight of history then it can have consequences - intended or unintended.

In reply to Alan James - Rockfax:


OK, that was poorly phrased, but the grade from one of your guides, which are in the large part, selective to a greater or lesser degree depending on the area.


Are you actually suggesting that this process makes your grades more accurate than whatever process underlies the grading of routes in definitive guides? Because that's what it sounds like you're saying.

In reply to John Postlethwaite:

Climbers should recognise that climbing is full of grey areas. Even in a perfect world an E3 could logically be anywhere from borderline E2 to borderline E4. But include the human factors, and the fact that we all have different strengths, weaknesses and skillsets, and we should all be aware that our experience of an E3 could range anywhere from E1 to E5.

I know I've been guilty of this in the past, but climbers are far too obsessed with grades. We have to accept that they are merely guesswork, indications of difficulty for the average climber and not to be regarded as a measuring tool.

Computers are a different matter - in the end everything boils down to boolean switches.

Synching the UKC logbook with the Rockfax DB is long overdue and a single repository of votes will improve the quality of the data. Making sure the Rockax DB is synched with the latest Rockfax guide is another no-brainer. And if we get a Rockfax app out of it that's another bonus.

In reply to Alan James - Rockfax:

I'd urge you not to do this for the North York Moors. Me and a few other people in the Moors have provided you (for free!) with THE most up-to-date info on routes - available nowhere else appart from on UKC. As such a huge number of the climbs are new, the grades change month on month, with concensus ect. If it becomes difficult to re-grade routes, then you'll just end up with a database that is out of date and no use. I'm aware that most of the Moors does not feature in a Rockfax guidebook, but even those that do need re-grading fairly regularly at the moment. The ability to easily regrade and sort routes into comprehensible chunks is the main reason I moderate crags.

Thanks,

Franco

In reply to Alan James - Rockfax:


For me, this is one of the two crucial points to how well the new system will work - if you can implement a "Request grade change" button or form on the "Edit route details" page then useability for moderators won't suffer much.

The other key point is how you perform the merge. If you simply overwrite ukc db grades with rf db grades, then a lot of correct changes/grade values will be lost - I certainly can't remember the grades of every route I've done at all the crags I moderate. Please can you consider how to handle conflicts, especially differences of more than one grade (changing a 6a+ to a 6a isn't really going to upset anyone, but changing a 7c+ to a 7b+ might).

Thanks.

In reply to Chris the Tall:


Couldn't agree more Chris. That's a part of what I am saying.

An element of the challenge is over reliance. Just look at what happens when people become GPS dependent!

It's not grade obsession which is the problem here, but grade 'certainty'.

Franco makes a compelling point.

In reply to Franco Cookson:

Perhaps it would be possible for the software to generate an email to the crag moderator informing them of any changes to the grades that are to be made.

I can appreciate you don't want your hard work being undone by an automated process.

I have similar concerns over Stanage Plantation, where I put in a lot of effort to reconcile data from 3 different guides, each with different names and grading systems. In that case I'm more concerned with names being changed, since I suspect that will result in people adding problems which they don't think are listed.

In reply to John Postlethwaite:



So you reckon if there's less 'certainty' over grades, then people will place less reliance on them.

Sorry I'd prefer consistency across the Rockfax empire

In reply to victim of mathematics:


No, however I think our grades in many case analysis is as accurate as BMC guides when viewed as a whole (obviously there will be individual exceptions). I have long said that a database with over hundred grade votes can't be ignored and I would stand up for its value against a grade chosen by a small group of researchers working on a definitive guide.

Alan

In reply to TimB:


Well we have over 30,000 routes to process so I am not going to promise that we will be looking at everyone of them. However we changed the voting system on both UKC and Rockfax to maintain votes a few years ago, so even if changed grades do get re-set to older values in some cases, the votes will still be there so no actual data will be lost. We will try not to do this though.

Alan

In reply to Franco Cookson:

As you acknowledge, the Northern England book only covers a small proportion of the routes you are concerned about and most of what it does contain are the older more established routes. These are the only routes that will be affected since we won't be bolting down grades changes for any route not linked to a Rockfax route.

If you have any particular concerns then email me a list of grades that you don't want changing, or better still, wait until we do it then see whether the process has worked and where there are anomalies, email and we can sort them.

Alan

In reply to Alan James - Rockfax:

Let's cut to the chase and ask the important questions:

What grade is the LPT classic, I've been a bad bad boy? 7c or 7c+?

(This is the sport climbers equivalent to "Is 3 pebble slab HVS or E1?)

In reply to Chris the Tall:




Not the point I am making at all.

Part of what I am say is that grading, by it's very nature, is a subjective judgement, inherently uncertain. Got no problem with consistency, so long as it is not conflated with certainty.

In reply to Ally Smith:

7c+ on RF, 7c on UKC but votes pointing significantly towards 7c+.

So this would be a 'go with the current RF grade' example, but with voting maintained.

Alan

In reply to Chris Craggs:



But you've admitted before that you don't have time to check all the routes in your books. What happens when the book you copy the grades from gets it wrong?

In reply to Alan James - Rockfax:


Random example
http://www.ukclimbing.com/logbook/c.php?i=28322
You say VD. Voting says hard VD. Not sure what the new Yorkshire guide is going to give it, but I'd be surprised if it's anything less than HS. But you'' prevent the grade being corrected.

This isn't an isolated example. In Yorkshire, many routes have had their grades copied from one guide to the next. The two new guides have checked the majority of routes, and there are a lot of grade changes as a result. My impression (apologies if I'm wrong) is the majority of routes to this area were not checked for the RF guide. Yet you'll be taking these grades in preference to the newly checked ones?

In reply to Alan James - Rockfax:


Will there be a quick means of doing this, or will it be a question of finding the relevant Contact page and typing in details, copying and pasting URLs, etc? If the latter, can I make a request for the former!

In reply to Toreador:


I have never admitted that we don't check all the routes in our books, I have said that we don't check them all by climbing each one between editions, but that would be impossible for a small team of authors with 33,000 routes to maintain.

We do check every route by accessing the world's most extensive and popular database of routes, as well as all other available sources.


What is actually far more likely to happen is that the scrap of paper, or unclear scribbled topo we had to work from gets the grade wrong. When that happens then the grade will be wrong and people can vote on it to correct it.

Not really sure what your point here is.

Alan

In reply to Toreador:


Voting says VDiff, RF gave it VDiff, we would stick with VDiff. I see no reason not to if people are happy to vote it at that grade.

You assume that you know what the correct grade is; we never make that assumption.


Yes, this went on for years with definitive guidebooks. UKC Logbooks has helped iron out a lot of these problems especially with lower grade routes.


You are beginning to annoy me now. We do check routes but we don't check them by climbing them between editions since that is impossible.

The grades we will be using are the ones voted for by user of UKC logbooks and I will defend the integrity of these grades against any group of guidebook writers. Yes there will be anomalies, and I am not saying they are all correct, but overall the system gives us some of the best information about grades ever assembled.

Point me at the people/organisation who have put more effort into documenting and opening grade opinions up to everyone than UKC and Rockfax.

Alan

In reply to Toreador:


I don't know I haven't designed the system yet. However I will be sinking tens of thousands of pounds into carrying out the research and development required to produce the best international database for climbers anywhere in the world. That is in addition to the tens of thousands of pounds I have already spent.

Alan

In reply to Alan James - Rockfax:


Sorry, my reply was meant to have been directed to Chris, who I'm sure has said such a thing.

In reply to Alan James - Rockfax:

I have long said that a database with over hundred grade votes can't be ignored and I would stand up for its value against a grade chosen by a small group of researchers working on a definitive guide.

Alan, "a grade chosen by a small group of researchers working on a definitive guide" is perhaps a slight misrepresentation of how the grading system works.

As the message from Franco Cookson indicates, the system is reliant on experienced and knowledgeable enthusiasts who are also subject to peer scrutiny. I can recall for example the recent history of adjustments made to some of the first ascent grades given by James Pearson; adjustments made by climbers of comparable ability and experience.

Voting on grades is fun, I do it all the time; but it's not quite the same as what is in essence a peer review process. It's also helpful to understand that a Don Whillans E1 is likely to be quite different to a Joe Brown E1 etc etc.

Now if the folks at Rockfax want to adopt a voting system for grades, that's OK by me; so long as they make it clear that this is what they are doing. That is, introducing a different, not necessarily better, system. Indeed, it is a pity that you infer that the definitive system, on which you have been so reliant, is slightly substandard.

In reply to Toreador:


You are wrong at least in the details - I have 'checked' most of the routes in the Yorkshire section of the North of England guide - the 'problem' is that I did it over a 40 year period. I have always been fairly meticulous at keeping records, and most serious anomalies will have been noted along the way. Your VDiff that may be HS obvious slipped through my net!

Chris

In reply to Alan James - Rockfax:


If I were the only one who thought this I'd agree. Offwidth for one agrees with me (he created a thread on this route). And if you look at the logbook comments it's clear that several think the same, but have not bothered voting. I think you are too trusting of the accuracy of grade voting, having no idea of the background or experience of those who vote. It's an important input, and I've made use of it, but it's far from infallible. As you know there have been a few threads about this over the years, where examples have been given of votes that are clearly wrong. Anyway, this isn't what the thread is about


That's exactly my point. You don't check them. But the new Yorkshire guide writers have mostly managed it. Yet you're taking the old unchecked grade in preference to the new checked grades.


You do a great job, I've not said otherwise. Just because I've got reservations about your plans doesn't mean I don't appreciate all your work. Just look back through some of the "Rockfax are evil" threads to see what I think.

Sorry if my opinions annoy you.

In reply to Toreador:


Well it depends on the book, for my UK guides I have done the majority of the routes up to E4/5 (yes there are exceptions - Ravenstones West for example) - for the French/Spanish ones, I have done as many as I can!


Chris

In reply to Toreador:


It maybe worth pointing out that much of the info in the UKC Databases may be dated (not saying it is, just it could be) and with, for example, the shiny new Yorkshire guide, they may need looking at again. I assume some moderators may be doing this already?


Chris

In reply to John Postlethwaite:


I struggle to think of something we have been more clear about over the last 12 years since we introduced the Rockfax database.


The current practice in the Peak, and it appears in Yorkshire, does seem to involve a lot of route checking. I have my reservations as to whether this is actually a sustainable system over a period of years, but I take my hat off to those doing it and acknowledge a great effort being put in.

The older system practiced in much of the 1990s and 80s did no such thing as this. Some were certainly more diligent than others but I have examples of definitive guidebooks that went through three editions with barely a flicker in grade change. One particular crag I remember visiting had the most macho grades of anywhere I have ever been to which we did a radical reassessment of in our book. I did a meticulous comparison between editions at this crag which was supposed to have been 'checked' and not only were all the grades identical to the previous edition, so was the text. The only difference was the name of the researcher at the beginning of the chapter.

Not all guidebooks were researched in that way, and we owe a great debt to the diligent people who have documented routes over the years, however it wasn't just the presentation of traditional guidebooks that struggled to keep up in the 1980s and 1990s, there was huge complacency amongst the producers that turned out some very lazy books.

Much of this has been turned around in recent years by the great work of the BMC, the CC and the FRCC who are all now producing excellent well researched guidebooks.

Alan

In reply to Alan James - Rockfax:
As Toreador says, he's not assuming: he's talked to me (and I'm sure others) and I've talked to other experienced guidebook workers who are more interested in good grades and sorting out the dangerous old sandbags than publisher turf wars. The grade and text in that example he gave was a nasty sandbag that will be dealt with when the new YMC guide is out. The problem with your extensive databases is they are mostly used where there is little grade uncertainty; yet quite often in the areas of most grade uncertainty, like some of your Yorkshire and Western grit moorland pages (particularly places like Ravenstones West or less popular bits of Crookrise), there are very few logbook comments/votes and some of those I wouldnt trust (the people climbing these areas are pretty hardcore compared to your averege Stanage voter and worse still it's not beyond a few people who dislike Rockfax to try and skew your votes). The fix is easy... Toreador and people like him can tell you the relatively few cases where the definitive (or logbook) grades are likely better en massse and when to use best local info to over-rule the existing Rockfax grades. My money would be on Franco being right as well on his home turf (from my limited Northumberland climbing neither guidebook I used was great sub HVS). I'd say grade changes due to broken holds, typos etc, are best picked up using emailed forms.

Moff and I have our Peak grit grading views public on Offwidth... they are not always the same as the BMC grades but we've never let anything through that was two or more grades different (unless we checked them post publication). Like any guidebook worker we too make mistakes and change our minds from time to time (based on feedback... even from Toreador, the low grade trad chimney sandbag fiend).

http://offwidth.uptosummit.com/guides.html

In reply to Alan James - Rockfax:


A system where grades are chosen by vote is very much not the same as a system where grades are chosen using a range of information, which may include voting data. I mean it's well-evidenced that UKC voting on grades does not obey the conventional rules about the wisdom of crowds (see the voting on Goliath's Groove as one of countless examples where people think en masse that a route is harder than it actually is).

I presume that you intend to keep the latter system, rather than use voting as your primary mechanism for assigning grades in new guides. John seems to presume otherwise. Perhaps you could clear that one up?

In reply to Offwidth:


And I see no reason why the new system won't allow for this to take place.

Alan

In reply to victim of mathematics:


Yes, the latter version.

Alan

In reply to Toreador:


Ok, slight over-reaction I admit. Sorry about that.

But one thing that is guaranteed to annoy me is when people suggest that Rockfax authors don't work hard. The main reason for our success is because we work so very, very hard.

Alan

In reply to victim of mathematics:
Rockfax have deliberately overruled forum votes in the past, the evidence is in Eastern Grit. I have nothing against making such sensible editorial decisions, as votes on too many routes (with huge numbers voting) are dumb and we all know that and chosing this to define Rockfax grades would be dumb.

Some Stanage examples:

Experienced concensus bottom end VS:

Inverted V: voted mid grade by UKC
http://www.ukclimbing.com/logbook/c.php?i=10321

Straight Crack: voted mid grade by UKC
http://www.ukclimbing.com/logbook/c.php?i=10325

Experienced concensus lower end VS:

Via Media: voted mid grade VS by UKC
http://www.ukclimbing.com/logbook/c.php?i=9864

Central Trinity: voted mid-grade by UKC
http://www.ukclimbing.com/logbook/c.php?i=10345

Hell Crack: voted mid grade by UKC
http://www.ukclimbing.com/logbook/c.php?i=10243

Concensus middle (to just above) Stanage VS:

Mississippi Buttress Direct: voted mid-grade by UKC
http://www.ukclimbing.com/logbook/c.php?i=10252

Hargreaves Original: voted mid grade VS by UKC
http://www.ukclimbing.com/logbook/c.php?i=10340

High Neb Buttress: voted mid-grade by UKC
http://www.ukclimbing.com/logbook/c.php?i=9995

Concensus top half Stanage VS:

Green Streak: voted mid grade VS by UKC
http://www.ukclimbing.com/logbook/c.php?i=9864

Paradise Arete: voted just slightly above average VS by UKC
http://www.ukclimbing.com/logbook/c.php?i=10137

Count's Crack: voted just slightly above average VS by UKC
http://www.ukclimbing.com/logbook/c.php?i=10022

Now I know we all have our individual quirks but good graders try and account for these, yet even ignoring this on average data it simply cannot be that the same populations are climbing and voting on these routes.

In reply to victim of mathematics:


So is Goliath's Groove easier than HVS 5a then?

Alan

In reply to Offwidth:

Your post only makes me think how good the voting is on UKC logbooks.

Alan

In reply to Offwidth:

I'm sorry, I'm not sure what your point is? Do you think I'm advocating a vote-led system (which I'm not), or are you making a point on the inability of people to vote accurately which I'm missing?

In reply to Alan James - Rockfax:



Are you being wilfully obtuse?

It's benchmark low-end HVS, yet the votes are split almost 50/50 between mid- and high-end HVS.

In reply to Alan James - Rockfax:
Then I despair of your ignorance (or elitism?)... the idea that VS leaders find these routes the same is a joke. A good thing I thought Rockfax stood for was fair grading for people leading at the grade. That lots of relative beginners are distorting the lower graded VS classics upwards and that too many extreme leaders vote for midgrade without thinking might explain the obvious distortions to me.

In reply to Alan James - Rockfax:

I'm genuinely confused Alan, I thought you "have long said that a database with over a hundred grade votes can't be ignored and I would stand up for its value against a grade chosen by a small group of researchers working on a definitive guide".

And that you "struggle to think of something we have been more clear about over the last 12 years".

I thought that you were trying to make clear that a voting system was both more sustainable and preferable to system based on "macho" and sometimes 'complacent' "producers" of "lazy books"? Or are you saying something different?

I know I slightly over egg the point here, but I think some of your words may be ill chosen. Words which at the very least over simplify what has been and continues to be a complex picture.

Rockfax has made a brilliant contribution to the development of guidebooks; but that should not obscure our view of the contribution of others and other factors; such as the evolution of grades themselves.

This takes me back to my original point about loosing sight of our history. It can lead to hubris.

In reply to victim of mathematics:


But that's the thing, I don't think it is benchmark low HVS, I think it is more middle of the grade and I know I am a bit of a harsh grader.

Whatever, it seems to me that if the votes are inaccurate, then they aren't far out.

BTW when doing the grade assessment for the recent North Wales Climbs guide we were amazed at the quality of the grade feedback from UKC Logbooks for Welsh mountain routes. Contrary to popular opinion (including my own before this) we actually found that as many routes were being voted down as up. That may say something about Welsh grading but it also revealed that people are more prepared than I thought to vote down a route they found easy.

Alan

In reply to victim of mathematics:

All are roughly the same mid VS grade according to UKC votes yet they differ by concensus by well over half a grade (I'd personally say a grade as I'd give some middle to top end HS). So UKC votes are not even accurate on half the classics on the most popular top-lead grade on the most popular crag in the UK.

I think you and I may need to readjust on GG as its probably no longer benchmark low grade HVS as too many people can't climb offwidths. I hope things havent got so bad that its a top-end HVS.

Nor is Queersville top end HVS: http://www.ukclimbing.com/logbook/c.php?i=10360

Nor is Hardings Superdirect mid grade: http://www.ukclimbing.com/logbook/c.php?i=10360

BAWS is borderline with VS when the votes put it a quarter grade up.

Nor are Surgeons Saunter, Right Unconquerable and Terrazza Crack pretty much the same at a quarter grade harder than Queersville or GG for HVS leaders.

In reply to John Postlethwaite:



Ok, I can see the understandable way you have interpreted what I said. To clarify, I think UKC logbook votes are an excellent way of assessing grades but they are just one of a number of methods we use to choose the final printed grade. I feel that the Logbook votes get dismissed by some as almost worthless, which explains my defensiveness.


It is certainly more sustainable than any other system since it will only get better with time. The criticisms were aimed at an era of guidebook writing which I am confident doesn't exist any more.

Alan

In reply to Offwidth:

So we agree with each other, but evidently not Alan, that in at least some cases, UKC votes are not a particularly sensible representation of the 'truth'. I do find this kind of fascinating.

As for GG, well I must admit to having chosen it as an example precisely because I know so many people get so exercised about it. I don't think I can live in a world where it and Queersville are supposed to be the same difficulty as Surgeons Saunter, Terrazza Crack or Masochism. It's bad enough that they're all the same grade to start with!

In reply to Alan James - Rockfax:
I don't think logbook grades are worthless I just think they are (almost predictably) inaccurate within a grade on popular routes. I also think on popular routes they are less useful than the results of the modern teams of checkers I've worked with and pretty much useless in determining where routes lie in an approximate graded list. As we know the grades and the rough grade order of famous stanage VS classics, hundreds of votes for each are not adding any value other than as a bit of fun. Like Victim I'm surprised given the sample size just how wrong they are and why this might be is interesting.

On obscure sandbag routes like Face, Arete and Crack they are all too often just plain wrong. On famous sandbags they often undergrade... Masochism easy E1 anyone out there who thinks Inverted V is mid-grade VS or GG a really tough HVS?

On the plus side, as a guidebook worker, I do find UKC logbook comments made by experienced regular posters useful.

In reply to Alan James - Rockfax:


I don't agree with this. The examples that Offwidth is giving are probably some of the most voted on routes in the database, yet they're still wrong. I don't see how you can argue that they're wrong because of a small sample size. There's clearly something else going on here, be that subtle ego-inflation or whatever else you'd care to hypothesise.

Anyway, there's a law of diminishing returns when it comes to the additional information you get with each additional vote. you soon reach the point where more votes don't tell you anything new. And they certainly don't address issues like whether or not you should redefine grades according to changing skill sets of climbers, advances in protection or changes in prevailing ethics. These all need human input from people who know what they're talking about.

In reply to Offwidth:

Hey offwidth.

As guidebooks don't print "Low-VS", "Mid-VS" or "Hard-VS" I am not sure how important 1/3 grade discrepancies really are. It seems to only matter when that quarter or 1/3 grade increase takes a route over the threshold into another grade.

My local climbing centre seems to get all the grades a little mixed up. There's a 7a+/b that's really a 6c+ and 6b's that are really 7a. Often routes are about 3 grades out. I know trad grades are a lot broader but if the routes were within quarter grades of what's stated then that would be comparatively dead-on accurate.

I don't think you have much of a case when you are arguing that a 'top end' Very Severe has been given a 'slightly harder than average' on the UKC logbook system. I think that's pretty damn goood, and perhaps you may be also 1/3 of a grade out, or perhaps you could meet in the middle with the voters at 1/6 of a grade each .

Any large discrepancies that you can show me (i.e sandbagging), I will be just as annoyied as you about. But they will probably have to cross into another grade for it to actually make a difference at the crag.

In reply to victim of mathematics:


Maybe you are after too much precision. The set of routes chosen are all VS, are voted VS and have been published as VS. This isn't a great indicator of something not working.

As for sustainability: it is certainly more sustainable than other systems, whether it gets better with time is probably what you were disputing though.


I agree with you here.

Alan

In reply to FreshSlate:

VS and HVS are wide grades: it can make a difference when a graded list is rather random. Its not sandbagging as such but the votes on such large numbers should have given greater accuracy. Do they show a classic Stanage VS is a VS, normally yes (except when its probably a HS compared to the average elsewhere) but that is a real 'no shit sherlock' result. Those errors add up: whats the real grade difference between Masochism and GG, is it really just a third of a grade? Is that Crookrise route VD??

Why do I go on about it, possibly because its stated all too often that the democratic accuracy of UKC voting gives better grades than 'elitist' definitive guidebook workers. The reality was until Rockfax took grading more seriously, definitive producers had got lazy in the lower grades (where most people buying guides climb) and definitive producers have changed due to the competition. It would be nice if Alan acknowledged that we experienced guidebook workers (and I'd include Chris and himself in that) can do better than UKC votes in producing a graded list on classic Stanage VS climbs.

In reply to FreshSlate:


Agreed. While routes tend to make quite an impression if they're really really soft or really really hard for the grade, I think many would struggle to say whether many routes were low or mid, or mid or high, even at their max grade! Unless something seems really soft or harsh I would probably just vote mid-grade. Grades don't need to be that exact.

In reply to Offwidth:



Better than this?

http://www.rockfax.com/databases/graded.php?area=1

How?


Chris

In reply to Offwidth:

This isn't about graded lists though, its about choosing an appropriate grade for a guidebook app, no?

In reply to Offwidth:

This makes me wonder, if people are so bad at voting within a grade, why aren't they similarly bad across grades? Why, for example, aren't loads of people voting for Terrazza Crack at E1?

In reply to Offwidth:


Actually I agree with Chris, the graded list he linked to is better than I could produce for sure.

I do acknowledge that there are some areas where experience of grades makes a difference in selecting the final grade for a route, which is why we don't always go with votes. There are quirks in online voting for sure.

Easy boulder problems, as an example, are very badly graded virtually everywhere, and that includes the online votes. In this case though it is because people don't know what they are voting on because of the grade confusion. There are problems of V0 5c UK tech, and V1 4c UK tech in our system, both of which are nonsensical. We are aiming to improve this dramatically in the new Peak Bouldering book by standardising the conversions and making sure people vote on grades in a system they understand.

There are actually remarkably few votes relative to ascents on easy boulder problems mainly because people just tick them I think.

Alan

In reply to victim of mathematics:


>I mean it's well-evidenced that UKC voting on grades does not obey the conventional rules about the wisdom of crowds (see the voting on Goliath's Groove as one of countless examples where people think en masse that a route is harder than it actually is).


Just read this thread with interest. I picked out the above quote, but I have also read the subsequent debate between Victim, Offwidth and AJ. If people "en masse" think a route is harder than it actually is, then surely we should query the judgement of the person or people who've determined what the grade "actually is" rather than the judgement of people "en masse".

I think from later posts that what Victim means by "actually is" is what a consensus of experienced guidebook writers/contributors thinks the grade is (rather than what Victim himself thinks). But I'm not sure this consensus will always be more accurate than the "en-masse" view of those who report their experience of the route.

Goliath's Groove is a good example. I've done it three times, each 15 - 20 years apart - so effectively an on-site each time. On the first of these occasions I think it was graded VS. I've found it pretty desperate each time, even though I've been reasonably comfortable leading E1 at the time. I don't much like offwidths (nothing against Offwidth) and grit isn't my "home" rock. So perhaps my opinion (as one of the assembled "en masses") shouldn't count - but surely guidebooks should inform climbers from out of the area as much as the local clientele. It may feel bottom end HVS to locals who probably get on the route most years and have it wired, but I'm not sure that makes it bottom end HVS to average punters, who want the guidebook to inform them how hard they will find it.

Mine is only one opinion, but with most of my climbing not on gritstone, I find other classic HVSs such as Queersville and Eliminator easier than GG, and Chequers Buttress (another benchmark bottom end HVS) a great deal easier). If the "en masses" agree with me, am I necessarily wrong?

In reply to Offwidth:

So because the UKC voting doesn't match what you think, you decide the votes are wrong.

Maybe there's another reason .....

In reply to Martin Hore:

If we listen to you and trust this wisdom of crowds, then Queersville is approximately as hard as Terrazza Crack. Does that sound like a sensible conclusion to you?

I've also no idea what you mean by locals having routes wired. If you think that gritstone grades are all a bit on the stiff side because the guidebook writers have forgotten that the grades are for the onsight, then you're doing them a massive disservice.

In reply to victim of mathematics:


So which one do you think is harder? They are both solid HVS in my world, one short and well protected but thuggy, the other longer, reachy and a bit bold.

Chris

In reply to Chris Craggs:

Agree - very different beasts. Both solid HVS.

Mick

In reply to Chris Craggs:

Well, Queersville has a couple of bold, easy moves and one 5a move with bomber gear right next to it. It was one of my first HVSs and it regularly comes up when people ask for recommendations of first HVSs on grit. The other one is Terrazza Crack.

Do you believe the logbook votes should be trusted absolutely as a measure of difficulty? You could save yourself a lot of work if you do...

In reply to victim of mathematics:


Terrazza is a standard HVS grit jamming crack - a 'mare if you don't have the right skill-set but aren't they all?


I start with the given grade, check the database (and all comments) for anomalies then come to a decision. As for saving myself a lot of work - it takes bloody hours to cross check them all, one of the more onerous tasks involved with the books!

Chris

In reply to victim of mathematics:



I've great respect for guidebook writers, particularly those who give their time voluntarily, and I've helped out in a small way where I can (eg Lundy).

I don't think for a moment that guidebook writers have forgotten that the grades are supposed to be for the onsight, but it is surely possible that in some cases their years of experience have mellowed their memories of how hard they actually found the route first time around?

I think it's generally acknowledged that grit grades feel on the hard side to those whose "home rock" is not gritstone. Perhaps grit aficionados find the grades harder in Pembroke, though I'm not so sure. But I accept this and take it into account. Of the grit routes I mentioned, only Chequers Buttress would definitely not merit E1 in another area IMO, but that doesn't mean I'd be voting E1 for these routes on grit. I've only seconded Terrazza Crack. That's one I would only contemplate leading it with my "E1 hat" firmly in place.

In reply to Alan James - Rockfax:

Well, I've really enjoyed reading this thread, nothing beats a good grade discussion! It is particularly satisfying that there will finally be a definitive (?!) grade for Three Pebble Slab - benchmark HVS 5a

In reply to Martin Hore:


Top tip - don't get suckered by your mates into leading it as the warmup, with the place covered in snow and a raw, icy wind blowing across the edge. They pissed themselves laughing!

Mick

In reply to Mark Kemball:


Actually I have realised over the years that the way to sell guidebooks is upgrade routes every edition and increase their star rating, so next time round TPS will be E2 5a ***, Tody's Wall E1 5b *** Chequer's Crack E2 5c *** and Valkyrie E1 5b, 5b ***.


Chris

In reply to Martin Hore:
I find grades elsewhere noticably easier than gritstone and northumberland sandstone so Im not grading from finding my local crags easy. Yorkshire perhaps being the hardest of the grit areas for me esp where pro gets sketchy on highballs.

For about 15 years I taught beginners and helped improvers numbering in the hundred's. I worked on the definitive guides with input from many sources. It may be regarded as arrogance but feel I know those Stanage classic grade designations I made from witnessing the experiences of all those climbers and thousands more ascents of strangers. With those partners I knew, it was often leading most of the batch and then the same with a new partner time and so on time and time again with similar results. It was not uncommon for them to say Inverted V felt solid VS as one of thier first leads at the grade but they changed their mind by the time they finished the list. Instructors we worked with told us the same thing.

Over the years the crack climbing skills involving chimney, wide crack or jamming I've witnessed in the population have got progressively worse on average which is why I think Chris is a little out on Terraza. I'm also really only a bog standard solid VS leader myself and like many of my experience onsight the odd extreme on a good day.

In reply to Chris Craggs:


Trying to work out if this is a parody or not... Strangely; I have the same dilemma when comparing your guidebooks to the definitive versions...

In reply to Toreador: Alan, Chris and Steve.




However, Face, Arete and Wall at Crookrise is an interesting example. It was first climbed pre 1923 and has been graded V.Diff in every guidebook since. I have recorded 3 ascents of it. First in 1971, then in 1988 (during work on the 1989 guide) and again in August 1997 (during work on the 1998 guide). Interestingly this last occasion was only a couple of weeks after I started climbing again after losing the fingers on my left hand. The notes column of my written logs for those dates make no comment about the grade and on both the later dates I soloed the route (including the 4c direct start in 1988). However, I note that I did up the P grade from P1 in the '89 guide to P3 in the '98? So I must have noticed something.

To be honest my recollections of the intricacies of the route now are vague but I suspect that the technicalities are low but the protection is poor and/or not easy to place above a big drop. P grades have now been dropped from the new series (not my decision - I liked them) but this is probably just the sort of route where they did add value. Without a P grade and given today's risk averse, climbing wall trained climbers, a general upgrade may be appropriate.

I've upgraded many routes during my 25 years involvement with guidebook production so I have no problem with it but, as I said at the start 'It's not an exact science' and the consensus does shift with time. So long as a 'right of appeal' or periodic review is built in to future guides, apps or databases then that is OK. I am sure that mistakes and anomalies will continue to made, found, debated and corrected long after Alan's new system has been installed. The great grade debate will never end - I hope!

Dave

In reply to Chris Cragg

I'm sure there is truth behind that humour... I've seen it work well in some indoor walls. Anyway Eastern Grit says 3PS is HVS (3rd from top of the graded list) and the earlier PGE said E1 (6 places up from the bottom) so if anything you are trying to cut sales. The UKC votes always said solidly into easy E1.

The EG graded list isnt bad either but it doesnt match UKC votes as for instance most of the easy (green) VS climbs have votes for mid-grade. Thank goodness for editorial common sense.

In reply to Dave Musgrove:
That 4c direct start is pretty special too...its not often I have to try hard second go on a genuine 4c problem. I don't think its all yorkshire's fault, a lot of it is the rest of the UK going soft. Plus several experienced bumblies who are keen on clearing out bold sandbags have suddenly arrived all at once during this guide's prep, a bit like buses.

In reply to Offwidth:


Thanks for that. Yes, I certainly agree that watching beginners/improvers adds a really valuable perspective, particularly in the easier grades. And I do appreciate the work you have put into improving the accuracy of the classic grades on eastern grit.

I entered this debate because I felt Victim's comment about people "en masse thinking that a route is harder than it actually is" was under-valuing the evidence of voted grades on UKC. I think voted grades have a part to play, for example by bringing to the table the opinions of those who come from out of area, but I'm certainly not suggesting that a voted grade should trump all the other evidence.

It's easy to misinterpret people's meaning on here, but I took it that Victim was suggesting that the "correct grade" as determined by the local guidebook activists, is forever the correct grade even in the face of "en-masse" evidence to the contrary in voted grades from the wider climbing public. I think that's probably going too far in the other direction.

There's a balance to be struck somewhere.

In reply to Martin Hore:

Well I've never argued that one should disregard the voting evidence out of hand, but there are some times when it is still right to ignore it, even when there's lots of votes. I mean over 2/3 of people who've voted on Inverted V think it's mid-grade VS or harder. Unless all the holds fall off it, or the polish becomes Stoney-esque, those 2/3 are never going to be right, no matter how many of them there are. I suspect Offwidth's suggestion that as a popular early VS lead, people find it hard and vote it up, even if they subsequently realise they were wrong. It's important to acknowledge these distortions.

I think we're (all) basically making the same point, just from different angles.

In reply to Martin Hore:




I think this is an excellent summary of the situation as I see it as well.

Alan

In reply to Martin Hore:

"I think voted grades have a part to play, for example by bringing to the table the opinions of those who come from out of area".

I'd hoped that too, but there is no way you can tell who voted for what on UKC (other sites like Mountain Project allow this) and sadly the evidence to me shows that the inside or outside influence of wise heads is ruined by bad data.

Mass UKC votes tell us what we already know (slightly less accurately) and where we have few votes they can be plain wrong. Yet if I noticed a chap called Martin Hore had said this particular VS seemed rather more like E1, for the following clear reasons, in a guidebook area I was working on, I would likely be out having a look. So please comment on less well climbed lines, especially if something is wrong but even if you think something is spot on. The votes do keep us honest in definitive work: where we place genuinely innapropriate grade markers, they become obvious all too soon.

In reply to Offwidth:




Your are right of course, but most consensus or majority voting systems are likely to be subject to this type of flaw.

The majority of people who take part in any survey are unlikely to acquire sufficient knowledge that will rival that as obtained from a minority of experts.


It depends on what the aim is,

For a decision to be acceptable by most, then a consensus or majority vote works well.
The cost in terms of time, effort and expense in obtaining the result is relatively low. The downside is that accuracy can be low.

For a decision to be accurate, there are better ways. But the cost in terms of time, effort and expense is usually high.
And then if the decision is not accepted by the majority well…….


There is one thing worse than getting a consensus or majority decision and that’s getting one you don’t agree with!

In reply to Alan James - Rockfax:

Yes Alan is correct, very very very hard!!
A lot of people have no idea what it takes to produce a guide book. Its like those people who download films for free and then decide to watch the making of, and then they see how much things cost and how much work goes in and what sacrifices are made.