Charlie ([info]ccfinlay) wrote,
@ 2007-03-01 10:16:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
The Nebula Process
I just posted these comments in a fit of passion over on Rich Horton's newsgroup in sff.net (on a thread that begins here), and I thought I should take ownership of them. If I'm going to stir the pot, I should do it here in my own kitchen. Now that the final ballot for the Nebula Awards, which includes the Norton, is announced, and I have forever taken off my Norton jury member hat, I have some thoughts on the Nebula process.

Both the rolling eligibility rule and the mix of recommendations/jury picks are counterproductive to the best interests of the Nebula.

With rolling eligibility, 4 out of 6 novels, 2 of 4 novellas, 4 of 5 novelettes, and 5 of 6 short stories were published in 2005. (And one of the two 2006 novels and the only 2006 short story were jury selections.) So, over 70% of the final ballot is from 2005, meaning that the 2007 Nebula Award is, for all practical intents and purposes, given to the best fiction of 2005. Some of these stories and books have already been on other award ballots in 2006 (To Crush The Moon was one of our finalists for the P. K. Dick Award, I know.) It makes the Nebula look out of date and, I suspect, rather irrelevant to readers -- outside the core group of voters -- as a reading list of current exceptional fiction.

The mixed system of jury/recs leads to similar problems. We ask publishers to provide books and magazines to a variable number of jury members throughout the year, changing the jury members from year to year, as if the juries have power to set the ballots the way they do in other awards. On the one hand, there's an expectation, as you state, later in the thread, that the jury will add work from more obscure sources (but not, fer goshsakes, Tor!), and yet at the same time, it is only the major publishers, like Tor, that seem to be able to afford to go to the trouble of sending out all the extra reading material to juries. To me it looks like, "We want your free books, but we mostly aren't going to pick them, and if we do, there will be a controversy about it." It just makes the process look silly. Not even bothering to add a story to the novella category, when it is short, and when half the stories on it are from 2005, makes the jury appear irrelevant. I like the Shunn and Melko novellas a very great deal, but do you mean to say that there were no other novellas published in 2006 that deserve to be on the ballot with them? Oh, wait, rolling eligibility... if they're really any good, members will rec them in time for the 2008 ballot so there's no need for the jury to act. Never mind that the shelf life of short fiction is even shorter than that of novels, and that the two-year delay makes the short fiction ballot look even less relevant as a reading list.

I think we should stick with the rec system but fix it. Make the Nebula only eligible for works in the immediate preceding year. Extend the deadline for recs from the last day of December to the last day of February, to give late-in-the-year publications a fair chance. In any category where not enough pieces have ten recs, the pieces with the next highest number of recs are taken to fill out the preliminary ballot. Move the date of the Awards if necessary, but let us come out ahead of the Hugos, for the same year, and same fiction as covered in the Hugos, so that the Nebula ballot can have some relevance. Trust the members of SFWA, and the rec process, so that we don't need the juries as a safety net.

If SFWA's not willing to fix the rec system, then get rid of it and turn the job completely over to the juries.

Either way, make the Nebula timely and relevant again.

Edited to remove the comment that short fiction markets send fiction to the Nebula jury: based on comments by former short fiction jury members below, apparently they don't.



(Post a new comment)


[info]lmarley
2007-03-01 03:30 pm UTC (link)
Just my humble opinion, but I think juries do a better job. All you have to do is look at the Nebula recommendation list and you see the way this particular business gets done. It's embarrassing and sort of silly.

Thanks for working on this, by the way. I sat on the P.K. Dick jury once myself, and it was a huge job. Pat yourself on the back.

(Reply to this) (Thread)


[info]ccfinlay
2007-03-01 03:49 pm UTC (link)
Geoff Landis and some others have expressed frustration with the inaction of the short fiction jury this year; but, yes, in general, I think the juries do a very good job. Though I feel obligated to point out that if the rec system functioned properly, the whole membership would be acting as a jury.

And thanks. But I think... I... pulled... my... shoulder... out... of... socket!

(Reply to this) (Parent)


[info]the_flea_king
2007-03-01 03:30 pm UTC (link)
hear HEAR.

(Reply to this)


[info]wistling
2007-03-01 03:37 pm UTC (link)
I was wondering why there were so many works from 2005!

(Reply to this)


[info]papersky
2007-03-01 03:43 pm UTC (link)
This is a situation where I shouldn't say anything, isn't it?

Could you give a link to the original discussion?

(Reply to this) (Thread)


[info]ccfinlay
2007-03-01 03:45 pm UTC (link)
http://webnews.sff.net/read?cmd=read&group=sff.people.richard-horton&artnum=3909

I'll add it to the main post as well.

(Reply to this) (Parent)(Thread)


[info]papersky
2007-03-01 03:55 pm UTC (link)
Thank you.

(Reply to this) (Parent)


[info]ccfinlay
2007-03-01 03:52 pm UTC (link)
Also, I think that part of the problem is that, since members both vote for and are eligible for the Nebula, and since many of the most prominent members in our field, who are spokespeople in other cases, appear on the Nebula ballot in any given year, the result is that many of the people most qualified to speak out, and most capable of effecting change, end up being silent. To which I say, Grrrr.

(Reply to this) (Parent)(Thread)


[info]papersky
2007-03-01 04:01 pm UTC (link)
I am not a member of SFWA.

And while I think you're making some really good points, I still think it would be inappropriate for me to say anything at all right now.

Large weather we're having this morning.

(Reply to this) (Parent)(Thread)


[info]ccfinlay
2007-03-01 04:45 pm UTC (link)
I was woken in the wee hours this morning by a spring thunderstorm, with a lightning bolt that hit so close it set off car alarms outside my window. I was going to blog about that instead.

(Reply to this) (Parent)


[info]sallytuppence
2007-03-01 03:47 pm UTC (link)
YES.

(Reply to this)


[info]buymeaclue
2007-03-01 03:55 pm UTC (link)
Why _not_ a novel from Tor (f'g'sake)?

I see Horton's explanation, and I'm all for promoting work from quirky and unexpected sources. But I'm still utterly baffled by the idea that an award ballot shouldn't be made up of the best fiction published in the eligibility period (which, yeah, I think would make more sense if it was tweaked), full stop.

Which is to say, if an overlooked book is the best contender for that jury spot, that's great. But if the best book not on the ballot comes out from Tor, why on earth should the jury turn it down in favor of a less awesome book from someone littler?

Because, I would guess, (general-)you assume that the folks who rec stories would have seen the Tor book and judged it to be lacking. But I'm not sure that's a safe assumption to make.

Recommending really good but not best-of-the-period work that people might not have heard of seems to me the job of a shortlist.

(Reply to this) (Thread)


[info]buymeaclue
2007-03-01 03:56 pm UTC (link)
(I would hope it's not necessary to say this, but just in case. I haven't read Farthing; this is not a defense of that particular book.)

(Reply to this) (Parent)


[info]ccfinlay
2007-03-01 04:02 pm UTC (link)
"Why _not_ a novel from Tor (f'g'sake)?"

I'm not the right person to ask, if you mean this as a serious and not a rhetorical question. I think a jury should pick whatever they think are the best works -- that's a jury's job. But it is my belief that the fiction juries are explicitly charged to, and expected to, find works that have been overlooked by the membership through the rec process. So any work that has some recs (but not enough), as works from major magazines and publishers often do, is likely to get passed over.

(Reply to this) (Parent)(Thread)


[info]buymeaclue
2007-03-01 04:08 pm UTC (link)
I don't ask questions if I'm not curious about the answers. (I may, as here, ask a question that the poster may not be able to answer, because I figure someone who sees the question may be able to.)

I mean, in this case I've clearly got my own opinion. But yeah, if anyone can offer up some reasons, I'd be absolutely keen to hear 'em. (Which is why I just posted a trimmed version of that comment over at the sff.net discussion. Now if I can just remember to check back and see what Rich says, if anything.)

I think this:

>But it is my belief that the fiction juries are explicitly charged to, and expected to, find works that have been overlooked by the membership through the rec process.

Does make a whole lot of sense.

(Reply to this) (Parent)(Thread)


[info]ccfinlay
2007-03-01 04:26 pm UTC (link)
Yes, but with rolling eligibility, how does that work? Juries are only allowed to look at works from, say, 2006. But many works from 2006 continue to be eligible throughout 2007. So they are faced with the problem that members may end up not overlooking the works... which, based on the age of most of the things on the ballot, is a very real possibility. And while they could easily tell which works from 2005 have been overlooked, but the juries are not allowed to consider those, even though everything else on the ballot in a category may be from that year.

(Reply to this) (Parent)(Thread)


[info]buymeaclue
2007-03-01 04:27 pm UTC (link)
You deleter, you. :-p

Since I agree the rolling eligibility thing is a little silly, I think I'd say, "It works after that gets adjusted."

(Reply to this) (Parent)


[info]tanaise
2007-03-01 08:52 pm UTC (link)
I think I could see the argument being "Tor's a big publisher, enough other people will read this book, see its virtues, and rec it on their own, so it will end up on the list in time without Jury help."

In that case, it's a logical extension of that argument to say, 'if this book will make it to the list on its own in time, maybe we should give this ballot spot to something more obscure now.'

But at the same time, if you *do* say that, you then run the risk of that book somehow not making the necessary number of recs, and thus never making the ballot at all.

(Reply to this) (Parent)(Thread)


[info]buymeaclue
2007-03-01 08:57 pm UTC (link)
I can see the argument, but as you say, it leans on assumptions that I just don't think are safe to make.

And even if they are safe...a system that encourages leaving work off the ballot in favor of inferior work still strikes me as deeply, deeply weird.

(Reply to this) (Parent)(Thread)


[info]tanaise
2007-03-01 09:09 pm UTC (link)
I'm now building a giant house of assumptions here, but he could also be saying, "don't just go with the easy choice/big publisher, shop around, make sure it's what you really want."

Except that, as you're mentioning, since we're outside of the process, we don't know what the novel group also looked at, and which they rejected, or why they picked that book. I'm remembering the discussion of the WFC awards that I went to which explained why it gave the award to the collection it gave it to, which many had disagreed with.

(Reply to this) (Parent)


(Anonymous)
2007-03-02 01:53 am UTC (link)
I see the point you're making -- why not have the "best possible book" be the one added?

But who defines "best book"? I really think the implication of the Nebula process is that this choice should be made by the membership as a whole. Sure, we might disagree! Of course we do, sometimes! That's why I think the jury should not be, as it were, second-guessing SFWA members. Rather, they should be giving SFWA members a chance to consider works that might have missed.

And of course, in the case of a book like FARTHING, the membership was not necessarily done speaking ... Recommendations could have been made through August.

All that said, of course it's a wonderful book, and I'll be very happy if it ends up winning.

The other point you make -- that it's dangerous to assume that just because a book is put out by Tor (or Bantam Spectra, or Ace, or Baen) the SFWA membership will all have noted its existence -- is fair enough. But I think the solution to that is not to have the jury's one pick fix things -- there were several outstanding Tor novels from 2006 that didn't make the preliminary ballot, after all. The solution is to make more sweeping changes to the overall process.

--
Rich Horton

(Reply to this) (Parent)(Thread)


[info]buymeaclue
2007-03-02 02:00 pm UTC (link)
Thanks for taking the time to respond; it's much appreciated.

>But who defines "best book"?

In terms of the award or in terms of the jury selection (if any)? The active members of SFWA and the jury, respectively.

>That's why I think the jury should not be, as it were, second-guessing SFWA members. Rather, they should be giving SFWA members a chance to consider works that might have missed.

I guess I'm just not sure what you're seeing as a second-guess, or more of one than adding a book from a big genre publisher.

Also, having a jury at all seems to imply the possibility that the voting membership may miss something or leave out a worthy book. What's wrong with acknowledging that?

Or do I misunderstand what you mean below by, "Dummies, you made a mistake?"

>The solution is to make more sweeping changes to the overall process.

Agreed.

(Reply to this) (Parent)


[info]oracne
2007-03-01 04:01 pm UTC (link)
I've been thinking about this, and have come to no conclusions yet. I do like the jury system, but I also wish the membership were more proactive in reccing. Except I worry that no one is READING, so no one has anything to rec. Hmm.

(Reply to this)


[info]14theditch
2007-03-01 04:02 pm UTC (link)
Charlie: My novel has even made the final ballot this year, a 2005 pub., and I totally agree with you.

(Reply to this)


[info]secritcrush
2007-03-01 04:03 pm UTC (link)
So, a couple of comments:

1) My year on the short fiction jury (2 years ago) I got not a free bit of anything - I have no idea what was provided, but I wouldn't chastise people or suggest they look silly for what they may not have received.

2) I think it's really important for the jury to come to a consensus - I think no addition is better than one that some member were violently against. Further, there is no mandate to add a story in every category - the juries can only read as much as they read. I know there were a number of stories I suggested that no one else on the jury read so they ended up being non-starters as far as jury discussion went.

3) I think "overlooked" is a better term for what the juries are supposed to add rather than obscure. And overlooked can come from a may come from a major house, especially if a writer is not well liked in the community or is perceived as lightweight. I also think little known writers who aren't active in the SF community tend to get overlooked - look how long it took Mary to make the ballot. She's been writing award quality fiction for a long time and the idea that just because it was published in F&SF it should be considered by the jury, well I don't agree.

The more bounds you set on a jury the less well they'll picking things to add.

ps. I think it's time for paul to introduce his awesome plan for revamping the Nebulas.

(Reply to this) (Thread)


[info]ccfinlay
2007-03-01 04:08 pm UTC (link)
Ah, thanks, chance. In a way, I'm glad to hear that the short fiction members weren't getting any free reading material.

I qualified obscure to overlooked in one of the comments above, but that point is also well made, and well, yeah, I agree with the rest of it.

(Reply to this) (Parent)(Thread)


[info]tim_pratt
2007-03-01 05:18 pm UTC (link)
I didn't get anything free when I was on the short fiction jury. I work at Locus, and we get almost all the magazines there, so I read them that way. I don't know how anybody else managed.

(Reply to this) (Parent)(Thread)


[info]ccfinlay
2007-03-01 05:25 pm UTC (link)
Yeah, maybe I should change that in the top post. I thought I recalled seeing short fiction jury members ask who they should contact to get their fiction; I assumed that it worked like the Norton did (which is the jury I have experience with), where we received eligible works.

(Reply to this) (Parent)


[info]autopope
2007-03-01 04:44 pm UTC (link)
I think this doesn't go far enough.

It's time to scrap the Nebula rules completely and reboot them from scratch. Start by defining the award categories, the timespan for eligibility, and the other basics. Then either clone the Hugo rules (for nominations and voting) using the SFWA membership base instead of the WSFS membership, or try and figure out a way of mangling the Clarke Award rules (for a jury based award) to cope with the larger volume of eligible works in the US market.

Or merge the two. (Hugo style nominations open to all, then a shortlist of ten items is selected and a jury -- selected by lot, with shortlisted writers excluded -- get to vote on a winner.)

Right now, the Nebulas are far too susceptible to log-rolling at the rec stage, eligibility is all over the map, then it turns into a procedural dog's dinner. I've been trying to get my head around it for a couple of years now and I still don't understand how the system works; either I'm thick, or the system's too damn baroque. And it's delivering perverse shortlists and winners which undermine the credibility of the award.

(Reply to this)


[info]tanaise
2007-03-01 06:21 pm UTC (link)
That's the sort of crazy talk that starts slapfights!

(Reply to this) (Thread)


[info]buymeaclue
2007-03-01 11:21 pm UTC (link)
If only!

(Reply to this) (Parent)


[info]clarkesworld
2007-03-01 06:24 pm UTC (link)
I agree completely. Does anyone happen to know the history behind the current rules?

(Reply to this)


(Anonymous)
2007-03-02 01:38 am UTC (link)
OK, here's a more complete look at what I think about the whole Nebula process.

First, I want to say -- as I already have a couple of times -- that I think FARTHING is a first rate book, very possibly the best novel on the shortlist. Nothing I will say is directed at FARTHING (nor at "The End of All Things") -- only at the Nebula nomination process. (And I note that my respect for FARTHING is well-documented in the past -- see my review at Amazon (and at my SFF.Net newsgroup), or for that matter my recent list of 2006 novels including my Hugo nomination ballot -- which included FARTHING.)

I have twice been a member of the Nebula Short Fiction jury. As such, I do know something of how jury deliberations go. And I have an impression of the purpose of the jury that was, I believe, held largely in common by the members of the juries on which I participated. That doesn't mean I'm right, of course. But what I will say is, I'll note, something I've seen said by other SFWA members -- I don't think my opinions here are unusual.

First, I do not know very much about the historical genesis of the Nebula jury, nor what was said at that time about its purpose. I'd be thrilled if someone could enlighten me. (I have a notion this subject may have come up before and that I've forgotten what was said!)

The first thing to note is that the Nebula is NOT primarily a juried award. The bulk of the nominations are, and always have been, determined by membership recommendations, and then by membership votes trimming this recommendations to a manageable shortlist. (Some details of this process have changed over time -- for instance, at the first I believe there was no preliminary ballot to final ballot step, and some early "shortlists" were pretty long.) And of course the winner is determined by membership vote. The only jury contribution is that each jury has the option to add one (and only one) story in each Nebula category.

What should that story be? Should it be the best story not on the preliminary ballot? I can see the argument for that -- but it bothers me. Because, in essence, that is overriding the preferences of the voting membership. It's saying, "Dummies, you made a mistake, and we're fixing it." And, of course, if we are doing this -- adding one worthy story -- why not more? Why not add as many Nebula worthy stories as possible? OK, I'm being a bit facetious -- but not entirely. Is the jury's function is only to "improve" the ballot by fixing what it must see as recommendation process mistakes?

--
Rich Horton
[More on next rock]

(Reply to this)


(Anonymous)
2007-03-02 01:39 am UTC (link)
Here's the rest of my long comment ...

All this is further complicated by the "rolling eligibility" rule. The jury can only consider stories from one calendar year -- 2006, in this case. But (as a look at the preliminary and final ballots from any recent year will show), the actual ballots usually consist of more stories from the previous year (2005, in this case) than from the current year. So in a sense the jury and the membership body are dealing with apples and oranges ... or, put another way, the membership body has not been proven to have "made a mistake" in not putting a 2006 story on the 2007 Preliminary Ballot. As of December 31, FARTHING had 3 recommendations. Might it not have gotten 7 more by August 2007? I certainly hope so -- I confess, to be sure, that there is no guarantee that it would have done. Had it not, that would have been a terrible shame, because it is a worthy novel. But that would at least have represented, in some amorphous sense, "the will of the membership".

Then why have a jury at all? That's a pretty good question. Maybe we don't need one.

But it there is one thing that a jury CAN do, that it seems to me dovetails well with the overall system. And that is to bring to the membership's attention worthy works that will have had a hard time accumulating 10 recommendations, because people may not have seen them. I called these "obscure" books, but I agree that the term someone else suggested, "overlooked", is better. Why might a book be overlooked? That's obviously a fuzzy concept, but two categories leap to my mind. One is that a book (or story, let's not forget story) might have been published by a small press (or perhaps a less well-known online venue), and not gotten much distribution or publicity. The other is that the book might have been published in another genre, so that the membership simply didn't notice it, or think of it as a Nebula candidate. That's what I have long thought is the appropriate category of story/book for jury picks.

This year that might have meant a chance for a novel like AGAINST THE DAY, or THE ROAD, or THE BRIEF HISTORY OF THE DEAD, to be on the Nebula shortlist. (The last, I will note, is based on a short story that the jury (in a year I was on the jury) added to the shortlist.) Now mind you, I have no argument with anyone who thinks that FARTHING (or any other novel) is better than those three novels -- it just seems to me that they represent book the membership might have overlooked. But why would the membership overlook a Tor novel? (Though to be fair, THE BRIEF HISTORY OF THE DEAD had 2 recs as of December 31, and THE ROAD had 3 -- with until next September to accumulate more.)

One more point -- some have noted that a very small novella category was not augmented by the jury. And in my view there were some fine stories from potentially "overlooked" categories -- stories like "Botch Town" and "Map of Dreams" and "The Maid on the Shore" and "Missile Gap" and "Incident on a Small Colony". But I agree that the jury should err on the side of not including stories just to include a story -- and that they should nominate based on consensus. So I won't dispute in the least a decision to not nominate a novella (or novelette, for that matter).

In reality, this is all a small subset of the overall set of problems (in my opinion) with the current Nebula process. I think much greater concerns are -- 1) a relative lack of recommendations, or more particularly perhaps, lack of recommenders; 2) the vulnerability of the process to logrolling; and 3) the confusion caused by the rolling eligibility rule. I have my obvious reservations about the jury picks, but in reality the result this year (and in most years) has been a better overall ballot. And there is no question that the inclusion of FARTHING on the novel shortlist improves the overall ballot. It's just that I think it would have improved next year's ballot, too ...

I agree with the concerns expressed by Charlie and Charlie ... and I recall Paul Melko's suggestions from a year or so back. I thought them interesting and well worth consideration. I think the process is pretty much broken -- something ought to change.

--
Rich Horton

(Reply to this)


(Anonymous)
2007-03-02 01:44 am UTC (link)
One minor note, concerning short fiction juries and free stuff.

In my two stints on the jury, I did get very occasional free stuff. One major magazine sent me free copies one year, but not the second year. And some authors sent typescripts of stories to consider.

But for the most part, no, short fiction jury members don't get much free stuff ...

One thing we tried to do on the juries I participated in was recommend to the rest of the jury stories we thought particularly worth consideration. (I tried to include stories that I might not have picked myself but that I thought likely to interest other people -- a chancy process, I agree.) The chairman (Bud Webster, both years, as it happens) did attempt to get copies of some of these stories for jury members, with mixed success.

That's one way, at any rate, to try to get potentially "overlooked" stories to be seen more widely.


I should add that

(Reply to this) (Thread)


(Anonymous)
2007-03-02 01:54 am UTC (link)
Darned if I can remember what I meant to add.

Except that I should have signed the post ...

--
Rich Horton

(Reply to this) (Parent)


Create an Account
Forgot your login?
Login w/ OpenID
English • Español • Deutsch • Русский…