Flock voting issues

[Josh Boyer]

Hello,

Over the past few weeks the Flock staff has dealt with a number of
issues regarding talks and votes.  We'd like to address these shortly,
so as to limit similar issues in the immediate future.

It has come to our attention that at least one Flock talk received a
large number of votes from people that have no realistic intention of
attending the conference.  We have also had requests to swap out
accepted talks with a talk that wasn't even on the ballot, or to
combine the scores of two low voted talks into one talk about both
topics.  While none of these actions or requests are against any
rules, they certainly go against the spirit of voting for Flock talks.
The vote is to allow people attending the conference to have a large
hand in setting the agenda for it, therefore making it the best
possible conference for the participants.

None of the aforementioned actions or requests have had any impact on
the actual Flock agenda, but they are still disappointing.  The Flock
staff will be reevaluating talk votes, the process we use for them,
and the eligibility of voters for the next Flock.  We welcome
discussion on these topics so that we can reach a plan that best meets
the Fedora Project's goals.

josh, for the Flock staff.

[Matthew Miller]

On Thu, Jun 04, 2015 at 03:56:11PM -0400, Josh Boyer wrote:
>  topics.  While none of these actions or requests are against any
>  rules, they certainly go against the spirit of voting for Flock talks.
>  The vote is to allow people attending the conference to have a large
>  hand in setting the agenda for it, therefore making it the best
>  possible conference for the participants. 
I've always thought of it as a general gauge of community interest in
the topic, which hopefully corresponds to the interest of the people
going. We could try simply messaging that better — and by "we", I
definitely include myself, because I know I've said things like
"voting is open everyone come vote, and hope to see you there". 

On the other side of things, I see 136 voters for talks, and currently
171 registered attendees — especially if a large number of those 136
aren't attendees, that ratio could be better.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Ruth Suehle]

On Thu, Jun 4, 2015 at 5:07 PM, Matthew Miller <mattdm(a)fedoraproject.org&gt;
wrote:

>  I've always thought of it as a general gauge of community interest in
>  the topic, which hopefully corresponds to the interest of the people
>  going. We could try simply messaging that better — and by "we", I
>  definitely include myself, because I know I've said things like
>  "voting is open everyone come vote, and hope to see you there".
>  

While conceptually that works, "I got all my friends to vote for me" isn't
a good gauge of overall interest. This is also partly why the final
schedule is 99% community vote with 1% room for something like if not
enough people voted on hearing the Fedora Council get together, but we know
that's still important, we put it on the schedule anyway.

[Matthew Miller]

On Thu, Jun 04, 2015 at 05:32:12PM -0400, Ruth Suehle wrote:
>  > I've always thought of it as a general gauge of community
> interest in
>  > the topic, which hopefully corresponds to the interest of the people
>  > going. We could try simply messaging that better — and by "we", I
>  > definitely include myself, because I know I've said things like
>  > "voting is open everyone come vote, and hope to see you there".
>  While conceptually that works, "I got all my friends to vote for me" isn't
>  a good gauge of overall interest. This is also partly why the final
>  schedule is 99% community vote with 1% room for something like if not
>  enough people voted on hearing the Fedora Council get together, but we know
>  that's still important, we put it on the schedule anyway. 
I guess also my corresponding assumption was that it was more like 20%
community vote.


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Josh Boyer]

On Thu, Jun 4, 2015 at 5:53 PM, Matthew Miller <mattdm(a)fedoraproject.org&gt; wrote:
>  On Thu, Jun 04, 2015 at 05:32:12PM -0400, Ruth Suehle wrote:
> > > I've always thought of it as a general gauge of community interest in
> > > the topic, which hopefully corresponds to the interest of the people
> > > going. We could try simply messaging that better — and by "we", I
> > > definitely include myself, because I know I've said things like
> > > "voting is open everyone come vote, and hope to see you there".
> > While conceptually that works, "I got all my friends to vote for me"
> isn't
> > a good gauge of overall interest. This is also partly why the final
> > schedule is 99% community vote with 1% room for something like if not
> > enough people voted on hearing the Fedora Council get together, but we know
> > that's still important, we put it on the schedule anyway.
>
>  I guess also my corresponding assumption was that it was more like 20%
>  community vote. 
Really?  Did you even really think about this before you made such an
assumption?  Let's break some stuff down here.

1) The number of talks submitted, while encouragingly growing year
over year, is not so plentiful as to allow us to pick and choose talks
on a whim.

2) If you compare the set schedule with the voting results, you will
find an extremely high correlation of high voted talks on the agenda.
To the point where some of the talks were accepted even with large
reservations from the planning committee.  Multiple times.

3) Why would we even bother holding a vote if the results were going
to be essentially meaningless?  What would be the point of the hassle
and arguing about it?

So, to sum it up, the votes matter, they matter probably the most out
of everything, and the committee tries very very hard to not overrule
or deviate from the results without a very good reason.  Which means
that people that are attending need to vote and that we need to avoid
gamesmanship involving people who otherwise do not care.

josh

[Matthew Miller]

On Fri, Jun 05, 2015 at 08:22:20AM -0400, Josh Boyer wrote:
>  > I guess also my corresponding assumption was that it was more
> like 20%
>  > community vote.
>  Really?  Did you even really think about this before you made such an
>  assumption?  Let's break some stuff down here. 
Uh... yes. Yes I did. Where is this hostility coming from? You asked
for comments!

>  1) The number of talks submitted, while encouragingly growing year
>  over year, is not so plentiful as to allow us to pick and choose talks
>  on a whim. 
Who said anything about "on a whim"?


>  2) If you compare the set schedule with the voting results, you will
>  find an extremely high correlation of high voted talks on the agenda.
>  To the point where some of the talks were accepted even with large
>  reservations from the planning committee.  Multiple times. 
I haven't gone and compared. I see all of the past voting results are
open now, but last year I rememer them being embargoed and I assumed
that was intentional. If I'm misremembering or misunderstood, I'm
sorry.


>  3) Why would we even bother holding a vote if the results were going
>  to be essentially meaningless?  What would be the point of the hassle
>  and arguing about it? 
A lot has been made over how Flock differs from recent FUDCons in that
it has an intentional, pre-planned scedule rather than a bar-camp
schedule -- which is to say, a schedule primarily made based on votes
of the attendees. I don't think it's unreasonable at all to take that
to mean that other factors are considered heavily.

That doesn't mean that it's _meaningless_, though. I would have said
"0%" if I meant meaningless. I said "20%", which, sure, is an
arbitrary
number I made up, but by that I mean: given significant weight, but
only one of a number of important factors considered by the committee.



-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Tom Callaway]

On 06/05/2015 09:38 AM, Matthew Miller wrote:
>  I said "20%", which, sure, is an arbitrary
>  number I made up, but by that I mean: given significant weight, but
>  only one of a number of important factors considered by the committee 
It would be interesting to hear from you (and others) what factors that
you (and others) think should be included in the decision making process.

I think in the past (including this year), we have looked objectively at
the talks submitted (with the advantage of being able to see the
speaker) and tried to ensure that the accepted talks are the most
useful/valuable/interesting to the flock community. But we've really
only had to make that choice for two years now (Flock 1 was a "accept
everything" year, due to the number of submissions received).

~tom

==
Red Hat

[Matthew Miller]

On Fri, Jun 05, 2015 at 09:48:41AM -0400, Tom Callaway wrote:
>  > I said "20%", which, sure, is an arbitrary
>  > number I made up, but by that I mean: given significant weight, but
>  > only one of a number of important factors considered by the committee
>  It would be interesting to hear from you (and others) what factors that
>  you (and others) think should be included in the decision making process. 
Definitely. I think the "useful/valuable/interesting to the Flock
community" factor is a large one, and the vote gives one indication of
that but the expert judgement of the committee can help with that too.
Additionally:

- Balancing speakers we know are well-respected and big draws with new
  voices and diverse perspectives

- Quality of and thought put into the proposal submission

- When there are submissions covering overlapping topics, choosing the
  best — when there are many talks proposed about a topic interesting
  to me, I might vote them all up, but really I just want to make sure
  there's one good talk on that topic

- And expanding on that, do we have good coverage across the project?

- Alignment with Flock's intended development/planning focus

- Alignment with current Fedora Project 12-18 month objectives (which
  are themselves intended to be selected by community process)


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Matthew Miller]

I came across this post from the planners of PostgreSQL's conference,
where they explain _their_ process
<https://www.postgresql.us/node/150> — maybe there's something to take
from that.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Josh Boyer]

On Fri, Jun 5, 2015 at 11:05 AM, Matthew Miller
<mattdm(a)fedoraproject.org&gt; wrote:
>  I came across this post from the planners of PostgreSQL's
> conference,
>  where they explain _their_ process
>  <https://www.postgresql.us/node/150> — maybe there's something to take
>  from that. 
Possibly.

However, before we go and evaluate a bunch of other selection
processes, it would be helpful to know what problem we're solving.  In
other words, what shortcomings or issues do you see with how Flock
talks are chosen today?  We've already highlighted the potential for
gaming, and perhaps a broader lack of understanding around importance
of the votes.  What else do you see as something that needs fixing?

josh

[Matthew Miller]

On Fri, Jun 05, 2015 at 11:34:11AM -0400, Josh Boyer wrote:
>  However, before we go and evaluate a bunch of other selection
>  processes, it would be helpful to know what problem we're solving.  In
>  other words, what shortcomings or issues do you see with how Flock
>  talks are chosen today?  We've already highlighted the potential for
>  gaming, and perhaps a broader lack of understanding around importance
>  of the votes.  What else do you see as something that needs fixing? 
Well, first I think it's come out really well in practice. But of the
things on my list:

  1) reduce gaming,

  2) make sure we separate out the attendee-interest and
     general-community-interest factors so they can considered
     separately, and

  3) prioritize talks which focus on Fedora
     project development, contributor community, and planning.

For 3, to pick on something in specific as an example (but not an
exclusive one), "Sysadmin in the age of docker and nodejs" sounds like
an awesome, interesting talk that I care about — but it seems really
user-focused. Something maybe better for a FUDCon, or for a general
Linux user conference, or for a sysadmin conference like LISA.
On the other hand, that same talk refocused as "How can Fedora Cloud
[or Server] best meet sysadmins' needs in this crazy new docker/nodejs
world?" might be right on target.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Josh Boyer]

On Fri, Jun 5, 2015 at 9:38 AM, Matthew Miller <mattdm(a)fedoraproject.org&gt; wrote:
>  On Fri, Jun 05, 2015 at 08:22:20AM -0400, Josh Boyer wrote:
> > > I guess also my corresponding assumption was that it was more like 20%
> > > community vote.
> > Really?  Did you even really think about this before you made such an
> > assumption?  Let's break some stuff down here.
>
>  Uh... yes. Yes I did. Where is this hostility coming from? You asked
>  for comments!
>
> > 1) The number of talks submitted, while encouragingly growing year
> > over year, is not so plentiful as to allow us to pick and choose talks
> > on a whim.
>
>  Who said anything about "on a whim"?
>
>
> > 2) If you compare the set schedule with the voting results, you will
> > find an extremely high correlation of high voted talks on the agenda.
> > To the point where some of the talks were accepted even with large
> > reservations from the planning committee.  Multiple times.
>
>  I haven't gone and compared. I see all of the past voting results are
>  open now, but last year I rememer them being embargoed and I assumed
>  that was intentional. If I'm misremembering or misunderstood, I'm
>  sorry.
>
>
> > 3) Why would we even bother holding a vote if the results were going
> > to be essentially meaningless?  What would be the point of the hassle
> > and arguing about it?
>
>  A lot has been made over how Flock differs from recent FUDCons in that
>  it has an intentional, pre-planned scedule rather than a bar-camp
>  schedule -- which is to say, a schedule primarily made based on votes
>  of the attendees. I don't think it's unreasonable at all to take that
>  to mean that other factors are considered heavily. 
Yes, it's a pre-planned schedule.  Almost entirely because bar-camp style meant:

a) We wasted a lot of time making the schedule on the first day, and
then adjusting it almost daily as the conference went on.

b) People had no idea if they wanted to attend because they had no
idea what the conference talks would actually be.

c) People had no idea if their talk would get enough votes, so they
didn't prepare which led to some pretty poor presentations on what
otherwise could have been very good material.

d) Sponsors had no idea who was going to attend, what the content
would be, or why it might be worth sponsoring.

What bar-camp _did_ provide was a conference generated by the
attendees, which is actually a great way to make sure people feel
engaged and excited.  By having the vote on talks ahead of time we
retain that high level of participation and we can set the schedule
well in advance and drum up further interest.  Pre-planned doesn't
mean anything more than "removing inefficiencies of bar-camp" really.

>  That doesn't mean that it's _meaningless_, though. I would
> have said
>  "0%" if I meant meaningless. I said "20%", which, sure, is an
> arbitrary
>  number I made up, but by that I mean: given significant weight, but
>  only one of a number of important factors considered by the committee. 
Your math and my math differ.  20% is not significant.

josh

On Jun 5, 2015 1:26 AM, "Josh Boyer" <jwboyer(a)fedoraproject.org&gt; wrote:
>  Hello,
>
>  Over the past few weeks the Flock staff has dealt with a number of
>  issues regarding talks and votes.  We'd like to address these shortly,
>  so as to limit similar issues in the immediate future.
>
>  It has come to our attention that at least one Flock talk received a
>  large number of votes from people that have no realistic intention of
>  attending the conference.  We have also had requests to swap out
>  accepted talks with a talk that wasn't even on the ballot, or to
>  combine the scores of two low voted talks into one talk about both
>  topics.  While none of these actions or requests are against any
>  rules, they certainly go against the spirit of voting for Flock talks.
>  The vote is to allow people attending the conference to have a large
>  hand in setting the agenda for it, therefore making it the best
>  possible conference for the participants. 
exactly. if particular talk get 100 votes and none of them going to attend
flock. Then it does not make sense to select that talk and have empty room
during session.

I think initially forming small committee of different domains experts to
shortlist talk will be very helpful. May be 3-4 members. it will help to
evaluate topic strength/imp, speakers credibility.
May be then we can open it for voting.

-
Pravin Satpute

[Neville A. Cross]

On Fri, 2015-06-05 at 07:23 +0530, pravin.d.s(a)gmail.com wrote:
>  
>  <skip>
>  I think initially forming small committee of different domains 
>  experts to shortlist talk will be very helpful. May be 3-4 members. 
>  it will help to evaluate topic strength/imp, speakers credibility. 
>  May be then we can open it for voting.
>  -
>  Pravin Satpute 
I like the general direction of this idea. Lot of room for boil down
details of who, how, when....

On the other hand speaker credibility may be a good thing for
sponsoring the best people to talk... I hope having a better list of
speakers may help to sponsor people that may have a multiplying effect
in their local communities. Even that they do not provide talks. Not
really a well formed idea. Just top of my head.

In any case, I will be looking forward to a session on how to improve
premier events.

Neville

[Truong Anh Tuan]

----- Original Message -----
>  From: "Neville A. Cross" <yn1v(a)taygon.com&gt;
>  To: flock-planning(a)lists.fedoraproject.org
>  Sent: Friday, June 5, 2015 10:29:54 PM
>  Subject: Re: Flock voting issues 
>  On Fri, 2015-06-05 at 07:23 +0530, pravin.d.s(a)gmail.com wrote:
> > 
> > <skip>
> > I think initially forming small committee of different domains
> > experts to shortlist talk will be very helpful. May be 3-4 members.
> > it will help to evaluate topic strength/imp, speakers credibility.
> > May be then we can open it for voting.
>  
>  I like the general direction of this idea. Lot of room for boil down
>  details of who, how, when....
>  
>  On the other hand speaker credibility may be a good thing for
>  sponsoring the best people to talk... I hope having a better list of
>  speakers may help to sponsor people that may have a multiplying effect
>  in their local communities. Even that they do not provide talks. Not
>  really a well formed idea. Just top of my head. 
I think the idea to sponsor some most active people in local/regional
communities is good. It is not about an award for their active
contributions or things like that (we are almost volunteers and we do
not expect those kinds of prize when we do contributing to the project)
but it is about they have got a lot of things in their minds to share
with and to improve the whole community.

Following this way, I think it is better to sponsor for people who most
active in their specific areas, e.g. devel, packaging, docs, ambassadors,
globalization, marketing, design, etc. in short, FOSCo/FAmSCo and FESCo
members and some other people.

We will have some other ways to choose the right person to sponsor
(voting is one of these). Nothing is perfect and we may sometimes miss
some people. But I think mixing all of these, we will have the best list
of person to sponsor to each Flock.

Rgds,
Tuan

[Stephen Gallagher]

On Thu, 2015-06-04 at 15:56 -0400, Josh Boyer wrote:
>  Hello,
>  
>  Over the past few weeks the Flock staff has dealt with a number of
>  issues regarding talks and votes.  We'd like to address these 
>  shortly,
>  so as to limit similar issues in the immediate future.
>  
>  It has come to our attention that at least one Flock talk received a
>  large number of votes from people that have no realistic intention of
>  attending the conference. 
Does this account for people who cannot attend but plan to view the
recordings later? Because I'd argue those people *should* still have a
vote.

[Josh Boyer]

On Fri, Jun 5, 2015 at 8:07 AM, Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  On Thu, 2015-06-04 at 15:56 -0400, Josh Boyer wrote:
> > Hello,
> >
> > Over the past few weeks the Flock staff has dealt with a number of
> > issues regarding talks and votes.  We'd like to address these
> > shortly,
> > so as to limit similar issues in the immediate future.
> >
> > It has come to our attention that at least one Flock talk received a
> > large number of votes from people that have no realistic intention of
> > attending the conference.
>
>  Does this account for people who cannot attend but plan to view the
>  recordings later? Because I'd argue those people *should* still have a
>  vote. 
In this specific situation, doubtful.

In general, your point is valid.  However, streaming and recording is
always one of the more troublesome items to cover.  We'll need to make
sure it is actually going to be done before the vote is opened so
people are aware if that is even an option.

josh

[Neville A. Cross]

On Fri, 2015-06-05 at 08:15 -0400, Josh Boyer wrote:
>  On Fri, Jun 5, 2015 at 8:07 AM, Stephen Gallagher <
>  sgallagh(a)redhat.com&gt; wrote:
>  > On Thu, 2015-06-04 at 15:56 -0400, Josh Boyer wrote:
>  > > Hello,
>  > > 
>  > > Over the past few weeks the Flock staff has dealt with a number 
>  > > of
>  > > issues regarding talks and votes.  We'd like to address these
>  > > shortly,
>  > > so as to limit similar issues in the immediate future.
>  > > 
>  > > It has come to our attention that at least one Flock talk 
>  > > received a
>  > > large number of votes from people that have no realistic 
>  > > intention of
>  > > attending the conference.
>  > 
>  > Does this account for people who cannot attend but plan to view the
>  > recordings later? Because I'd argue those people *should* still 
>  > have a
>  > vote.
>  
>  In this specific situation, doubtful.
>  
>  In general, your point is valid.  However, streaming and recording is
>  always one of the more troublesome items to cover.  We'll need to 
>  make
>  sure it is actually going to be done before the vote is opened so
>  people are aware if that is even an option.
>   
Real time streaming with recording is really complex. Recording it less
complex, the weak link is the hard work of collecting and uploading the
material after the adrenaline is gone. For what is worth, I have been
going back to flock 2014 talks, some that I was not able to attend
because I was in another room or even review those that I did attend.

Apart from voting issue, I think recording should be done. There is a
lot of effort/budget in put a team of great people together. The videos
will add value to the results of the event.

Neville

[Toshio Kuratomi]

On Fri, Jun 05, 2015 at 09:38:22AM -0600, Neville A. Cross wrote:
>  On Fri, 2015-06-05 at 08:15 -0400, Josh Boyer wrote:
>  > On Fri, Jun 5, 2015 at 8:07 AM, Stephen Gallagher <
>  > sgallagh(a)redhat.com&gt; wrote:
>  > > 
>  > > Does this account for people who cannot attend but plan to view the
>  > > recordings later? Because I'd argue those people *should* still 
>  > > have a
>  > > vote.
>  > 
>  > In this specific situation, doubtful.
>  > 
>  > In general, your point is valid.  However, streaming and recording is
>  > always one of the more troublesome items to cover.  We'll need to 
>  > make
>  > sure it is actually going to be done before the vote is opened so
>  > people are aware if that is even an option.
>  > 
>   
If we don't solve the recording issues we should remember to update the
election description for next year:

"Even if you cannot attend Flock in person, these sessions will be streamed
and recorded, so your vote is appreciated!"

- https://admin.fedoraproject.org/voting/about/flock-2015

>  Real time streaming with recording is really complex. Recording it
> less
>  complex, the weak link is the hard work of collecting and uploading the
>  material after the adrenaline is gone. For what is worth, I have been
>  going back to flock 2014 talks, some that I was not able to attend
>  because I was in another room or even review those that I did attend.
>  
>  Apart from voting issue, I think recording should be done. There is a
>  lot of effort/budget in put a team of great people together. The videos
>  will add value to the results of the event.
>   
What's needed to get the recordings into the hands of remote viewers?
Collecting the videos from different pieces of hardware?  Do we
do postprocessing?  Upload on limited conference center bandwidth?  What
services are we uploading to?

-Toshio

[Kushal Das]

On Fri, Jun 5, 2015 at 5:37 PM, Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  On Thu, 2015-06-04 at 15:56 -0400, Josh Boyer wrote:
> > Hello,
> >
> > Over the past few weeks the Flock staff has dealt with a number of
> > issues regarding talks and votes.  We'd like to address these
> > shortly,
> > so as to limit similar issues in the immediate future.
> >
> > It has come to our attention that at least one Flock talk received a
> > large number of votes from people that have no realistic intention of
> > attending the conference.
>
>  Does this account for people who cannot attend but plan to view the
>  recordings later? Because I'd argue those people *should* still have a
>  vote.
>  
This one was like "vote for my talk, or else I will not move out of
your desk", the
speaker was too interested to get the talk selected.


Kushal
-- 
Fedora Cloud Engineer
CPython Core Developer
http://kushaldas.in